What Text to Speech Actually Is

Text to speech (TTS) is the process of converting written text into spoken audio using machine learning models. Under the hood, a modern TTS system takes a string of text, runs it through a neural network trained on thousands of hours of recorded human speech, and outputs an audio waveform that mimics the cadence, intonation, and timbre of a target voice.

ElevenLabs sits at the higher end of quality for this category. The company trains large voice models that can either reproduce a specific speaker’s voice from a short sample, generate new synthetic voices from text descriptions, or clone a voice you upload as a clean audio file. The output is typically returned as MP3 or PCM audio, either streamed or as a complete file.

The technical contract is straightforward. You send text plus a voice identifier plus some settings, and you get back an audio file. The interesting parts are the settings, the voice library, and how the model handles edge cases like numbers, acronyms, and code.

For developers, the value proposition is that you no longer need to record voice talent, book studio time, or maintain audio files. You make an API call and receive broadcast-ready audio in seconds. The trade-off is that you inherit the model’s quirks, so understanding those quirks is the difference between amateur and production-grade output.

Setup and Authentication

ElevenLabs uses a standard API key model. Sign up at elevenlabs.io, navigate to Profile then API Keys, and generate a new key. Store it as an environment variable rather than hardcoding it in source control.

The base URL is https://api.elevenlabs.io/v1. The two endpoints you’ll use most are /v1/text-to-speech/{voice_id} for synthesis and /v1/voices for the voice catalog. A third endpoint, /v1/text-to-speech/{voice_id}/stream, returns audio chunks in real time for latency-sensitive applications.

For local development, set the key in your shell:

export ELEVENLABS_API_KEY=“your_key_here”

If you prefer Python, install the official SDK with pip install elevenlabs. If you prefer raw HTTP, the requests library or curl work fine and give you more visibility into the actual request and response.

There’s also a free tier that gives you a limited number of characters per month, which is enough to test the API and validate a workflow before committing to a paid plan. For team usage, expect to move to a paid tier quickly because character consumption grows faster than people estimate.

First Working Example

Here is a complete working example using curl. Replace the voice_id with one you grab from the /v1/voices endpoint, and you have a working text to speech call.

curl -X POST “https://api.elevenlabs.io/v1/text-to-speech/21m00Tcm4TlvDq8ikWAM”
-H “xi-api-key: $ELEVENLABS_API_KEY”
-H “Content-Type: application/json”
-d ’{ “text”: “The quarterly revenue increased by twelve percent compared to the previous quarter.”, “model_id”: “eleven_multilingual_v2”, “voice_settings”: { “stability”: 0.5, “similarity_boost”: 0.75 } }’
—output speech.mp3

That voice_id belongs to Rachel, one of the default voices. The response is an audio binary that gets written straight to speech.mp3 on disk. Open the file in any audio player and you should hear a clean, natural-sounding read of the sentence.

In Python with the official SDK, the equivalent is:

from elevenlabs import generate, play, set_api_key set_api_key(“your_key_here”) audio = generate( text=“The quarterly revenue increased by twelve percent.”, voice=“Rachel”, model=“eleven_multilingual_v2” ) play(audio)

The play helper requires the simpleaudio package and will output the audio through your default speaker. If you want to save the file instead, swap in the save function:

from elevenlabs import generate, save audio = generate(text=“Hello world”, voice=“Rachel”) save(audio, “output.mp3”)

Once you have a working file, play it back and listen for pronunciation, pacing, and tone. If anything sounds off, that is where the voice settings come in, and that is where most of the tuning time goes.

Key Settings That Matter

The default voice settings produce decent output, but you should understand what each dial does before shipping audio to a real audience.

Stability controls how expressive the voice is. Low values around 0.2 make the model more emotional and varied, which can sound dramatic or occasionally unstable. High values around 0.8 make the output more consistent and predictable, which is what you want for narration where a steady reading style is appropriate. A safe starting point is 0.5.

Similarity boost controls how closely the model adheres to the original voice sample. Crank it up and the output sounds more like the reference, sometimes at the cost of audio artifacts. Drop it down and you get a cleaner but more generic result. A safe starting point is 0.75.

Style exaggeration is a third dial that controls how much the model emphasizes the stylistic qualities of the voice. It is most useful with expressive voices and can be left at 0 for clean narration. Push it above 0.5 only when you have tested the result and confirmed it sounds intentional rather than forced.

Model selection matters more than most people realize. The multilingual v2 family handles around 29 languages and produces natural results across all of them. If you only need English, the monolingual models can be slightly faster and cleaner. Turbo models prioritize latency over absolute quality and are the right choice for real-time voice agents.

Output format is set via the output_format parameter and supports options like mp3_44100_128, pcm_16000, and pcm_44100. For phone systems, use 8kHz PCM. For podcast production, use 44.1kHz MP3 at high bitrate. For web playback, mp3_44100_128 is a good default.

Streaming is available via the /stream endpoint and returns chunks as they are generated, which lets you start playback before the full audio is ready. For most batch use cases the standard endpoint is fine, but for any application where the user is waiting on the audio to respond to a prompt, use the streaming variant.

Where It Shines

Text to speech is a genuine productivity multiplier in a handful of specific scenarios.

Audiobook production at scale is the obvious one. If you have a long-form text catalog, you can convert hundreds of titles to audio without coordinating narrators. Some publishers use TTS for first drafts, then re-record specific chapters with a human narrator if the final product demands it.

Internal training content is another strong fit. Compliance videos, onboarding walkthroughs, and policy explainers can all be generated in the same afternoon, in multiple languages, without booking recording sessions. The voice quality is more than good enough for content employees watch once and move on.

Product demos and explainer videos benefit from a consistent voice across many short clips. Instead of recording 50 variations of the same intro, you generate them all and keep the audio identical. This is also useful for A/B testing where you want to swap voice characteristics without rebooking talent.

Real-time applications like voice agents, IVR systems, and accessibility tools use streaming TTS to respond to user input in under a second. The latency is low enough on the turbo models that the experience feels conversational rather than stilted.

Localization is where multilingual models earn their keep. A single English script can become Spanish, French, German, or Japanese output without changing the voice identity. For product teams shipping globally, this compresses what used to be a months-long translation and recording project into a few hours of prompt work.

Where It Fails

Despite the quality improvements, TTS still has real limitations that you should plan around rather than discover in production.

Pronunciation of unusual proper nouns, technical jargon, and brand names is hit or miss. The model interprets text phonetically and occasionally picks the wrong syllable stress or mispronounces an acronym. You can fix this with phoneme overrides or by spelling words phonetically, but it is an extra step that does not scale well across large content libraries.

Emotion control is limited. You can shift the tone of voice through stability and style settings, but you cannot reliably ask for “frustrated” or “whispered” output and get a consistent result across the whole clip. The model has a baseline emotional range and stays within it.

Long-form consistency is a known weak point. When you generate 30 minutes of audio, the voice can drift slightly in pitch or pacing partway through. Some providers address this with chunking and re-anchoring, but you should listen to long outputs in full before publishing anything mission critical.

Latency for the highest-quality models can run several seconds for a single paragraph. For applications that need sub-second response, you need the turbo variants and accept a small quality drop. The difference is noticeable on careful listening but rarely matters in conversational contexts.

Cost adds up at scale. Pricing is typically per character, and while the per-character rate is low, generating millions of characters per month for a popular product becomes a meaningful line item. Build a usage monitoring step into your pipeline and set hard limits so a runaway script cannot burn through your monthly budget overnight.

Licensing for cloned voices is a legal gray area. The platform terms restrict certain uses, and you generally cannot clone a voice you do not have explicit rights to. For commercial work, use only voices from the official library or voices you have created with documented consent from the speaker.

Practical Workflow Pattern

A production TTS pipeline usually looks like the same set of stages regardless of the use case, so it is worth building the structure once and reusing it.

Step one is voice selection. Browse the voice library and pick a base voice that matches your brand. Test it with a paragraph that includes the most common edge cases you will encounter, including numbers, acronyms, punctuation, and at least one foreign word. The same voice sounds different on different content, so the test paragraph matters.

Step two is building a pre-processing layer. Write a small function that takes raw text, expands abbreviations (Dr. becomes Doctor), converts numbers to words when needed, and adds appropriate pauses. This single function will fix most of your pronunciation issues and is the highest-leverage piece of the entire pipeline.

Step three is the API call itself, wrapped in a module that handles authentication, retries, and error logging. Treat the API like any other external dependency and never trust it to be available all the time. Network blips and rate limits will happen, and your pipeline should handle them gracefully.

Step four is post-processing. Most workflows route the output MP3 through a normalization step using ffmpeg’s loudnorm filter so all your audio plays at consistent volume. A short script that calls ffmpeg on every generated file is enough.

Step five is storage and delivery. Save the audio to object storage with a deterministic filename based on a hash of the input text. This gives you caching for free and avoids regenerating identical audio when the same content is requested twice.

For a typical business workflow, the pattern becomes: text content lives in your CMS, a job pulls new entries and pushes them through the TTS pipeline, audio files land in a CDN, and your product serves them next to the original text. The whole loop runs on a schedule and is monitored like any other background job.

The realistic expectation is that TTS replaces about 70% of your audio production needs. The remaining 30%, usually hero content like a brand anthem or a major launch video, still benefits from a real voice actor. The trick is knowing which content goes in which bucket and building the pipeline so the boundary is clear and the handoff is clean.

If this is the kind of problem agents can help with, the free Working With Claude field guide is the practical next step. Thirty-two pages, no fluff. Get the free guide.