> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.6.1) (default): https://docs.dynamo.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.dynamo.nvidia.com/dynamo/dev/llms.txt
> - v1.6.1: https://docs.dynamo.nvidia.com/dynamo/v1.6.1/llms.txt
> - v1.6.0: https://docs.dynamo.nvidia.com/dynamo/v1.6.0/llms.txt
> - v1.5.1: https://docs.dynamo.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.dynamo.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.dynamo.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.dynamo.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.dynamo.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.dynamo.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.dynamo.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.dynamo.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.dynamo.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.dynamo.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.dynamo.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.dynamo.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.dynamo.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.dynamo.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.dynamo.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.dynamo.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.dynamo.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.dynamo.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.dynamo.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Voice Pipelines

Dynamo serves voice workloads through OpenAI-compatible endpoints. Run separate
automatic speech recognition (ASR), LLM, and text-to-speech (TTS) workers for a
cascaded pipeline, or one vLLM-Omni model that takes audio in and returns audio
out.

![A voice agent application chooses between a cascaded pipeline of streaming ASR, an LLM, and text to speech, each on its own endpoint, or one full-duplex audio-in, audio-out model on the realtime endpoint.](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/8758b03770530492a096f289d3a2bd42f2ba6a846e023a20c779319c5f15a2ba/assets/img/voice-pipeline-stages.svg)

## Endpoints

| Stage | Endpoint | Output | Guide |
| --- | --- | --- | --- |
| Streaming ASR | `/v1/realtime` WebSocket, `session.type="transcription"` | Transcript events | [Speech to Text](/dynamo/dev/speech/speech-to-text) |
| LLM | `/v1/chat/completions` with `stream: true` | Text deltas | [Local Guide](/dynamo/dev/cli/getting-started/quickstart) |
| TTS | `/v1/audio/speech` | Audio, streamed for `wav` and `pcm` | [Text-to-Speech](/dynamo/dev/speech/text-to-speech) |
| Audio-in, audio-out model | `/v1/realtime` WebSocket, `session.type="realtime"` | Text and audio events | [Speech-to-Speech](/dynamo/dev/speech/speech-to-speech) |

For a cascaded voice agent, connect an application such as Pipecat to the three
stage endpoints. The application owns conversation history, turn detection,
interruptions, and orchestration; Dynamo serves and routes each stage.

## Set up the backend

<Tabs>
<Tab title="vLLM and vLLM-Omni">

The speech pages in this section run on vLLM (speech to text) and vLLM-Omni (TTS
and speech-to-speech).

- Speech to text needs a working
  [vLLM backend setup](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/overview#installation)
  plus vLLM's audio dependencies:

  ```bash
  pip install 'vllm[audio]'
  ```

- Text-to-speech and speech-to-speech need vLLM-Omni:


[vLLM-Omni](https://github.com/vllm-project/vllm-omni) generates images, video, and audio through `python -m dynamo.vllm.omni`.

**Prerequisites**

- A working vLLM backend setup.
- An `amd64` host. Dynamo container builds do not install vLLM-Omni on `arm64`.

Dynamo container images include vLLM-Omni. A PyPI installation of `ai-dynamo[vllm]` does not include it. For a source installation, install the vLLM-Omni release that matches your vLLM version:

```bash
pip install git+https://github.com/vllm-project/vllm-omni.git@<version>
```

The `agg_omni_*.sh` and `disagg_omni_glm_image.sh` launchers start the frontend and workers directly on one host. They are intended for local development and testing and do not create a `DynamoGraphDeployment`.

Generated media is written through [fsspec](https://filesystem-spec.readthedocs.io/) storage. The default is `file:///tmp/dynamo_media`. Set `--media-output-fs-url` for S3, GCS, or Azure Blob storage, and optionally `--media-output-http-url` to rewrite the URLs returned in responses.

vLLM-Omni ships built-in stage configurations for supported models. Set `--stage-configs-path` only to override them.


See the [vLLM backend setup](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/overview) and the [vLLM-Omni Configuration reference](/dynamo/dev/reference/backends/v-llm-omni-configuration) for every flag.

</Tab>
<Tab title="Nemotron Speech NIMs (Kubernetes)">

The Nemotron Speech cascade deploys Nemotron ASR and Magpie TTS Speech NIMs with
Dynamo adapters, plus a Nemotron LLM served by vLLM. ASR, LLM, and TTS run in
separate pods.

Follow the
[deployment and smoke-test instructions](https://github.com/ai-dynamo/dynamo/tree/main/examples/nemotron_speech_cascaded_pipeline#deploy-on-kubernetes)
in the example. It requires compatible source-built Dynamo images; check the
example's runtime-image requirements before you deploy. The deployment smoke test
checks the speech endpoints, not a complete browser voice agent. See
[external orchestration](https://github.com/ai-dynamo/dynamo/tree/main/examples/nemotron_speech_cascaded_pipeline#external-orchestration)
for client integration.

</Tab>
</Tabs>

## Run a Pipeline Locally

Bring up and test each stage on its own before you connect them:

1. Start an ASR worker and stream an audio file to it:
   [Speech to Text](/dynamo/dev/speech/speech-to-text).
2. Start an LLM with any vLLM, SGLang, or TensorRT-LLM launcher, for example
   `bash examples/backends/vllm/launch/agg.sh`.
3. Start a TTS worker and synthesize a reply:
   [Text-to-Speech](/dynamo/dev/speech/text-to-speech).

Each launcher starts its own frontend on port 8000, so test the stages one at a
time on a single host.

## Incremental Text Input

Incremental text input lets an orchestrator send ASR text to the LLM while the user is still speaking. The worker starts prefilling the text it has received so far, in full KV cache blocks, so those tokens are already in the prefix cache when the final request arrives. The client sends `input_text.append` events with the partial text, then `input_text.commit` with the final text to start generation, or `input_text.clear` to discard the prefill work.

![Without incremental text input, LLM prefill waits for the final text from ASR. With it, prefill runs on the partial text while the user is still speaking, so the first token arrives earlier. Illustrative timings.](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/ab61e4e090d08aa3a884c13d0927bc906f0401d6b40c1eb7d742d2d4c0ee8dc2/assets/img/voice-incremental-text-input.svg)

It applies to text-only `/v1/realtime` sessions on aggregated vLLM workers, without tools or disaggregation, and it needs prefix caching. Short or revised input can yield no benefit. The input events are Dynamo extensions, not standard OpenAI events, and realtime text does not yet apply `--dyn-reasoning-parser`. ASR and TTS orchestration remain external. Session recovery across worker failures is a further goal and is not provided by the examples above.