> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Multimodal Model Serving

Dynamo supports multimodal inference across multiple LLM backends, enabling models to process images, video, and audio alongside text.

## Which Feature to Use

Dynamo provides support for improving latency and throughput for multimodal workloads, with image and video inputs, through the following features. Use them together or separately, depending on your workload characteristics:

| Workload | Feature | Benefit |
|----------|---------|---------|
| Workload where significant time is spent preprocessing. | [Parallel media decoding](/dynamo/dev/multimodal/parallel-media-decoding) | Move image fetching and decompression off the worker's critical path. |
| Workload includes repeated multimodal content across requests. | [Embedding cache](/dynamo/dev/multimodal/embedding-cache) | Skip re-encoding repeated multimodal content. |
| Workload includes repeated multimodal content across requests, and multiple backend workers serve multimodal requests. | [Multimodal KV routing](/dynamo/dev/multimodal/multimodal-kv-routing) | Maximize KV cache hit rates for multimodal content. |
| Workload where media encoding is a bottleneck. | [Encoder disaggregation](/dynamo/dev/multimodal/encoder-disaggregation) | Scale encoders independently of LLM workers. |


<Info>
These features currently support image and video inputs only. Support for audio modalities will be added in upcoming releases.
</Info>

## Multimodal Performance Optimization Features

<CardGroup cols={2}>
  <Card title="Parallel Media Decoding" icon="regular image" href="/dynamo/dev/multimodal/parallel-media-decoding">
    Decode image inputs concurrently in the Rust frontend and transfer pixels through NIXL
  </Card>
  <Card title="Embedding Cache" icon="regular database" href="/dynamo/dev/multimodal/embedding-cache">
    Cache vision encoder embeddings to skip re-encoding repeated multimodal content
  </Card>
  <Card title="Multimodal KV Routing" icon="regular arrows-split-up-and-left" href="/dynamo/dev/multimodal/multimodal-kv-routing">
    Include multimodal identity in cache-aware, load-balanced worker selection
  </Card>
  <Card title="Encoder Disaggregation" icon="regular microchip" href="/dynamo/dev/multimodal/encoder-disaggregation">
    Separate vision encoding into a dedicated worker for independent scaling
  </Card>
</CardGroup>

## Example Workflows

Reference implementations for deploying multimodal models for each backend:

- [Voice Pipelines](/dynamo/dev/multimodal/voice-pipelines): streaming speech endpoints and cascaded voice agents
- [SGLang Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)
- [TensorRT-LLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)
- [vLLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)

To use an author-provided custom vision tower or projector, see [Custom Vision Encoders](/dynamo/dev/multimodal/custom-vision-encoders).

## Shared Image Download Cache

The optional shared cache stores encoded bytes from HTTP and HTTPS image URLs
before image decoding. Workers can reuse those bytes across processes while
keeping their local decoded-image caches independent. Data URLs,
frontend-decoded NIXL inputs, video, and audio do not use this cache.

### Backend Support

| Backend | Mode | Shared encoded-image cache |
| --- | --- | --- |
| vLLM | Aggregated and disaggregated paths that load URLs through Dynamo `ImageLoader` | Supported |
| TensorRT-LLM | Aggregated and disaggregated paths that load URLs through Dynamo `ImageLoader` | Supported |
| SGLang | Standard URL loading | Not supported. Dynamo passes URL strings directly to SGLang. |
| SGLang | `--frontend-decoding` | Only URL items that still reach Dynamo `ImageLoader` can use the cache. NIXL-decoded items bypass it. |

Dynamo does not provision or discover a cache service. Deploy Redis Cluster or
Dragonfly separately and provide a cluster endpoint that every participating
URL-loading worker can reach. The connection URL carries the endpoint,
authentication, and TLS choice. See [Cache Service Deployment](#cache-service-deployment)
for single-node and multi-node layouts.

Set the following environment variables on every participating worker:

| Environment variable | Default | Description |
| --- | --- | --- |
| `DYN_MM_SHARED_IMAGE_CACHE_ENABLED` | `0` | Set to `1` to enable the shared cache. |
| `DYN_MM_SHARED_IMAGE_CACHE_URL` | None | Redis Cluster or Dragonfly connection URL. Required when the cache is enabled. |
| `DYN_MM_SHARED_IMAGE_CACHE_TTL_SECS` | `3600` | Positive cache-entry lifetime in seconds. |
| `DYN_MM_SHARED_IMAGE_CACHE_CONNECT_TIMEOUT_SECS` | `0.1` | Positive connection timeout in seconds. |
| `DYN_MM_SHARED_IMAGE_CACHE_IO_TIMEOUT_SECS` | `2.0` | Positive Redis operation timeout in seconds. |
| `DYN_MM_MAX_FILE_SIZE_MB` | `64` | Maximum encoded image size downloaded from the origin, in MiB. The limit applies whether or not the shared cache is enabled, so oversized images are never stored in it. |
| `DYN_MM_IMAGE_CACHE_SESSION_SCOPED` | `0` | Set to `1` to partition Dynamo image and image-embedding caches by session affinity. |

Store the full connection URL in a Kubernetes Secret and inject it into each
worker rather than placing credentials in a manifest:

```yaml
env:
  - name: DYN_MM_SHARED_IMAGE_CACHE_ENABLED
    value: "1"
  - name: DYN_MM_SHARED_IMAGE_CACHE_URL
    valueFrom:
      secretKeyRef:
        name: multimodal-image-cache
        key: connection-url
  - name: DYN_MM_IMAGE_CACHE_SESSION_SCOPED
    value: "1"
```

<Warning>
Use a `rediss://` URL unless Redis traffic is protected by equivalent
transport encryption, such as an encrypted service mesh. A `redis://` URL
sends cache data and any URL credentials without TLS and should be used only
on an appropriately secured in-cluster network.
</Warning>

Cache reads and fills run on the request path. A miss waits for Redis `SET` so
the fill completes before the request returns. Redis errors fail open, and each
cache operation is attempted once. While a cache node is unreachable, an image
load pays one failed `GET` and one failed `SET`: each is bounded by
`DYN_MM_SHARED_IMAGE_CACHE_CONNECT_TIMEOUT_SECS` when a new connection is
refused or times out, and by `DYN_MM_SHARED_IMAGE_CACHE_IO_TIMEOUT_SECS` when
an open connection stops responding.
Warnings are emitted initially and at most once per minute; individual
operation failures remain available at debug level. Dynamo does not currently
use a bounded asynchronous write queue.

### Cache Service Deployment

Dynamo uses the Redis Cluster protocol for both backends. The client uses
`DYN_MM_SHARED_IMAGE_CACHE_URL` only to discover the cluster: it reads the slot
map from that endpoint and then connects to each node at the address the node
announces. When a node stops responding, the client rediscovers through the same
URL, so point it at an address that survives cache restarts, such as a
Kubernetes Service that selects every cache node, not at an individual pod.
After a restart, the nodes announce their new addresses and workers pick them
up without restarting. Dynamo requires redis-py 6.2 or later.

**Single-node Dragonfly.** Run Dragonfly with `--cluster_mode=emulated` so that
it answers Redis Cluster commands, and keep its default announced address (the
pod IP). Do not set `--cluster_announce_ip` to the host in
`DYN_MM_SHARED_IMAGE_CACHE_URL`: with redis-py 6.2 through 7.1, a node that
announces the same host and port as the URL removes the URL from the client's
rediscovery list after one failed operation.

**Redis Cluster.** Run the nodes as a StatefulSet with a headless Service for
per-pod DNS names, plus a regular Service that selects every node for the URL.
Configure each node with:

- `cluster-announce-ip` set to the pod IP, or `cluster-announce-hostname` set to
  the pod's DNS name together with `cluster-preferred-endpoint-type hostname`
  (Redis 7.0 or later).
- `cluster-require-full-coverage no`, so the remaining nodes keep serving while
  one node is down. Images that hash to the unavailable node fall back to the
  origin.
- A `nodes.conf` that persists across pod restarts, so a restarted node keeps its
  identity and slots. A rolling restart then keeps the cluster available. If
  every node restarts at once, all peer addresses change and the nodes need
  their peers' new addresses before they can re-form the cluster.

**Multi-node Dragonfly.** With `--cluster_mode=yes`, Dragonfly nodes do not
discover each other. A cluster orchestrator must push the slot map with
`DFLYCLUSTER CONFIG` and push it again whenever a node restarts, because the
map is kept in memory. Point the URL at a Service that selects every node, as
for Redis Cluster.

### Session Scoping

Enable `DYN_MM_IMAGE_CACHE_SESSION_SCOPED=1` when the same URL can resolve to
different content for different sessions. Dynamo derives the
scope from these headers, in precedence order:

1. `x-dynamo-session-id`
2. Recognized agent headers: Claude Code `x-claude-code-session-id` (or
   `x-claude-code-agent-id` for a child agent), Codex `thread-id`, then OpenCode
   `x-session-id`

Use `x-dynamo-session-id` to provide an explicit Dynamo session identity.
Dynamo normalizes it and the recognized agent headers into both an
`AgentContext` and the session affinity used for routing and image-cache
scoping.

<Warning>
Session scoping partitions cache entries; it is not an authentication
boundary. Unless a trusted proxy sets or overwrites `x-dynamo-session-id`,
the caller controls the scope. Configure the proxy to derive this header
from the authenticated identity on every request, and treat scope identifiers
as secrets. A caller that learns another identifier can reuse cached content
stored under that scope for the same URL.
</Warning>

When session scoping is enabled, a missing or blank scope bypasses the local
decoded-image cache, in-flight request deduplication, the shared encoded-image
cache, and Dynamo-owned image embedding caches. A valid scope partitions all of
those caches. The scope does not partition backend KV cache keys; affinity can
route a session back to the same worker, but KV identity remains content-based.