Compatibility
Compatibility by version
CUDA 12 container images discontinued; EFA variants go multi-arch as -efa; GA wheels published as 1.3.0.post1 (containers stay :1.3.0); UCX 1.20.x.
Backend versions listed are the versions tested and supported for the selected release. TensorRT-LLM does not support Python 3.11.
For extended driver compatibility beyond the listed minimums, including forward compatibility and cuda-compat packages, see the CUDA Compatibility documentation.
See Release Artifacts for the full artifact inventory — container images, wheels, Helm charts, and crates — and Model Early Access Builds for per-model early access container builds.
Find a Compatible Release
Pick your backend and the CUDA generation your host driver supports to see which Dynamo releases you can run — and what to pull for the current one.
What runs where
Releases that match your backend and driver
Driver floors from the CUDA & driver history; pull commands shown for the current release only.
Platform Notes
Dynamo ships multi-arch (x86_64 + ARM64) container images. Wheels are built in a manylinux_2_28-compatible environment and validated on CentOS Stream 9 and Ubuntu 22.04/24.04; other Linux distributions are expected to work but are not officially verified.
Cloud Service Providers
Amazon Linux 2023 (AWS) · x86_64 · Supported
AL2023 TensorRT-LLM limitation: there is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with docker run --network host ... due to a bug in mpi4py. Replace the --network host flag with precise networking configuration by mapping only the necessary ports (4222 for NATS, 2379/2380 for etcd, 8000 for the frontend).
Feature Support
- Disaggregated Serving · vLLM: Prefill/decode separation with NIXL KV transfer
- KV Block Manager · SGLang: Work in progress across all combinations
- Multimodal (Image) · SGLang: Not compatible with KV-aware routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D)
- Multimodal (Image) · TRT-LLM: Image URLs + pre-computed embeddings. Disagg: EP/D + E/P/D. KV-aware routing via dedicated MM Router Worker (requires KV event publishing)
- Multimodal (Image) · vLLM: With KV-aware routing, image-aware routing on documented paths
- Multimodal (Video) · vLLM: Video input with frame sampling
- Multimodal (Audio) · vLLM: Qwen2-Audio, experimental
- Request Migration · TRT-LLM: Work in progress with multimodal
- Request Cancellation · SGLang: Remote-prefill-phase cancellation not supported in disaggregated mode
- Request Cancellation · TRT-LLM: Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue)
- LoRA · vLLM: Dynamic load/unload; KV-aware routing supports adapter affinity
- Speculative Decoding · SGLang: Code hooks exist; no examples or docs yet
- Speculative Decoding · vLLM: Eagle3
- GPU Memory Service · SGLang: Weights and KV; upstream integration remains in progress
- GPU Memory Service · TRT-LLM: Weights only; multinode and upstream integration remain in progress
- GPU Memory Service · vLLM: Weights and KV; upstream integration remains in progress
- Shadow Engine Failover · SGLang: No KV-cache reuse or hardware fault tolerance
- Shadow Engine Failover · TRT-LLM: No KV-cache reuse or hardware fault tolerance
- Shadow Engine Failover · vLLM: Software-process failover only; no KV-cache reuse or hardware fault tolerance
- Dynamo Snapshot · SGLang: Single-GPU supported; multi-GPU and multinode remain in progress
- Dynamo Snapshot · TRT-LLM: Single-GPU aggregated text-worker path only
- Dynamo Snapshot · vLLM: Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress
Superscripts reference the numbered notes above; full per-backend detail follows.
Per-Backend Detail
vLLM
SGLang
TensorRT-LLM
vLLM offers the broadest feature coverage in Dynamo, with full support for disaggregated serving, KV-aware routing, KV block management, LoRA adapters, and multimodal inference including video and audio.
Source: docs/backends/vllm/README.md
Feature Interactions
Pairwise feature-by-feature compatibility within each backend. Each cell reports whether the row feature works together with the column feature. A — marks the diagonal or a combination that does not apply; blank cells are the mirror of the populated lower triangle.
Legend: ✓ Supported · WIP Work in Progress / Experimental / Limited
vLLM Feature Interactions
Notes:
- Multimodal + KV-Aware Routing: Image-aware KV routing is supported in the documented vLLM paths. The default Rust frontend path supports model families handled by
llm-multimodal; the Python chat-processor path delegates to vLLM’s multimodal processor. (Source)- KV-Aware LoRA Routing: vLLM supports routing requests based on LoRA adapter affinity.
- Audio Support: vLLM supports audio models like Qwen2-Audio (experimental). (Source)
- Video Support: vLLM supports video input with frame sampling. (Source)
- Speculative Decoding: Eagle3 support documented. (Source)
SGLang Feature Interactions
Notes:
- Multimodal + KV-Aware Routing: Not supported. (Source)
- Multimodal Patterns: Supports simple Aggregated EPD, E/PD, and E/P/D patterns. Traditional Disagg EP/D is not supported. (Source)
- Request Cancellation: Cancellation during the remote prefill phase is not supported in disaggregated mode. (Source)
- Speculative Decoding: Code hooks exist (
spec_decode_statsin publisher), but no examples or documentation yet.
TensorRT-LLM Feature Interactions
Notes:
- Multimodal Disaggregation: Supports EP/D (Traditional) and E/P/D (Full Disaggregation) image flows, including image URLs and pre-computed embeddings. (Source)
- Multimodal + KV-Aware Routing: The native Rust frontend routes supported models using image-aware KV overlap. TRT-LLM workers must publish KV events with block reuse enabled. (Source)
- Request Cancellation: Due to known issues, the TensorRT-LLM engine is temporarily not notified of request cancellations, meaning allocated resources for cancelled requests are not freed.