DocumentationThis document provides the support matrix for Dynamo, including hardware, software and build instructions.
If you are using a GPU, the following GPU models and architectures are supported:
Dynamo is compatible with the following platforms:
[!Note] Wheels are built using a manylinux_2_28-compatible environment and they have been validated on CentOS 9 and Ubuntu (22.04, 24.04).
Compatibility with other Linux distributions is expected but has not been officially verified yet.
[!Caution] KV Block Manager is supported only with Python 3.12. Python 3.12 support is currently limited to Ubuntu 24.04.
[!Important] Specific versions of TensorRT-LLM supported by Dynamo are subject to change. Currently TensorRT-LLM does not support Python 3.11 so installation of the ai-dynamo[trtllm] will fail.
[!Caution] There is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with
docker run --network host ...due to a bug in mpi4py. To avoid this issue, replace the--network hostflag with more precise networking configuration by mapping only the necessary ports (e.g., 4222 for nats, 2379/2380 for etcd, 8000 for frontend).
Dynamo currently provides build support in the following ways:
Wheels: We distribute Python wheels of Dynamo and KV Block Manager:
Dynamo Runtime Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Runtime for each of the LLM inference frameworks on NGC:
Dynamo Kubernetes Operator Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Operator on NGC:
Helm Charts: NGC hosts the helm charts supporting Kubernetes deployments of Dynamo:
Rust Crates:
Once you’ve confirmed that your platform and architecture are compatible, you can install Dynamo by following the instructions in the Quick Start Guide.
