Dynamo Support Matrix

This document provides the support matrix for Dynamo, including hardware, software and build instructions.

Hardware Compatibility

GPU Compatibility

If you are using a GPU, the following GPU models and architectures are supported:

Platform Architecture Compatibility

Dynamo is compatible with the following platforms:

[!Note] Wheels are built using a manylinux_2_28-compatible environment and they have been validated on CentOS 9 and Ubuntu (22.04, 24.04).

Compatibility with other Linux distributions is expected but has not been officially verified yet.

[!Caution] KV Block Manager is supported only with Python 3.12. Python 3.12 support is currently limited to Ubuntu 24.04.

Software Compatibility

Runtime Dependency

Build Dependency

[!Important] Specific versions of TensorRT-LLM supported by Dynamo are subject to change. Currently TensorRT-LLM does not support Python 3.11 so installation of the ai-dynamo[trtllm] will fail.

CUDA Support by Framework

Cloud Service Provider Compatibility

AWS

[!Caution] There is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with docker run --network host ... due to a bug in mpi4py. To avoid this issue, replace the --network host flag with more precise networking configuration by mapping only the necessary ports (e.g., 4222 for nats, 2379/2380 for etcd, 8000 for frontend).

Build Support

Dynamo currently provides build support in the following ways:

• Wheels: We distribute Python wheels of Dynamo and KV Block Manager:

  • ai-dynamo
  • ai-dynamo-runtime
  • New as of Dynamo v0.7.0: kvbm as a standalone implementation.

• Dynamo Runtime Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Runtime for each of the LLM inference frameworks on NGC:

  • SGLang
  • TensorRT-LLM
  • vLLM

• Dynamo Kubernetes Operator Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Operator on NGC:

  • kubernetes-operator to simplify deployments of Dynamo Graphs.

• Helm Charts: NGC hosts the helm charts supporting Kubernetes deployments of Dynamo:

  • Dynamo CRDs
  • Dynamo Platform
  • Dynamo Graph

• Rust Crates:

  • dynamo-runtime
  • dynamo-async-openai
  • dynamo-parsers
  • dynamo-llm

Once you’ve confirmed that your platform and architecture are compatible, you can install Dynamo by following the instructions in the Quick Start Guide.