For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
This page is for version v0.9.0.
For other versions, use one of these documentation indexes:
- Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
- dev: https://docs.nvidia.com/dynamo/dev/llms.txt
- v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
- v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
- v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
- v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
- v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
- v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
- v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
- v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
- v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
- v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
- v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
- v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
- v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
- v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
- v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
- v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
- v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
- v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
- v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt
This guide demonstrates how to deploy google/gemma-3-1b-it with Variable Sliding Window Attention (VSWA) using Dynamo. Since google/gemma-3-1b-it is a small model, each aggregated, decode, or prefill worker only requires one H100 GPU or one GB200 GPU.
VSWA is a mechanism in which a model’s layers alternate between multiple sliding window sizes. An example of this is Gemma 3, which incorporates both global attention layers and sliding window layers.
[!Note]
Ensure that required services such as nats and etcd are running before starting.
Request access to google/gemma-3-1b-it on Hugging Face and set your HF_TOKEN environment variable for authentication.
It’s recommended to continue using the VSWA feature with the Dynamo 0.5.0 release and the TensorRT-LLM dynamo runtime image nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.5.0. The 0.5.1 release bundles TensorRT-LLM v1.1.0rc5, which has a regression that breaks VSWA.
