> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Rust API Dynamo publishes 25 Rust crates for release `1.4.0`. #### [dynamo-kv-router](https://docs.rs/dynamo-kv-router/1.4.0) KV-aware request routing library #### [dynamo-llm](https://docs.rs/dynamo-llm/1.4.0) LLM inference engine #### [dynamo-memory](https://docs.rs/dynamo-memory/1.4.0) Memory management utilities #### [dynamo-runtime](https://docs.rs/dynamo-runtime/1.4.0) Core distributed runtime library #### [kvbm-logical](https://docs.rs/kvbm-logical/1.4.0) Logical layer for the KV Block Manager #### [dynamo-bench](https://docs.rs/dynamo-bench/1.4.0) Lightweight HTTP benchmarks for Dynamo endpoints #### [dynamo-config](https://docs.rs/dynamo-config/1.2.1) Configuration management #### [dynamo-data-gen](https://docs.rs/dynamo-data-gen/1.4.0) Schemas and primitives for Dynamo data generation and replay traces #### [dynamo-kv-hashing](https://docs.rs/dynamo-kv-hashing/1.4.0) Request-to-lineage-hash contract for KV cache identity #### [dynamo-parsers](https://docs.rs/dynamo-parsers/7.0.1) Protocol parsers (SSE, JSON streaming) #### [dynamo-parsers-v2](https://docs.rs/dynamo-parsers-v2/0.1.23) Successor parser line to dynamo-parsers, consumed by the Frontend #### [dynamo-protocols](https://docs.rs/dynamo-protocols/5.0.1) Async OpenAI-compatible API client #### [dynamo-renderer](https://docs.rs/dynamo-renderer/4.0.0) Chat-template rendering used by the Dynamo Frontend #### [dynamo-rl](https://docs.rs/dynamo-rl/1.4.0) Dynamo RL worker discovery API #### [dynamo-tokenizers](https://docs.rs/dynamo-tokenizers/1.5.4) Tokenizer library for LLM inference #### [dynamo-tokens](https://docs.rs/dynamo-tokens/1.4.0) Tokenizer bindings for LLM inference #### [dynamo-truthy](https://docs.rs/dynamo-truthy/1.4.0) Canonical truthy/falsy boolean flag parsing #### [fastokens](https://docs.rs/fastokens/0.2.0) Rust BPE tokenizer backend consumed by the Frontend #### [kvbm-common](https://docs.rs/kvbm-common/1.4.0) Shared types for the KV Block Manager #### [kvbm-config](https://docs.rs/kvbm-config/1.4.0) KVBM configuration for Tokio, Rayon, and Messenger runtimes #### [kvbm-engine](https://docs.rs/kvbm-engine/1.4.0) Distributed coordination primitives for KVBM #### [kvbm-kernels](https://docs.rs/kvbm-kernels/1.4.0) CUDA kernels for the KV Block Manager #### [kvbm-physical](https://docs.rs/kvbm-physical/1.4.0) Physical block layer for the KV Block Manager #### [dynamo-mocker](https://docs.rs/dynamo-mocker/1.4.0) Inference engine simulator for benchmarking #### [dynamo-async-openai](https://docs.rs/dynamo-async-openai/1.0.2) Legacy OpenAI client; use dynamo-protocols ## Core Crates | Crate | Summary | Version | Install | | ------------------------------------------------------------ | -------------------------------------- | ------- | ---------------------------------- | | [`dynamo-kv-router`](https://docs.rs/dynamo-kv-router/1.4.0) | KV-aware request routing library | `1.4.0` | `cargo add dynamo-kv-router@1.4.0` | | [`dynamo-llm`](https://docs.rs/dynamo-llm/1.4.0) | LLM inference engine | `1.4.0` | `cargo add dynamo-llm@1.4.0` | | [`dynamo-memory`](https://docs.rs/dynamo-memory/1.4.0) | Memory management utilities | `1.4.0` | `cargo add dynamo-memory@1.4.0` | | [`dynamo-runtime`](https://docs.rs/dynamo-runtime/1.4.0) | Core distributed runtime library | `1.4.0` | `cargo add dynamo-runtime@1.4.0` | | [`kvbm-logical`](https://docs.rs/kvbm-logical/1.4.0) | Logical layer for the KV Block Manager | `1.4.0` | `cargo add kvbm-logical@1.4.0` | ## Supporting Crates | Crate | Summary | Version | Install | | --------------------------------------------------------------- | ------------------------------------------------------------------- | -------- | ------------------------------------ | | [`dynamo-bench`](https://docs.rs/dynamo-bench/1.4.0) | Lightweight HTTP benchmarks for Dynamo endpoints | `1.4.0` | `cargo add dynamo-bench@1.4.0` | | [`dynamo-config`](https://docs.rs/dynamo-config/1.2.1) | Configuration management | `1.2.1` | `cargo add dynamo-config@1.2.1` | | [`dynamo-data-gen`](https://docs.rs/dynamo-data-gen/1.4.0) | Schemas and primitives for Dynamo data generation and replay traces | `1.4.0` | `cargo add dynamo-data-gen@1.4.0` | | [`dynamo-kv-hashing`](https://docs.rs/dynamo-kv-hashing/1.4.0) | Request-to-lineage-hash contract for KV cache identity | `1.4.0` | `cargo add dynamo-kv-hashing@1.4.0` | | [`dynamo-parsers`](https://docs.rs/dynamo-parsers/7.0.1) | Protocol parsers (SSE, JSON streaming) | `7.0.1` | `cargo add dynamo-parsers@7.0.1` | | [`dynamo-parsers-v2`](https://docs.rs/dynamo-parsers-v2/0.1.23) | Successor parser line to dynamo-parsers, consumed by the Frontend | `0.1.23` | `cargo add dynamo-parsers-v2@0.1.23` | | [`dynamo-protocols`](https://docs.rs/dynamo-protocols/5.0.1) | Async OpenAI-compatible API client | `5.0.1` | `cargo add dynamo-protocols@5.0.1` | | [`dynamo-renderer`](https://docs.rs/dynamo-renderer/4.0.0) | Chat-template rendering used by the Dynamo Frontend | `4.0.0` | `cargo add dynamo-renderer@4.0.0` | | [`dynamo-rl`](https://docs.rs/dynamo-rl/1.4.0) | Dynamo RL worker discovery API | `1.4.0` | `cargo add dynamo-rl@1.4.0` | | [`dynamo-tokenizers`](https://docs.rs/dynamo-tokenizers/1.5.4) | Tokenizer library for LLM inference | `1.5.4` | `cargo add dynamo-tokenizers@1.5.4` | | [`dynamo-tokens`](https://docs.rs/dynamo-tokens/1.4.0) | Tokenizer bindings for LLM inference | `1.4.0` | `cargo add dynamo-tokens@1.4.0` | | [`dynamo-truthy`](https://docs.rs/dynamo-truthy/1.4.0) | Canonical truthy/falsy boolean flag parsing | `1.4.0` | `cargo add dynamo-truthy@1.4.0` | | [`fastokens`](https://docs.rs/fastokens/0.2.0) | Rust BPE tokenizer backend consumed by the Frontend | `0.2.0` | `cargo add fastokens@0.2.0` | | [`kvbm-common`](https://docs.rs/kvbm-common/1.4.0) | Shared types for the KV Block Manager | `1.4.0` | `cargo add kvbm-common@1.4.0` | | [`kvbm-config`](https://docs.rs/kvbm-config/1.4.0) | KVBM configuration for Tokio, Rayon, and Messenger runtimes | `1.4.0` | `cargo add kvbm-config@1.4.0` | | [`kvbm-engine`](https://docs.rs/kvbm-engine/1.4.0) | Distributed coordination primitives for KVBM | `1.4.0` | `cargo add kvbm-engine@1.4.0` | | [`kvbm-kernels`](https://docs.rs/kvbm-kernels/1.4.0) | CUDA kernels for the KV Block Manager | `1.4.0` | `cargo add kvbm-kernels@1.4.0` | | [`kvbm-physical`](https://docs.rs/kvbm-physical/1.4.0) | Physical block layer for the KV Block Manager | `1.4.0` | `cargo add kvbm-physical@1.4.0` | ## Development and Testing | Crate | Summary | Version | Install | | ------------------------------------------------------ | ------------------------------------------- | ------- | ------------------------------- | | [`dynamo-mocker`](https://docs.rs/dynamo-mocker/1.4.0) | Inference engine simulator for benchmarking | `1.4.0` | `cargo add dynamo-mocker@1.4.0` | ## Deprecated Crates | Crate | Summary | Version | Install | | ------------------------------------------------------------------ | ------------------------------------------ | ------- | ------------------------------------- | | [`dynamo-async-openai`](https://docs.rs/dynamo-async-openai/1.0.2) | Legacy OpenAI client; use dynamo-protocols | `1.0.2` | `cargo add dynamo-async-openai@1.0.2` | ## Language Bindings | Binding | Language | Summary | Source | | ---------------- | -------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | `dynamo-codegen` | Python | PyO3 code generation for the dynamo.\_core module. | [`lib/bindings/python/codegen`](https://github.com/ai-dynamo/dynamo/tree/main/lib/bindings/python/codegen) | | `libdynamo_llm` | C | C bindings for the Dynamo LLM library. | [`lib/bindings/c`](https://github.com/ai-dynamo/dynamo/tree/main/lib/bindings/c) |