> This page is for version v1.4.1.
> For other versions, use one of these documentation indexes:
> - Latest (v1.6.0) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.6.0: https://docs.nvidia.com/dynamo/v1.6.0/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Dynamo v1.6.0

Dynamo v1.6.0 is the 19th feature release of the open-source distributed inference platform, spanning 654 merged PRs from 148 contributors. **Dynamo Router** gains pluggable flow control: request classifiers can defer, reorder, or steer requests, and every selection path now runs on one shared core with scoring knobs consolidated into one policy file. On Kubernetes, LPX brings LPU-backed inference to DynamoGraphDeployments, multinode components take role-specific pod templates, and Dynamo Snapshot backs engine failover. **Dynamo Planner** reconfigures GPU budgets at runtime, and the engines move to SGLang v0.5.21, TensorRT-LLM v1.3.0rc29, and vLLM v0.31.0.

> **Note**
>
> Breaking changes and deprecations for this release are tracked on the [Deprecations](/dynamo/reference/releases/deprecations#v160) ledger; known issues on the [Known Issues](/dynamo/reference/releases/known-issues#v160) page. Key dependency pins live on [Compatibility](/dynamo/reference/compatibility); shipped artifacts on [Release Artifacts](/dynamo/reference/release-artifacts). Model early access builds (`vX.Y.Z-<model>-dev.N`) are tracked in [Model Early Access Builds](/dynamo/reference/model-early-access-builds).

## Get v1.6.0

Pull, deploy, and install with every artifact pinned to the v1.6.0 release set.

## Highlights

## Features & Improvements

### Frontend & Agents

* **Responses Tool Selection Hardening:** Rejected unsupported tools and `tool_choice` variants on `/v1/responses` with an actionable HTTP 400 instead of dropping them silently ([#14440](https://github.com/ai-dynamo/dynamo/pull/14440)), supported function-only `tool_choice.allowed_tools` ([#14990](https://github.com/ai-dynamo/dynamo/pull/14990)), removed text-based tool call inference from non-streaming responses ([#14846](https://github.com/ai-dynamo/dynamo/pull/14846)), and restored original function names and namespaces ([#15401](https://github.com/ai-dynamo/dynamo/pull/15401)).
* **Parsers V2 Structural Tags:** Added opt-in V2 parsers for GLM, DeepSeek V4/V4.1, Kimi K2/K3, Gemma 4, Muse, and Qwen3 ([#14950](https://github.com/ai-dynamo/dynamo/pull/14950)), configured through `--dyn-structural-tag` ([#16217](https://github.com/ai-dynamo/dynamo/pull/16217)). Structural tags are on by default for supported parsers ([#15916](https://github.com/ai-dynamo/dynamo/pull/15916)), and vLLM tool-call tags can start inside reasoning ([#16220](https://github.com/ai-dynamo/dynamo/pull/16220)).
* **KServe Tensor Protocol Fidelity:** Added half-precision `Float16` and `BFloat16` KServe tensor variants ([#14150](https://github.com/ai-dynamo/dynamo/pull/14150)), reported `ModelMetadata` datatypes by KServe v2 wire names ([#14957](https://github.com/ai-dynamo/dynamo/pull/14957)), and translated Triton `ModelConfig.dims` to KServe shapes ([#14996](https://github.com/ai-dynamo/dynamo/pull/14996)).
* **Semantic Error HTTP Mapping:** Centralized semantic error mapping to HTTP status codes in Dynamo Frontend for protocol-correct OpenAI and Anthropic failures without internal diagnostics ([#14396](https://github.com/ai-dynamo/dynamo/pull/14396)), and extended the JSON error envelope to the remaining OpenAI routes ([#14002](https://github.com/ai-dynamo/dynamo/pull/14002)).
* **DeepSeek V4.1 Frontend Support:** Added mainline frontend support for DeepSeek V4.1 Flash reasoning and tool calling, enabled when workers pass both `--dyn-tool-call-parser deepseek_v41` and `--dyn-reasoning-parser deepseek_v41` ([#15030](https://github.com/ai-dynamo/dynamo/pull/15030)), with native V4.1 image inputs and tool-result image ordering aligned to the rendered prompt by consuming the selected formatter's media order from `dynamo-renderer` ([#15097](https://github.com/ai-dynamo/dynamo/pull/15097)).
* **SSE Streaming Lifecycle Hardening:** Reworked SSE streams to end on a hard kill, not a graceful stop, so delayed events still finish ([#14645](https://github.com/ai-dynamo/dynamo/pull/14645)), and made streaming handlers wait for the first backend event so an exception before output returns the non-streaming HTTP status ([#14662](https://github.com/ai-dynamo/dynamo/pull/14662)).
* **Tokenizer Cache Lifecycle Hardening:** Replaced the process-global tokenizer pool with a weak-reference registry that shuts down each tokenizer's thread pool once collected ([#13368](https://github.com/ai-dynamo/dynamo/pull/13368)), and turned `DYN_TOKENIZER_CACHE_BYTES` into one combined budget per process, keeping the 64 MiB default ([#15387](https://github.com/ai-dynamo/dynamo/pull/15387)).
* **Root-Level Thinking Token Budget:** Added a root-level `thinking_token_budget` field to chat completion and Responses requests ([#12624](https://github.com/ai-dynamo/dynamo/pull/12624)) and carried it into SGLang, which requires strict-thinking mode and a reasoning parser ([#15120](https://github.com/ai-dynamo/dynamo/pull/15120)).
* **SGLang Cross-Encoder Reranking Endpoint:** Added an OpenAI-compatible `POST /v1/rerank` endpoint for SGLang text-only cross-encoder models with a dedicated `--rerank-worker` mode and support for `top_n` ([#14032](https://github.com/ai-dynamo/dynamo/pull/14032)), and fixed scheduler crashes in rerank workers serving XLM-RoBERTa cross-encoders ([#16286](https://github.com/ai-dynamo/dynamo/pull/16286)).
* **Frontend HTTP Mutual TLS:** Added a `--tls-client-ca-cert-path` flag and `DYN_TLS_CLIENT_CA_CERT_PATH` environment variable to the frontend HTTP server, requiring a CA-signed client certificate when set, with validation that rejects a client CA supplied without both a server certificate and key. Server-only TLS behavior remained unchanged ([#14173](https://github.com/ai-dynamo/dynamo/pull/14173)).
* **Opt-In Jemalloc Preload:** Added an opt-in jemalloc allocator preload for `python -m dynamo.frontend`, activated by `DYN_FRONTEND_JEMALLOC=1`, that continues with the default allocator when the library is missing ([#14678](https://github.com/ai-dynamo/dynamo/pull/14678)), with `libjemalloc2` installed in the SGLang, TensorRT-LLM, and vLLM runtime images so the switch takes effect ([#13101](https://github.com/ai-dynamo/dynamo/pull/13101)) and documentation on bounding frontend host memory growth ([#13099](https://github.com/ai-dynamo/dynamo/pull/13099)).
* **Typed Chunk Metrics Path:** Carried chunk metrics in the typed `llm_metrics` field instead of annotation comments, reducing mean inter-token latency by 6.21% with a 5.57% gain in output token throughput in a local A/B run ([#15115](https://github.com/ai-dynamo/dynamo/pull/15115)).
* **Opaque Agent Header Preservation:** Replaced the agent-compaction field with an `agent_headers` map of the `x-claude-code-*` and `x-codex-*` header families, so new coding-agent hints no longer require an envelope schema change ([#15367](https://github.com/ai-dynamo/dynamo/pull/15367)).
* **Raw Completion Stop Preservation:** Added an optional `no_stop_trim` field to `/v1/completions` that retains matched stop strings and stop/EOS token text so callers can see terminal markers, defaulting to false ([#15519](https://github.com/ai-dynamo/dynamo/pull/15519)).

### Scheduling

* **Session Prefix Indexing and Lineage:** Added an opt-in session prefix index (`--enable-session-prefix-index`) that tracks the KV blocks each session has built per worker and DP rank from vLLM KV events ([#13807](https://github.com/ai-dynamo/dynamo/pull/13807), [#13806](https://github.com/ai-dynamo/dynamo/pull/13806)), hardened worker-removal handling and capped session identifiers at 256 bytes ([#15362](https://github.com/ai-dynamo/dynamo/pull/15362)), and grouped subagent requests under their parent session so they land on one worker ([#14521](https://github.com/ai-dynamo/dynamo/pull/14521)).
* **Standalone EPP Routing Enablement:** Passed `expected_output_tokens` and `policy_class` from the standalone endpoint picker (EPP) into its selection service ([#13595](https://github.com/ai-dynamo/dynamo/pull/13595)), routed runtime EPP requests by the `x-dynamo-meta-policy-class` header ([#14480](https://github.com/ai-dynamo/dynamo/pull/14480)), added cache namespace and salt resolution ([#13714](https://github.com/ai-dynamo/dynamo/pull/13714)) and `sglang-renderer` tokenizer support ([#14501](https://github.com/ai-dynamo/dynamo/pull/14501)), and made native EPP atomically reserve KV-routed prefill capacity ([#13042](https://github.com/ai-dynamo/dynamo/pull/13042)).
* **Pluggable Router Flow Control:** Added a pluggable request-classifier hook to the KV Router that can defer a request, override scheduling inputs, or suggest a worker without overriding explicit pins ([#14123](https://github.com/ai-dynamo/dynamo/pull/14123)), with a deadline-ordered policy queue for every request ([#14176](https://github.com/ai-dynamo/dynamo/pull/14176)), async classifier lifecycle events ([#14177](https://github.com/ai-dynamo/dynamo/pull/14177)), and one plugin registration catalog ([#14149](https://github.com/ai-dynamo/dynamo/pull/14149)). Skipped capacity checks for policy classes with queueing disabled ([#15423](https://github.com/ai-dynamo/dynamo/pull/15423)).
* **ThunderAgent Builtin Router Plugin:** Shipped ThunderAgent as a builtin router plugin enabled through `--router-policy-config` ([#15164](https://github.com/ai-dynamo/dynamo/pull/15164)), made its cost model DP-rank aware so SGLang data-parallel attention workers no longer fail program-lifecycle requests ([#14000](https://github.com/ai-dynamo/dynamo/pull/14000)), and made it honor the Router's pinned worker and check that worker's live capacity ([#15290](https://github.com/ai-dynamo/dynamo/pull/15290)).
* **KV Router Slot Tracker Recovery:** Cut slot tracker overhead by sharding its locks ([#15531](https://github.com/ai-dynamo/dynamo/pull/15531)), tied standalone replica sync to each request's reservation so peer routers no longer see a re-added request's prefill as complete ([#15578](https://github.com/ai-dynamo/dynamo/pull/15578)), reworked the recovery snapshot cache ([#15551](https://github.com/ai-dynamo/dynamo/pull/15551)), and restored buffer-first gap recovery that keeps the existing index on a KV event gap ([#14627](https://github.com/ai-dynamo/dynamo/pull/14627)).
* **Planner GPU Budget Enforcement:** Added runtime reconfiguration of endpoint floors and `min_gpu_budget`/`max_gpu_budget` bounds through `PATCH /v1/min-endpoints` ([#14218](https://github.com/ai-dynamo/dynamo/pull/14218)), treated `min_gpu_budget` as a scale-down guard ([#14231](https://github.com/ai-dynamo/dynamo/pull/14231)), and preserved the decode replica count when a raised GPU floor would reduce it ([#15466](https://github.com/ai-dynamo/dynamo/pull/15466)). Applied `min_endpoint` in single-pool load scaling ([#15530](https://github.com/ai-dynamo/dynamo/pull/15530)) and distinguished startup from drain and rollout so falling demand cannot cancel pending capacity or drain serving workers ([#14671](https://github.com/ai-dynamo/dynamo/pull/14671)).
* **Planner Gym Autoscaler Evaluation:** Added an experimental Planner Gym under `gyms/planner-gym` that compares Planner, KEDA/HPA-style, reactive, and static autoscaling policies ([#15576](https://github.com/ai-dynamo/dynamo/pull/15576)), resolved its defaults through the production Planner configuration ([#15678](https://github.com/ai-dynamo/dynamo/pull/15678)), and removed stale offline replay restrictions so Planner policies run on agentic Mooncake and Dynamo or Weka traces ([#15435](https://github.com/ai-dynamo/dynamo/pull/15435)).
* **Phase-Aware Prefill Accounting:** Gated prefill-start recording off for decode-only requests ([#14019](https://github.com/ai-dynamo/dynamo/pull/14019)), moved SGLang prefill completion to after the background prefill stream ends ([#14968](https://github.com/ai-dynamo/dynamo/pull/14968)), and fixed output-block undercounting across block boundaries and batched those updates ([#14613](https://github.com/ai-dynamo/dynamo/pull/14613), [#15448](https://github.com/ai-dynamo/dynamo/pull/15448)).
* **Direct ZMQ Subscriber Multiplexing:** Added opt-in fan-in via `DYN_ROUTER_ZMQ_ENDPOINTS_PER_SUB` so direct-ZMQ subscribers share one SUB socket across many publisher endpoints ([#14219](https://github.com/ai-dynamo/dynamo/pull/14219)), and coalesced pending KV load metrics ([#15587](https://github.com/ai-dynamo/dynamo/pull/15587)).
* **Standalone Indexer Lifecycle Delegates:** Added ownership callbacks to the standalone KV indexer that notify consumers when a block hash gains its first owner or its last owner releases it ([#14945](https://github.com/ai-dynamo/dynamo/pull/14945)), made cleanup safe for absent blocks and backpressure ([#14652](https://github.com/ai-dynamo/dynamo/pull/14652)), and bounded the KV state-agent discovery wait at vLLM startup ([#14003](https://github.com/ai-dynamo/dynamo/pull/14003)).
* **Unified Selection Core:** Unified Dynamo Frontend, standalone selection service, and EPP worker selection on one shared core ([#14570](https://github.com/ai-dynamo/dynamo/pull/14570)) and reduced scheduler and dispatch overhead for large worker pools ([#14666](https://github.com/ai-dynamo/dynamo/pull/14666)).
* **TokenSpeed Disaggregation with KV Routing:** Added prefill/decode disaggregation for the TokenSpeed backend over Mooncake and published native KV-cache events, so the Router can send prompts to the prefill worker already holding their prefix. Included bootstrap validation and transfer-error handling ([#9237](https://github.com/ai-dynamo/dynamo/pull/9237)).
* **Native KV Cache Events:** Added a native TensorRT-LLM KV cache event mode that subscribes directly to each attention-DP rank's ZMQ stream, bypassing the legacy polling and republishing path ([#12167](https://github.com/ai-dynamo/dynamo/pull/12167)).
* **Throughput Scaling Replica Cap:** Added the `max_throughput_scaling_replicas` Planner configuration option, defaulting to 8, and applied it per component to both throughput scale-up and scale-down targets, with capped targets fitted against GPU and power ceilings while preserving explicit endpoint and out-of-band budget recovery ([#14296](https://github.com/ai-dynamo/dynamo/pull/14296)).
* **Relay WAN gRPC Transport:** Added a `dynamo.kvrelay.v1` Protobuf contract and WAN adapter so consumers outside a data center can subscribe to cache locality, readiness, and KV occupancy streams through an opt-in listener ([#14379](https://github.com/ai-dynamo/dynamo/pull/14379)).
* **Router KV Hints Envelope:** Passed Router-produced KV hints to vLLM through vLLM's dedicated KV-hints request argument, keeping hint data separate from the KV connector's `kv_transfer_params` on token, text, and prefill paths. In prefill/decode deployments the hint is applied once on the prefill worker, and decode handoff requests no longer replay it ([#14890](https://github.com/ai-dynamo/dynamo/pull/14890)).
* **Centralized Router Policy Configuration:** Moved Router scoring knobs into `--router-policy-config` with a `router` section for shared cache settings, deprecated the six scoring flags and `--load-aware` while their behavior remains available for migration, and removed `router_credit_blocks` from the overlap-query response ([#15202](https://github.com/ai-dynamo/dynamo/pull/15202)). Added a `soft-pin-repin` policy example for soft session-affinity rebinding ([#14152](https://github.com/ai-dynamo/dynamo/pull/14152)).
* **Worker KV Reuse Telemetry:** Added router metrics for KV cache reuse per routing attempt across SGLang, TensorRT-LLM, and vLLM, labeled by `phase` and `model`, so operators can compare routed prompt tokens against engine cache hits ([#15443](https://github.com/ai-dynamo/dynamo/pull/15443)).

### KV Block Manager

* **File-Based Relay Source Mode:** Added a `--sources-file` option to KV DC Relay that selects Dynamo namespaces from a versioned JSON file with live reload, and an `/engine/state` route reporting readiness ([#14807](https://github.com/ai-dynamo/dynamo/pull/14807)).

### Multimodal & Diffusion

* **Nemotron Omni Multimodal Routing:** Added exact multimodal KV routing for Nemotron 3 Nano Omni images by modeling vLLM dynamic-resolution geometry and model-specific delimiters ([#14564](https://github.com/ai-dynamo/dynamo/pull/14564)), and for video by sampling and decoding frames and grouping them into temporal tubelets ([#14653](https://github.com/ai-dynamo/dynamo/pull/14653)).
* **Cascaded Nemotron Speech Example:** Added a self-contained deployment example exposing Nemotron ASR, LLM, and TTS endpoints through `/v1/realtime` transcription and streaming `/v1/audio/speech`, with a three-GPU DynamoGraphDeployment and a smoke client ([#12567](https://github.com/ai-dynamo/dynamo/pull/12567)), and added a client-driven ASR-to-LLM handoff for speculative prefix-cache warming ([#14327](https://github.com/ai-dynamo/dynamo/pull/14327)).
* **Omni Video Parameter Plumbing:** Preserved joint video-and-audio output from vLLM-Omni with per-output frame rate and audio sample rate metadata ([#13707](https://github.com/ai-dynamo/dynamo/pull/13707)), cloned sampling parameters so request-specific video controls do not leak across requests ([#13708](https://github.com/ai-dynamo/dynamo/pull/13708)), and forwarded `response_format` so `b64_json` is no longer dropped ([#14844](https://github.com/ai-dynamo/dynamo/pull/14844)). Also fixed the HTTP 500 on video requests carrying `nvext.negative_prompt` and carried the value to the first-stage engine prompt ([#15402](https://github.com/ai-dynamo/dynamo/pull/15402), [#15541](https://github.com/ai-dynamo/dynamo/pull/15541)).
* **MiniMax-H3 FastH3 Video Enablement:** Added a four-B200 aggregated vLLM-Omni launcher for MiniMax-H3 text-to-video-and-audio generation with the four-step Dense FastH3 startup adapter ([#13589](https://github.com/ai-dynamo/dynamo/pull/13589)), then enabled MiniMax-H3 video sparse attention (VSA) on vLLM-Omni with the `--fastvideo-vsa-topk` option and B200 and GB300 hardware guards ([#14702](https://github.com/ai-dynamo/dynamo/pull/14702)).
* **Shared Encoded-Image Cache Hardening:** Added an opt-in shared cache of encoded image bytes in Redis Cluster or Dragonfly, failing open on any cache error ([#14676](https://github.com/ai-dynamo/dynamo/pull/14676)). Fixed case-differing S3 and GCS paths collapsing onto one image cache entry ([#15101](https://github.com/ai-dynamo/dynamo/pull/15101)), made the embedding cache evict only while it needs space ([#15497](https://github.com/ai-dynamo/dynamo/pull/15497)), and stopped the NIXL embedding ring buffer from overlapping a live embedding ([#15498](https://github.com/ai-dynamo/dynamo/pull/15498)).
* **vLLM External Encoder Handoff:** Added a public `ExternalEncoderHandoff` API so external encoders built on `VisionEncoderBackend` reuse Dynamo's image extraction and prompt reconstruction ([#13293](https://github.com/ai-dynamo/dynamo/pull/13293)), forwarded preprocessed multimodal features to vLLM's gRPC API ([#14244](https://github.com/ai-dynamo/dynamo/pull/14244)), and moved multimodal tensor transfer from pickle to typed msgpack ([#14433](https://github.com/ai-dynamo/dynamo/pull/14433)).
* **Multimodal UUID and Media Deduplication:** Extended client-provided multimodal UUID forwarding from images to all modalities for vLLM-based deployments ([#14300](https://github.com/ai-dynamo/dynamo/pull/14300)), and stopped inline `data:` media from being sent twice on the TCP request plane ([#14801](https://github.com/ai-dynamo/dynamo/pull/14801)).
* **Nemotron Audex Speech Synthesis:** Added Nemotron Audex text-to-speech support to the vLLM-Omni backend on the existing `/v1/audio/speech` endpoint, covering `nvidia/Nemotron-Labs-Audex-2B` and `nvidia/Nemotron-Labs-Audex-30B-A3B`. Added `cfg_scale` to the frontend and worker request models for classifier-free guidance, and fixed cumulative waveform assembly so Audex requests no longer return silent or over-length audio ([#12788](https://github.com/ai-dynamo/dynamo/pull/12788)).
* **SGLang Qwen Video KV Routing:** Added video-aware KV cache routing for Qwen models on SGLang, mirroring SGLang's frame sampling in the frontend so repeated video requests hit cached prefix blocks ([#15014](https://github.com/ai-dynamo/dynamo/pull/15014)). Extended the Multimodal KV Routing guide with video identity handling, documenting vLLM support for Qwen3-VL, Qwen3.5, and Nemotron 3 Nano Omni ([#14958](https://github.com/ai-dynamo/dynamo/pull/14958)).

### Kubernetes Deployment

* **LPX Integration and Partition Selection:** Added the LPX component type for LPU-backed inference across API, controller, runtime, scheduler, Grove, and Helm ([#15062](https://github.com/ai-dynamo/dynamo/pull/15062)), let hybrid XT and HX (LP30) builds run some or all partitions on the Cyborg GPU via `lpx.experimental.localPartitions` ([#15563](https://github.com/ai-dynamo/dynamo/pull/15563), [#15706](https://github.com/ai-dynamo/dynamo/pull/15706)), packed LPU-only PropSync partitions into whole-node allocations ([#15583](https://github.com/ai-dynamo/dynamo/pull/15583)), and fixed HX hybrid deployments ([#15804](https://github.com/ai-dynamo/dynamo/pull/15804)).
* **Rolling LPX Build Updates:** Routed compatible `lpx.buildId` changes through Grove rolling updates so remaining replicas keep serving ([#15607](https://github.com/ai-dynamo/dynamo/pull/15607)), and excluded `RollingUpdate` from the layout comparison so an unchanged PodCliqueSet is no longer recreated ([#15672](https://github.com/ai-dynamo/dynamo/pull/15672)).
* **Role-Specific Multinode PodTemplate API:** Activated the `roles[].podTemplate` API for multinode `worker`, `prefill`, and `decode` components so leader and worker Pods can come from different complete PodTemplates ([#14468](https://github.com/ai-dynamo/dynamo/pull/14468), [#14544](https://github.com/ai-dynamo/dynamo/pull/14544)), added an explicit `roles[].replicas` surface for role cardinality ([#14477](https://github.com/ai-dynamo/dynamo/pull/14477)), and restricted `multinode` to inference-engine component types ([#14533](https://github.com/ai-dynamo/dynamo/pull/14533)).
* **Snapshot CUDA Shared Memory Capture:** Enabled CUDA shared-memory Snapshot capture so workers no longer disable NCCL cuMem, NVLS or TensorRT-LLM zero-copy, and existing checkpoints are recaptured because the compatibility hash includes the annotation ([#15791](https://github.com/ai-dynamo/dynamo/pull/15791)), with SGLang, TensorRT-LLM, and vLLM (v0.31.0 and later) releasing only the KV cache before capture instead of moving weights to host memory, which shrinks checkpoints and speeds restores ([#16358](https://github.com/ai-dynamo/dynamo/pull/16358)).
* **Snapshot-Backed Engine Failover:** Added snapshot-backed engine failover for SGLang and vLLM, including single-node multi-GPU workers, that replenishes a failed engine without replacing the worker Pod, with unsupported snapshot failover backends rejected at admission ([#15924](https://github.com/ai-dynamo/dynamo/pull/15924)), and propagated the compatibility hash to `WaitForCheckpoint` references ([#16687](https://github.com/ai-dynamo/dynamo/pull/16687)).
* **Intel XPU GMS Support:** Added Intel XPU support to the GPU Memory Service (GMS), using Level-Zero interop for inter-process memory sharing, a C++ extension for tensor reconstruction, and SGLang and vLLM integration paths. XPU support runs on native torch.xpu with no IPEX dependency and requires PyTorch 2.13 or newer ([#12484](https://github.com/ai-dynamo/dynamo/pull/12484)).
* **GA Helm OCI Publishing:** Added a push of GA Helm charts to the NGC OCI registry alongside the existing ChartMuseum push, so consumers on OCI-only registries such as ACR, ECR, and Harbor can mirror released charts ([#13657](https://github.com/ai-dynamo/dynamo/pull/13657)).
* **KVCR Resiliency Deployment Example:** Added a two-node vLLM KV Cache Reuse (KVCR) example over RDMA. The memory-service sidecar keeps the KVCR Guard available across engine restarts ([#14695](https://github.com/ai-dynamo/dynamo/pull/14695)).
* **EPP Multi-Replica Support:** Removed the admission rule that restricted the Dynamo-mode endpoint picker (EPP) to exactly one replica, so native Rust EPP deployments can scale out and tolerate pod loss like the Frontend-hosted Router, while legacy Go EPP deployments still require a single replica. `DYN_ROUTER_REPLICA_SYNC` remained off by default, matching the Frontend behavior ([#14758](https://github.com/ai-dynamo/dynamo/pull/14758)).
* **Portable Multinode Topology Variables:** Added `DYNAMO_RANK` and `DYNAMO_LEADER_ADDRESS` to new multinode components so SGLang and vLLM commands no longer depend on Grove or LeaderWorkerSet syntax. Injection is gated on the operator origin version, so existing deployments keep their current environment and command lines, and Grove was upgraded to `v0.1.0-alpha.14` ([#15374](https://github.com/ai-dynamo/dynamo/pull/15374), [#15704](https://github.com/ai-dynamo/dynamo/pull/15704)).

### Dynamo Sidecar

> **Note**
>
> Dynamo Sidecar remains experimental in Dynamo v1.6.0.

* **vLLM Sidecar LoRA Lifecycle:** Added dynamic LoRA load, unload, and per-request adapter selection to the vLLM sidecar, with local, Hugging Face, and S3 adapters and no silent base-model fallback for unavailable adapters ([#13068](https://github.com/ai-dynamo/dynamo/pull/13068)). Added aggregated and NIXL prefill/decode launchers for local LoRA serving ([#14574](https://github.com/ai-dynamo/dynamo/pull/14574)).
* **TensorRT-LLM OpenEngine gRPC Sidecar:** Moved the TensorRT-LLM sidecar onto TensorRT-LLM's OpenEngine (`openengine.v1`) gRPC API and added disaggregated serving ([#14349](https://github.com/ai-dynamo/dynamo/pull/14349)). Raised the shared sidecar gRPC startup deadline from 300 to 1,800 seconds for large model loads ([#14825](https://github.com/ai-dynamo/dynamo/pull/14825)).
* **vLLM Sidecar EPD Encode Role:** Added opt-in Encoder-Prefill-Decode support to the native vLLM gRPC sidecar with E+PD and E+P+D launchers ([#13966](https://github.com/ai-dynamo/dynamo/pull/13966)). Extended the Encode worker to accept video and combined image plus video requests ([#15287](https://github.com/ai-dynamo/dynamo/pull/15287)), and shipped the FFmpeg binaries video decode needs for VP8 and VP9 ([#16556](https://github.com/ai-dynamo/dynamo/pull/16556)).
* **SGLang Sidecar KV Routing:** Added a disaggregated KV-aware routing launcher for SGLang sidecars ([#13862](https://github.com/ai-dynamo/dynamo/pull/13862)) and node-local KV event relay for multinode headless sidecars ([#14908](https://github.com/ai-dynamo/dynamo/pull/14908)). Published HiCache capacity so capacity-aware routers no longer treat HiCache workers as device-only ([#14313](https://github.com/ai-dynamo/dynamo/pull/14313)) and an EAGLE flag so KV routers use the engine's bigram hashing ([#14374](https://github.com/ai-dynamo/dynamo/pull/14374)), and moved the prefill bootstrap handoff into a nonterminal response ([#14260](https://github.com/ai-dynamo/dynamo/pull/14260)).
* **Multimodal-Aware vLLM KV Routing:** Resolved the image placeholder token for the vLLM KV event publisher, so repeated image requests match cached image blocks instead of only the text prefix. The token resolves once at startup with a 30 second fetch limit, and image requests fall back to text-prefix routing if resolution fails ([#13360](https://github.com/ai-dynamo/dynamo/pull/13360)).
* **Standalone Decode Sidecar Contract:** Added a standalone decode sidecar for the inference gateway that validates `x-prefiller-host-port` metadata so client-supplied prefill metadata cannot reach endpoint selection ([#13669](https://github.com/ai-dynamo/dynamo/pull/13669)).
* **Engine-Primary Dynamo Native Sidecars:** Added operator support for separate inference engine and Dynamo runtime containers, selected per component via an init container named `runtime`, while combined mode remains the default ([#14927](https://github.com/ai-dynamo/dynamo/pull/14927)).
* **Shared Sidecar Transport Flags:** Added `--discovery-backend`, `--request-plane`, `--response-plane`, and `--event-plane` to the SGLang, TensorRT-LLM, and vLLM sidecars, so workers select `file`, `tcp`, `quic`, or `zmq` directly ([#15653](https://github.com/ai-dynamo/dynamo/pull/15653)).

### SGLang

* **Sharded SGLang Worker Gateway:** Added `--gateway-workers` to shard the `dynamo.sglang` gateway across processes so tokenizer intake and output relay no longer share one GIL ([#15078](https://github.com/ai-dynamo/dynamo/pull/15078)), and fixed a startup `AttributeError` ([#16337](https://github.com/ai-dynamo/dynamo/pull/16337)).
* **SGLang Cache Salt Routing:** Added cache salt support for SGLang text workers so differently salted requests use separate KV cache entries. Non-empty salts are rejected when the installed engine or request type cannot honor them. Upgrade all workers and routers before enabling salted traffic ([#14803](https://github.com/ai-dynamo/dynamo/pull/14803)).

### vLLM

* **Generate Endpoint RL Parity:** Added reinforcement learning (RL) serving parity for the `dynamo.vllm` worker in aggregated deployments, adapting native Generate token-in/token-out requests and publishing RL administration routes ([#15179](https://github.com/ai-dynamo/dynamo/pull/15179)). Carried typed `/inference/v1/generate` controls such as sampling, stop-token IDs, and logprobs into Dynamo's internal request, preserving requests from v1.4 frontends ([#14398](https://github.com/ai-dynamo/dynamo/pull/14398)), with a fix that sent `top_k` and `min_p` as unset when clients did not disable them ([#15286](https://github.com/ai-dynamo/dynamo/pull/15286)).
* **LMCacheMPConnector PD Support:** Registered a connector adapter so vLLM prefill/decode disaggregation works with `LMCacheMPConnector`, which previously failed at request time with an unsupported connector error. Wrapper resolution now prefers a connector that produces KV transfer parameters, so `MultiConnector` setups combining LMCacheMP with NixlConnector still resolve to NIXL ([#13258](https://github.com/ai-dynamo/dynamo/pull/13258)).
* **Incremental Realtime Text Input:** Added text-only `/v1/realtime` sessions to aggregated vLLM workers, with experimental input events so clients can forward ASR text during transcription and commit the final transcript ([#14326](https://github.com/ai-dynamo/dynamo/pull/14326)).

### Triton

* **FP16/BF16 Tensor Input Support:** Dropped the `dynamo.triton` guard that rejected FP16 and BF16 inputs ([#14835](https://github.com/ai-dynamo/dynamo/pull/14835)), reinterpreted their values from raw `uint16` bit patterns ([#16683](https://github.com/ai-dynamo/dynamo/pull/16683)), and passed BF16 as a raw-byte tensor ([#16558](https://github.com/ai-dynamo/dynamo/pull/16558)).
* **KServe Classification Output Support:** Added support for the KServe `classification` requested-output parameter on the Triton worker and forwarded requested outputs from the frontend ([#14618](https://github.com/ai-dynamo/dynamo/pull/14618)), so the worker returns only the named tensors ([#14783](https://github.com/ai-dynamo/dynamo/pull/14783)).
* **Defunct Triton Example Removal:** Removed the out-of-date Triton Inference Server backend example, including its Dockerfile, launch scripts, model repository, and worker implementation. The Dynamo Triton Runtime under `components/src/dynamo/triton` replaces it ([#15170](https://github.com/ai-dynamo/dynamo/pull/15170)).
* **Triton Classification Endpoint Support:** Added OpenAI-compatible `POST /v1/classify` serving to the Triton backend, opt-in through `--task classify` with input and output tensor names auto-detected from `config.pbtxt` and overridable via `--classify-input-name` and `--classify-output-name`. The default `--task tensor` KServe path is unchanged, and text-only input is supported while token-ID variants and `encoding_format` requests return HTTP 400 ([#15396](https://github.com/ai-dynamo/dynamo/pull/15396)).

### Model Loading & LoRA

* **Native LoRA Capacity Reporting:** Exposed native vLLM LoRA capacity through shared model registration, so a worker reporting `max_loras=4` carries `max_gpu_lora_count=4` into discovery ([#14754](https://github.com/ai-dynamo/dynamo/pull/14754)), and corrected LoRA allocation simulation with capacity-bounded adapter residency and churn, hit, and occupancy metrics ([#13758](https://github.com/ai-dynamo/dynamo/pull/13758)).
* **Shared NGC Model Fetching:** Added shared NGC model fetching through ModelExpress for SGLang and vLLM, reusing cached models ([#15441](https://github.com/ai-dynamo/dynamo/pull/15441)). Local models served under custom names no longer trigger `hf://` metadata fetches on offline hosts ([#15964](https://github.com/ai-dynamo/dynamo/pull/15964)).

### Performance Modeling & Replay

* **Typed Agentic Replay Adapters:** Added a streaming Weka importer in `dynamo-data-gen`, lowered Weka and Claude/Codex/Dynamo request traces into versioned Agentic Mooncake v2 rows, and added Dynamo replay entrypoints for typed agentic lanes ([#13586](https://github.com/ai-dynamo/dynamo/pull/13586)).
* **Codex and Claude Trace Export:** Added an offline Codex trace exporter for agentic replay alongside the Claude exporter and fixed Claude export so a full local corpus replays. Claude export on a 216 MB project dropped from 49 s to 2.6 s and peak RSS from 2.3 GB to 1.3 GB ([#15601](https://github.com/ai-dynamo/dynamo/pull/15601)).
* **Mocker Host-Memory KV Offload:** Wired AISimulate's vLLM host-memory KV offload into the Mocker with tier-aware KV routing, so the Router tracks host-resident blocks and `--num-host-blocks` alone enables host offload ([#15647](https://github.com/ai-dynamo/dynamo/pull/15647)).
* **AISimulate Canonical Configuration:** Moved performance-model consumers across the Planner, Profiler, Router, Frontend, Mocker, and replay to AISimulate's canonical configuration interface, carrying complete configurations through Planner roles and the related CLI consumers ([#15063](https://github.com/ai-dynamo/dynamo/pull/15063)).

### Fault Tolerance & Observability

* **Frontend NUMA Affinity Warning:** Added a startup check that warns when the frontend's allowed CPUs span multiple NUMA nodes, a likely source of degraded performance ([#11184](https://github.com/ai-dynamo/dynamo/pull/11184)).
* **Tokio Queue Overload Warnings:** Added detection of sustained Tokio runnable-queue pressure in the runtime metrics loop, with worker-scaled thresholds, hot-worker detection, hysteresis, and sample-gap checks. Warnings log once per overload episode and repeat at most every five minutes while it continues, and the count is exported as `dynamo_tokio_queue_overload_warnings_total` for operators ([#13027](https://github.com/ai-dynamo/dynamo/pull/13027)).
* **OTLP Metrics Export:** Added a periodic OTLP exporter that ships Dynamo and engine metrics to a collector over gRPC, configured through standard `OTEL_*` variables and off unless `OTEL_METRICS_EXPORTER=otlp` is set ([#14071](https://github.com/ai-dynamo/dynamo/pull/14071)).
* **Per-Request Lifecycle Instrumentation:** Added opt-in runtime tracing that models each inference request as a finite-state machine, recording state timing and terminal outcome for aggregated and disaggregated serving ([#14101](https://github.com/ai-dynamo/dynamo/pull/14101)).
* **Buffered JSONL FPM Recording:** Added `--output`, `--log-metrics`, and `--flush-interval` options to the forward pass metrics (FPM) receiver, writing one JSON record per message to a new file and flushing on an interval ([#14511](https://github.com/ai-dynamo/dynamo/pull/14511)).
* **NVTX Ranges in Wheels:** Compiled the runtime's NVTX ranges into published wheels, so the `preprocess.*`, `route.*`, `transport.*`, and `detokenize` ranges appear in Nsight reports when `DYN_ENABLE_RUST_NVTX` is set, and removed the dependency on `libnvToolsExt.so`, which CUDA 12.9 dropped ([#15242](https://github.com/ai-dynamo/dynamo/pull/15242)).

### Runtime & Transport

* **Request-Plane Transport Configuration Hardening:** Added `DYN_EVENT_PLANE_HOST` to override the address advertised by direct ZMQ event publishers ([#11870](https://github.com/ai-dynamo/dynamo/pull/11870)). Restricted `DYN_REQUEST_PLANE` to default to TCP only when absent, so invalid values surface as startup errors ([#12612](https://github.com/ai-dynamo/dynamo/pull/12612)), and defaulted the event-plane context to four libzmq I/O threads with a `DYN_ZMQ_IO_THREADS` override ([#14517](https://github.com/ai-dynamo/dynamo/pull/14517)).
* **Configurable Etcd Startup Retry:** Made the etcd startup retry timeout configurable, keeping 120 seconds as the default and bounding each individual connection attempt by the remaining retry window ([#13742](https://github.com/ai-dynamo/dynamo/pull/13742)).
* **Batched QUIC Response Transport:** Added an opt-in batched QUIC response plane selected with `--response-plane quic` or `DYN_RESPONSE_PLANE=quic` on each frontend and worker. TCP remains the default ([#11996](https://github.com/ai-dynamo/dynamo/pull/11996)).
* **Packed Int32 Token IDs:** Added an opt-in `DYN_TOKEN_IDS_AS_BYTES` flag that serializes `token_ids` as a packed int32 blob on request-plane codecs. It must stay off until all msgpack workers run a release containing the new reader ([#15326](https://github.com/ai-dynamo/dynamo/pull/15326)).
* **Unary Endpoint Helper APIs:** Added unary endpoint utilities that serve a handler result and collect unary responses and LLM token streams while preserving request context ([#12920](https://github.com/ai-dynamo/dynamo/pull/12920)), then promoted them to `dynamo.runtime` and `dynamo.llm` ([#15195](https://github.com/ai-dynamo/dynamo/pull/15195)).
* **Shared Prompt Token Storage:** Made copies of a preprocessed request share one prompt-token buffer, reducing clone cost for disaggregated prefill and migration, though direct Rust consumers of `token_ids` may need adaptation ([#14701](https://github.com/ai-dynamo/dynamo/pull/14701)).
* **Unary Chat Model Publisher:** Added `UnaryChatModel` to `dynamo.llm`, which registers and serves a Chat model endpoint, plus `LLMUnaryClient.connect`, which waits for a routable instance ([#15552](https://github.com/ai-dynamo/dynamo/pull/15552)).

## Recipes

### New Recipes

* **DeepSeek-V4-Pro-0813 vLLM Recipes:** The GB200 recipes require the custom [`vllm-runtime:1.5.0-deepseek-v4-pro-0813-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added aggregated and disaggregated recipes for `deepseek-ai/DeepSeek-V4-Pro-0813` on H200 ([#13783](https://github.com/ai-dynamo/dynamo/pull/13783)), relaxed GB200 prefill liveness probing so vLLM JIT kernel compilation no longer triggers a crashloop ([#14735](https://github.com/ai-dynamo/dynamo/pull/14735)), and replaced the removed `use_fp4_indexer_cache` with `indexer_kv_dtype: mxfp4` ([#15427](https://github.com/ai-dynamo/dynamo/pull/15427)).
* **A.X-K2 vLLM Recipes:** Requires the custom [`vllm-runtime:1.4.1-a.x-k2-post.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added FP8 H200 targets with 4xTP8 aggregated and 2P2D TP8 disaggregated layouts, KV-aware routing, and EAGLE3 ([#14318](https://github.com/ai-dynamo/dynamo/pull/14318)), and A.X-K2-NVFP4 B200 recipes on vLLM v0.26.0 with TP4 aggregated and disaggregated deployments ([#14635](https://github.com/ai-dynamo/dynamo/pull/14635)).
* **Solar Open2 250B NVFP4 Recipes:** Requires the custom [`vllm-runtime:1.4.1-solar-open2-250b-post.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added aggregated and disaggregated B200 recipes with a reasoning parser fix for the checkpoint's special tokens ([#14376](https://github.com/ai-dynamo/dynamo/pull/14376)), then repointed benchmark manifests at the public AIPerf v0.12.0 image ([#14956](https://github.com/ai-dynamo/dynamo/pull/14956)).
* **K-EXAONE 2.0 vLLM Recipes:** Requires the custom [`vllm-runtime:1.4.1-k-exaone-2.0-750b-post.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added vLLM recipes for `LGAI-EXAONE/K-EXAONE-2.0-750B-A37B-NVFP4` on B200 with aggregated TP=4 and disaggregated 1P1D variants ([#14822](https://github.com/ai-dynamo/dynamo/pull/14822)), then moved recipe notes into the Fern page and corrected the InfiniBand component, image, and resource settings ([#15015](https://github.com/ai-dynamo/dynamo/pull/15015)).
* **Gemma 4 31B Recipes:** Requires the custom [`tensorrtllm-runtime:1.5.0-gemma-4-31b-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime/tags) container. Added Gemma 4 31B deployment recipes for B200, GB200, and H200 GPUs with TensorRT-LLM aggregated configurations, model cache setup, and benchmarking workflows, plus documentation in the recipe catalog and a Google provider filter ([#13221](https://github.com/ai-dynamo/dynamo/pull/13221)).
* **Inkling-NVFP4 GB300 Agentic Recipes:** Requires the custom [`vllm-runtime:1.5.0-inkling-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added aggregated (2 replicas x TP4) and disaggregated (1P1D, TP4 per role) Dynamo plus vLLM recipes for thinkingmachines/Inkling-NVFP4 on GB300, tuned for an agentic workload with KV-aware routing, MTP-8 speculative decoding, and 1M context, along with an AIPerf trace-replay benchmark Job and Kustomize overlays for generic and AWS RoCE clusters ([#13798](https://github.com/ai-dynamo/dynamo/pull/13798)).
* **Qwen3.6-35B-A3B SGLang Recipes:** Added SGLang multi-token prediction deployment recipes for Qwen3.6-35B-A3B on B200, GB200, and H200 GPUs, along with model-download configurations, smoke tests, benchmark results, and catalog and documentation entries ([#13958](https://github.com/ai-dynamo/dynamo/pull/13958)).
* **Qwen3.8-Flash-Next vLLM Recipes:** Requires the custom upstream [`vllm/vllm-openai:qwen38-flash-next`](https://hub.docker.com/r/vllm/vllm-openai/tags?name=qwen38-flash-next) image, which installs `ai-dynamo` at pod startup. Added aggregated (4-GPU TP4, 8-GPU 2xTP4, 12-GPU 3xTP4) and disaggregated 1P1D and 2P1D vLLM deployment recipes for Qwen3.8-Flash-Next on B200, including model-cache setup, AIPerf trace-replay benchmarks, and a docs catalog entry ([#14033](https://github.com/ai-dynamo/dynamo/pull/14033)).
* **DeepSeek-V4.1-Flash SGLang Recipes:** Requires the custom [`sglang-runtime:1.6.0-deepseek-v4.1-flash-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime/tags) container. Added day-0 aggregated and disaggregated recipes on 8x GB200 with KV-aware routing on the aggregated target. Both serve up to 1,048,576 tokens of context and neither is benchmarked ([#14682](https://github.com/ai-dynamo/dynamo/pull/14682)).
* **Motif-3 NVFP4 Serving Recipe:** Requires the custom [`vllm-runtime:1.5.0-motif-3-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) container. Added the Motif-3 NVFP4 aggregated serving recipe for vLLM on B200, along with model-cache and benchmark manifests, a Fern documentation page, and a catalog entry that brings the recipe catalog to 25 model families and 125 deployable configurations ([#15534](https://github.com/ai-dynamo/dynamo/pull/15534)).
* **Deployment Recipe Starter Templates:** Added a catalog of copyable deployment recipe templates covering aggregated and disaggregated serving for SGLang, TensorRT-LLM, and vLLM, along with Kustomize components for caching, placement, scheduling, probes, networking, and registry credentials, plus contribution and validation guidance for building new recipes ([#13724](https://github.com/ai-dynamo/dynamo/pull/13724)).

### Recipe Updates

* **Provider Networking Kustomize Components:** Ported the disaggregated provider-networking Kustomize Components for GKE RoCE and InfiniBand to the v1beta1 copy-and-fill scaffold, and extended the recipe validator to enforce the networking-slot ordering and worker-only networking-delta contract, so recipes carry no cluster-specific values in the base ([#14142](https://github.com/ai-dynamo/dynamo/pull/14142)).
* **Kimi Recipe Configuration Fixes:** Set `DYN_SELF_HOST_METADATA=1` on every Kimi K3 worker role in SGLang and vLLM configurations ([#14617](https://github.com/ai-dynamo/dynamo/pull/14617)), and removed `--request-plane nats` from the Kimi K2.5 aggregated recipe to use the default TCP transport ([#15428](https://github.com/ai-dynamo/dynamo/pull/15428)).
* **Dynamo Frontend Image for Recipes:** Switched the GLM-5.3-Flash recipes' Frontend to the `nvcr.io/nvidia/ai-dynamo/dynamo-frontend` image, dropping the inline `pip install ai-dynamo` step ([#14229](https://github.com/ai-dynamo/dynamo/pull/14229)), and applied it to Qwen3.6 SGLang MTP recipes with `--dyn-reasoning-parser qwen3` so reasoning lands in `message.reasoning_content` ([#14472](https://github.com/ai-dynamo/dynamo/pull/14472)).
* **GB300 Max Model Length:** Corrected the SGLang chat configuration in the Qwen3 recipe README to document a 262,144-token context length for the GB300 aggregated and disaggregated profiles, while GB200 values remained unchanged ([#14048](https://github.com/ai-dynamo/dynamo/pull/14048)).
* **Shared DGD Environment Merging:** Fixed the generated Kustomize OpenAPI schema so environment variables on DynamoGraphDeployment resources merge by name instead of producing duplicate entries, preserving existing values when recipes patch shared environment variables ([#15064](https://github.com/ai-dynamo/dynamo/pull/15064)).
* **Qwen3-32B-FP8 Decode Mode Flag:** Added the `--disaggregation-mode decode` flag to the decode worker in the Qwen3-32B-FP8 vLLM disaggregated recipe, which previously started as an aggregated worker and left the Frontend reporting no model despite a `Ready` deployment ([#16821](https://github.com/ai-dynamo/dynamo/pull/16821)).
* **Qwen3-VL Worker Startup:** Fixed the Qwen3-VL-32B-FP8 vLLM recipes so their worker containers pass command-line arguments correctly; the disaggregated encode and decode workers previously exited cleanly without ever starting, leaving no error explaining the missing workers ([#16855](https://github.com/ai-dynamo/dynamo/pull/16855)).
* **Refreshed Kimi-K3 Recipes:** Requires the custom [`vllm-runtime:1.5.0-kimi-k3-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime/tags) or [`sglang-runtime:1.5.0-kimi-k3-dev.1`](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime/tags) container. Expanded the Kimi-K3 recipes with SGLang and vLLM aggregated and disaggregated targets for H200, GB200, and GB300 and added Mooncake trace benchmarking workflows ([#12944](https://github.com/ai-dynamo/dynamo/pull/12944)).

## Bug Fixes

### Frontend & Agents

* **Anthropic Tool-Call Conversion Fidelity:** Fixed Anthropic Messages conversion so buffered tool calls are emitted after an early finish reason ([#15129](https://github.com/ai-dynamo/dynamo/pull/15129)) and the single-call limit from `tool_choice.disable_parallel_tool_use` is applied ([#14605](https://github.com/ai-dynamo/dynamo/pull/14605)). Preserved `tool.strict` ([#16206](https://github.com/ai-dynamo/dynamo/pull/16206)), mapped `output_config.format` to `response_format` ([#16207](https://github.com/ai-dynamo/dynamo/pull/16207)), and suppressed tool calls whose `tool_use.input` is not a JSON object ([#16214](https://github.com/ai-dynamo/dynamo/pull/16214)).
* **Anthropic Messages Protocol Corrections:** Reworked Anthropic streaming so text after a tool call opens a new block ([#15467](https://github.com/ai-dynamo/dynamo/pull/15467)), reported `stop_reason: "stop_sequence"` with the matched string ([#16417](https://github.com/ai-dynamo/dynamo/pull/16417)), and stripped the Claude Code billing preamble by default ([#15826](https://github.com/ai-dynamo/dynamo/pull/15826)).
* **SGLang Structured-Output Reasoning Gating:** Extended the reasoning gate to structured response formats so JSON no longer returns entirely in `reasoning_content` ([#14465](https://github.com/ai-dynamo/dynamo/pull/14465)), gave forced `tool_choice` precedence over `response_format` ([#14179](https://github.com/ai-dynamo/dynamo/pull/14179)), gated `tool_choice="auto"` grammars on reasoning to avoid responses that contain neither an answer nor a tool call ([#15227](https://github.com/ai-dynamo/dynamo/pull/15227)), and bypassed the tool parser for ungated answer JSON ([#15515](https://github.com/ai-dynamo/dynamo/pull/15515)).
* **SGLang Chat Processor Parser Fixes:** Passed the checkpoint tokenizer to SGLang parsers so reasoning and tool calls are detected when checkpoint delimiters differ from defaults ([#15509](https://github.com/ai-dynamo/dynamo/pull/15509)), retained parser-declared tool closing delimiters during stop handling ([#15510](https://github.com/ai-dynamo/dynamo/pull/15510)), honored explicit `skip_special_tokens=false` ([#14602](https://github.com/ai-dynamo/dynamo/pull/14602)), and returned chat-template errors as invalid-argument errors ([#15511](https://github.com/ai-dynamo/dynamo/pull/15511)).
* **Truncated Tool-Call Finish Reasons:** Preserved `finish_reason: "length"` at `max_tokens` rather than a completed tool call ([#13791](https://github.com/ai-dynamo/dynamo/pull/13791)), suppressed the incomplete final tool call in Chat Completions and Anthropic Messages ([#13792](https://github.com/ai-dynamo/dynamo/pull/13792)), and reported streams ending inside GLM-4.7 tool-call blocks as length-limited ([#15184](https://github.com/ai-dynamo/dynamo/pull/15184)).
* **Chat and Completion Logprobs Reporting:** Returned prompt logprobs at the root of non-streaming chat completion responses ([#13957](https://github.com/ai-dynamo/dynamo/pull/13957)), emitted completion `top_logprobs` as token-to-logprob maps ([#15521](https://github.com/ai-dynamo/dynamo/pull/15521)), and mapped chat logprobs onto vLLM sampling parameters instead of rejecting them ([#15670](https://github.com/ai-dynamo/dynamo/pull/15670)).
* **Request Parameter Validation Hardening:** Returned HTTP 400 instead of generic 500 for conflicting guided-decoding options ([#13790](https://github.com/ai-dynamo/dynamo/pull/13790)), and aligned stop-sequence limits so excess `stop` or `stop_token_ids` entries fail with 400 ([#14462](https://github.com/ai-dynamo/dynamo/pull/14462)). Also rejected an explicit empty `tools` array ([#16231](https://github.com/ai-dynamo/dynamo/pull/16231)) and `logit_bias` keys that are not non-negative 32-bit integers ([#16322](https://github.com/ai-dynamo/dynamo/pull/16322)), and validated `strict: true` function parameter schemas before inference ([#16219](https://github.com/ai-dynamo/dynamo/pull/16219)).
* **Configurable HTTP Listen Backlog:** Replaced the frontend HTTP listener's 128-entry queue with a configurable backlog defaulting to 4096, preventing `Connection reset by peer` during connection bursts ([#15080](https://github.com/ai-dynamo/dynamo/pull/15080)), and restored the TCP call-home listener's 4096 backlog ([#15270](https://github.com/ai-dynamo/dynamo/pull/15270)).
* **Multimodal Media Forwarding Fixes:** Fixed SGLang and vLLM chat processors to read media from `tool` messages too, avoiding image count mismatches ([#15237](https://github.com/ai-dynamo/dynamo/pull/15237)), converted Responses `function_call_output` image parts instead of rejecting them ([#15306](https://github.com/ai-dynamo/dynamo/pull/15306)), and closed validation for audio, image, and video requests ([#15420](https://github.com/ai-dynamo/dynamo/pull/15420)).
* **Raw Reasoning Output Preservation:** Preserved backend `reasoning_content` in the `/v1/responses` API, with non-streaming responses returning `reasoning_text` content instead of `summary_text` and streaming responses emitting `response.reasoning_text.delta` plus the corresponding added and done events. Reasoning, text, and tool-call ordering is retained across interleaved reasoning spans, and `summary` remains as a compatibility fallback ([#11604](https://github.com/ai-dynamo/dynamo/pull/11604)).
* **Scoped Tool-Call Array Bound:** Fixed guided decoding so the `maxItems=1` constraint applied when `parallel_tool_calls=False` is limited to `tool_choice="required"` schemas instead of also hitting named function choices. Array-typed function parameters no longer get truncated to a single element, and the caller's request dictionary is no longer mutated in place ([#13242](https://github.com/ai-dynamo/dynamo/pull/13242)).
* **Speculative Prefill Warmup Lifetime:** Bounded prefill warmup tasks to their preprocessor's lifetime with a 60-second warmup deadline after the client response finishes and a process-wide limit of four concurrent speculative preprocessing operations, so stalled backends or never-ending response streams no longer retain request resources indefinitely ([#13438](https://github.com/ai-dynamo/dynamo/pull/13438)).
* **Incremental Tool-Call Streaming:** Fixed vLLM preprocessing to stream tool-call deltas as the parser emits them instead of buffering them until the end of a completion, so consumers receive incremental SSE frames ([#13975](https://github.com/ai-dynamo/dynamo/pull/13975)).
* **Client Token ID Bounds:** Added client token id validation for `nvext.token_data`, completion `prompt`, and embedding `input`, so an id at or above the vocabulary bound returns HTTP 400 instead of reaching the engine ([#14475](https://github.com/ai-dynamo/dynamo/pull/14475)).
* **SGLang Worker Routing Hints:** Fixed the SGLang chat processor to forward `backend_instance_id`, `decode_worker_id`, `prefill_worker_id`, `dp_rank`, and `prefill_dp_rank` from `nvext` into the request routing fields. Requests pinned to a specific worker or data-parallel rank previously reached the Router unpinned and could be sent to a different worker ([#14696](https://github.com/ai-dynamo/dynamo/pull/14696)).
* **Kimi Dynamic Tool Declarations:** Added support for Kimi K3 message extensions so tools declared only in `system.tools` are treated like top-level OpenAI tools across request validation, tool-choice handling, guided decoding, parser definitions, stop-token behavior, and response gating, with dynamic declarations preserved at their original message positions to maintain prompt ordering and prefix-cache behavior ([#14755](https://github.com/ai-dynamo/dynamo/pull/14755)).
* **Null Optional Completion Parameters:** Fixed the frontend to treat explicit `null` values as omitted for recognized passthrough fields such as `cache_salt` and `stop_token_ids`, and for `stream_options.include_usage` and `stream_options.continuous_usage_stats` on `/v1/chat/completions` and `/v1/completions`. OpenAI-compatible clients that serialize unset optional parameters as `null` no longer receive HTTP 400 validation errors ([#14847](https://github.com/ai-dynamo/dynamo/pull/14847)).
* **Responses API Metadata Echo:** Fixed the Responses API to echo request `metadata` in both non-streaming and streaming responses, which was previously replaced with an empty map when Dynamo built the response. Requests without metadata continue to return an empty object ([#15102](https://github.com/ai-dynamo/dynamo/pull/15102)).
* **Renderer Message Rewrite Preservation:** Made the frontend render transformed messages, preserving tool argument normalization and the `continue_final_message` marker instead of falling back to the original history. Ordinary requests keep the typed fast path ([#15163](https://github.com/ai-dynamo/dynamo/pull/15163)).
* **Reordered Message Continuation Guard:** Rejected `continue_final_message` requests with an `InvalidArgument` error before tokenization when the prompt formatter moves the final input message ahead of other messages, preventing silently truncated prompts and discarded tool results. Continuation still works for ordered tool results and assistant prefixes ([#15247](https://github.com/ai-dynamo/dynamo/pull/15247)).
* **Anthropic Hosted Tool Rejection:** Changed `/v1/messages` to return HTTP 400 with `invalid_request_error` instead of HTTP 501 for unsupported hosted tools, so clients stop retrying a deterministic rejection, and aligned the behavior with the `/v1/responses` hosted-tool policy while preserving `NotImplemented` request metrics ([#15468](https://github.com/ai-dynamo/dynamo/pull/15468)).
* **Refusal Text Preservation:** Fixed aggregated non-streaming chat responses dropping refusal text by accumulating refusal fragments per choice and retaining them in the final message, so clients receive the full refusal content. Responses without refusal text remain unchanged ([#15516](https://github.com/ai-dynamo/dynamo/pull/15516)).
* **Streaming Cached-Token Usage:** Fixed SGLang and vLLM streaming so sparsely reported cached-token usage reaches the client and explicit zero cache counts are preserved. Streaming usage now follows `stream_options.include_usage` and `continuous_usage_stats` ([#16944](https://github.com/ai-dynamo/dynamo/pull/16944)).

### Scheduling

* **Planner Load Predictor Correctness:** Clamped Kalman forecasts at zero so they no longer go negative ([#14497](https://github.com/ai-dynamo/dynamo/pull/14497)), scored failed simulation presets honestly ([#14515](https://github.com/ai-dynamo/dynamo/pull/14515)), weighted frontend histogram averages by observation count ([#14705](https://github.com/ai-dynamo/dynamo/pull/14705)), and reported unreadable request counts as unknown rather than zero ([#14766](https://github.com/ai-dynamo/dynamo/pull/14766)).
* **Per-Rank DP Capacity Handling:** Normalized aggregate vLLM KV capacity to per-rank values and rejected zero data-parallel rank counts ([#14258](https://github.com/ai-dynamo/dynamo/pull/14258)), and hardened the Router for empty prompts, DP ranges, and large KV events ([#15303](https://github.com/ai-dynamo/dynamo/pull/15303)).
* **KV Router Lookup Repair:** Fixed lazy cross-worker reverse-lookup repair in the KV Router compressed radix tree so repair updates only existing entries, preventing scrubbed keys from being recreated and reviving phantom tracked blocks. Store paths retained their insertion behavior for newly stored blocks ([#11891](https://github.com/ai-dynamo/dynamo/pull/11891)).
* **Non-Blocking Prometheus Metrics Collection:** Moved Planner traffic, KV-only, and accept-length collection to a background thread so slow Prometheus queries no longer block the event loop, keeping control API requests responsive. Shared metrics are published on the event loop only after a successful collection, and `collect_accept_length()` is now async across provider and environment protocols ([#14377](https://github.com/ai-dynamo/dynamo/pull/14377)).
* **Atomic Planner Decision Publishing:** Fixed the virtual Planner to publish worker counts and decision ID in a single etcd transaction, so clients no longer observe partial scaling decisions ([#14809](https://github.com/ai-dynamo/dynamo/pull/14809)).
* **Pending Virtual Connector Decisions:** Fixed `VirtualConnectorClient.wait()` in the Planner to return immediately when an unacknowledged scaling decision already exists in etcd, instead of only reporting decisions published after the watch started. Consumers no longer miss a decision published before they began waiting ([#14841](https://github.com/ai-dynamo/dynamo/pull/14841)).
* **Rust Worker Topology Policy:** Added topology policy loading in `dynamo-backend-common` so Rust workers read topology settings before model registration and publish the same topology fields as Python workers. The loader waits up to 30 seconds for delayed projection ([#14909](https://github.com/ai-dynamo/dynamo/pull/14909)).
* **KV Router Lease Leak Fix:** Fixed a reference cycle between the KV Router scheduler and lease manager, so dropped or failed routers no longer leak scheduler and lease state ([#15258](https://github.com/ai-dynamo/dynamo/pull/15258)).
* **Non-Finite Router Weight Rejection:** Made Router configuration reject infinite and NaN values for `overlap_score_credit_decay`, `prefill_load_scale`, `router_temperature`, and `decode_active_request_weight` at validation time instead of failing on every request after startup. Finite values, including the largest representable value, behave as before ([#15529](https://github.com/ai-dynamo/dynamo/pull/15529)).
* **SGLang Mooncake Key Prefix:** Fixed shared-cache lookups with `--shared-cache-type hicache` so the Router matches SGLang's Mooncake keys, which are prefixed with the backend tag and model name ([#16427](https://github.com/ai-dynamo/dynamo/pull/16427)).
* **Prefill Activation Fail-Closed:** Fixed disaggregated serving during rollouts so requests arriving while the prefill binding is activating receive a retryable 503 instead of being forwarded straight to decode without `bootstrap_info`, so clients receive the 503 instead of a 500 ([#16942](https://github.com/ai-dynamo/dynamo/pull/16942)).
* **Stale Router Discovery Record Removal:** Removed the obsolete `router-discovery` advertisement, whose stale transport could prevent a KV-routing frontend from initializing after a container restart. Worker discovery and request and response formats are unchanged, and a KV frontend without another RPC endpoint no longer binds `DYN_TCP_RPC_PORT`. The last release that consumed the record was Dynamo v1.3.1 ([#16828](https://github.com/ai-dynamo/dynamo/pull/16828)).

### Multimodal & Diffusion

* **Qwen3-ASR Realtime Example Support:** Reworked the vLLM realtime launch example to use vLLM's native `Qwen3ASRRealtimeGeneration` instead of forcing the Voxtral architecture ([#15481](https://github.com/ai-dynamo/dynamo/pull/15481)), and set `Qwen/Qwen3-ASR-0.6B` as the default realtime ASR model ([#16438](https://github.com/ai-dynamo/dynamo/pull/16438)).
* **Multimodal Request Error Propagation:** Surfaced image generation failures instead of returning HTTP 200 with empty data ([#13000](https://github.com/ai-dynamo/dynamo/pull/13000)), mapped timeouts and connection failures on user-supplied image URLs to 408 and 400 ([#14240](https://github.com/ai-dynamo/dynamo/pull/14240)), preserved malformed `data:` URIs as 400 rather than collapsing to 500 ([#14603](https://github.com/ai-dynamo/dynamo/pull/14603)), and returned 400 for multimodal requests to text-only vLLM prefill workers ([#15305](https://github.com/ai-dynamo/dynamo/pull/15305)). The Rust frontend with `--frontend-decoding` now returns HTTP 400 naming the modality instead of HTTP 500 for `image_url` and `video_url` parts that fail ([#16829](https://github.com/ai-dynamo/dynamo/pull/16829)).
* **Shared HTTP Client Media Fetching:** Routed remote media downloads through the shared HTTP client with URL validation and size limits for TensorRT-LLM `.safetensors` embedding URLs ([#14373](https://github.com/ai-dynamo/dynamo/pull/14373)) and vLLM-Omni `ref_audio` voice-cloning downloads ([#14372](https://github.com/ai-dynamo/dynamo/pull/14372)), and enforced the `DYN_MM_MAX_FILE_SIZE_MB` cap, 64 MiB by default, on remote image, audio, and video downloads ([#15005](https://github.com/ai-dynamo/dynamo/pull/15005)).
* **EPD Encode Worker Hardening:** Fixed the vLLM E/P/D encode worker answering 503 forever instead of reporting healthy ([#15434](https://github.com/ai-dynamo/dynamo/pull/15434)), and enabled per-worker port isolation and readiness checks in the multimodal launch scripts ([#14751](https://github.com/ai-dynamo/dynamo/pull/14751)).
* **Qwen DeepStack Embedding Size:** Fixed Qwen multimodal placeholder generation in vLLM to use the model's actual vision embedding dimension instead of a fixed value, resolving tensor splitting errors when running DeepStack-based models such as the Qwen3-VL series ([#11244](https://github.com/ai-dynamo/dynamo/pull/11244)).
* **Multimodal Processor Kwargs Forwarding:** Fixed the TensorRT-LLM backend dropping per-request `mm_processor_kwargs`, which were hardcoded to an empty dictionary before reaching `generate_async` while the request still returned HTTP 200. The value is now read from the top-level field with an `extra_args` fallback, matching the precedence used by the vLLM backend ([#13856](https://github.com/ai-dynamo/dynamo/pull/13856)).
* **Data URI Video Decoding:** Fixed TensorRT-LLM multimodal handling so inline `data:` URI videos are base64-decoded and passed through the same codec probe and NVDEC path as fetched HTTP(S) bytes, instead of failing with an undetected codec error. Malformed or non-base64 URIs and payloads exceeding `max_file_size_bytes` now return HTTP 400 ([#15219](https://github.com/ai-dynamo/dynamo/pull/15219)).
* **Idle Media-Loader CPU Fix:** Fixed media-loader NIXL agent busy-polling by setting a 1 ms progress-thread delay, dropping idle frontend CPU use from 200% to 1.9% per multimodal worker set, and drained per-read notifications that leaked about 110 bytes per image ([#15289](https://github.com/ai-dynamo/dynamo/pull/15289)).
* **Multimodal Decode KV Identity:** Fixed non-Qwen vLLM decode workers clearing `multi_modal_data`, which let same-size images collide in the decode prefix cache so a request could reuse another request's image KV ([#15371](https://github.com/ai-dynamo/dynamo/pull/15371)).
* **TTS Task Type Validation:** Fixed vLLM Omni to reject a `task_type` the Qwen3-TTS checkpoint cannot serve with a 400 instead of terminating the stage, so later valid requests no longer fail ([#15591](https://github.com/ai-dynamo/dynamo/pull/15591)).
* **Unsupported Diffusion Layer Count:** Fixed vLLM-Omni image and video worker startup by omitting the obsolete `layerwise_num_gpu_layers` field by default, which the diffusion configuration validator rejected before engine initialization completed. Explicit legacy CLI or environment values are now rejected during Dynamo configuration validation with guidance to use `--enable-layerwise-offload` without a layer-count override ([#16238](https://github.com/ai-dynamo/dynamo/pull/16238)).

### Kubernetes Deployment

* **Unified vLLM Flag Parsing:** Consolidated the Operator's vLLM argument readers so every flag spelling resolved identically ([#15028](https://github.com/ai-dynamo/dynamo/pull/15028)), and resolved repeated flags to the last occurrence, correcting world size derivation ([#15224](https://github.com/ai-dynamo/dynamo/pull/15224)).
* **DGDR GPU SKU Discovery:** Added canonical SKU identifiers to DGDR GPU discovery, starting with `b300_sxm` for `NVIDIA B300` variants, so an omitted `spec.hardware.gpuSku` no longer persisted an enum-invalid product name ([#15216](https://github.com/ai-dynamo/dynamo/pull/15216)). Canonicalized GB200 to `gb200`, retaining `gb200_sxm` as a deprecated value with an admission warning ([#15711](https://github.com/ai-dynamo/dynamo/pull/15711)), and added Vera Rubin NVL72 support as `vr_nvl72` ([#15996](https://github.com/ai-dynamo/dynamo/pull/15996)).
* **SGLang Snapshot Memory Saver Fixes:** Enabled the SGLang memory saver during argument parsing when Dynamo snapshot mode is active and suppressed raw forward-pass metrics during snapshot startup including restored pods carrying `DYN_FORWARDPASS_METRIC_PORT` ([#14555](https://github.com/ai-dynamo/dynamo/pull/14555)).
* **DGD Adoption and Metadata Validation:** Hardened DGDR adoption to reject an unrelated DGD with the same name and refuse to replace its confirmed deployment ([#13885](https://github.com/ai-dynamo/dynamo/pull/13885)), and validated durable metadata on terminating DGD updates, which previously accepted every mutation ([#13486](https://github.com/ai-dynamo/dynamo/pull/13486)).
* **Oversized Metadata Header Status:** Changed the standalone endpoint picker provider (EPP) to return HTTP 431 Request Header Fields Too Large for oversized scheduling metadata headers instead of 400, and kept the too-many-entries and too-large cases distinguishable, matching the integrated frontend behavior ([#14612](https://github.com/ai-dynamo/dynamo/pull/14612)).
* **Worker Hash Input Normalization:** Fixed the Operator worker hash computation to canonicalize semantically equivalent configurations, including empty Grove wrappers, `forceScalingGroup: false`, disabled checkpoint settings, and explicit backend-default compilation-cache mount paths. Representation-only differences no longer change the `nvidia.com/dynamo-worker-hash` label, though deployments using the explicit forms may see a one-time hash change after upgrading ([#14721](https://github.com/ai-dynamo/dynamo/pull/14721)).
* **Init Container Pull Secrets:** Fixed image pull secret discovery in the Operator to scan init container images in addition to regular containers, so components with a public main image and a private initialization image no longer fail to start without explicitly configured credentials. Explicit pull secrets, deduplication, and discovery-disable behavior were preserved ([#14922](https://github.com/ai-dynamo/dynamo/pull/14922)).
* **Bounded Dynamo Namespace Labels:** Fixed derived Dynamo namespaces that exceeded the 63-byte Kubernetes label limit by shortening them to a readable prefix plus a hash, preventing Grove from rejecting a generated PodCliqueSet ([#16414](https://github.com/ai-dynamo/dynamo/pull/16414)).
* **Shadow-Mode KV Sizing Fix:** Reserved the co-located peer engine's non-KV memory in the shadow-mode vLLM scratch-KV budget, charging each engine 2 GiB for allocations outside PyTorch's allocator plus 1.25 GiB for the CUDA context, so the failover winner's KV cache fits instead of timing out while the peer is still resident ([#16710](https://github.com/ai-dynamo/dynamo/pull/16710)).
* **Authored Frontend Command Handling:** Fixed the Operator so an authored `command` on a frontend container or frontend sidecar owns the entire argv, dropping the default frontend `Args` that previously duplicated `-m dynamo.frontend` and crash-looped the pod. Legacy Go EPP deployments keep their generated pool, config-file, and gRPC arguments ([#16814](https://github.com/ai-dynamo/dynamo/pull/16814)).
* **v1alpha1 envFromSecret Preservation:** Fixed DGDR overrides that set v1alpha1 `envFromSecret` on a service whose v1beta1 main container already had `envFrom`, such as `hf-token-secret`, which previously dropped the requested Secret during conversion. `envFrom` is now additive like `env` and `volumeMounts`, so the result matches the equivalent v1beta1 override ([#16966](https://github.com/ai-dynamo/dynamo/pull/16966)).
* **GMS V1 Snapshot Loader Restart Loop:** Fixed the `gms-loader` container exiting after a successful load with `DYN_GMS_USE_V1=true`, which made the restarted loader crash-loop and left the pod `Ready=False` ([#16915](https://github.com/ai-dynamo/dynamo/pull/16915)).

### Dynamo Sidecar

* **vLLM Sidecar Parser Passthrough:** Accepted `--dyn-tool-call-parser` and `--dyn-reasoning-parser` in the vLLM sidecar so the frontend parses output as in-process workers do ([#15539](https://github.com/ai-dynamo/dynamo/pull/15539)), and started vLLM with `--reasoning-parser none` so thinking requests no longer returned 400 ([#15684](https://github.com/ai-dynamo/dynamo/pull/15684)).
* **Sidecar Streaming Payload Fidelity:** Fixed the vLLM sidecar to forward buffered empty text deltas as empty strings instead of dropping them, so the frontend waits for vLLM to release its stop-string buffer rather than detokenizing the token IDs and duplicating completion text ([#14699](https://github.com/ai-dynamo/dynamo/pull/14699)), with corrected hidden stop-token reporting and preserved cancelled prefill responses ([#15243](https://github.com/ai-dynamo/dynamo/pull/15243)).
* **Effective Attention Block Size:** Fixed the vLLM sidecar to register and publish KV events with the engine-reported effective attention block size, so routing retains KV cache information under decode context parallelism (DCP) ([#14673](https://github.com/ai-dynamo/dynamo/pull/14673)).
* **Decode Worker KV Indexing:** Fixed the shared Rust worker forcing `enable_local_indexer` off for decode workers, so sidecars and other callers can create local KV indexes and recovery endpoints for declared event sources with matching model metadata. Explicit `enable_local_indexer=false` and the encode-role exclusion remain unchanged ([#14981](https://github.com/ai-dynamo/dynamo/pull/14981)).
* **Early Sidecar Health Probes:** Started sidecar health probes before waiting for the inference engine, so Kubernetes can pass the sidecar startup probe and launch the engine container while readiness stays independent of model loading ([#15051](https://github.com/ai-dynamo/dynamo/pull/15051)).
* **SGLang Incremental Streaming Validation:** Added a discovery check that rejects SGLang sidecar startup unless `GetServerInfo` reports incremental streaming output as enabled, with an error directing users to relaunch SGLang with `--incremental-streaming-output`. This prevents duplicated output and inflated completion-token counts caused by cumulative output IDs being treated as new tokens ([#15150](https://github.com/ai-dynamo/dynamo/pull/15150)).
* **SGLang Sidecar Refusal Messages:** Fixed SGLang sidecar request validation so refusals return their specific reason in the HTTP 400 response body instead of a generic "Invalid request" ([#15285](https://github.com/ai-dynamo/dynamo/pull/15285)).
* **SGLang Cached Token Reporting:** Fixed the SGLang sidecar to read `cached_tokens` from the engine's `meta_info` and report it as `usage.prompt_tokens_details.cached_tokens`, clamped to the prompt length, so OpenAI completions and chat responses reflect prefix-cache hits instead of always reporting zero ([#15602](https://github.com/ai-dynamo/dynamo/pull/15602)).
* **Guided Decoding for Tool Choice:** Fixed TensorRT-LLM sidecar launch scripts, deploy examples, and the README engine command to set `guided_decoding_backend: xgrammar`, so chat completions requests with a required or named `tool_choice` return a parsed tool call instead of a 500 error ([#15649](https://github.com/ai-dynamo/dynamo/pull/15649)).
* **Explicit Sidecar Launcher Resolution:** Applied a shared `resolve_sidecar` helper across all 15 SGLang, TensorRT-LLM, and vLLM sidecar launch scripts, so each prefers a native sidecar command and falls back to `python3 -m dynamo.<backend>.sidecar` ([#16059](https://github.com/ai-dynamo/dynamo/pull/16059)).
* **Cancelled Sidecar Request Aborts:** Fixed the vLLM native gRPC sidecar to abort cancelled requests, so queued requests are released instead of occupying an engine slot until output is produced ([#16201](https://github.com/ai-dynamo/dynamo/pull/16201)).
* **TensorRT-LLM Sidecar Request Semantics:** Fixed the TensorRT-LLM sidecar to forward conversation identity across prefill and decode, disable engine detokenization when Dynamo handles it, advertise the configured default thinking mode, and accept logical rank zero while still rejecting unsupported nonzero rank targeting, so conversation-scoped KV cache reuse works and valid rank-zero requests reach the engine ([#16315](https://github.com/ai-dynamo/dynamo/pull/16315)).
* **Hybrid DP Rank Ownership:** Fixed the vLLM sidecar so a frontend owning only part of a data parallel group, such as ranks 4 to 7 of an eight-rank group, is no longer rejected at discovery. Each worker now registers its local rank range ([#14697](https://github.com/ai-dynamo/dynamo/pull/14697)).

### SGLang

* **SGLang Cancellation Lifecycle Hardening:** Added a bounded cancellation lifecycle for SGLang decode and aggregated workers that aborts requests cancelled before the first output token ([#14781](https://github.com/ai-dynamo/dynamo/pull/14781)), extended it to disaggregated prefill workers and their paired decode-side state ([#14782](https://github.com/ai-dynamo/dynamo/pull/14782)), and returned SGLang validation aborts as request errors before placeholder tokens or a successful finish could be emitted ([#15065](https://github.com/ai-dynamo/dynamo/pull/15065)).
* **DCP Attention Backend Validation:** Added a startup check that rejects `--dcp-size` on attention backends that do not read the DCP parallel state, where the SGLang worker would kill its scheduler ([#14800](https://github.com/ai-dynamo/dynamo/pull/14800)), and extended it to MLA decode context parallel, suggesting `--attention-backend flashinfer`, `trtllm_mla`, or `cutedsl_mla` ([#16255](https://github.com/ai-dynamo/dynamo/pull/16255)).
* **SGLang Guided Decoding Fix:** Removed `--skip-tokenizer-init` from SGLang manifests and docs, because the flag caused `response_format: json_schema` requests to return content that did not satisfy the schema ([#13605](https://github.com/ai-dynamo/dynamo/pull/13605)).
* **Disaggregated Parallel Sampling Rejection:** Added validation that rejects SGLang disaggregated requests with `n > 1` using an HTTP 400 error instead of letting them hang indefinitely. Aggregated parallel sampling and disaggregated `n=1` requests continue to work ([#14112](https://github.com/ai-dynamo/dynamo/pull/14112)).
* **SGLang API Compatibility Shims:** Added compatibility shims for `get_model_config` and `use_mla_backend`, which SGLang moved from `ServerArgs` to its `overrides` module, fixing an `AttributeError` that prevented the SGLang worker from starting against newer SGLang versions ([#14234](https://github.com/ai-dynamo/dynamo/pull/14234)).
* **Prefill DP Rank Forwarding:** Fixed SGLang disaggregated serving so the Router forwards the prefill data-parallel rank to decode, dropping decode `bootstrap_duration` from 150 ms to 650 ms down to 0.3 ms to 0.6 ms on GLM-5.2 ([#15246](https://github.com/ai-dynamo/dynamo/pull/15246)).
* **Prefill First-Token Logprobs:** Fixed disaggregated prefill in SGLang to forward the generated-token logprobs option through the shared helper, so the first generated token now carries its log-probability data. Prompt-wide logprobs remain excluded because prefill discards that metadata and the Router does not merge it into the decode response ([#15512](https://github.com/ai-dynamo/dynamo/pull/15512)).
* **Text-Only Engine Media Rejection:** Added SGLang worker validation that returns a 400 error for image, video, or audio content on a text-only model, instead of silently dropping the media ([#15520](https://github.com/ai-dynamo/dynamo/pull/15520)).
* **Attention-DP Rank Counting Fix:** Fixed SGLang DP rank counting to match `dp_size * attn_dp_size`, so attention-DP workers register every rank instead of only rank 0 ([#16598](https://github.com/ai-dynamo/dynamo/pull/16598)).
* **SGLang Crash Exit Delay:** Fixed a 60-second delay in worker exit and pod restart after an SGLang child process died by defaulting `SGLANG_CUDA_COREDUMP_BEFORE_CRASH_WAIT_SECS` to 0 unless the user sets it, sets `CUDA_ENABLE_USER_TRIGGERED_COREDUMP=1`, or passes `--crash-dump-folder`. On one A100, worker exit after a killed detokenizer dropped from 66 seconds to 6 seconds ([#16952](https://github.com/ai-dynamo/dynamo/pull/16952)).

### TensorRT-LLM

* **Request-Local Pre-Submit Validation Errors:** Fixed TensorRT-LLM handling of pre-submit `ValueError`, `TypeError`, and `NotImplementedError` failures so they return a request-scoped error instead of shutting down a healthy engine, while worker shutdown is preserved for unhealthy engines and errors raised during result iteration ([#14006](https://github.com/ai-dynamo/dynamo/pull/14006)).
* **Decode Cached Token Reporting:** Fixed `usage.prompt_tokens_details.cached_tokens` on the TensorRT-LLM generation-only path, which reported the transferred prompt as a cache hit, so benchmarks, dashboards, and routing decisions that read the usage block get accurate cache-hit data when `return_perf_metrics` is enabled ([#15389](https://github.com/ai-dynamo/dynamo/pull/15389)).
* **Salted TensorRT-LLM Health Checks:** Fixed TensorRT-LLM workers aborting when attention data parallelism and the V2 KV cache manager were enabled by giving the health-check probe request its own cache salt, so it no longer shares a reuse namespace with the engine's dummy requests. A caller-supplied salt still takes precedence ([#16688](https://github.com/ai-dynamo/dynamo/pull/16688)).
* **MPI Child Failure Exit:** Fixed TensorRT-LLM workers hanging until Kubernetes killed the container after an MPI engine child died. The worker now exits promptly with code 1 ([#16931](https://github.com/ai-dynamo/dynamo/pull/16931)).
* **Snapshot Restore MPI Transport:** Fixed restored TensorRT-LLM engines with TP greater than 1 failing their first collective with `ucx send failed: Endpoint timeout`. Snapshot mode now defaults `OMPI_MCA_pml=ob1` and `OMPI_MCA_btl=tcp,self`, because Open MPI's UCX transport enables TCP keepalive probes that abort inter-rank connections during the checkpoint dump. Explicit settings still take precedence ([#16941](https://github.com/ai-dynamo/dynamo/pull/16941)).

### vLLM

* **ModelExpress RL Version Declaration:** Reworked vLLM weight version reporting to report `version: null` with `version_declared: false` until declared, adding a `set_weight_version` route ([#13041](https://github.com/ai-dynamo/dynamo/pull/13041)). Workers loading through the ModelExpress RL loader now declare `MX_REFIT_DESIRED_VERSION_UID` at startup ([#15252](https://github.com/ai-dynamo/dynamo/pull/15252)).
* **Prefill Worker Lifecycle Hardening:** Shut down EngineCore when a prefill worker exits and made FPM self-benchmark inputs reproducible with retained input, timing, execution, preparation, and worker provenance ([#16789](https://github.com/ai-dynamo/dynamo/pull/16789)), with prefill health probes no longer requesting a KV connector handoff, which previously left unclaimed KV transfer state on idle prefill workers and blocked KV cache resets ([#16825](https://github.com/ai-dynamo/dynamo/pull/16825)).
* **Elastic EP Grow Rollback:** Fixed vLLM `scale_elastic_ep` so a failed grow rolls back to the previous data-parallel size and keeps serving instead of wedging the engine ([#12991](https://github.com/ai-dynamo/dynamo/pull/12991)).
* **Engine-Expanded Prompt ID Metadata:** Fixed vLLM `nvext.engine_data` to report the engine-processed prompt tokens, so training consumers can align them with completion tokens and log probabilities ([#15470](https://github.com/ai-dynamo/dynamo/pull/15470)).

### Model Loading & LoRA

* **LoRA Card Registration Fix:** Fixed LoRA adapter registration in the SGLang and vLLM handlers by passing `ignore_weights=True`, so card registration no longer downloads base-model weights on a cache miss and fails before publishing the adapter ([#11164](https://github.com/ai-dynamo/dynamo/pull/11164)).

### Performance Modeling & Replay

* **Forward-Pass Metrics KV Warm-Up:** Settled KV warm-up so decode sweeps no longer abort when attention-DP ranks report READY in different steps, with hybrid KDA/Mamba fixes ([#14728](https://github.com/ai-dynamo/dynamo/pull/14728), [#14614](https://github.com/ai-dynamo/dynamo/pull/14614)). Snapshot-restored vLLM replicas now stay visible to the Planner ([#12545](https://github.com/ai-dynamo/dynamo/pull/12545)).

### Fault Tolerance & Observability

* **Request Trace Record Accuracy:** Reworked audit records to derive `payload_complete` and `payload_drop_reason` from the actual response ([#14143](https://github.com/ai-dynamo/dynamo/pull/14143)), and wrote `request_end` for requests cancelled before the first response item ([#14799](https://github.com/ai-dynamo/dynamo/pull/14799)).
* **Histogram Bucket Configuration Fixes:** Raised the default inter-token latency bucket ceiling from 2.0s to 80.0s ([#14839](https://github.com/ai-dynamo/dynamo/pull/14839)).
* **Etcd and NATS Connection Resilience:** Added etcd watch retry with backoff, enabled HTTP/2 keepalive by default with a 15s ping interval and 10s acknowledgment timeout, and raised the default lease TTL from 10s to 30s ([#15266](https://github.com/ai-dynamo/dynamo/pull/15266)). Startup now retries temporary NATS connection failures for up to 120 seconds ([#15269](https://github.com/ai-dynamo/dynamo/pull/15269)).
* **System Status Listener Rebinding:** Fixed the system status server to rebind its TCP listener after accept failures instead of retrying a stale descriptor indefinitely, keeping status endpoints reachable after process restore while retaining the originally bound address, including an assigned ephemeral port ([#12059](https://github.com/ai-dynamo/dynamo/pull/12059)).
* **Prefill Worker KV Gauges:** Fixed missing `dynamo_component_total_blocks` and `dynamo_component_gpu_cache_usage_percent` samples on disaggregated vLLM prefill workers, which now publish configured total capacity at startup ([#12559](https://github.com/ai-dynamo/dynamo/pull/12559)).
* **Pending Handshake Cancellation:** Fixed requests hanging when a worker left discovery before sending the response prologue, so worker removal unblocks the requester with a migratable `Disconnected` error ([#13095](https://github.com/ai-dynamo/dynamo/pull/13095)).
* **Worker Error Preservation:** Fixed finish-reason handling in the backend so a worker's error finish reason is no longer overwritten by local stop detection, token limits, or user-defined stop strings, preventing failed requests from returning HTTP 200 with `finish_reason: "stop"` and malformed output ([#15094](https://github.com/ai-dynamo/dynamo/pull/15094)).
* **Migration Handshake Cancellation Accounting:** Fixed the Frontend to record client cancellation during a migration handshake as `outcome="cancelled"` instead of `outcome="success"` in `dynamo_frontend_model_migration_duration_seconds`, so migration metrics match the HTTP outcome for the same request ([#16440](https://github.com/ai-dynamo/dynamo/pull/16440)).
* **Invalid Request Refusal Handling:** Fixed worker routability after a worker refused an invalid request before opening the response stream, returning the worker's own error instead of wrapping it in `CannotConnect`. Clients now receive HTTP 400 and the worker stays routable rather than being marked down and causing HTTP 503 on the next valid request ([#16597](https://github.com/ai-dynamo/dynamo/pull/16597)).

### Runtime & Transport

* **IPv6-Only Networking Support:** Reworked local address resolution to prefer non-loopback IPv4, then IPv6, then loopback ([#13126](https://github.com/ai-dynamo/dynamo/pull/13126)), with the ZMQ event plane reusing it ([#15324](https://github.com/ai-dynamo/dynamo/pull/15324)). Narrowed `ZMQ_IPV6` to bracketed IPv6 endpoints, fixed remaining IPv6-only gaps in listeners, SGLang KV events, and vLLM multinode ([#15383](https://github.com/ai-dynamo/dynamo/pull/15383)), and changed automatic host selection to prefer the routed source address ([#15628](https://github.com/ai-dynamo/dynamo/pull/15628)).
* **QUIC Response Plane Reliability:** Carried typed pre-stream errors through the QUIC response plane, mapping `InvalidArgument` to HTTP 400 ([#14789](https://github.com/ai-dynamo/dynamo/pull/14789)), and updated Quinn to v0.11.12 to fix `too many gaps in stream buffer` failures ([#14876](https://github.com/ai-dynamo/dynamo/pull/14876)).
* **Endpoint Identity-Scoped TCP Routing:** Scoped TCP endpoint routing and cleanup by full endpoint identity, so endpoints registered under the same name in one process no longer overwrite or tear down each other's handlers ([#14723](https://github.com/ai-dynamo/dynamo/pull/14723), [#14271](https://github.com/ai-dynamo/dynamo/pull/14271)).
* **Oversized TCP Frame Rejection:** Added a preflight size check in the TCP client that rejects requests exceeding `DYN_TCP_MAX_MESSAGE_SIZE` before connection lookup and admission, returning `InvalidArgument` which the HTTP layer maps to a 400 response instead of letting the oversized frame reach the worker and close the connection during decode ([#14110](https://github.com/ai-dynamo/dynamo/pull/14110)).
* **ZMQ Socket Exhaustion Guidance:** Added error messages when event-plane socket creation fails with `EMFILE`, so operators hitting file-descriptor limits see advice to reduce direct-ZMQ peers or raise `ulimit -n` ([#14145](https://github.com/ai-dynamo/dynamo/pull/14145)).
* **Shared Fetch Model Runtime:** Fixed `fetch_model()` creating a second Tokio runtime when called before a Dynamo runtime existed, and the Python Worker mutating the process environment when applying transport overrides ([#14534](https://github.com/ai-dynamo/dynamo/pull/14534)).
* **ZMQ Publisher Port Race:** Fixed a race during concurrent ZMQ publisher startup by binding the publisher socket directly to an ephemeral port instead of probing with a temporary TCP listener, and read the assigned address back from the socket for logging and discovery, so concurrent publishers no longer collide on the same port ([#14566](https://github.com/ai-dynamo/dynamo/pull/14566)).
* **Discovery Watch Establishment Ordering:** Fixed discovery list-and-watch to return only after the backend watch is established, preventing missed `Removed` events ([#14769](https://github.com/ai-dynamo/dynamo/pull/14769)). Every watch now opens with one Added event per matching instance and one Resync snapshot, so a reopened watch replaces state without a separate list call ([#15143](https://github.com/ai-dynamo/dynamo/pull/15143)).

### Container Images

* **Root Cache Mount Target:** Fixed `container/run.sh` so that user overrides with a group suffix, such as `--user root:root` or `0:1000`, mount the Hugging Face cache at `/root/.cache/huggingface` instead of the Dynamo user cache directory. Non-root users with group 0 continue to use the Dynamo user cache ([#14820](https://github.com/ai-dynamo/dynamo/pull/14820)).

## Dependency Changes

Engine and NIXL pins are on [Compatibility](/dynamo/reference/compatibility). The other dependency changes in this release:

* **NATS Server:** Upgraded from v2.12.14 to v2.14.8 in the runtime images ([#16845](https://github.com/ai-dynamo/dynamo/pull/16845)).
* **Frontend Runtime Libraries:** Upgraded `dynamo-renderer` from v5.1.2 to v8.0.3 and `dynamo-tokenizers` from v1.8.1 to v3.1.1, both major-version jumps ([#16686](https://github.com/ai-dynamo/dynamo/pull/16686)).
* **Planner Image Python:** Moved the `dynamo-planner` container from CPython 3.12 to 3.13 ([#14924](https://github.com/ai-dynamo/dynamo/pull/14924)).
* **LMCache:** Raised the floor to v0.5.6 in the vLLM CUDA runtime image ([#16845](https://github.com/ai-dynamo/dynamo/pull/16845)).
* **vLLM-Omni:** Upgraded from v0.28.0rc1 to v0.31.0rc1 on CUDA images ([#15643](https://github.com/ai-dynamo/dynamo/pull/15643)), and from v0.28.0rc1 (CPU) and v0.27.0rc1 (XPU) to v0.30.0rc1 ([#15182](https://github.com/ai-dynamo/dynamo/pull/15182), [#16673](https://github.com/ai-dynamo/dynamo/pull/16673)).
* **vLLM on CPU and XPU:** Upgraded from v0.28.0 (CPU) and v0.27.1 (XPU) to v0.30.0 ([#15182](https://github.com/ai-dynamo/dynamo/pull/15182), [#16673](https://github.com/ai-dynamo/dynamo/pull/16673)).
* **SGLang on XPU:** Upgraded from v0.5.11 to v0.5.19, built against oneAPI 2026.0 ([#15186](https://github.com/ai-dynamo/dynamo/pull/15186)).
* **libfabric:** Upgraded from v2.4.0 to v2.6.0 ([#15908](https://github.com/ai-dynamo/dynamo/pull/15908)).
* **etcd:** Upgraded from v3.5.33 to v3.5.34 ([#15697](https://github.com/ai-dynamo/dynamo/pull/15697)).
* **AIPerf and AISimulate:** Upgraded both from v0.12.0 to v0.13.0 ([#15495](https://github.com/ai-dynamo/dynamo/pull/15495), [#16949](https://github.com/ai-dynamo/dynamo/pull/16949)).
* **Grove:** Upgraded from v0.1.0-alpha.13 to v0.1.0-alpha.14 ([#15704](https://github.com/ai-dynamo/dynamo/pull/15704)).
* **ModelExpress:** Upgraded from v0.6.0 to v0.7.0 ([#15656](https://github.com/ai-dynamo/dynamo/pull/15656)).
* **Triton Runtime Image:** Moved from `26.07-py3` (CUDA 13.2) to `26.09-py3` (CUDA 13.4) ([#15499](https://github.com/ai-dynamo/dynamo/pull/15499)).

## Documentation

### New Content

* **EPP Mode Boundary Documentation:** Clarified which EPP mode operator-generated resources apply to and the RBAC boundary with standalone mode ([#14756](https://github.com/ai-dynamo/dynamo/pull/14756), [#14620](https://github.com/ai-dynamo/dynamo/pull/14620)). Documented the `FailClose` endpoint-picker failure mode, required a `VLLM_IMAGE` variable in the on-ramp example, and corrected docs left over from the Go EPP deprecation ([#14811](https://github.com/ai-dynamo/dynamo/pull/14811), [#14481](https://github.com/ai-dynamo/dynamo/pull/14481), [#14263](https://github.com/ai-dynamo/dynamo/pull/14263)).
* **Operator Deployment Caveat Docs:** Added a warning that `compilationCache` PVC contents must be cleared after a backend, torch, or CUDA upgrade ([#14381](https://github.com/ai-dynamo/dynamo/pull/14381)), and documented that creating a DGDR carries the same authority as creating a Job or Pod in that namespace, while Pod-level security stays enforced by Pod Security Admission ([#14429](https://github.com/ai-dynamo/dynamo/pull/14429)).
* **Snapshot API Documentation:** Replaced the `DynamoCheckpoint Auto mode` wording with the `SnapshotJob` to `PodSnapshot` workflow ([#16037](https://github.com/ai-dynamo/dynamo/pull/16037)), with a Kubernetes capture and restore quickstart including TensorRT-LLM examples for Qwen3-0.6B on one GPU and pinned GLM 5.3 NVFP4 on eight B200s ([#16044](https://github.com/ai-dynamo/dynamo/pull/16044)).
* **KV Cache Transfer Docs Corrections:** Corrected the TensorRT-LLM docs to state `TRTLLM_USE_UCX_KVCACHE` enables direct UCX even with `cache_transceiver_config.backend: DEFAULT`, and added a warning that mounting host `/opt/amazon/efa` breaks the NIXL LIBFABRIC backend ([#13048](https://github.com/ai-dynamo/dynamo/pull/13048)).
* **Secure Deployment Guidelines Documentation:** Added a Secure Deployment Guidelines page documenting Dynamo's trust model, securing the external inference API, authenticating the discovery, event, and request planes, and running with least privilege ([#13505](https://github.com/ai-dynamo/dynamo/pull/13505)).
* **Nightly Install Selector Accuracy:** Fixed the nightly install selector rows to show the shipped backend versions, or "not currently available" instead of the stable tag ([#13599](https://github.com/ai-dynamo/dynamo/pull/13599)).
* **Planner Mode Selection Guide:** Added a Choose a Planner Mode guide, split the overview by whether users have TTFT/ITL SLA targets, and documented KV cache reuse and speculative decoding ([#14335](https://github.com/ai-dynamo/dynamo/pull/14335)).
* **EPD Disaggregation Blog Post:** Added a blog post on when to use encode-prefill-decode (EPD) disaggregation for multimodal serving, and listed the Pinterest Engineering article on building a VLM serving stack with Dynamo in the external publications page ([#14630](https://github.com/ai-dynamo/dynamo/pull/14630)).
* **LongCat TokenSpeed Disaggregation Example:** Added an experimental LongCat-Flash TokenSpeed disaggregated example with KV-aware routing, and its GB300 configuration raises sequence and KV-cache capacity (`--max-num-seqs` 4 to 32, `--max-total-tokens` 262144 to 917504) and selects `trtllm_mla` to address long-context decode slowdown ([#15013](https://github.com/ai-dynamo/dynamo/pull/15013)).
* **Nemotron-3-Ultra Performance Table:** Restored the Expected Performance table, covering all 12 aggregated and disaggregated profiles across B200, GB200, and H200 at 256K and 1M context lengths ([#15194](https://github.com/ai-dynamo/dynamo/pull/15194)).
* **SGLang Speculative Decoding Guide:** Added a documentation page for EAGLE3 speculative decoding with SGLang, covering aggregated quick start, disaggregated serving with the KV router, and `sglang:spec_*` metrics, and marked SGLang speculative decoding as supported in the speculative decoding overview, the SGLang feature matrix, and the compatibility tables ([#15268](https://github.com/ai-dynamo/dynamo/pull/15268)).
* **Many-Model GPU Sharing Examples:** Added a Many-Model Serving section and `examples/gpu-sharing/` directory benchmarking Qwen3-4B on one 8x A100 40GB node. At equal load per GPU, GPU fractions retained 60-88% of dedicated-GPU throughput versus 45-73% for HAMi, about 23% cheaper per output token at light per-model load ([#15918](https://github.com/ai-dynamo/dynamo/pull/15918)).
* **Sidecar Documentation Simplification:** Simplified the sidecar READMEs and Fern pages to one example per engine, added topology diagrams for SGLang, TensorRT-LLM, and vLLM, and corrected stale claims: the `dynamo-sidecar` image is published to NGC starting with v1.6.0 ([#16062](https://github.com/ai-dynamo/dynamo/pull/16062)).
* **Tied-Embedding Policy Update Limitation:** Documented that the first policy update fails for Qwen2.5 and other tied-embedding models on vLLM v0.29.0 and later, with links to the upstream verl fixes, which no reviewed recipe snapshot selects yet ([#16682](https://github.com/ai-dynamo/dynamo/pull/16682)).

## Looking Ahead

### Runtime Router Policy Management

A `/router_policy` API on the Frontend reads and updates live worker-selection parameters and busy thresholds without a restart.

### Planner Reconfiguration & Batch Scheduling

Runtime reconfiguration extends beyond GPU budgets to a running Planner's SLA targets and load predictor. An opt-in batch scheduling policy reserves capacity for online traffic, with the Batch API served through the Frontend.

### Generation Artifacts for RL

vLLM workers can deliver bounded generation artifacts inline or to object stores for reinforcement learning pipelines.

### Kubernetes Deployment Search

An experimental `v1beta2` API runs simulation-backed deployment searches, and DisaggregatedSet joins Grove as an opt-in workload provider.

## Dynamo Special Interest Groups (SIGs)

Dynamo now has six Special Interest Groups (SIGs). Each SIG runs a public meeting, reviews the Dynamo Enhancement Proposals (DEPs) and PRs in its area, and sets that area's roadmap. Anyone in the community can join, bring a PR or DEP, and be involved in the technical direction of Dynamo.

* **#sig-router:** KV-aware routing, Global Router, and Gateway API routing.
* **#sig-engines:** SGLang, TensorRT-LLM, vLLM, TokenSpeed, and Dynamo Sidecar.
* **#sig-frontend:** Frontend APIs, tool calling, and parsers.
* **#sig-deploy:** Kubernetes Operator, Helm charts, DRA, and Grove.
* **#sig-agents:** Agentic inference, agent hints, and request tracing.
* **#sig-rl:** RL integration, rollouts, and weight updates.

**How to Join**

1. Join the [ai-dynamo Slack](https://ai-dynamo.org/slack) and find the channel.
2. Subscribe to the [community calendar](/dynamo/dev/community) for meeting times.

For how to contribute code, docs, and recipes, see the [Contribution Guide](/dynamo/dev/contributing/contribution-guide).

#### Open-source contributions and new contributors

Between v1.5.1 and v1.6.0 the project merged 654 PRs from 148 contributors, 58 of them from outside NVIDIA. Thank you to the external community contributors in this release (organization identified through the commit-author email domain or the public GitHub profile):

* **DaoCloud:** [@bzsuni](https://github.com/bzsuni), [@carlory](https://github.com/carlory), [@cyclinder](https://github.com/cyclinder), [@flpanbin](https://github.com/flpanbin), [@git-jxj](https://github.com/git-jxj), [@panpan0000](https://github.com/panpan0000), [@weizhoublue](https://github.com/weizhoublue), [@yankay](https://github.com/yankay).
* **Intel:** [@Jianhong-Zhang](https://github.com/Jianhong-Zhang), [@sandeep-maddipatla](https://github.com/sandeep-maddipatla), [@tthakkal](https://github.com/tthakkal), [@VincyZhang](https://github.com/VincyZhang), [@zxue2](https://github.com/zxue2).
* **CoreWeave:** [@alexeldeib](https://github.com/alexeldeib), [@c2w-sea](https://github.com/c2w-sea), [@houshengbo](https://github.com/houshengbo), [@surajssd](https://github.com/surajssd).
* **Aliyun:** [@xianlubird](https://github.com/xianlubird).
* **Altera Labs:** [@aylonsk](https://github.com/aylonsk).
* **Apple:** [@dundysm](https://github.com/dundysm).
* **Baseten:** [@jacksonrnewhouse](https://github.com/jacksonrnewhouse), [@michaelfeil](https://github.com/michaelfeil), [@Stelath](https://github.com/Stelath).
* **Datadog:** [@walkoss](https://github.com/walkoss).
* **Docker:** [@ericcurtin](https://github.com/ericcurtin).
* **Fireworks AI:** [@zhaohuizh](https://github.com/zhaohuizh).
* **Gcore:** [@Kaonael](https://github.com/Kaonael).
* **HKUST:** [@ZenAlexa](https://github.com/ZenAlexa).
* **IIT Kharagpur:** [@shwetha-krishnamurthy](https://github.com/shwetha-krishnamurthy).
* **Inferact:** [@ywang96](https://github.com/ywang96).
* **LinkedIn:** [@chanh](https://github.com/chanh).
* **Microsoft:** [@Jont828](https://github.com/Jont828).
* **New York University:** [@JasperChan929](https://github.com/JasperChan929).
* **PaddlePaddle:** [@ShaneGZhu](https://github.com/ShaneGZhu).
* **Prime Intellect:** [@qimcis](https://github.com/qimcis).
* **Rimo:** [@Atharva-Kanherkar](https://github.com/Atharva-Kanherkar).
* **San Jose State University:** [@ayaangazali](https://github.com/ayaangazali).
* **SK Telecom:** [@changhyeonnam](https://github.com/changhyeonnam), [@SeungminHeo](https://github.com/SeungminHeo).
* **T-Systems:** [@RealNicolasBourbaki](https://github.com/RealNicolasBourbaki).
* **UC San Diego:** [@nihalmaddala](https://github.com/nihalmaddala).
* **UCAS:** [@gitover22](https://github.com/gitover22).
* **UIUC:** [@livingshade](https://github.com/livingshade).
* **xAI:** [@sherlockwu](https://github.com/sherlockwu).
* **Independent:** [@AmeenP](https://github.com/AmeenP), [@awesome-pro](https://github.com/awesome-pro), [@bojiang-li](https://github.com/bojiang-li), [@Champollion9012](https://github.com/Champollion9012), [@charan-rathore](https://github.com/charan-rathore), [@cmdy](https://github.com/cmdy), [@Efeselimsurekli](https://github.com/Efeselimsurekli), [@elizabetht](https://github.com/elizabetht), [@host452b](https://github.com/host452b), [@jorgeantonio21](https://github.com/jorgeantonio21), [@junuxyz](https://github.com/junuxyz), [@neilkg](https://github.com/neilkg), [@Prudctual](https://github.com/Prudctual), [@Sumit884-byte](https://github.com/Sumit884-byte).

**New Contributors:** welcome to the 45 first-time contributors in this release:

* [@Victor49152](https://github.com/Victor49152) matched the Qwen vision utilities to the DeepStack vision embedding size ([#11244](https://github.com/ai-dynamo/dynamo/pull/11244)).
* [@Stelath](https://github.com/Stelath) preserved raw reasoning output in the Responses API ([#11604](https://github.com/ai-dynamo/dynamo/pull/11604)).
* [@RealNicolasBourbaki](https://github.com/RealNicolasBourbaki) propagated the worker ID into snapshot-restored vLLM engine cores ([#12545](https://github.com/ai-dynamo/dynamo/pull/12545)).
* [@gitover22](https://github.com/gitover22) published KV cache gauges from vLLM prefill workers ([#12559](https://github.com/ai-dynamo/dynamo/pull/12559)).
* [@aswinvisva](https://github.com/aswinvisva) forwarded `mm_processor_kwargs` to the TensorRT-LLM engine and enabled NVDEC decoding of `data:` URI videos ([#13856](https://github.com/ai-dynamo/dynamo/pull/13856), [#15219](https://github.com/ai-dynamo/dynamo/pull/15219)).
* [@ShaneGZhu](https://github.com/ShaneGZhu) made the ThunderAgent cost model aware of data-parallel ranks ([#14000](https://github.com/ai-dynamo/dynamo/pull/14000)).
* [@aylonsk](https://github.com/aylonsk) bounded the KV router state-agent discovery wait at worker startup and four more Planner and router fixes ([#14003](https://github.com/ai-dynamo/dynamo/pull/14003)).
* [@Champollion9012](https://github.com/Champollion9012) recorded the router prefill start only for phases that run prefill ([#14019](https://github.com/ai-dynamo/dynamo/pull/14019)).
* [@sbak5](https://github.com/sbak5) added per-request lifecycle instrumentation to the runtime ([#14101](https://github.com/ai-dynamo/dynamo/pull/14101)).
* [@ZenAlexa](https://github.com/ZenAlexa) rejected oversized TCP request frames before send ([#14110](https://github.com/ai-dynamo/dynamo/pull/14110)).
* [@Vinya567](https://github.com/Vinya567) added FP16 and BF16 tensor protocol variants and Triton `/v1/classify` support ([#14150](https://github.com/ai-dynamo/dynamo/pull/14150), [#15396](https://github.com/ai-dynamo/dynamo/pull/15396)).
* [@cyclinder](https://github.com/cyclinder) preserved forced SGLang tool guidance in the frontend ([#14179](https://github.com/ai-dynamo/dynamo/pull/14179)).
* [@trevor-m](https://github.com/trevor-m) fixed SGLang compatibility for `get_model_config` and `use_mla_backend` ([#14234](https://github.com/ai-dynamo/dynamo/pull/14234)).
* [@SeungminHeo](https://github.com/SeungminHeo) added A.X-K2 H200 vLLM recipes with a disaggregated benchmark ([#14318](https://github.com/ai-dynamo/dynamo/pull/14318)).
* [@Efeselimsurekli](https://github.com/Efeselimsurekli) documented that the operator `compilationCache` is version-specific ([#14381](https://github.com/ai-dynamo/dynamo/pull/14381)).
* [@git-jxj](https://github.com/git-jxj) rejected unsupported tools in the Responses API before dispatch ([#14440](https://github.com/ai-dynamo/dynamo/pull/14440)).
* [@diffractiongoo](https://github.com/diffractiongoo) shared the process runtime with model fetching ([#14534](https://github.com/ai-dynamo/dynamo/pull/14534)).
* [@nvjullin](https://github.com/nvjullin) forwarded `session_id` and `parent_session_id` to the SGLang engine ([#14611](https://github.com/ai-dynamo/dynamo/pull/14611)).
* [@tianhaox](https://github.com/tianhaox) fixed the decode warm-up for hybrid KDA and Mamba models, validated on GLM-5.3-Flash ([#14614](https://github.com/ai-dynamo/dynamo/pull/14614)).
* [@jacksonrnewhouse](https://github.com/jacksonrnewhouse) drained SSE streams after graceful generation stops ([#14645](https://github.com/ai-dynamo/dynamo/pull/14645)).
* [@zhaohuizh](https://github.com/zhaohuizh) shared prompt tokens across request clones ([#14701](https://github.com/ai-dynamo/dynamo/pull/14701)).
* [@Arsene12358](https://github.com/Arsene12358) settled vLLM KV warm-up stages through a per-step round on every attention data-parallel rank ([#14728](https://github.com/ai-dynamo/dynamo/pull/14728)).
* [@Prudctual](https://github.com/Prudctual) carried typed pre-stream errors on the QUIC response plane and passed `response_format` into vLLM-Omni video inputs ([#14789](https://github.com/ai-dynamo/dynamo/pull/14789), [#14844](https://github.com/ai-dynamo/dynamo/pull/14844)).
* [@Sumit884-byte](https://github.com/Sumit884-byte) sent inline media once on the TCP request plane and prepended the KServe batch dimension for Triton metadata ([#14801](https://github.com/ai-dynamo/dynamo/pull/14801), [#14996](https://github.com/ai-dynamo/dynamo/pull/14996)).
* [@bojiang-li](https://github.com/bojiang-li) used the root cache for container user overrides with a group and three more operator, cache, and multimodal fixes ([#14820](https://github.com/ai-dynamo/dynamo/pull/14820)).
* [@host452b](https://github.com/host452b) enforced media download limits ([#15005](https://github.com/ai-dynamo/dynamo/pull/15005)).
* [@andrewpaprotsky](https://github.com/andrewpaprotsky) added the operator LPX integration and four follow-up LPX changes ([#15062](https://github.com/ai-dynamo/dynamo/pull/15062)).
* [@Atharva-Kanherkar](https://github.com/Atharva-Kanherkar) preserved URL path case in the multimodal image cache key ([#15101](https://github.com/ai-dynamo/dynamo/pull/15101)).
* [@sherlockwu](https://github.com/sherlockwu) gated SGLang auto tool-call grammars on reasoning and forwarded tool-message media in Python chat processors ([#15227](https://github.com/ai-dynamo/dynamo/pull/15227), [#15237](https://github.com/ai-dynamo/dynamo/pull/15237)).
* [@livingshade](https://github.com/livingshade) added SGLang EAGLE3 speculative decoding tests and a guide ([#15251](https://github.com/ai-dynamo/dynamo/pull/15251), [#15268](https://github.com/ai-dynamo/dynamo/pull/15268)).
* [@c2w-sea](https://github.com/c2w-sea) declared the ModelExpress startup weight version for vLLM ([#15252](https://github.com/ai-dynamo/dynamo/pull/15252)).
* [@junuxyz](https://github.com/junuxyz) moved vLLM runtime configurations, examples, and recipes to explicit NIXL roles ([#15256](https://github.com/ai-dynamo/dynamo/pull/15256), [#15366](https://github.com/ai-dynamo/dynamo/pull/15366)).
* [@surajssd](https://github.com/surajssd) broke a KV router request-lease ownership cycle and retried NATS connections at startup ([#15258](https://github.com/ai-dynamo/dynamo/pull/15258), [#15269](https://github.com/ai-dynamo/dynamo/pull/15269)).
* [@JasperChan929](https://github.com/JasperChan929) charged and gated explicit hard pins at ThunderAgent admission ([#15290](https://github.com/ai-dynamo/dynamo/pull/15290)).
* [@dundysm](https://github.com/dundysm) kept `multi_modal_data` on non-Qwen vLLM decode for image KV identity ([#15371](https://github.com/ai-dynamo/dynamo/pull/15371)).
* [@charan-rathore](https://github.com/charan-rathore) placed the vLLM-Omni video `negative_prompt` in the prompt entry ([#15402](https://github.com/ai-dynamo/dynamo/pull/15402)).
* [@awesome-pro](https://github.com/awesome-pro) generated the nightly release ledger for the docs ([#15410](https://github.com/ai-dynamo/dynamo/pull/15410)).
* [@sunil1511](https://github.com/sunil1511) added shared NGC model fetching with SGLang and vLLM integration ([#15441](https://github.com/ai-dynamo/dynamo/pull/15441)).
* [@neilkg](https://github.com/neilkg) passed the checkpoint tokenizer to SGLang parsers and eight more frontend and SGLang fidelity fixes ([#15509](https://github.com/ai-dynamo/dynamo/pull/15509)).
* [@kathleenfrench](https://github.com/kathleenfrench) dropped the stale `dynamoGraphDeploymentRef` from LPX bindings ([#15522](https://github.com/ai-dynamo/dynamo/pull/15522)).
* [@geelen](https://github.com/geelen) let recipe cards without logos use the full header width ([#15546](https://github.com/ai-dynamo/dynamo/pull/15546)).
* [@joshhay95](https://github.com/joshhay95) selected GPU-local partitions for hybrid and HX LPX builds ([#15563](https://github.com/ai-dynamo/dynamo/pull/15563), [#15706](https://github.com/ai-dynamo/dynamo/pull/15706)).
* [@ericcurtin](https://github.com/ericcurtin) rejected vLLM-Omni TTS tasks the checkpoint cannot serve ([#15591](https://github.com/ai-dynamo/dynamo/pull/15591)).
* [@mattwittwer](https://github.com/mattwittwer) rejected non-numeric `logit_bias` keys in the frontend ([#16322](https://github.com/ai-dynamo/dynamo/pull/16322)).
* [@Jianhong-Zhang](https://github.com/Jianhong-Zhang) upgraded SGLang XPU to v0.5.19 on oneAPI 2026.0 ([#16684](https://github.com/ai-dynamo/dynamo/pull/16684)).