> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Dynamo v1.4.0

Dynamo v1.4.0 is the 17th feature release of the open-source distributed inference platform, spanning 640 merged PRs from 127 contributors. It takes **Dynamo Router** multi-datacenter with a sequenced KV relay, peer reservation replay, and endpoint-scoped event transport. It opens an experimental vLLM-compatible **`/inference/v1/generate`** token-in/token-out endpoint on the Dynamo Frontend alongside a default-on tokenizer prefix cache. **vLLM-Omni** pipelines gain NIXL RDMA disaggregation for true multi-node autoregressive-to-diffusion serving, the **Spica** discrete-event simulation framework joins the repository, and the engine fleet moves to vLLM v0.26.0, SGLang v0.5.16, TensorRT-LLM 1.3.0rc22, and NIXL v1.3.x. The release also ships a rebuilt **documentation website** with tab-based navigation and native Simplified Chinese localization.

Breaking changes and deprecations for this release are tracked on the [Deprecations](/dynamo/reference/releases/deprecations#v140) ledger; known issues on the [Known Issues](/dynamo/reference/releases/known-issues#v140) page. Key dependency pins live on [Compatibility](/dynamo/reference/compatibility); shipped artifacts on [Release Artifacts](/dynamo/reference/release-artifacts). Model early access builds (`vX.Y.Z-<model>-dev.N`) are tracked in [Model Early Access Builds](/dynamo/reference/model-early-access-builds).

## Get v1.4.0

Pull, deploy, and install with every artifact pinned to the v1.4.0 release set.

## Highlights

## Features & Improvements

### Frontend

#### Generate Endpoint

* **Experimental /inference/v1/generate Endpoint:** Added a vLLM-compatible `/inference/v1/generate` token-in/token-out endpoint on the Dynamo frontend ([#11108](https://github.com/ai-dynamo/dynamo/pull/11108)), then dispatched real requests through it with Generate capability gating, tokenizer-free frontend registration, and sanitized cancellations and errors ([#11167](https://github.com/ai-dynamo/dynamo/pull/11167)). The endpoint remains experimental and unary-only.

#### Tokenizer & Chat Templates

* **Tokenizer Prefix Cache Enablement:** Enabled the process-local L1 tokenizer prefix cache by default unless `DYN_TOKENIZER_CACHE` is exactly `0`, raising the default byte budget from 50 MiB to 64 MiB ([#11078](https://github.com/ai-dynamo/dynamo/pull/11078)). Extended the cache to supported TikToken models by forwarding exact registered special-token boundaries ([#11499](https://github.com/ai-dynamo/dynamo/pull/11499)).
* **Chat-Template Thinking Defaults:** Added deployment-level thinking defaults for chat templates ([#12440](https://github.com/ai-dynamo/dynamo/pull/12440)), and honored `reasoning_effort` for templates that gate reasoning through `enable_thinking` by deriving `enable_thinking=false` for `reasoning_effort="none"` and `true` for other supplied values, with explicit `chat_template_args.enable_thinking` values taking precedence and requests without `reasoning_effort` left unchanged ([#11691](https://github.com/ai-dynamo/dynamo/pull/11691)).
* **Dynamo Protocols 3.1.0 Upgrade:** Updated Dynamo to dynamo-protocols 3.1.0 with compatible parser, renderer, and tokenizer versions, propagated tokenizer-cache initialization failures instead of hiding them, and adapted Anthropic, OpenAI Responses, and media preprocessing to protocol 3.x optional media URL fields ([#11978](https://github.com/ai-dynamo/dynamo/pull/11978)).

#### OpenAI API & Validation

* **Configurable Overload Rejection Status:** Added `DYN_HTTP_OVERLOAD_STATUS_CODE` so operators can choose the HTTP status returned for Frontend overload and admission-control rejections, keeping `529` as the default and accepting `503` ([#11249](https://github.com/ai-dynamo/dynamo/pull/11249)). Rejected informational status codes in the parser, falling back to `529` instead of surfacing HTTP 500 ([#12738](https://github.com/ai-dynamo/dynamo/pull/12738)).

#### Request Plane & Extensions

* **Python Frontend Route Extensions:** Added a `dynamo.llm.FrontendRoute` hook so external packages can register GET routes on the Frontend via `--frontend-route-extension` or `DYN_FRONTEND_ROUTE_EXTENSIONS`, without a custom binary. Routes run on a bounded pool and shed load with 503s, keeping handlers off the hot path ([#11626](https://github.com/ai-dynamo/dynamo/pull/11626)).

#### General

* **Squeeze-Evolve Multi-Model Service:** Added an experimental `dynamo.squeeze_evolve` component that serves verifier-free evolutionary test-time scaling as a chat model, routing candidate groups across model tiers via per-tier KvRouter instances. It is opt-in and not wired into any default frontend path ([#10785](https://github.com/ai-dynamo/dynamo/pull/10785)).

### Multimodal & Diffusion

* **vLLM-Omni Realtime Worker:** Added a realtime bidirectional vLLM-Omni worker exposing a WebSocket `/v1/realtime` endpoint that streams transcript and output audio deltas ([#10166](https://github.com/ai-dynamo/dynamo/pull/10166)), and fixed an `AttributeError` on newer vLLM-Omni versions when reading `multimodal_output` ([#12667](https://github.com/ai-dynamo/dynamo/pull/12667)).
* **Unified vLLM Multimodal Pipeline:** Added image, video, and audio handling to the aggregated unified vLLM backend ([#11267](https://github.com/ai-dynamo/dynamo/pull/11267)) with modality-aware multimodal routing metadata ([#10375](https://github.com/ai-dynamo/dynamo/pull/10375)). Enabled the vLLM multimodal CPU embedding cache ([#11268](https://github.com/ai-dynamo/dynamo/pull/11268)), added frontend media discovery plus SHM and NIXL transfer fallback ([#11269](https://github.com/ai-dynamo/dynamo/pull/11269)), and brought multimodal prefill and decode parity to the unified backend ([#11270](https://github.com/ai-dynamo/dynamo/pull/11270)).
* **Pluggable Custom Vision Encoder:** Added a `VisionEncoderBackend` ABC, the `build_mixed_embeds` assembler, and a `--custom-encoder-class` flag ([#10910](https://github.com/ai-dynamo/dynamo/pull/10910)), then wired the contract into serving so an aggregated `dynamo.vllm` worker runs a user-supplied encoder in-process without a separate encode worker ([#10832](https://github.com/ai-dynamo/dynamo/pull/10832)). Added adapters that build the final vLLM `EmbedsPrompt` or `TokensPrompt` directly ([#11847](https://github.com/ai-dynamo/dynamo/pull/11847)).
* **Cached Multimodal UUID Passthrough:** Added `UuidOnly` and `multi_modal_uuids` support, preserving user-provided image UUIDs and image-slot alignment through the vLLM worker ([#11943](https://github.com/ai-dynamo/dynamo/pull/11943)), rejected non-null multimodal UUIDs on the SGLang and TensorRT-LLM paths ([#11944](https://github.com/ai-dynamo/dynamo/pull/11944)), and parsed canonical and deprecated image UUID forms in the Rust and Python frontends ([#11958](https://github.com/ai-dynamo/dynamo/pull/11958)).
* **Media Decoder Installer And Diagnostics:** Added an operator-run installer for the media-decoder packages the runtime images omit, invoked as `python -m dynamo.common.utils.install_media_decoders <vllm|sglang|trtllm>` with `--packages`, `--pip-args`, `--timeout-s`, and `--dry-run` flags ([#12051](https://github.com/ai-dynamo/dynamo/pull/12051)). Reworked decode errors to name the missing package, installer command, and hardware alternative ([#12725](https://github.com/ai-dynamo/dynamo/pull/12725)).
* **NVDEC Hardware Video Decode:** H.264/H.265 video inputs now decode on the GPU via NVDEC through PyNvVideoCodec, keeping decode work off the SMs beyond YUV-to-RGB conversion. `DYN_DISABLE_NVDEC` opts out and `DYN_NVDEC_GPU_ID` selects the decode GPU ([#11836](https://github.com/ai-dynamo/dynamo/pull/11836)).
* **Encode-Worker Request Routing:** Added a backend-agnostic encoder routing hop that sends multimodal requests to a surface-less Encode worker before aggregated or prefill execution, with an optional round-robin `EncoderRouter` ([#11460](https://github.com/ai-dynamo/dynamo/pull/11460)). Forced the vLLM V1 model runner in the disaggregated EPD encode worker to work around a vLLM 0.25.1 startup crash ([#11913](https://github.com/ai-dynamo/dynamo/pull/11913)).
* **Video Embedding Cache Benchmark:** Added a video-understanding benchmark workload for SGLang multimodal embedding cache reuse, including `video-single-turn` and `video-sliding-window` JSONL generation and an E/PD sweep config comparing cache-off vs cache-on, letting users evaluate SGLang video cache performance ([#10000](https://github.com/ai-dynamo/dynamo/pull/10000)).
* **NIXL Connector for vLLM-Omni:** Added a NIXL RDMA-based tensor transport connector for vLLM-Omni, enabling autoregressive-to-diffusion (AR->DIT) pipelines such as GLM-Image to run disaggregated across multiple nodes and device types instead of requiring shared memory on a single machine ([#10071](https://github.com/ai-dynamo/dynamo/pull/10071)).
* **Cross-Request Vision Batching:** Added a ThreadedMicroBatcher and batcher-backed AsyncVisionEncoder that combine image forwards from concurrent requests into thread-affine, cost-bounded batches, improving GPU utilization for the custom encoder path. Canceled requests are tombstoned before dispatch so abandoned input cannot affect in-flight requests sharing a later batch ([#11037](https://github.com/ai-dynamo/dynamo/pull/11037)).
* **TensorRT-LLM Video URL Support:** Added `video_url` multimodal input handling to the TensorRT-LLM backend, decoding video URLs into `VideoData` instead of silently dropping them, mirroring the `image_url` path's validation and controlling frame count via `DYN_MM_VIDEO_NUM_FRAMES` ([#11896](https://github.com/ai-dynamo/dynamo/pull/11896)).

### Scheduling

#### Multi-DC Indexing & Relay

* **Radix Tree Memory Reduction:** Replaced each positional radix bucket's inner DashMap with FxHashMap, since the outer DashMap already served as the concurrency boundary and inner sharding added no benefit. This reduced peak Mocker RSS by 5.51 GiB (35.3%) in a 32-worker A/B test ([#12161](https://github.com/ai-dynamo/dynamo/pull/12161)).
* **CRTC Compressed Radix Tree Indexer:** Replaced each CRTC node's eager `DashMap` with adaptive child storage, improving full-trace throughput from 9.04M to 13.49M block ops/s ([#11459](https://github.com/ai-dynamo/dynamo/pull/11459)). Added a router-local Relay boundary for exact per-datacenter aggregation ([#11435](https://github.com/ai-dynamo/dynamo/pull/11435)), renamed `tenant_id` to `routing_group` across APIs ([#11471](https://github.com/ai-dynamo/dynamo/pull/11471)), closed stale-shape races ([#11169](https://github.com/ai-dynamo/dynamo/pull/11169)), and skipped shape locks for already-internal nodes ([#11102](https://github.com/ai-dynamo/dynamo/pull/11102)).
* **Arena-Backed Block Tracker Rework:** Replaced per-block `Arc` ownership in the KV Router block tracker with a `SlotMap`-arena block chain ([#11508](https://github.com/ai-dynamo/dynamo/pull/11508)), removed the PromptRegistry mutation lookup ([#11515](https://github.com/ai-dynamo/dynamo/pull/11515)), and moved to compressed radix-tree edges ([#11644](https://github.com/ai-dynamo/dynamo/pull/11644)). Streamlined `BlockTracker::release` ([#11568](https://github.com/ai-dynamo/dynamo/pull/11568)), fixed a leak in `ConcurrentRadixTreeCompressed` ([#11785](https://github.com/ai-dynamo/dynamo/pull/11785)), and preserved worker and DP-rank coverage on cold restores ([#11190](https://github.com/ai-dynamo/dynamo/pull/11190)).
* **Missing KV Event Publisher Diagnostics:** Added the optional `kv_event_publishing_enabled` runtime capability across backends with a warning-only source-health monitor and stable structured codes ([#12095](https://github.com/ai-dynamo/dynamo/pull/12095)), plus a Router diagnostic that logs an actionable error when expected worker or DP query endpoints never appear on the local-indexer recovery path ([#11194](https://github.com/ai-dynamo/dynamo/pull/11194)).
* **Endpoint-Scoped Relay Event Transport:** Added endpoint-scoped `EventPublisher`/`EventSubscriber` constructors that thread namespace/component/endpoint identity through discovery and transport ([#11841](https://github.com/ai-dynamo/dynamo/pull/11841)), added a domain-scoped KV Relay with a lane-sticky global indexer for multi-DC prefix routing ([#11793](https://github.com/ai-dynamo/dynamo/pull/11793)), and fixed a startup TypeError in the relay's health endpoint ([#12948](https://github.com/ai-dynamo/dynamo/pull/12948)).
* **Planner Predictor Data Inputs:** Added a KV-cache-read dimension to the vLLM prefill self-benchmark grid, gated behind `--benchmark-prefill-kv-read-granularity` and `DYN_BENCHMARK_PREFILL_KV_READ_GRANULARITY` ([#11150](https://github.com/ai-dynamo/dynamo/pull/11150)). Added native `dynamo.request.trace.v1` support to Planner load-predictor warmup with schema auto-detection and rejection of malformed traces ([#11842](https://github.com/ai-dynamo/dynamo/pull/11842)).
* **Pool-Scoped KV DC Relay:** Added a pool-scoped KV DC Relay runtime mapping each serving endpoint to one Dynamo PoolId, with a PoolRegistry owning per-pool Relay actors and CKF state. This keeps independent endpoints serving the same model isolated and recovering from stream failures ([#12199](https://github.com/ai-dynamo/dynamo/pull/12199)).

#### Selection & Reservation Replay

* **Active Sequence Replica Event Pipeline:** Batched frontend active-sequence replica-sync lifecycle events on ZMQ with the shared 256-event and 1 ms policy ([#11920](https://github.com/ai-dynamo/dynamo/pull/11920)), added a lifecycle-owned direct-ZMQ fan-in preserving per-source FIFO ([#11967](https://github.com/ai-dynamo/dynamo/pull/11967)), and replaced the async publication API with a bounded synchronous enqueue that drops the newest event on overflow ([#11992](https://github.com/ai-dynamo/dynamo/pull/11992)).
* **Standalone EPP Selector Mode:** Added a standalone EPP mode selected by `DYN_EPP_MODE` with `EppStandaloneConfig` validation ([#11070](https://github.com/ai-dynamo/dynamo/pull/11070)), embedded the `dynamo-kv-router` `SelectionService` with optional multi-EPP replication via `DYN_EPP_PEER_SERVICE` ([#11541](https://github.com/ai-dynamo/dynamo/pull/11541)), wired an in-process `EppRouter` without DistributedRuntime, etcd, or NATS ([#11542](https://github.com/ai-dynamo/dynamo/pull/11542)), and preserved token-id completion prompts in the EPP scorer request ([#10872](https://github.com/ai-dynamo/dynamo/pull/10872)).
* **Selection Booking Replay By Id:** Added a `SelectionCache` keyed by `(model, routing_group, selection_id)` so a follow-up `POST /reservations` replays the cached `POST /select` booking instead of re-sending and re-hashing a long prompt ([#11416](https://github.com/ai-dynamo/dynamo/pull/11416)), with `select_and_reserve` returning the normalized booking fields so peer ingress replicas can replicate it ([#12335](https://github.com/ai-dynamo/dynamo/pull/12335)).
* **Session Affinity Across Replicas:** Propagated `agent_context.session_id` through live router selection and Mocker replay into `SchedulingRequest` ([#11089](https://github.com/ai-dynamo/dynamo/pull/11089)), and synchronized session-affinity bindings automatically whenever the affinity TTL is configured, with first-live-binding-wins semantics ([#11750](https://github.com/ai-dynamo/dynamo/pull/11750)). Moved replica tasks under the affinity coordinator and upgraded the end-to-end test to an adversarial two-frontend scenario ([#11750](https://github.com/ai-dynamo/dynamo/pull/11750)).
* **Decode Affinity Scoring Path:** Added decode affinity scoring to the Router's worker selection ([#11720](https://github.com/ai-dynamo/dynamo/pull/11720)), with an experimental block-equivalent decode cost term plumbed through config, CLI, and bindings ([#12158](https://github.com/ai-dynamo/dynamo/pull/12158)). Stamped `request_id` and `worker_type` on cost-breakdown log rows ([#12370](https://github.com/ai-dynamo/dynamo/pull/12370)) and extended the same stamping to the decode-affinity branch ([#12955](https://github.com/ai-dynamo/dynamo/pull/12955)).
* **Provider-Keyed Tracking Hashes:** Added an opt-in `keyed-xxh3-v1` identity mode for router tracking, backed by a zeroized 32-byte provider key and managed key ID, preserving hashes and wire formats. Standalone selection honors `router_assume_kv_reuse=false` with random tracking identities; rotation requires coordinated key changes and restarts ([#11548](https://github.com/ai-dynamo/dynamo/pull/11548)).
* **ISL-Based Aggregate Routing:** Added an optional input sequence length (ISL) dimension to aggregate GlobalRouter pool selection, allowing requests to route across pools based on token count in addition to existing TTFT x ITL configurations. Existing 2D TTFT x ITL routing remains unchanged when ISL settings are not configured ([#11835](https://github.com/ai-dynamo/dynamo/pull/11835)).

#### KV Events & Source Health

* **Concurrent KV Event Ingestion:** Delivered direct-ZMQ KV events through one FIFO task per publisher, bypassing the lossy broadcast hop ([#11938](https://github.com/ai-dynamo/dynamo/pull/11938)). Preserved native SGLang and vLLM event-list boundaries ([#11466](https://github.com/ai-dynamo/dynamo/pull/11466)), published ordered RouterEvent vectors in partitioned batches ([#11776](https://github.com/ai-dynamo/dynamo/pull/11776)), batched deferred Mocker visibility boundaries ([#11860](https://github.com/ai-dynamo/dynamo/pull/11860)), and restored KV-aware routing for multinode SGLang workers on Kubernetes ([#12185](https://github.com/ai-dynamo/dynamo/pull/12185)).
* **Cache-Salt Multi-Tenant KV Isolation:** Added cache-salt-aware KV routing so requests carrying a tenant namespace (via `x-tenant-id`, `nvext.cache_salt`, or a compatibility `cache_salt` field) hash and route independently across the Router, vLLM, and TensorRT-LLM KV events, preventing cross-tenant cache reuse while preserving existing unsalted behavior ([#8197](https://github.com/ai-dynamo/dynamo/pull/8197)).
* **Forward-Pass Metrics Tracing:** Added opt-in, best-effort persistence of forward-pass metrics to bounded rotating gzip JSONL traces for vLLM, SGLang, TensorRT-LLM, and Mocker publishers, exposed via `--fpm-trace`/`--no-fpm-trace` and `DYN_FPM_TRACE`, giving operators durable forward-pass records without adding disk I/O to the inference path ([#11110](https://github.com/ai-dynamo/dynamo/pull/11110)).

#### Routing Policies

* **Router Queue Admission Strategy:** Added exact-worker lanes to the policy-class queue ([#11363](https://github.com/ai-dynamo/dynamo/pull/11363)) with deferred blocked-lane rechecks ([#11484](https://github.com/ai-dynamo/dynamo/pull/11484)), a public `PolicyClassAdmissionStrategy` contract ([#11434](https://github.com/ai-dynamo/dynamo/pull/11434)), and a live `RequestProgress` reader ([#11615](https://github.com/ai-dynamo/dynamo/pull/11615)). Kept Router queueing off unless a threshold is configured ([#11599](https://github.com/ai-dynamo/dynamo/pull/11599)) and replaced `--admission-control` with opt-in thresholds, ignoring the legacy flag and `DYN_ADMISSION_CONTROL` with a warning ([#11276](https://github.com/ai-dynamo/dynamo/pull/11276)).
* **Conditional Disaggregation Routing Policy:** Added a conditional disaggregation policy kernel to the Router ([#11718](https://github.com/ai-dynamo/dynamo/pull/11718)) with backend handling for the conditional disagg bypass annotation ([#11719](https://github.com/ai-dynamo/dynamo/pull/11719)). Deprecated `--enforce-disagg` and `DYN_ENFORCE_DISAGG` as accepted no-ops ([#11160](https://github.com/ai-dynamo/dynamo/pull/11160)), emitting the deprecation warning immediately for both forms ([#11800](https://github.com/ai-dynamo/dynamo/pull/11800)).
* **Configurable Active Request Expiry:** Added the `DYN_ROUTER_ACTIVE_REQUEST_EXPIRY_SECS` environment variable to configure the KV-router's stale active-request cleanup guard, preventing long-context agentic requests from having their live record removed before completion when they exceed the previous hard-coded 300-second limit ([#11303](https://github.com/ai-dynamo/dynamo/pull/11303)).
* **Native KV Offloading Capacity:** Added a backend-neutral `native_offloading_capacity.total_tokens` metadata contract for Rust and Python routing consumers, accepting SGLang capacity only from the scheduler's realized host pool and translating it by write policy to avoid double counting GPU capacity in router budgets ([#11321](https://github.com/ai-dynamo/dynamo/pull/11321)).
* **Amplified Overlap Score Credit:** Allowed the KV Router's `--router-kv-overlap-score-credit` value to exceed 1.0 and removed the zero floor on adjusted prefill blocks, so amplified overlap credit can offset decode load and improve cache-hit routing without adding new configuration surface ([#11714](https://github.com/ai-dynamo/dynamo/pull/11714)).
* **Native Multimodal KV Routing:** Moved TensorRT-LLM aggregated multimodal KV routing onto Dynamo's Rust Frontend and KV Router, removing the separate Python MM-router sidecar and updating worker integration for TensorRT-LLM v1.3.0rc22. This simplifies deployment to a single native request path while preserving KV cache reuse across repeated image requests ([#11737](https://github.com/ai-dynamo/dynamo/pull/11737)).

#### General

* **HiCache Host Capacity Budget:** Updated ThunderAgent to derive its retention budget from GPU KV plus native HiCache host capacity instead of device KV alone, aligning pause/resume behavior with the memory SGLang can actually retain ([#11185](https://github.com/ai-dynamo/dynamo/pull/11185)).

### Performance Modeling & Replay

#### Spica Simulation

* **Experimental Spica Simulator Integration:** Moved the experimental Spica simulator from aiconfigurator into Dynamo as an independent `aisimulate` distribution with import `aisimulate.spica` and CLI `python -m aisimulate.spica` ([#11923](https://github.com/ai-dynamo/dynamo/pull/11923)), and stopped emitting synthetic arrival intervals for closed-loop workloads so Replay receives only `replay_concurrency` ([#12824](https://github.com/ai-dynamo/dynamo/pull/12824)).
* **SATF Request-Trace Conversion:** Added a Rust API and CLI (`request_trace_to_satf`) that converts Dynamo request-trace JSONL and gzip JSONL shards into SATF 2.0 performance-layer artifacts, preserving token counts, timing, KV block identity, request metrics, and tool-call structure for downstream simulation and analysis without altering the existing capture format ([#11375](https://github.com/ai-dynamo/dynamo/pull/11375)).

#### Mocker

* **Native G1 Block Manager:** Added a selectable native G1 KV manager for the shared vLLM and TensorRT-LLM mock schedulers ([#12033](https://github.com/ai-dynamo/dynamo/pull/12033)), defaulted both backends to native G1 with fallback to KVBM when G2/G3/G4 offload is required ([#12222](https://github.com/ai-dynamo/dynamo/pull/12222)), flattened native G1 request token representation ([#12248](https://github.com/ai-dynamo/dynamo/pull/12248)), and fixed a build failure via `resolved_g1_backend()` ([#12336](https://github.com/ai-dynamo/dynamo/pull/12336)).
* **SGLang Mocker Scheduling Fidelity:** Streamlined SGLang radix scheduling in the Mocker by keeping each request in one append-only `u32` token sequence so decode caching borrows the retained prefix ([#12043](https://github.com/ai-dynamo/dynamo/pull/12043)), and modeled SGLang KV allocation by page instead of individual token-slot IDs ([#12108](https://github.com/ai-dynamo/dynamo/pull/12108)).
* **Mocker KV Bookkeeping Overhead:** Reduced Mocker KV-management overhead in high-concurrency disaggregated workloads by batching terminal dereference signals and using canonical immutable block handles ([#11095](https://github.com/ai-dynamo/dynamo/pull/11095)). Replaced the native `VllmBlockPool` inactive-copy `BTreeMap` with an intrusive linked LRU for constant-time operations, with a balanced AgentX A/B measuring a 22.2% median output-token throughput improvement ([#12244](https://github.com/ai-dynamo/dynamo/pull/12244)).
* **Mocker Max Model Length:** Added explicit `max_model_len` support to the vLLM Mocker and replay so they match the frontend, Router, and real worker sequence-length behavior. Requests exceeding the context limit are rejected up front, while mid-generation hits complete with `finish_reason="length"` ([#11069](https://github.com/ai-dynamo/dynamo/pull/11069)).

#### Trace Replay & Benchmarks

* **Poisson Synthetic Replay Arrivals:** Added Poisson synthetic replay driven by request rate, alongside fixed-interval and concurrency modes; an independent arrival seed means switching modes changes only timestamps. Replay now requires one of request rate, interval, or concurrency, removing the implicit 1 ms fallback ([#12062](https://github.com/ai-dynamo/dynamo/pull/12062)).

### Kubernetes Deployment

#### Inference Gateway (EPP)

* **Standalone EPP Worker Discovery:** Added discovery of standalone EPP workers from the gateway-targeted GAIE `InferencePool` via a kube-rs reflector index of Ready, selector-matching workers ([#11072](https://github.com/ai-dynamo/dynamo/pull/11072)), with a topology adapter that converts each ready worker into a `WorkerRegistration` and reconciles the worker set with the in-process Selector ([#11074](https://github.com/ai-dynamo/dynamo/pull/11074)).

#### Operator & CRDs

* **Grove Update Strategy Controls:** Added `spec.grove.updateStrategy.type` to DynamoGraphDeployment with `RollingRecreate` and `OnDelete` values, surfacing OnDelete rollout progress through a DGD condition and events ([#11085](https://github.com/ai-dynamo/dynamo/pull/11085)), and added component-scoped `Recreate` support via the `nvidia.com/deployment-strategy` pod template annotation ([#11909](https://github.com/ai-dynamo/dynamo/pull/11909)).
* **CRD apiVersion Converter Utility:** Added `deploy/utils/convert_api_version.py`, a CLI that converts `nvidia.com/v1alpha1` manifests to `nvidia.com/v1beta1` by POSTing a ConversionReview to the Operator's existing conversion webhook ([#10657](https://github.com/ai-dynamo/dynamo/pull/10657)). Fixed the power-agent Helm chart, where a whitespace trim broke the default `helm install` ([#12651](https://github.com/ai-dynamo/dynamo/pull/12651)).
* **Grove Volcano Scheduler Support:** Added Operator-controlled Volcano scheduler integration for Grove-backed DynamoGraphDeployment workflows, setting `schedulerName: volcano` on Grove clique pods and mapping the `nvidia.com/volcano-queue` annotation to Grove's PodCliqueSet metadata, with startup validation preventing simultaneous KAI-Scheduler and Volcano configuration ([#8225](https://github.com/ai-dynamo/dynamo/pull/8225)).
* **DynamoGraphDeployment Placement Score Status:** Added `status.placementScore` and `status.placementScoreState` fields to the DynamoGraphDeployment API, mirrored across v1alpha1 and v1beta1 with conversion wiring, and reported non-failing placement states during reconcile on the Grove pathway. Full score reporting awaits Grove exposing a consumable score field. ([#11057](https://github.com/ai-dynamo/dynamo/pull/11057)).
* **Fine-Grained DGD Readiness Reasons:** Added classification of the DynamoGraphDeployment `Ready` condition into specific reasons (insufficient\_capacity, pods\_not\_ready, updating, mixed\_not\_ready\_reasons) derived from Grove status fields, plus an optional `scheduledReplicas` field on component status, so operators can diagnose why a Grove-backed deployment is stuck without inspecting Grove resources directly ([#11131](https://github.com/ai-dynamo/dynamo/pull/11131)).
* **Operator RuntimeVersionOverride Backport:** Added `runtimeVersionOverride` support for DGD (DynamoGraphDeployment) components and DGDR-generated deployments, backported to release/1.4.0, letting users pin a specific runtime version for these components ([#12427](https://github.com/ai-dynamo/dynamo/pull/12427)).

#### GPU Memory Service & Snapshot

* **GMS Shadow Failover Hardening:** Added `dynamo_component_engine_failover_*` counters and state gauges for intra-pod shadow-engine failover with a Grafana dashboard ([#11664](https://github.com/ai-dynamo/dynamo/pull/11664)), aliased all scratch KV allocations to a single granule of device memory ([#11911](https://github.com/ai-dynamo/dynamo/pull/11911)), and surfaced scratch-KV engagement in shadow worker logs ([#11724](https://github.com/ai-dynamo/dynamo/pull/11724)).
* **VMM Device Abstraction Layer:** Introduced a device-agnostic Virtual Memory Management (VMM) abstraction in the GPU Memory Service with a `--device-kind` CLI option and VMMDevice protocol, preserving existing CUDA behavior while laying groundwork for XPU support, which raises NotImplementedError until Phase 2 ([#9788](https://github.com/ai-dynamo/dynamo/pull/9788)).

#### General

* **XPU DRA Deployment Examples:** Added XPU Dynamic Resource Allocation deployment YAMLs for aggregated and disaggregated vLLM serving, including tracing and Planner variants, giving users ready-made configs for aggregated serving, OpenTelemetry tracing, NIXL KV transfer, and Planner-based disaggregation on XPU. ([#9253](https://github.com/ai-dynamo/dynamo/pull/9253)).

### Fault Tolerance & Observability

* **OTLP Log Export Sink:** Added an opt-in `otel` audit sink that exports chat-completion audit records as OTLP `LogRecord`s, enabled via `DYN_AUDIT_SINKS` ([#9390](https://github.com/ai-dynamo/dynamo/pull/9390)), and decoupled `OTEL_EXPORT_ENABLED` from `DYN_LOGGING_JSONL` so OTLP export initializes regardless of the local log format ([#10817](https://github.com/ai-dynamo/dynamo/pull/10817)).
* **Unified Request-Trace Payload Capture:** Folded audit-style payload logging into the request trace pipeline, emitting a single `dynamo.request.trace.v1` stream with payload rows selected via `DYN_REQUEST_TRACE_RECORDS` ([#11180](https://github.com/ai-dynamo/dynamo/pull/11180)), and added allowlisted HTTP request-header capture through `DYN_REQUEST_TRACE_HTTP_HEADER_CAPTURE_LIST` ([#11386](https://github.com/ai-dynamo/dynamo/pull/11386)).
* **Frontend Multimodal Input Metrics:** Added per-request counts of `image_url`, `video_url`, and `audio_url` content parts as frontend Prometheus histograms and request-completed log fields ([#11166](https://github.com/ai-dynamo/dynamo/pull/11166)), followed by the `dynamo_frontend_image_tokens_per_request` histogram of calculated image-placeholder token counts ([#12065](https://github.com/ai-dynamo/dynamo/pull/12065)).
* **Local Resource Monitor Caching:** Cached process selection and aggregation in the local resource monitor so dashboard deltas no longer rescan every retained process series, and disconnected hidden dashboard tabs from Socket.IO until shown again, reducing background traffic and CPU use ([#11336](https://github.com/ai-dynamo/dynamo/pull/11336)).
* **Tokenizer Cache Token Metrics:** Added per-model `dynamo_frontend_tokenizer_cache_cached_tokens_total` and `dynamo_frontend_tokenizer_cache_uncached_tokens_total` counters alongside the existing aggregate hit and miss metrics, letting operators measure token-level cache reuse per served model instead of inferring it from aggregate counts ([#11353](https://github.com/ai-dynamo/dynamo/pull/11353)).
* **Buffered ITL Metrics Collection:** Reduced inter-token latency metrics overhead by buffering observations in a request-local Prometheus histogram, flushing every 64 samples instead of synchronizing on every output token, cutting the ITL histogram on-CPU cost by 71.9% while preserving the existing Prometheus schema ([#11569](https://github.com/ai-dynamo/dynamo/pull/11569)).
* **Dynamic Log Filter Warning:** Added a stderr warning emitted when a dynamic span or field selector forces logging to fall back from the lock-free Targets filter to EnvFilter, explaining the streaming-performance risk and how to restore the fast path ([#11974](https://github.com/ai-dynamo/dynamo/pull/11974)).
* **Configurable Worker Inhibition Duration:** Added `DYN_RUNTIME_INHIBITED_DURATION_SECS` to configure the local worker inhibition window, keeping the five-second default when unset and allowing `0` to disable local inhibition([#11993](https://github.com/ai-dynamo/dynamo/pull/11993)).

### Engines

#### vLLM

* **Graph-Aware Self-Benchmark Sampling:** Replaced linear per-request sweeps in the vLLM self-benchmark with deterministic prefill and decode sampling derived from the resolved PIECEWISE/FULL CUDA Graph configuration ([#11509](https://github.com/ai-dynamo/dynamo/pull/11509)). Added an option to override the benchmark grid ([#11713](https://github.com/ai-dynamo/dynamo/pull/11713)) and filtered infeasible EAGLE prefill KV-read coordinates ([#12836](https://github.com/ai-dynamo/dynamo/pull/12836)).
* **Unified Backend Elastic EP:** Added headless multi-node worker support and live elastic expert-parallel (EP) scaling via `scale_elastic_ep` to the vLLM unified backend, bringing it to parity with the legacy path and letting deployments scale data-parallel size up or down without restarting service ([#10834](https://github.com/ai-dynamo/dynamo/pull/10834)).
* **vLLM Multiple Served Names:** Enabled the vLLM backend to accept multiple `--served-model-name` values, registering the first as primary and the rest as aliases, matching SGLang. A single worker now serves several model names, with `/v1/models` and chat completions resolving aliases to the primary ([#11611](https://github.com/ai-dynamo/dynamo/pull/11611)).
* **vLLM Render Client Support:** Added a pooled HTTP client in the standalone External Processor (EPP) that calls vLLM's `/v1/chat/completions/render` endpoint for tokenization, configurable via `DYN_EPP_TOKENIZER_SERVICE_URL` and `DYN_EPP_TOKENIZATION_TIMEOUT_MS`, replacing an earlier offline tokenizer approach and classifying failures as typed Unavailable, Timeout, UpstreamStatus, or InvalidResponse errors ([#11827](https://github.com/ai-dynamo/dynamo/pull/11827)).

#### SGLang

* **Multiple Served Model Names:** Added support for registering an SGLang worker under multiple served model names via `--served-model-name`, canonicalizing alias requests to the primary name so engine routing, Prometheus metrics, and the OpenAI `response.model` field stay consistent ([#11141](https://github.com/ai-dynamo/dynamo/pull/11141)).

#### TensorRT-LLM

* **Engine-Owned Conversation-Aware ADP Routing:** Added opt-in TensorRT-LLM engine-owned conversation-affinity attention-DP routing, deriving the conversation id from `agent_context.session_id` and passing it as `ConversationParams` ([#11609](https://github.com/ai-dynamo/dynamo/pull/11609)), with a `--conversation-affinity` flag and `DYN_ENGINE_CONV_AFFINITY` environment variable forcing engine-side assignment ([#11705](https://github.com/ai-dynamo/dynamo/pull/11705)).
* **Prefill Handoff And Event Batching:** Added generic Rust and Python `KvEventPublisher.publish_batch` APIs so one ordered source batch reaches the KV event processor as a single input ([#11783](https://github.com/ai-dynamo/dynamo/pull/11783)), and skipped duplicate prompt metadata in the legacy TensorRT-LLM prefill handoff for text-only requests ([#11202](https://github.com/ai-dynamo/dynamo/pull/11202)).

### Infrastructure Modernization

* **Dependency and Service Refresh:** Aligned aiohttp to 3.14.3 or later across all container images, including the TensorRT-LLM runtime's system-site install ([#13132](https://github.com/ai-dynamo/dynamo/pull/13132), [#13195](https://github.com/ai-dynamo/dynamo/pull/13195), [#13227](https://github.com/ai-dynamo/dynamo/pull/13227), [#13256](https://github.com/ai-dynamo/dynamo/pull/13256)), bumped etcd to v3.5.33 and NATS Server to v2.12.14 ([#13016](https://github.com/ai-dynamo/dynamo/pull/13016), [#13225](https://github.com/ai-dynamo/dynamo/pull/13225)), moved vLLM images to NIXL v1.3.2 ([#12882](https://github.com/ai-dynamo/dynamo/pull/12882), [#13185](https://github.com/ai-dynamo/dynamo/pull/13185)), and adopted the published AIConfigurator v0.11.0 release ([#13280](https://github.com/ai-dynamo/dynamo/pull/13280)).
* **TCP Request-Plane Fast Path:** Removed two large request-payload copies from the TCP request-plane frontend send path for payloads of at least 4096 bytes via vectored socket writes ([#10519](https://github.com/ai-dynamo/dynamo/pull/10519)), and replaced the Tokio mutex guarding the TCP stream registry with `parking_lot::Mutex` ([#11065](https://github.com/ai-dynamo/dynamo/pull/11065)).
* **Python Request-Plane Codec Transcoding:** Added a statically dispatched ingress payload adapter that transcodes request-plane wire bytes directly to and from Python objects, making MessagePack the default with JSON as an explicit fallback ([#11104](https://github.com/ai-dynamo/dynamo/pull/11104)), interned the six annotated-response envelope keys ([#11331](https://github.com/ai-dynamo/dynamo/pull/11331)), and added per-worker codec negotiation via discovery metadata ([#12658](https://github.com/ai-dynamo/dynamo/pull/12658)).
* **Client TCP Address Lookup:** Added `Client.instance_tcp_addresses()` to the Python bindings, returning a dict mapping instance id to TCP transport address for an endpoint's registered instances, letting callers reach peer worker nodes without reading the discovery/etcd registry layout directly ([#11540](https://github.com/ai-dynamo/dynamo/pull/11540)).

### Hardware

* **XPU Deployment Sanity Checks:** Extended `dev/sanity_check.py` to detect and report Intel XPU systems alongside NVIDIA systems, and added `xpu-smi` to XPU Docker images, giving users GPU, framework, and filesystem diagnostics for Intel XPU deployments ([#7702](https://github.com/ai-dynamo/dynamo/pull/7702)).
* **vLLM XPU Support Enablement:** Resolved the accelerator device from vLLM's current platform in the multimodal embedding cache connector so CPU-cached embedding loads work on both CUDA and XPU ([#9938](https://github.com/ai-dynamo/dynamo/pull/9938)), and updated the XPU aggregated examples for vLLM XPU 0.24.0 by removing the `--block-size` override ([#11229](https://github.com/ai-dynamo/dynamo/pull/11229)).

### KV Block Manager

* **Unified KV Block Clearing:** Added a `POST /engine/control/clear_kv_blocks` endpoint for unified vLLM workers that resets the prefix cache and connector state without preempting active requests. Supported aggregated, prefill, and decode workers, and propagated cached-token usage so clients can verify cache hits and post-clear misses ([#11451](https://github.com/ai-dynamo/dynamo/pull/11451)).

### General

* **vLLM v0.26.0 Upgrade:** Upgraded the pinned vLLM to v0.26.0 with refreshed CUDA and CPU runtime images, FlashInfer 0.6.14, and NIXL v1.3.1, later moved to v1.3.2 ([#12202](https://github.com/ai-dynamo/dynamo/pull/12202), [#11606](https://github.com/ai-dynamo/dynamo/pull/11606)), and moved the XPU container to the `vllm/vllm-openai-xpu` v0.26.0 image ([#12190](https://github.com/ai-dynamo/dynamo/pull/12190)). The upgrade brings reasoning-parser configuration and structured-output handling to the Rust vLLM backend.
* **TensorRT-LLM v1.3.0rc22 Upgrade:** Upgraded TensorRT-LLM to v1.3.0rc22 across supported runtime configurations and platforms with updated baseline SBOM records ([#11086](https://github.com/ai-dynamo/dynamo/pull/11086), [#11763](https://github.com/ai-dynamo/dynamo/pull/11763), [#12071](https://github.com/ai-dynamo/dynamo/pull/12071)), including improved multimodal request handling and image-token range detection for TensorRT-LLM KV event processing.
* **SGLang v0.5.16 Upgrade:** Upgraded the SGLang backend and CUDA 13 runtime to v0.5.16 on the `lmsysorg/sglang:v0.5.16-cu130-runtime` image with NIXL v1.3.0 pinned for disaggregated serving ([#11596](https://github.com/ai-dynamo/dynamo/pull/11596), [#12081](https://github.com/ai-dynamo/dynamo/pull/12081)), setting `incremental_streaming_output` before SGLang resolves its immutable startup record and removing the expired 0.5.14 profiling compatibility path.
* **dynamo-parsers 7.0.1 Upgrade:** Upgraded `dynamo-parsers` to 7.0.1 in the workspace and Python bindings ([#12147](https://github.com/ai-dynamo/dynamo/pull/12147), [#12451](https://github.com/ai-dynamo/dynamo/pull/12451), [#12509](https://github.com/ai-dynamo/dynamo/pull/12509)), aligning the frontend crate graph on `dynamo-protocols 4.0.0` and `dynamo-renderer 3.0.0` and picking up the DeepSeek fix that stops tool-call markup from leaking into reasoning content.
* **Inline Compliance Artifact Coverage:** Extended the inline license/SBOM compliance pipeline to the Planner and Operator images ([#11328](https://github.com/ai-dynamo/dynamo/pull/11328)) and the frontend and snapshot images ([#11533](https://github.com/ai-dynamo/dynamo/pull/11533)). Added per-build OSRB CSV diffs against context-dependent baselines ([#11240](https://github.com/ai-dynamo/dynamo/pull/11240)) and passed the IRSA sccache secrets to compliance-extract rebuilds ([#11181](https://github.com/ai-dynamo/dynamo/pull/11181)).
* **Power Agent DCGM Actuator:** Added an opt-in DCGM actuator for the Power Agent, letting operators set `agent.actuator=dcgm` to write power caps through the cluster `nvidia-dcgm` hostengine, with NVML remaining the default and cap writes anchored to a transaction-verified GPU UUID ([#9790](https://github.com/ai-dynamo/dynamo/pull/9790)).
* **Self-Hosted MDC Default:** Flipped `DYN_SELF_HOST_METADATA` to on by default so workers advertise their model card (MDC) files over the `system_status_server` instead of requiring shared storage, and added a startup hard-fail when self-hosting is enabled but `DYN_SYSTEM_PORT` is unset instead of silently degrading `hf://` MDC advertisement ([#11417](https://github.com/ai-dynamo/dynamo/pull/11417)).
* **Hugging Face Hub LoRA Source:** Added native `hf://<repo>[@revision]` support for dynamic LoRA loading, downloading adapters into the standard Hugging Face Hub cache with revision pinning, timeout limits, and integrity checks instead of requiring a separate copy under `DYN_LORA_PATH` ([#11431](https://github.com/ai-dynamo/dynamo/pull/11431)).
* **Instance Transport Object Model:** Added `Client.instances()` to the Python bindings, returning read-only `Instance` snapshot objects with `instance_id`, `namespace`, `component`, `endpoint`, `device_type`, and `transport` attributes, preserving all transport types including NATS instead of flattening addresses to opaque TCP-only strings ([#11617](https://github.com/ai-dynamo/dynamo/pull/11617)).
* **Static Logging Filter Fast Path:** Switched to a metadata-only `Targets` filter for static `DYN_LOG` and configured log filter directives, falling back to `EnvFilter` only for dynamic span or field directives, removing a shared span-matcher lock from the request-span lifecycle and reducing frontend CPU overhead on streaming responses ([#11820](https://github.com/ai-dynamo/dynamo/pull/11820)).
* **EFA Installer Version Bump:** Updated the AWS EFA Installer to version 1.49.0 and reverted the prior libfabric override so the AWS container now uses the installer's provided libfabric, improving compatibility with EFA networking ([#12010](https://github.com/ai-dynamo/dynamo/pull/12010)).

## Recipes

* **Model-Agnostic Accuracy Check:** Added `recipes/accuracy/`, an AIPerf-based Kubernetes Job that runs a public benchmark against any deployed Dynamo recipe and grades the answers, with a methodology README ([#12495](https://github.com/ai-dynamo/dynamo/pull/12495), [#13215](https://github.com/ai-dynamo/dynamo/pull/13215)).
* **GPT-OSS-120B vLLM Deployment Recipes:** Added GPT-OSS-120B recipes for aggregated and disaggregated vLLM serving on B200 and H200 GPUs, covering speculative decoding, KV-cache routing and offload, reasoning, tool calling, automated model downloads, and an AIPerf trace-replay benchmarking workflow with documented prerequisites, quick start, performance results, and known limitations ([#11506](https://github.com/ai-dynamo/dynamo/pull/11506)). Fixed the disaggregated deployments so they no longer force etcd discovery at a hardcoded endpoint, letting the cluster discovery backend injected by the Operator apply, while keeping the NATS message bus setting ([#12070](https://github.com/ai-dynamo/dynamo/pull/12070)).
* **Hugging Face Hub Pin Bump:** Upgraded the `huggingface_hub` CLI pin from 1.11.0 to 1.16.4 in model-cache and vLLM LoRA helper job YAML files, fixing a missing `click` dependency that broke the `hf download` command ([#10986](https://github.com/ai-dynamo/dynamo/pull/10986)).
* **ARM Recipe Build Tools:** Added build-essential installation to ARM/GB200-facing benchmark recipes so aiperf>=0.8.0 can build its crick dependency from source, since crick lacks a Linux aarch64 wheel ([#11278](https://github.com/ai-dynamo/dynamo/pull/11278)).
* **DSV4 Agentic vLLM Recipes:** Added DeepSeek-V4-Flash and DeepSeek-V4-Pro deployment recipes combining Dynamo with vLLM, covering aggregated and P/D-disaggregated profiles on B200 and H200 for agentic workloads, plus model-cache manifests and an AIPerf Moontrace perf job. Disaggregated variants move KV over NIXL GPU-Direct RDMA ([#11429](https://github.com/ai-dynamo/dynamo/pull/11429)).
* **TokenSpeed FlashInfer Cache Fix:** Pinned the TokenSpeed recipe's base image by immutable digest, fixed the source revision, and aligned the FlashInfer JIT cache version required before building kernels, preventing version mismatches in kimi-k2.5 builds ([#11465](https://github.com/ai-dynamo/dynamo/pull/11465)).
* **Inkling NVFP4 B200 Recipe:** Added a recipe for thinkingmachines/Inkling-NVFP4 with Dynamo plus SGLang aggregated serving on one 8x B200 node, including NVFP4 quantization, FA4 attention, EAGLE speculative decoding, and a model-cache download job. It documents a required `DYN_FORWARDPASS_METRIC_PORT` override avoiding a scheduler crash ([#11743](https://github.com/ai-dynamo/dynamo/pull/11743)).
* **GLM-5.2 SGLang Recipes:** Added GLM-5.2 SGLang aggregated and disaggregated serving recipes for B200 and H200 clusters, including model-cache setup, EAGLE speculative decoding, HiCache CPU offload, NVFP4/FP8 quantization, and an AIPerf trace-replay benchmark workflow, along with fixes for reasoning-parser configuration, a non-stream logprobs 500 error, and trace-staging symlink handling. ([#11926](https://github.com/ai-dynamo/dynamo/pull/11926)).
* **Kimi-K3 Recipe Publication:** Brought the Kimi-K3 recipe assets (README, model-cache config, and vLLM deploy manifests for GB200/GB300 aggregated and disaggregated setups) onto main and added its row to the Available Recipes table, so users on main can now discover and deploy Kimi-K3 without switching to the release branch ([#12228](https://github.com/ai-dynamo/dynamo/pull/12228)).

## Bug Fixes

### Frontend

* **LoRA Unload Resource Release:** Fixed the Frontend to release the worker connection and associated memory after each LoRA unload, eliminating unbounded RSS growth of roughly 95 MB per load/unload cycle ([#12743](https://github.com/ai-dynamo/dynamo/pull/12743), [#13203](https://github.com/ai-dynamo/dynamo/pull/13203)).
* **FinishReason Error Propagation:** Fixed the Frontend to read the string form of `FinishReason` that workers emit, so request-level error messages reach the caller instead of a bare HTTP 500 ([#13118](https://github.com/ai-dynamo/dynamo/pull/13118), [#13212](https://github.com/ai-dynamo/dynamo/pull/13212)).
* **MDC Version Compatibility Fix:** Preserved compatibility for Model Discovery Card (MDC) versions across rolling upgrades by accepting legacy v1.2 `context_length` fallbacks and omitting default-valued post-v1.3 router fields so v1.3 frontends can deserialize v1.4 MDCs, while keeping removed v1.3 router behavior like `durable_kv_events=true` failing closed ([#12801](https://github.com/ai-dynamo/dynamo/pull/12801)).
* **Hidden Disabled Batch Routes:** Fixed the OpenAPI specification to omit Batch and Files API routes when their internal enable flag is off, so disabled surfaces no longer advertise themselves ([#12760](https://github.com/ai-dynamo/dynamo/pull/12760), [#13122](https://github.com/ai-dynamo/dynamo/pull/13122)).
* **Chat Template Kwargs Plumbing:** Fixed the vLLM chat processor so per-request `chat_template_kwargs` reach the chat template ([#11729](https://github.com/ai-dynamo/dynamo/pull/11729)), threaded the server-wide `--default-chat-template-kwargs` flag through with per-request values overriding defaults ([#11731](https://github.com/ai-dynamo/dynamo/pull/11731)), and rejected a `chat_template` nested inside kwargs, closing a validation bypass ([#11755](https://github.com/ai-dynamo/dynamo/pull/11755)).
* **Streaming Tool-Call Emission Fixes:** Reworked the streaming converter for the Anthropic Messages and OpenAI Responses formats to accumulate fragmented tool-call identities and argument deltas, publishing each complete block on finish ([#10988](https://github.com/ai-dynamo/dynamo/pull/10988)), and synthesized a `finish_reason: "tool_calls"` chunk when the upstream stream omitted its terminal reason ([#11045](https://github.com/ai-dynamo/dynamo/pull/11045)).
* **Guided JSON Reasoning Parsing:** Fixed guided decoding for force-reasoning parsers, preserving both `reasoning_content` and structured tool calls for `tool_choice=required` and named tool calls ([#11205](https://github.com/ai-dynamo/dynamo/pull/11205)), and routed `response_format: json_schema` output to `content` rather than `reasoning_content` with SGLang reasoning enabled ([#11512](https://github.com/ai-dynamo/dynamo/pull/11512)).
* **Token Usage Accounting Fixes:** Corrected Anthropic `/v1/messages` `input_tokens` accounting so cached tokens are no longer double-counted ([#11030](https://github.com/ai-dynamo/dynamo/pull/11030)), propagated `completion_tokens_details` including `reasoning_tokens` into Chat Completions responses ([#11027](https://github.com/ai-dynamo/dynamo/pull/11027)), and added `DYN_ENABLE_FORCE_INCLUDE_USAGE` to require usage in streaming responses even when clients omit `include_usage` ([#12017](https://github.com/ai-dynamo/dynamo/pull/12017)).
* **Frontend Request Validation Hardening:** Mapped `InvalidArgument` errors from the annotated first-event path to HTTP 400 ([#11453](https://github.com/ai-dynamo/dynamo/pull/11453)) and converted preserved aggregation errors on `/v1/audio/speech` so worker rejections return 4xx ([#12832](https://github.com/ai-dynamo/dynamo/pull/12832)). Restored boundary validation so `max_tokens: 0` returns HTTP 400 ([#11394](https://github.com/ai-dynamo/dynamo/pull/11394)), and tolerated raw control bytes in JSON string fields on the OpenAI endpoints ([#11114](https://github.com/ai-dynamo/dynamo/pull/11114)).
* **Responses API Event Semantics:** Mapped length-limited generation to incomplete response states, emitting `response.incomplete` with `max_output_tokens` as the reason ([#12182](https://github.com/ai-dynamo/dynamo/pull/12182)), emitted requested reasoning summaries through the full ordered streaming lifecycle ([#12183](https://github.com/ai-dynamo/dynamo/pull/12183)), and rejected `completion_token_ids` and `prompt_logprobs` requested via `nvext.extra_fields` with a field-specific 501 ([#11253](https://github.com/ai-dynamo/dynamo/pull/11253)).
* **Release 1.4.0 Frontend Backports:** Backported validation of converted OpenAI Responses and Anthropic requests, returning 400 for malformed inputs and 501 for unsupported features ([#12904](https://github.com/ai-dynamo/dynamo/pull/12904)), and rejected non-progressing guided JSON cycles ([#12853](https://github.com/ai-dynamo/dynamo/pull/12853)). Forwarded vLLM tool-call and structured-output constraints to workers ([#12877](https://github.com/ai-dynamo/dynamo/pull/12877)), preserved explicit `top_logprobs=0` ([#12681](https://github.com/ai-dynamo/dynamo/pull/12681)), and converted SGLang `log_probs` and `top_logprobs` to the OpenAI contract ([#12988](https://github.com/ai-dynamo/dynamo/pull/12988)).
* **Model Registry Reconciliation:** Added periodic reconciliation that queries all discovered models every 30 seconds and retries incomplete registrations, so a frontend's local model registry recovers from missed discovery events or transient registration failures without manual intervention ([#11092](https://github.com/ai-dynamo/dynamo/pull/11092)).
* **Async Tokenization Offload:** Moved prompt tokenization off the Frontend's tokio event loop onto the blocking thread pool, matching the embedding-path offload. This prevents BPE encoding of long prompts from stalling the event loop and wrongly marking healthy workers as down in disaggregated serving ([#11200](https://github.com/ai-dynamo/dynamo/pull/11200)).
* **MiniMax M2 Tool-Calling Fix:** Fixed forced and named tool calling for MiniMax M2 with reasoning by wiring `minimax_m2` into Dynamo's force-reasoning and guided-output parsing paths, so tool output is separated from reasoning instead of misparsed. Docs now recommend `minimax_m2` over deprecated `minimax_append_think` ([#11554](https://github.com/ai-dynamo/dynamo/pull/11554)).
* **Kimi Multi-Turn Reasoning Fix:** Fixed post-tool reasoning parsing for Kimi K2.5/2.6 so private reasoning and the `</think>` marker no longer leak into `content`, and normalized `thinking`/`enable_thinking` values consistently across prompt rendering, the SGLang reasoning gate, and the postprocessor ([#11653](https://github.com/ai-dynamo/dynamo/pull/11653)).
* **SGLang Reasoning Delimiter Fix:** Fixed the custom SGLang chat processor to preserve special delimiter tokens during incremental detokenization when a tool-call or reasoning parser is active, so reasoning content populates correctly in streaming output ([#11748](https://github.com/ai-dynamo/dynamo/pull/11748)).
* **Startup File-Descriptor Limit:** Raised the process's soft RLIMIT\_NOFILE at frontend startup, bounded by the environment's hard limit, to prevent the TCP accept() loop from spiraling into EMFILE errors and flooding logs under high connection concurrency ([#11802](https://github.com/ai-dynamo/dynamo/pull/11802)).
* **Tool-Result Media Preservation:** Fixed Anthropic tool results so embedded images are no longer reduced to text-only content, and added support for image, video, and audio URL parts in OpenAI-compatible tool messages, routing them through the existing user-media preprocessing path ([#12180](https://github.com/ai-dynamo/dynamo/pull/12180)).
* **Parallel Tool Calls Enforcement:** Added a post-parse fallback that caps chat completion responses to the first tool call when `parallel_tool_calls: false` is set, covering streaming and non-streaming responses across vLLM, SGLang, and TensorRT-LLM backends ([#12201](https://github.com/ai-dynamo/dynamo/pull/12201)).
* **Mistral Tokenizer Chat Template Fix:** Fixed an AttributeError in the vLLM chat-processor path where MistralTokenizer lacks a `chat_template` attribute, which previously blocked Mistral models from registering and caused chat requests to return HTTP 404 ([#12659](https://github.com/ai-dynamo/dynamo/pull/12659)).

### Multimodal & Diffusion

* **Preserved Media URL Client Errors:** Fixed audio and video loaders to preserve URL-validation and HTTP-status errors during per-item and batch handling, so intentional client rejections keep their 4xx status instead of flattening into generic failures ([#12407](https://github.com/ai-dynamo/dynamo/pull/12407), [#13193](https://github.com/ai-dynamo/dynamo/pull/13193)).
* **GLM-Image NIXL Deploy Config:** Migrated the GLM-Image AR to DiT NIXL disaggregation stage config off `vllm-omni`'s legacy `stage_args` schema to the stages deploy schema ([#12984](https://github.com/ai-dynamo/dynamo/pull/12984)), following a fix that declared the entry stage's `engine_input_source` after `vllm-omni` v0.26.0rc1 removed its default ([#12709](https://github.com/ai-dynamo/dynamo/pull/12709)).
* **Multimodal Trust-Remote-Code Fix:** Fixed multimodal and encode workers across vLLM, SGLang, and TensorRT-LLM to thread `trust_remote_code` from the engine config instead of hardcoding `True`, so custom model repo code no longer executes on load unless the operator explicitly enabled it ([#10738](https://github.com/ai-dynamo/dynamo/pull/10738)).
* **SSRF-Blocked Media URL Status:** Fixed multimodal image loading to return HTTP 400 instead of 500 when an image URL is blocked by the SSRF guard, preserving the `UrlValidationError` so clients get the block reason ([#11312](https://github.com/ai-dynamo/dynamo/pull/11312)).

### Scheduling

* **SGLang Global Router Support:** Fixed the prefill router's execute\_prefill logic to detect SGLang workers and return a Bootstrap outcome with bootstrap host, port, and room instead of hanging, so decode workers now receive requests when using SGLang with the global router ([#7897](https://github.com/ai-dynamo/dynamo/pull/7897)).
* **Planner Worker Discovery Hardening:** Added `runtimeNamespace` to replica status so Planner reads the runtime namespace from DGD status ([#11352](https://github.com/ai-dynamo/dynamo/pull/11352)), resolved workers by DGD component type ([#11578](https://github.com/ai-dynamo/dynamo/pull/11578)), and matched model cards against `.status.components[*].componentNames` ([#12047](https://github.com/ai-dynamo/dynamo/pull/12047)). GlobalPlanner read the v1beta1 `spec.components` schema ([#11990](https://github.com/ai-dynamo/dynamo/pull/11990)) and rejected scaling with `DynamoGraphDeploymentNotReadyError` on unready deployments ([#10526](https://github.com/ai-dynamo/dynamo/pull/10526)).
* **Exact Push Router Dispatch:** Fixed `PushRouter::direct()` to enforce exact-dispatch behavior, returning a typed `CannotConnect` error instead of silently rerouting to another worker when the selected instance disappears before transport resolution. Transport resolution now runs before overload checks, so stale overload metadata cannot mask a missing exact target ([#10429](https://github.com/ai-dynamo/dynamo/pull/10429)).
* **EAGLE3 KV Routing Fix:** Fixed `enable_eagle` detection in SGLang to derive from `SpeculativeAlgorithm.is_eagle()` instead of a hand-maintained name set, correcting KV-aware routing for EAGLE3 and FROZEN\_KV\_MTP workers that previously reported zero cached blocks due to hash misalignment ([#10982](https://github.com/ai-dynamo/dynamo/pull/10982)).
* **KvRouter Cancellation Scope Fix:** Fixed KvRouter to cancel a child token instead of the runtime's primary token on drop, so rebuilding the Router after all workers exit no longer tears down the entire runtime with an etcd lease error ([#11390](https://github.com/ai-dynamo/dynamo/pull/11390)).
* **Prefill EAGLE KV Routing Fix:** Fixed prefill KV routing to detect EAGLE/MTP support from the prefill endpoint's runtime config instead of always disabling EAGLE hash mode, preventing zero cache-block predictions and restoring correct prefix cache hits when SGLang prefill workers enable EAGLE ([#11764](https://github.com/ai-dynamo/dynamo/pull/11764)).

### Kubernetes Deployment

* **CRIU Checkpoint Image Mechanics:** Added an `imageIoMode` CRIU knob (`writeback` or `direct`, defaulting to `direct`) and bumped go-criu to v8.4.0 ([#11476](https://github.com/ai-dynamo/dynamo/pull/11476)), pinned the snapshot image CRIU build to a fixed revision ([#11659](https://github.com/ai-dynamo/dynamo/pull/11659)), made rootfs diff capture atomic with zero-byte-archive rejection ([#11883](https://github.com/ai-dynamo/dynamo/pull/11883)), and fixed the DRA `nvidia-smi` probe to enter the workload PID namespace ([#12227](https://github.com/ai-dynamo/dynamo/pull/12227)).
* **DGD Admission Webhook Migration:** Served both v1alpha1 and v1beta1 DynamoGraphDeployment admission endpoints via the v1beta1 hub ([#11117](https://github.com/ai-dynamo/dynamo/pull/11117)), restored pre-existing v1alpha1 behaviors with per-container CEL scoping ([#11126](https://github.com/ai-dynamo/dynamo/pull/11126)), and fixed volume-mount composition so service PVC mounts survived ([#11263](https://github.com/ai-dynamo/dynamo/pull/11263)). Made cluster-wide handlers Lease-aware ([#11597](https://github.com/ai-dynamo/dynamo/pull/11597)) and gated Operator readiness on the webhook server with a TLS-checked `/readyz` ([#12255](https://github.com/ai-dynamo/dynamo/pull/12255)).
* **DGDR Profiling And Status Hardening:** Added a reusable Go package that applies versioned `overrides.dgd` patches using Kubernetes structural merge semantics ([#11124](https://github.com/ai-dynamo/dynamo/pull/11124)), an idempotent profiling-to-deployment handoff so a DynamoGraphDeploymentRequest no longer sticks in `Deploying` ([#11250](https://github.com/ai-dynamo/dynamo/pull/11250)), and a `MessageValidationPassed` constant on the `Validation=True` condition ([#11261](https://github.com/ai-dynamo/dynamo/pull/11261)).
* **Explicit Service Mesh Override:** Added a `serviceMesh.enabled` option to the Operator configuration so users can explicitly enable or disable Istio integration, fixing startup failures that occurred when Istio's API group was partially present but the `DestinationRule` resource was unavailable ([#10773](https://github.com/ai-dynamo/dynamo/pull/10773)).
* **Docker Secret Index Resilience:** Fixed the Operator's controller manager crashing on startup when the initial Docker secrets index refresh encountered malformed `kubernetes.io/dockerconfigjson` secrets, now skipping invalid entries and logging aggregated errors while continuing to index valid ones ([#10789](https://github.com/ai-dynamo/dynamo/pull/10789)).
* **LWS Worker GPU Validation:** Removed the hardcoded GPU limit check from LWS worker pod template generation, letting Kubernetes and device admission decide pod validity so CPU-only deployments no longer fail to render ([#11115](https://github.com/ai-dynamo/dynamo/pull/11115)).
* **Duplicate Compilation Cache Mounts:** Fixed pod generation to reuse an existing writable compilation-cache mount instead of appending a duplicate one, and rejected mount-path conflicts and read-only cache mounts, preventing Grove PodCliqueSet validation failures on converted v1alpha1 components ([#11264](https://github.com/ai-dynamo/dynamo/pull/11264)).
* **EPP DiskPressure Resilience:** Set default CPU, memory, and ephemeral storage requests for the EndPoint Picker (EPP) and lowered the default RUST\_LOG level to info, preventing kubelet from evicting EPP under node DiskPressure and stalling InferencePool deployments ([#11550](https://github.com/ai-dynamo/dynamo/pull/11550)).
* **Multi-Node TensorRT-LLM Env Relay:** Fixed multi-node TensorRT-LLM launch commands to forward the CPATH and TRITON\_PTXAS\_PATH environment variables and avoid duplicate forwarding, preventing build and compilation failures on multi-node deployments ([#11999](https://github.com/ai-dynamo/dynamo/pull/11999)).
* **DGDSA Replica Ownership Fix:** Fixed a reconciliation race between the DynamoGraphDeployment and DynamoGraphDeploymentScalingAdapter (DGDSA) controllers by replacing full-spec DGDSA synchronization with a patch that preserves DGDSA-owned replica counts, preventing stale DGD values from overwriting scaling adapter replicas during serve, unserve, and scaling operations ([#12087](https://github.com/ai-dynamo/dynamo/pull/12087)).
* **Event Channel Metadata Compatibility:** Fixed DynamoWorkerMetadata deserialization by reconstructing legacy `namespace`/`component` fields alongside the current `scope` field for Kubernetes event channels, preventing rolling upgrades from rejecting worker records due to version mismatches ([#12708](https://github.com/ai-dynamo/dynamo/pull/12708)).

### Fault Tolerance & Observability

* **Bounded Endpoint Drain Shutdown:** Fixed a zombie-worker failure mode where a lost etcd lease with a stuck in-flight request left a worker Running but unable to serve; per-endpoint drain now respects the graceful shutdown timeout and lease loss routes through Runtime::shutdown() ([#11068](https://github.com/ai-dynamo/dynamo/pull/11068)).

### Engines

#### vLLM

* **Writable vLLM Cache Directories:** Pre-created the vLLM cache root in the container image with group 0 write access so pods running under a UID other than 1000 with gid 0 no longer fail engine initialization with EACCES on the first torch compile ([#13040](https://github.com/ai-dynamo/dynamo/pull/13040)), cherry-picked into the release branch ([#13045](https://github.com/ai-dynamo/dynamo/pull/13045)).
* **Lifecycle-Aware Disaggregated LoRA Prefill:** Registered and served LoRA lifecycle endpoints on vLLM prefill workers, with metadata-only initial registration, adapters activated from the inference-time `LoRARequest`, and endpoint deduplication by physical pod ([#11520](https://github.com/ai-dynamo/dynamo/pull/11520)). Fixed dynamic LoRA registration to publish adapter model cards with the engine-actual main-attention KV block size instead of the raw CLI `block_size` ([#11867](https://github.com/ai-dynamo/dynamo/pull/11867)).
* **vLLM Embedding Pooling Correctness:** Set the embedding worker's pooling task to `embed` in `PoolingParams` so vLLM returns one pooled, L2-normalized embedding per input ([#10248](https://github.com/ai-dynamo/dynamo/pull/10248)), and propagated `truncate_prompt_tokens` to vLLM `encode` as tokenizer kwargs for raw-text embedding inputs ([#11099](https://github.com/ai-dynamo/dynamo/pull/11099)).
* **Embeddings Dimensions Type Validation:** Fixed the `/v1/embeddings` endpoint to reject boolean values passed as the `dimensions` parameter, raising a `TypeError` instead of silently truncating embedding vectors to one dimension. Prevents corrupted embeddings from propagating undetected into downstream vector stores or RAG systems ([#9873](https://github.com/ai-dynamo/dynamo/pull/9873)).
* **Cache Salt Propagation Fix:** Fixed the vLLM text-mode path to propagate `nvext.cache_salt` from the request into the prompt object sent to the vLLM engine, ensuring KV cache keys reflect the requested cache salt instead of losing it. ([#11082](https://github.com/ai-dynamo/dynamo/pull/11082)).
* **Qwen3-TTS Prompt Estimator Fix:** Fixed Qwen3-TTS prompt-length estimation to call the estimator through `Qwen3TTSPromptEmbedsBuilder` instead of the stale talker-model import, avoiding the generic 2048-token fallback that could produce invalid or poor-quality audio ([#11595](https://github.com/ai-dynamo/dynamo/pull/11595)).
* **vLLM EngineCore Shutdown on Failed Startup:** Fixed vLLM decode workers to shut down the EngineCore process when Dynamo-side initialization fails after engine startup, so failed workers exit cleanly instead of leaving a live engine process behind ([#12867](https://github.com/ai-dynamo/dynamo/pull/12867)).

#### SGLang

* **HiCache Capacity Publication:** Fixed SGLang workers to publish ratio-based HiCache offload capacity, so Dynamo Router can account for SGLang CPU-offload cache tiers ([#12953](https://github.com/ai-dynamo/dynamo/pull/12953), [#13060](https://github.com/ai-dynamo/dynamo/pull/13060)).
* **Writable FlashInfer Cubin Directories:** Marked the installed `flashinfer_cubin/cubins` directories writable in the SGLang runtime image before switching to the non-root `dynamo` user ([#11154](https://github.com/ai-dynamo/dynamo/pull/11154)), added matching group-write permissions in the vLLM runtime images for arbitrary OpenShift UIDs in group 0 ([#12770](https://github.com/ai-dynamo/dynamo/pull/12770)), and carried the vLLM fix into the release branch ([#12976](https://github.com/ai-dynamo/dynamo/pull/12976)).
* **SGLang Token Usage Reporting:** Populated `prompt_tokens_details.cached_tokens` in terminal completion usage from final SGLang response metadata, restoring cache observability for warm requests including HiCache ([#10925](https://github.com/ai-dynamo/dynamo/pull/10925)), and treated streamed usage metadata as optional so final chunks without token counts no longer raise `KeyError` ([#11062](https://github.com/ai-dynamo/dynamo/pull/11062)).
* **SGLang Response Format Forwarding:** Fixed the SGLang chat processor to forward the OpenAI `response_format` parameter to decode and prefill engine calls, enabling SGLang-native JSON schema and JSON object structured output constraints when using `--dyn-chat-processor sglang`. ([#10259](https://github.com/ai-dynamo/dynamo/pull/10259)).
* **SGLang Multimodal Audio Forwarding:** Fixed forwarding of Dynamo's `--enable-multimodal` setting to SGLang's server arguments and added extraction and forwarding of `audio_url`/`audio_data` alongside image and video inputs across aggregate, disaggregated prefill, and disaggregated decode paths. Unextracted audio content is now rejected using the same guard applied to other modalities ([#11747](https://github.com/ai-dynamo/dynamo/pull/11747)).
* **SGLang Bootstrap Host Discovery:** Fixed prefill bootstrap endpoint discovery to prefer the concrete, non-loopback SGLang host from `GetServerInfo` before falling back to `dist_init_addr` or local-IP resolution. This prevents decode workers from receiving unreachable loopback or wildcard addresses for bootstrap ([#12026](https://github.com/ai-dynamo/dynamo/pull/12026)).
* **SGLang Conditional-Disagg Error Propagation:** Fixed the SGLang rejection of the `x-bypass-remote-prefill` annotation so clients receive the intended unsupported-feature message instead of a bare Internal Server Error ([#12616](https://github.com/ai-dynamo/dynamo/pull/12616)).
* **SGLang NVExt Token Validation:** Rejected out-of-vocabulary IDs from pre-tokenized `nvext.token_data` before reaching SGLang's embedding lookup, while preserving valid request-scoped multimodal sentinel token IDs, backported to release/1.4.0 ([#12852](https://github.com/ai-dynamo/dynamo/pull/12852)).

#### TensorRT-LLM

* **TensorRT-LLM Mpmath Version Cap:** Capped mpmath below 1.4 in the trtllm extra to prevent the resolver from pulling mpmath 1.5.0a0, which removed `bitcount` and caused the TensorRT-LLM backend to crash on import. ([#11402](https://github.com/ai-dynamo/dynamo/pull/11402)).
* **TensorRT-LLM Multimodal Request Handling:** Scoped NIXL Connect initialization to multimodal disaggregated workers so plain text workers no longer load the `nixl` bindings ([#11373](https://github.com/ai-dynamo/dynamo/pull/11373)), sized an omitted `max_tokens` from the expanded prompt length instead of the default 32 ([#11513](https://github.com/ai-dynamo/dynamo/pull/11513)), and routed `video_url` inputs through Dynamo's URL validation policy before local decoding ([#12002](https://github.com/ai-dynamo/dynamo/pull/12002)).
* **TensorRT-LLM RC22 Compatibility Fixes:** Updated disaggregated serving for TensorRT-LLM rc22's (node\_id, process\_id) machine-ID pair and stopped dispatching GEN when a context request finished during prefill without KV-cache handoff IDs ([#12203](https://github.com/ai-dynamo/dynamo/pull/12203)), with engine argument defaults read through the changed TensorRT-LLM API ([#11741](https://github.com/ai-dynamo/dynamo/pull/11741)).

### Infrastructure Modernization

* **ZMQ Subscriber Socket Leak:** Fixed a leak where a dropped `ZmqSubTransport` left its subscriber pump running and SUB socket open, accumulating sockets until "Too many open files". Pumps now stop when their final owner is dropped, and cancellation propagates to discovery watch tasks ([#11730](https://github.com/ai-dynamo/dynamo/pull/11730)).

### KV Block Manager

* **KVBM Decode KV Cache Alignment:** Configured KVBM prefill and decode workers with `--disable-hybrid-kv-cache-manager` so both sides share the same vLLM KV-cache layout, marking decode workers with `--disaggregation-mode decode` across launchers, examples, and the integration test ([#12821](https://github.com/ai-dynamo/dynamo/pull/12821)). Stopped suppressing an explicit `kv_events_config` on disaggregated decode workers and set up the Dynamo KV-event relay for them ([#12084](https://github.com/ai-dynamo/dynamo/pull/12084)).
* **HMA Disable For PdConnector:** Fixed KVBM and LMCache disaggregated serving startup failures by disabling vLLM's hybrid KV cache manager (HMA) for PdConnector configurations whose child connectors, such as DynamoConnector or LMCacheConnectorV1 paired with NixlConnector, do not all support HMA, preventing the MultiConnector assertion during EngineCore startup ([#11608](https://github.com/ai-dynamo/dynamo/pull/11608)).

### General

* **etcd Connection Resilience Fixes:** Added a cancellable startup retry loop for the initial etcd connection and primary lease creation, retrying up to 120s with exponential backoff ([#10799](https://github.com/ai-dynamo/dynamo/pull/10799)), and introduced a `WatchEvent::Resync` snapshot after watch reconnects so `KvCache`, `TypedPrefixWatcher`, and session-affinity claim watchers rebuild state and stale instances are removed ([#11146](https://github.com/ai-dynamo/dynamo/pull/11146)).
* **Shared ZMQ Context Fix:** Fixed the event plane's direct (broker-less) PUB/SUB transport to reuse a single process-wide ZMQ context instead of creating one per socket, preventing unbounded OS-thread growth on frontends as worker fleets scaled up, which had starved the async runtime and triggered liveness-probe restart loops ([#11246](https://github.com/ai-dynamo/dynamo/pull/11246)).
* **JSON5 Config Parsing Fix:** Fixed max context length derivation to parse config.json with JSON5 instead of strict JSON, preventing registration failures for models like Nemotron-H whose configs contain non-finite literals such as `Infinity` in unused fields ([#11445](https://github.com/ai-dynamo/dynamo/pull/11445)).
* **Triton CUDA 13 Image Fix:** Updated the Triton example's default server image to `tritonserver:25.10-py3` for CUDA 13.0.2 compatibility, fixed the DCGM library bind-mount path and permissions, and added a build-time `import tritonserver` smoke check ([#12577](https://github.com/ai-dynamo/dynamo/pull/12577)).

### Recipes

* **DeepSeek-V4 H200 Checkpoint Download:** Fixed the deepseek-v4 and deepseek-v4-pro H200 recipes to download the FP8 checkpoint instead of the B200 NVFP4 variant by parameterizing the model-download Job ([#13034](https://github.com/ai-dynamo/dynamo/pull/13034), [#13213](https://github.com/ai-dynamo/dynamo/pull/13213)).
* **DeepSeek-R1 Disagg Decode Mode:** Fixed the deepseek-r1 vLLM disaggregated recipe to declare `--disaggregation-mode decode` on the decode worker, which previously never became ready ([#13046](https://github.com/ai-dynamo/dynamo/pull/13046)).
* **Removed Stale Router Flags:** Removed the retired `--router-reset-states` flags from affected recipes so they launch cleanly against the shipped Router CLI ([#13116](https://github.com/ai-dynamo/dynamo/pull/13116), [#13117](https://github.com/ai-dynamo/dynamo/pull/13117)).

## Developer Experience

### Documentation Website Refactor

* **Tab-Based Documentation Navigation:** Restructured the documentation site from a single flat sidebar into top-level Home, User Guide, Developer Guide, Recipes, Reference, Blog, and Community tabs, with an explicit redirect table of 357 redirects so every previously live URL still resolves ([#10855](https://github.com/ai-dynamo/dynamo/pull/10855)). Restored the per-backend SGLang, TensorRT-LLM, and vLLM multimodal deployment guides to the navigation and fixed a stale anchor in their cross-references ([#11064](https://github.com/ai-dynamo/dynamo/pull/11064)), and consolidated Kubernetes content while correcting remaining navigation, cross-reference, styling, and release and observability pages missed by the initial pass ([#12243](https://github.com/ai-dynamo/dynamo/pull/12243)).
* **Simplified Chinese Docs Localization:** Registered all 13 existing zh-CN translations in the Fern nav under the hidden Additional Resources section so they publish at real URLs and the language switcher links resolve instead of serving soft-404 stubs ([#11088](https://github.com/ai-dynamo/dynamo/pull/11088)), then replaced the raw HTML switcher badge with a plain Markdown link and dropped duplicate body H1s on 5 translated pages ([#11148](https://github.com/ai-dynamo/dynamo/pull/11148)). Migrated the manual switcher to Fern native localization, moving translations to a `fern/translations/zh-CN` mirror tree with a `translations:` config that adds the header language dropdown and falls back to English for untranslated pages on the dev version ([#11161](https://github.com/ai-dynamo/dynamo/pull/11161)).
* **Tagged-Commit Release Snapshots:** Fixed the Fern docs release workflow to build versioned pages and navigation from the tagged commit's own docs, navigation, and converter instead of whatever content was on `main`, preventing release tags from publishing mismatched content, and added a local release dry-run script under `fern/` for pre-publish validation ([#11140](https://github.com/ai-dynamo/dynamo/pull/11140)).

## Documentation

* **Kubernetes & Deployment Docs:** Restructured the Gateway API (GAIE) documentation into overview and quickstart pages with standalone EPP on-ramp guidance ([#10956](https://github.com/ai-dynamo/dynamo/pull/10956), [#10993](https://github.com/ai-dynamo/dynamo/pull/10993), [#11119](https://github.com/ai-dynamo/dynamo/pull/11119), [#12480](https://github.com/ai-dynamo/dynamo/pull/12480)), corrected the Dynamo Operator's default termination grace period ([#11120](https://github.com/ai-dynamo/dynamo/pull/11120)), aligned install examples with current platform versions ([#11486](https://github.com/ai-dynamo/dynamo/pull/11486)), and documented DGDR `spec.overrides.dgd` merge semantics ([#11487](https://github.com/ai-dynamo/dynamo/pull/11487)), scale-to-zero behavior and its recovery limits ([#11536](https://github.com/ai-dynamo/dynamo/pull/11536)), and the TensorRT-LLM snapshot support matrix ([#11674](https://github.com/ai-dynamo/dynamo/pull/11674)).
* **Recipe & Model Pages:** Added recipe pages for the Inkling NVFP4 day-0 launch ([#11749](https://github.com/ai-dynamo/dynamo/pull/11749), [#11807](https://github.com/ai-dynamo/dynamo/pull/11807)), GLM-5.2 on SGLang across B200 NVFP4 and H200 FP8 targets ([#11972](https://github.com/ai-dynamo/dynamo/pull/11972)), Kimi-K3 ([#12210](https://github.com/ai-dynamo/dynamo/pull/12210)), and vLLM agentic targets for GPT-OSS-120B ([#11980](https://github.com/ai-dynamo/dynamo/pull/11980)), plus a Model Early Access Builds reference page standardizing early-access terminology ([#11898](https://github.com/ai-dynamo/dynamo/pull/11898)).
* **Router & KV Routing Reference:** Corrected the Router reference pages to match shipped v1.4.0 behavior ([#11892](https://github.com/ai-dynamo/dynamo/pull/11892), [#12949](https://github.com/ai-dynamo/dynamo/pull/12949), [#12983](https://github.com/ai-dynamo/dynamo/pull/12983)), clarified that KV routing requires explicit worker-side event publishing and distinguished the fallback modes ([#11505](https://github.com/ai-dynamo/dynamo/pull/11505)), added a KV offloading support matrix across backends and cache tiers ([#11973](https://github.com/ai-dynamo/dynamo/pull/11973)), updated SGLang HiCache minimum-version requirements ([#11183](https://github.com/ai-dynamo/dynamo/pull/11183)), and corrected session-radix ownership claims ([#11241](https://github.com/ai-dynamo/dynamo/pull/11241)).
* **Backend Setup & Troubleshooting:** Directed TensorRT-LLM users to the container-based install path ([#11066](https://github.com/ai-dynamo/dynamo/pull/11066)), recommended the PyTorch multiprocessing backend over Ray for vLLM multi-node deployments ([#11259](https://github.com/ai-dynamo/dynamo/pull/11259)), documented the CUDA 13 FlashInfer sampler failure and its workaround ([#10954](https://github.com/ai-dynamo/dynamo/pull/10954)), the NVENC hardware requirement for TensorRT-LLM video diffusion MP4 output ([#11457](https://github.com/ai-dynamo/dynamo/pull/11457)), corrected the TensorRT-LLM UCX environment-variable note and EFA image facts ([#13049](https://github.com/ai-dynamo/dynamo/pull/13049)), vLLM worker-role flags and `--kv-transfer-config` ([#12666](https://github.com/ai-dynamo/dynamo/pull/12666)), and XPU deployment paths for Intel GPUs including DRA requirements ([#11894](https://github.com/ai-dynamo/dynamo/pull/11894)).
* **Planner & Profiler Docs:** Documented rapid-profiling behavior when no SLA-feasible configuration exists and corrected the unachievable-SLA constraints claim ([#11612](https://github.com/ai-dynamo/dynamo/pull/11612), [#12774](https://github.com/ai-dynamo/dynamo/pull/12774)), and corrected the Planner configuration reference covering all four optimization targets and TLS verification behavior ([#11613](https://github.com/ai-dynamo/dynamo/pull/11613)).
* **Simulation Docs:** Reframed the DynoSim documentation around deterministic offline replay and live Mocker with a dedicated simulation-model page ([#12172](https://github.com/ai-dynamo/dynamo/pull/12172), [#11430](https://github.com/ai-dynamo/dynamo/pull/11430)), corrected the load-generator ISL/OSL preset semantics ([#10791](https://github.com/ai-dynamo/dynamo/pull/10791)), and corrected Spica's `kv_load_ratio` support guidance ([#12714](https://github.com/ai-dynamo/dynamo/pull/12714)).
* **Observability Docs:** Documented the actual OTLP endpoint resolution chain and sampling controls, removing a stale fallback claim that caused silent log loss ([#11337](https://github.com/ai-dynamo/dynamo/pull/11337)), and corrected tokenizer-cache metric absence semantics while adding the v1.4.0 deprecations section ([#12775](https://github.com/ai-dynamo/dynamo/pull/12775)), and marked the CKF counter as registered but never incremented in this release ([#13149](https://github.com/ai-dynamo/dynamo/pull/13149)).
* **Cleanup:** Replaced references to the removed experimental unified backend engines with current guidance ([#11833](https://github.com/ai-dynamo/dynamo/pull/11833)).

## Looking Ahead

### Standalone Router & Runtime-Free EPP

The standalone Router work continues toward a runtime-free selection service and inference-gateway EPP that deploy without the full Dynamo runtime, extending the endpoint-scoped transport and reservation-replay foundations shipped in this release. The Rust ext-proc EPP becomes the default gateway integration in an upcoming release, replacing the Go-based Endpoint Picker.

### Custom Routing Strategies

Router scoring and selection open up behind native `WorkerScore` and `WorkerPick` policies already merged on main, so deployments can bring a routing strategy tuned to their workload. The trace-capture and simulation tooling shipped in this release gives those strategies a place to be validated against recorded production traffic before they route real requests.

### Cross-Datacenter Routing

Cross-datacenter prefix routing shipped experimental in Dynamo v1.4.0. The work ahead hardens the datacenter-scoped KV relay and sequenced indexing toward production use, and workers move toward advertising their own router configuration so fleets negotiate routing behavior without frontend flag coordination.

### Voice & Custom Multimodal Encoders

Multimodal work extends toward custom encoder support, frontend-decoded encode-prefill-decode pipelines, and realtime voice serving, building on the unified vLLM multimodal serving and transcription infrastructure in this release.

#### Open-source contributions and new contributors

Dynamo v1.4.0 includes contributions from 127 developers, 29 of whom made their first merged contribution in this release. Welcome to our first-time contributors:

* [@nvyutwu](https://github.com/nvyutwu) added an OTLP audit-log sink exporting chat-completion audit records through an OpenTelemetry Collector ([#9390](https://github.com/ai-dynamo/dynamo/pull/9390))
* [@Broduker](https://github.com/Broduker) fixed chat-template override support in the SGLang chat processor ([#10249](https://github.com/ai-dynamo/dynamo/pull/10249))
* [@maheshrbapatu](https://github.com/maheshrbapatu) fixed unified multimodal routing for modality-aware KV-cache metadata ([#10375](https://github.com/ai-dynamo/dynamo/pull/10375))
* [@zidanehuang001](https://github.com/zidanehuang001) made GlobalPlanner reject scale operations while a deployment is not ready ([#10526](https://github.com/ai-dynamo/dynamo/pull/10526))
* [@xiaguan](https://github.com/xiaguan) exposed salt-hash computation for KV producers without a request context ([#10652](https://github.com/ai-dynamo/dynamo/pull/10652))
* [@Hrithvik-Alex](https://github.com/Hrithvik-Alex) preserved trace context in detached Python contexts ([#10783](https://github.com/ai-dynamo/dynamo/pull/10783))
* [@Nietzsche2000](https://github.com/Nietzsche2000) contributed the experimental squeeze-evolve multi-model evolutionary test-time scaling component ([#10785](https://github.com/ai-dynamo/dynamo/pull/10785))
* [@muma378](https://github.com/muma378) made Dynamo Operator startup resilient to docker-secret index refresh failures ([#10789](https://github.com/ai-dynamo/dynamo/pull/10789))
* [@aishwaryabandapelly-ai](https://github.com/aishwaryabandapelly-ai) propagated chat-completion token details end to end ([#11027](https://github.com/ai-dynamo/dynamo/pull/11027))
* [@YTGhost](https://github.com/YTGhost) corrected Anthropic /v1/messages input-token accounting ([#11030](https://github.com/ai-dynamo/dynamo/pull/11030))
* [@cpakkamisaac-sae](https://github.com/cpakkamisaac-sae) standardized shell code fences across the docs ([#11063](https://github.com/ai-dynamo/dynamo/pull/11063))
* [@panpan0000](https://github.com/panpan0000) removed premature GPU pre-validation for LeaderWorkerSet workers ([#11115](https://github.com/ai-dynamo/dynamo/pull/11115))
* [@fengyuli-dev](https://github.com/fengyuli-dev) added multiple served model names to the SGLang backend ([#11141](https://github.com/ai-dynamo/dynamo/pull/11141))
* [@nvpohanh](https://github.com/nvpohanh) added the SGLang profile request object API ([#11199](https://github.com/ai-dynamo/dynamo/pull/11199))
* [@joeltg](https://github.com/joeltg) shared one ZMQ context across all event-plane sockets ([#11246](https://github.com/ai-dynamo/dynamo/pull/11246))
* [@ssojrani-wq](https://github.com/ssojrani-wq) parsed config.json as JSON5 when deriving max context length ([#11445](https://github.com/ai-dynamo/dynamo/pull/11445))
* [@renflowerz](https://github.com/renflowerz) restored trace context across the Python chat-processor boundary ([#11480](https://github.com/ai-dynamo/dynamo/pull/11480))
* [@hdimmfh](https://github.com/hdimmfh) aligned Kubernetes installation examples with the shipped release ([#11486](https://github.com/ai-dynamo/dynamo/pull/11486))
* [@RomaA2000](https://github.com/RomaA2000) added GPT-OSS-120B vLLM recipes for B200 and H200 ([#11506](https://github.com/ai-dynamo/dynamo/pull/11506))
* [@aniskumar-nv](https://github.com/aniskumar-nv) added Python-loadable frontend route extensions ([#11626](https://github.com/ai-dynamo/dynamo/pull/11626))
* [@alexeldeib](https://github.com/alexeldeib) honored reasoning effort in chat templates ([#11691](https://github.com/ai-dynamo/dynamo/pull/11691))
* [@kulkarnisamr](https://github.com/kulkarnisamr) capped Kubernetes discovery RBAC label values to 63 characters ([#11709](https://github.com/ai-dynamo/dynamo/pull/11709))
* [@Shang-Pin](https://github.com/Shang-Pin) hardened KV Router lookup-entry cleanup on unresolved removals ([#11785](https://github.com/ai-dynamo/dynamo/pull/11785))
* [@chanh](https://github.com/chanh) disabled tokenizer.json baked-in truncation ([#11792](https://github.com/ai-dynamo/dynamo/pull/11792))
* [@mikekg](https://github.com/mikekg) raised the file-descriptor soft limit at startup to avoid accept() failure spirals ([#11802](https://github.com/ai-dynamo/dynamo/pull/11802))
* [@larroy](https://github.com/larroy) refined the Inkling SGLang aggregated B200 recipe ([#11807](https://github.com/ai-dynamo/dynamo/pull/11807))
* [@liyuanzhe1991](https://github.com/liyuanzhe1991) improved FPM prefill-sweep sampling ([#11824](https://github.com/ai-dynamo/dynamo/pull/11824))
* [@shwetha-krishnamurthy](https://github.com/shwetha-krishnamurthy) added vLLM agentic targets to the GPT-OSS-120B recipe ([#11980](https://github.com/ai-dynamo/dynamo/pull/11980))
* [@pskiran1](https://github.com/pskiran1) enforced parallel\_tool\_calls: false in chat completions ([#12201](https://github.com/ai-dynamo/dynamo/pull/12201))

If you would like to get involved, please see our [Contribution Guide](/dynamo/dev/contributing/contribution-guide).