Recipes

GLM-5.3-Flash

Z.AI

SGLang NVFP4 recipes with TP4 aggregated serving on B200/GB200 or GB200 2P1D Mooncake NVLink disaggregation with KV events, plus vLLM FP8 recipes on GB200 and H200.

GLM-5.3/5.2

Z.AI / NVIDIA

SGLang recipes for long-context agentic traffic on B200 (NVFP4) or H200 (FP8), aggregated or disaggregated, with KV-aware routing, EAGLE MTP speculation, and HiCache CPU offload.

GLM-5 NVFP4

NVIDIA / Z.AI

20x GB200 SGLang P/D recipe for 1K input / 8K output traffic with EAGLE speculative decoding, plus an AWS EFA variant.

Kimi-K3

Moonshot / NVIDIA

Kimi-K3 recipes for vLLM on H200, GB200, and GB300, plus SGLang on GB200 and GB300.

Kimi-K2.6

Moonshot / NVIDIA

Multi-target vLLM matrix covering B200 and H200 for chat and agentic traffic, with Eagle3 MLA speculation and KV-aware routing.

DeepSeek-V4.1-Flash

DeepSeek

vLLM aggregated and 1P1D disaggregated recipes on B200, GB200, and H200, with DSpark and KV-aware routing. B200 and GB200 use sparse-indexer logits; H200 uses CUTLASS MoE. SGLang recipes are available on GB200.

DeepSeek-V4-Pro

NVIDIA checkpoint / DeepSeek

vLLM agentic recipe — MoE 1.6T / 49B active, B200 (NVFP4, 1M ctx) and H200 (FP8), aggregated or disaggregated with MTP-2 and KV-aware routing.

DeepSeek-V4-Pro-0813

DeepSeek

vLLM agentic recipe — MoE 1.6T, MXFP4 experts + FP8 KV, GB200 and H200, aggregated or disaggregated with KV-aware routing. Full 1M context with no CPU KV offload.

DeepSeek-V4-Flash

NVIDIA checkpoint / DeepSeek

vLLM agentic recipe — MoE 284B / 13B active, B200 (NVFP4) and H200 (FP8), aggregated or disaggregated (2P1D / 4P3D) with KV-aware routing.

Qwen3.8-Flash-Next

Qwen

Experimental vLLM recipes for long-context agentic traffic on B200, aggregated or 1P1D disaggregated, with NVFP4 weights, MTP3 speculation, and KV-aware routing.

Qwen3.8-2.4T-A95B

Qwen

vLLM and SGLang recipes for chat and agentic traffic on GB300 or GB200, aggregated or disaggregated, with KV-aware routing, MTP speculation, and FP8 weights/KV over MNNVL.

Qwen3.6-35B-A3B

Qwen

SGLang recipes for agentic traffic on B200, GB200, or H200, with FP8 KV caching, prefix caching, and speculative decoding.

Qwen3.5-122B-A10B

Qwen

vLLM recipes for long-context agentic traffic on B200 NVFP4 and H200 FP8, aggregated or disaggregated 1P2D, with KV-aware routing and MTP speculation on H200 aggregated.

Qwen3-235B-A22B FP8

Qwen

16-GPU TensorRT-LLM recipe matrix for Hopper/Blackwell and aggregate/P-D serving.

Qwen3-32B

Qwen

Disaggregated vLLM serving with KV-aware routing on 16x H200, for multi-turn conversational traffic with prefix reuse.

Qwen3-32B FP8

Qwen

FP8 recipe set spanning TensorRT-LLM aggregate, TensorRT-LLM P/D, and vLLM P/D targets.

Nemotron 3.5 Lightning

NVIDIA

30B hybrid Mamba/Attention/MoE recipe matrix for H100, H200, B200, and GB200, with NVFP4 and BF16 variants, vLLM MTP/DFlash/DSpark targets, DSpark with KV-routing, and experimental TensorRT-LLM MTP/no-spec.

Nemotron 3 Ultra

NVIDIA

Aggregated vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and trace-backed benchmarks.

Nemotron 3 Super

NVIDIA

NVFP4 and FP8 vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and KV-aware routing.

Gemma-4-31B

Google / NVIDIA

Aggregated TensorRT-LLM recipes for long-context agentic traffic on B200, GB200, or H200, with KV-aware routing and CPU KV-cache offload.

Inkling NVFP4

Thinking Machines

Thinking Machines’ first open-weights model — a multimodal MoE with controllable reasoning effort. vLLM GB300 targets serve agentic traffic at 1M context with MTP speculation and KV-aware routing, aggregated or disaggregated; the Day-0 SGLang B200 target adds image and audio input.

GPT-OSS-120B

OpenAI

TensorRT-LLM GB200 targets for static traffic plus vLLM B200/H200 agentic targets (agg + disagg) with KV-aware routing and EAGLE3 speculative decoding.

K-EXAONE 2.0

LG AI Research

750B-A37B MoE in NVFP4 on B200, aggregated or disaggregated 1P1D, with MTP speculative decoding and KV-aware routing on an 8K/1K chat trace.

A.X-K2

SK Telecom

B200 profiles use NVFP4 weights and FP8 KV cache; H200 profiles use FP8 weights and BF16 KV cache. Both use EAGLE3. Choose aggregated or disaggregated serving; H200 uses TP8 workers and a one-Job concurrency 16–128 benchmark sweep.

NVFP4 checkpoint with a 1M-token context on B200: two KV-routed TP2 replicas, or disaggregated 1P:2D with an expert-parallel prefill worker.

vLLM recipes for MiniMax M3 on GB200: aggregated or prefill/decode disaggregated, with KV-aware routing, EAGLE3 speculation and NVFP4 quantization.

Experimental Motif vLLM recipes for aggregated chat on two B200 GPUs or 3P1D KV-aware serving on eight B200 GPUs, with TP2 and MTP2.