Skip to navigation
Model RecipesReflection

Beam-1.0-501B-A23B

Serve Beam-1.0-501B-A23B with NVIDIA Dynamo on H200, B200, B300, GB200, or GB300.

View as Markdown

Deploy Beam-1.0-501B-A23B with NVIDIA Dynamo on H200, B200, B300, GB200, or GB300. Choose your framework, GPU, and serving topology; the summary and commands update to match. SGLang and vLLM provide aggregated and disaggregated serving on all five GPU types, with KV-aware routing and NIXL prefill/decode transfer. All profiles enable DFlash2 with block length 8 and FP8 KV cache. H200 uses FP8 weights; B200, B300, GB200, and GB300 use NVFP4 weights.

Choose your deployment target

Framework
GPU
Topology
Checkpoint reflection/Beam-1.0-501B-A23B-FP8Precision FP8 + FP8 KVGPUs 8x H200, one workerParallelism 1 × TP8Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-FP8Precision FP8 + FP8 KVGPUs 16x H200, one prefill + one decodeParallelism 1P1D, TP8 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B200, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B200, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B300, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B300, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB200, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB200, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB300, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB300, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-FP8Precision FP8 + FP8 KVGPUs 8x H200, one workerParallelism 1 × TP8Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-FP8Precision FP8 + FP8 KVGPUs 16x H200, one prefill + one decodeParallelism 1P1D, TP8 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B200, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B200, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B300, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x B300, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB200, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB200, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB300, two workersParallelism 2 × TP4Routing KV-awareContext Up to 1,048,576 tokens
Checkpoint reflection/Beam-1.0-501B-A23B-NVFP4Precision NVFP4 + FP8 KVGPUs 8x GB300, one prefill + one decodeParallelism 1P1D, TP4 per roleRouting KV-aware, NIXL KV transferContext Up to 1,048,576 tokens

Prerequisites

  • Install the Dynamo Kubernetes Platform with the nvidia.com/v1beta1 CRD and the GPUs shown above.
  • The manifests use Dynamo 1.6.0 and NVIDIA’s Beam runtime images: nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0-beam-1.0-501b-dev.1 or nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0-beam-1.0-501b-dev.1.
  • Prepare the shared-model-cache PVC and Hugging Face checkpoints using the model-staging instructions. Configure storage, registry access, and GPU placement for your cluster.

Stage the FP8 checkpoint for H200 or the NVFP4 checkpoint for B200, B300, GB200, and GB300, plus the DFlash2 drafter for every target.

Provide RDMA device access and a working NIXL/UCX path between the prefill and decode workers. Set the device-plugin resources and UCX_NET_DEVICES for your cluster, and allow the runtime’s transport ports between workers. SGLang uses bootstrap port 8998; vLLM uses NIXL side-channel port 5600 at each worker’s pod IP. Both roles must use the same checkpoint, KV-cache format, and DFlash2 settings.

Deploy

From the repository root, set the profile for your selected target:

export PROFILE=sglang/agg-h200
export PROFILE=sglang/disagg-h200
export PROFILE=sglang/agg-b200-nvfp4
export PROFILE=sglang/disagg-b200-nvfp4
export PROFILE=sglang/agg-b300-nvfp4
export PROFILE=sglang/disagg-b300-nvfp4
export PROFILE=sglang/agg-gb200-nvfp4
export PROFILE=sglang/disagg-gb200-nvfp4
export PROFILE=sglang/agg-gb300-nvfp4
export PROFILE=sglang/disagg-gb300-nvfp4
export PROFILE=vllm/agg-h200
export PROFILE=vllm/disagg-h200
export PROFILE=vllm/agg-b200-nvfp4
export PROFILE=vllm/disagg-b200-nvfp4
export PROFILE=vllm/agg-b300-nvfp4
export PROFILE=vllm/disagg-b300-nvfp4
export PROFILE=vllm/agg-gb200-nvfp4
export PROFILE=vllm/disagg-gb200-nvfp4
export PROFILE=vllm/agg-gb300-nvfp4
export PROFILE=vllm/disagg-gb300-nvfp4

Before deploying, follow the model-staging instructions for your selected PROFILE. Use the namespace containing that staged model cache. Use your cluster overlay for GPU and CPU-architecture placement when nodes differ. On a cluster with matching hardware, apply the portable manifest below:

export NAMESPACE=beam
PROFILE_TARGET=${PROFILE#*/}
PROFILE_TARGET=${PROFILE_TARGET%-nvfp4}
export DEPLOYMENT="beam-${PROFILE%%/*}-${PROFILE_TARGET#*-}-${PROFILE_TARGET%%-*}"
kubectl apply -n "$NAMESPACE" -f "recipes/beam-1.0-501b/$PROFILE/deploy.yaml"
kubectl wait -n "$NAMESPACE" --for=condition=Ready "dgd/$DEPLOYMENT" --timeout=7200s
kubectl port-forward -n "$NAMESPACE" "svc/$DEPLOYMENT-frontend" 8000:8000

Smoke Test

In another terminal, send a streaming request through the Dynamo frontend:

curl --fail-with-body -N http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Beam-1.0-501B-A23B","messages":[{"role":"user","content":"What is 2 + 2?"}],"max_tokens":256,"temperature":0,"stream":true}'

Configurations

SGLang and vLLM share these defaults and deployment layouts. Blackwell includes B200, B300, GB200, and GB300; P/D means prefill/decode.

Shared settingDefault
ContextUp to 1,048,576 tokens
KV cache dtypefp8_e4m3
KV page / block size64 tokens
DFlash2Enabled, block length 8
RoutingKV-aware
SettingH200 aggregatedH200 disaggregatedBlackwell aggregatedBlackwell disaggregated
Weight precisionFP8FP8NVFP4NVFP4
TP per worker8844
Worker replicas11 P + 1 D21 P + 1 D
Total GPUs81688
KV transfer—NIXL—NIXL

The selected deploy.yaml contains the attention and MoE backends, batching, cache, and memory settings. Measured performance results are not included.

Browse the Feature Benchmarks for KV-routing, disaggregation, and speculative-decoding studies.

Source

Beam-1.0-501B-A23B recipe and manifests