> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Beam-1.0-501B-A23B

Deploy Beam-1.0-501B-A23B with NVIDIA Dynamo on H200, B200, B300, GB200, or GB300.
Choose your framework, GPU, and serving topology; the summary and commands
update to match. SGLang and vLLM provide aggregated and disaggregated serving
on all five GPU types, with KV-aware routing and NIXL prefill/decode transfer.
All profiles enable DFlash2 with block length 8 and FP8 KV cache.
H200 uses FP8 weights; B200, B300, GB200, and GB300 use NVFP4 weights.

Choose your deployment target

Framework

SGLang

<input type="radio" id="recipe-framework-vllm" name="recipe-framework" value="vllm" />

vLLM

GPU

H200

<input type="radio" id="recipe-sku-b200" name="recipe-sku" value="b200" />

B200

<input type="radio" id="recipe-sku-b300" name="recipe-sku" value="b300" />

B300

<input type="radio" id="recipe-sku-gb200" name="recipe-sku" value="gb200" />

GB200

<input type="radio" id="recipe-sku-gb300" name="recipe-sku" value="gb300" />

GB300

Topology

Aggregated

<input type="radio" id="recipe-variant-disagg" name="recipe-variant" value="disagg" />

Disaggregated

**Checkpoint** [reflection/Beam-1.0-501B-A23B-FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) **Precision** FP8 + FP8 KV **GPUs** 8x H200, one worker **Parallelism** 1 × TP8 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) **Precision** FP8 + FP8 KV **GPUs** 16x H200, one prefill + one decode **Parallelism** 1P1D, TP8 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B200, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B200, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B300, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B300, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB200, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB200, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB300, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB300, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) **Precision** FP8 + FP8 KV **GPUs** 8x H200, one worker **Parallelism** 1 × TP8 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) **Precision** FP8 + FP8 KV **GPUs** 16x H200, one prefill + one decode **Parallelism** 1P1D, TP8 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B200, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B200, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B300, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x B300, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB200, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB200, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB300, two workers **Parallelism** 2 × TP4 **Routing** KV-aware **Context** Up to 1,048,576 tokens

**Checkpoint** [reflection/Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) **Precision** NVFP4 + FP8 KV **GPUs** 8x GB300, one prefill + one decode **Parallelism** 1P1D, TP4 per role **Routing** KV-aware, NIXL KV transfer **Context** Up to 1,048,576 tokens

## Prerequisites

* Install the [Dynamo Kubernetes Platform](/dynamo/dev/kubernetes/getting-started/quickstart)
  with the `nvidia.com/v1beta1` CRD and the GPUs shown above.
* The manifests use Dynamo 1.6.0 and NVIDIA's Beam runtime images:
  `nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0-beam-1.0-501b-dev.1` or
  `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0-beam-1.0-501b-dev.1`.
* Prepare the `shared-model-cache` PVC and Hugging Face checkpoints using the
  [model-staging instructions](https://github.com/ai-dynamo/dynamo/blob/main/recipes/beam-1.0-501b/README.md#2-stage-the-model-and-drafter).
  Configure storage, registry access, and GPU placement for your cluster.

Stage the [FP8 checkpoint](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8)
for H200 or the [NVFP4 checkpoint](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4)
for B200, B300, GB200, and GB300, plus the
[DFlash2 drafter](https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2) for every target.

Provide RDMA device access and a working NIXL/UCX path between the prefill and
decode workers. Set the device-plugin resources and `UCX_NET_DEVICES` for your
cluster, and allow the runtime's transport ports between workers. SGLang uses
bootstrap port `8998`; vLLM uses NIXL side-channel port `5600` at each worker's
pod IP. Both roles must use the same checkpoint, KV-cache format, and DFlash2 settings.

## Deploy

From the repository root, set the profile for your selected target:

```bash
export PROFILE=sglang/agg-h200
```

```bash
export PROFILE=sglang/disagg-h200
```

```bash
export PROFILE=sglang/agg-b200-nvfp4
```

```bash
export PROFILE=sglang/disagg-b200-nvfp4
```

```bash
export PROFILE=sglang/agg-b300-nvfp4
```

```bash
export PROFILE=sglang/disagg-b300-nvfp4
```

```bash
export PROFILE=sglang/agg-gb200-nvfp4
```

```bash
export PROFILE=sglang/disagg-gb200-nvfp4
```

```bash
export PROFILE=sglang/agg-gb300-nvfp4
```

```bash
export PROFILE=sglang/disagg-gb300-nvfp4
```

```bash
export PROFILE=vllm/agg-h200
```

```bash
export PROFILE=vllm/disagg-h200
```

```bash
export PROFILE=vllm/agg-b200-nvfp4
```

```bash
export PROFILE=vllm/disagg-b200-nvfp4
```

```bash
export PROFILE=vllm/agg-b300-nvfp4
```

```bash
export PROFILE=vllm/disagg-b300-nvfp4
```

```bash
export PROFILE=vllm/agg-gb200-nvfp4
```

```bash
export PROFILE=vllm/disagg-gb200-nvfp4
```

```bash
export PROFILE=vllm/agg-gb300-nvfp4
```

```bash
export PROFILE=vllm/disagg-gb300-nvfp4
```

Before deploying, follow the
[model-staging instructions](https://github.com/ai-dynamo/dynamo/blob/main/recipes/beam-1.0-501b/README.md#2-stage-the-model-and-drafter)
for your selected `PROFILE`. Use the namespace containing that staged model cache.
Use your cluster overlay for GPU and CPU-architecture placement when nodes differ.
On a cluster with matching hardware, apply the portable manifest below:

```bash
export NAMESPACE=beam
PROFILE_TARGET=${PROFILE#*/}
PROFILE_TARGET=${PROFILE_TARGET%-nvfp4}
export DEPLOYMENT="beam-${PROFILE%%/*}-${PROFILE_TARGET#*-}-${PROFILE_TARGET%%-*}"
kubectl apply -n "$NAMESPACE" -f "recipes/beam-1.0-501b/$PROFILE/deploy.yaml"
kubectl wait -n "$NAMESPACE" --for=condition=Ready "dgd/$DEPLOYMENT" --timeout=7200s
kubectl port-forward -n "$NAMESPACE" "svc/$DEPLOYMENT-frontend" 8000:8000
```

## Smoke Test

In another terminal, send a streaming request through the Dynamo frontend:

```bash
curl --fail-with-body -N http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Beam-1.0-501B-A23B","messages":[{"role":"user","content":"What is 2 + 2?"}],"max_tokens":256,"temperature":0,"stream":true}'
```

## Configurations

SGLang and vLLM share these defaults and deployment layouts. Blackwell includes
B200, B300, GB200, and GB300; P/D means prefill/decode.

| Shared setting       | Default                 |
| -------------------- | ----------------------- |
| Context              | Up to 1,048,576 tokens  |
| KV cache dtype       | `fp8_e4m3`              |
| KV page / block size | 64 tokens               |
| DFlash2              | Enabled, block length 8 |
| Routing              | KV-aware                |

| Setting          | H200 aggregated | H200 disaggregated | Blackwell aggregated | Blackwell disaggregated |
| ---------------- | --------------- | ------------------ | -------------------- | ----------------------- |
| Weight precision | FP8             | FP8                | NVFP4                | NVFP4                   |
| TP per worker    | 8               | 8                  | 4                    | 4                       |
| Worker replicas  | 1               | 1 P + 1 D          | 2                    | 1 P + 1 D               |
| Total GPUs       | 8               | 16                 | 8                    | 8                       |
| KV transfer      | —               | NIXL               | —                    | NIXL                    |

The selected `deploy.yaml` contains the attention and MoE backends, batching,
cache, and memory settings. Measured performance results are not included.

## Related Feature Benchmarks

Browse the [Feature Benchmarks](/dynamo/dev/recipes/benchmarks)
for KV-routing, disaggregation, and speculative-decoding studies.

## Source

[Beam-1.0-501B-A23B recipe and manifests](https://github.com/ai-dynamo/dynamo/tree/main/recipes/beam-1.0-501b)