Knowledge BaseConceptsSimulation

Benchmark Planner Decisions with DynoSim

Compare topology and scaling decisions against a saved trace
View as Markdown

Run the Dynamo Planner inside an offline DynoSim prediction to compare aggregated and disaggregated topologies, service-level objective (SLO) targets, and worker startup delays without a live cluster.

The production Planner scales Kubernetes or Global Planner deployments; it does not autoscale a local deployment. aisimulate predict --stack dynamo runs locally but evaluates Planner decisions for the deployment described by its YAML input.

For the general workflow, see Run a DynoSim Simulation. For Planner field types and defaults, see the Planner Configuration reference. For the simulation adapter, see DynoSim Architecture.

Prerequisites

Build the Rust runtime bindings and install Dynamo from the repository root:

$source .venv/bin/activate
$maturin develop --release -m lib/bindings/python/Cargo.toml
$uv pip install -e .

Use a release build because repeated simulation runs are CPU-bound.

1

Run the aggregated baseline

Download the FAST’25 tool-agent trace:

$mkdir -p traces/mooncake-fast25
$curl -sL \
>  https://raw.githubusercontent.com/kvcache-ai/Mooncake/refs/heads/main/FAST25-release/traces/toolagent_trace.jsonl \
>  -o traces/mooncake-fast25/toolagent_trace.jsonl

Save this configuration as planner-aggregated.yaml:

1traffic:
2  source:
3    type: trace
4    format: mooncake
5    paths: [traces/mooncake-fast25/toolagent_trace.jsonl]
6    block_size: 512
7  load: {type: trace_timestamps, speedup: 1.0}
8  stop: {max_virtual_time_seconds: 3600}
9engine:
10  mode: aggregated
11  model: nvidia/Llama-3.1-8B-Instruct-FP8
12  hardware: h200_sxm
13  backend: vllm
14  context_length: max
15  workers:
16    aggregated:
17      parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
18      scheduler: {max_batched_tokens: 8192, max_sequences: 256}
19      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: default, memory_fraction: 0.9}}
20      timing: {type: default}
21      startup_seconds: 0
22router:
23  policy: round_robin
24  prefill_load_model: {type: none}
25planner:
26  policy: enabled
27  target: sla
28  enable_throughput_scaling: true
29  enable_load_scaling: true
30  throughput_adjustment_interval_seconds: 300
31  load_adjustment_interval_seconds: 10
32  load_scaling_down_sensitivity: 80
33  load_min_observations: 5
34  max_num_gpus: 2
35  min_workers: 1
36evaluation:
37  sla: {ttft_ms: 1500, itl_ms: 50}

Run the prediction:

$aisimulate predict \
>  --stack dynamo \
>  --config planner-aggregated.yaml \
>  --output-dir planner-reports/aggregated

The command prints the summary and writes Planner decisions and metrics to planner-reports/aggregated/prediction.json.

2

Run the disaggregated comparison

Copy the baseline, then replace its engine mapping with separate prefill and decode roles:

1engine:
2  mode: disaggregated
3  model: nvidia/Llama-3.1-8B-Instruct-FP8
4  hardware: h200_sxm
5  backend: vllm
6  context_length: max
7  kv_transfer: {bytes_per_token: auto, bandwidth_gb_per_second: 400, timing_mode: destination_missing}
8  workers:
9    prefill:
10      parallelism: {replicas: 1, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
11      scheduler: {max_batched_tokens: 8192, max_sequences: 64}
12      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: default, memory_fraction: 0.9}}
13      timing: {type: default}
14      startup_seconds: 0
15    decode:
16      parallelism: {replicas: 1, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
17      scheduler: {max_batched_tokens: 8192, max_sequences: 256}
18      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: default, memory_fraction: 0.9}}
19      timing: {type: default}
20      startup_seconds: 0

Also replace the planner mapping with role-specific minimums:

1planner:
2  policy: enabled
3  target: sla
4  enable_throughput_scaling: true
5  enable_load_scaling: true
6  throughput_adjustment_interval_seconds: 300
7  load_adjustment_interval_seconds: 10
8  load_scaling_down_sensitivity: 80
9  load_min_observations: 5
10  max_num_gpus: 2
11  min_workers: 1
12  prefill_min_workers: 1
13  decode_min_workers: 1

Save the result as planner-disaggregated.yaml, then run it:

$aisimulate predict \
>  --stack dynamo \
>  --config planner-disaggregated.yaml \
>  --output-dir planner-reports/disaggregated

Compare request metrics, scaling events, and cumulative GPU time between the two prediction.json files.

3

Sweep aggregated startup time

The startup_seconds field is concrete rather than a recommendation search dimension. Run the same prediction with a sequence of CLI overrides:

$for startup_seconds in $(seq 0 10 300); do
$  output_dir=$(printf "planner-reports/startup-%03d" "$startup_seconds")
$  aisimulate predict \
>    --stack dynamo \
>    --config planner-aggregated.yaml \
>    --set "engine.workers.aggregated.startup_seconds=$startup_seconds" \
>    --output-dir "$output_dir" \
>    --overwrite
$done

Read each prediction.json to compare TTFT, ITL, scaling events, and cumulative GPU time as startup delay increases.