Run a DynoSim Simulation | NVIDIA Dynamo Documentation

A DynoSim prediction evaluates one workload against one simulated Dynamo configuration. AISimulate drives the simulated engine cores directly without starting a frontend, registering workers, or sending HTTP requests. It runs on CPUs and writes an AIPerf-style summary and JSON report.

For live simulation with registered Mocker workers, see Simulate a Local Deployment with Mocker. The former public Replay online CLI remains unavailable, although the Python replay SDK retains online mode. For the configuration and output contract, see the DynoSim Replay CLI Reference. For the internal execution model, see DynoSim Architecture.

Prerequisites

Run the commands from the repository root. Build the runtime bindings and install Dynamo into the project virtual environment:

$ .venv/bin/maturin develop --release -m lib/bindings/python/Cargo.toml
$ uv pip install -e .

Use a release build because simulation is CPU-bound.

Run a synthetic workload

Save this configuration as /tmp/dynosim-synthetic.yaml:

1 traffic:
2   source: {type: synthetic, input_tokens: 2048, output_tokens: 128}
3   load: {type: concurrency, concurrency: 16}
4   stop: {requests: 100}
5 engine:
6   mode: aggregated
7   model: meta-llama/Meta-Llama-3.1-8B-Instruct
8   hardware: h200_sxm
9   backend: vllm
10   context_length: 8192
11   workers:
12     aggregated:
13       parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
14       scheduler: {max_batched_tokens: 8192, max_sequences: 256}
15       kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
16       timing: {type: polynomial}
17 router:
18   policy: round_robin
19   prefill_load_model: {type: none}
20 planner: {policy: disabled}

Run the prediction through the Dynamo stack:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-synthetic.yaml \
>   --output-dir /tmp/dynosim-synthetic

Confirm that all requests completed and that /tmp/dynosim-synthetic/prediction.json exists.

Add prefix reuse and multiple turns

Override the synthetic source with three-turn sessions. --set parses its right-hand side as YAML and applies it after loading the file:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-synthetic.yaml \
>   --set 'traffic.source={type: synthetic-session, new_input_tokens_per_turn: 1024, output_tokens_per_turn: 128, session: {turns: 3, shared_prefix_ratio: 0.5, prefix_groups: 8, inter_turn_delay_ms: 250}}' \
>   --set 'traffic.stop={sessions: 50}' \
>   --output-dir /tmp/dynosim-prefix

Compare the prefix-cache reuse and latency metrics with the independent-request run.

Replay a saved trace

Download the public FAST’25 tool-agent trace:

$ curl -sL \
>   https://raw.githubusercontent.com/kvcache-ai/Mooncake/refs/heads/main/FAST25-release/traces/toolagent_trace.jsonl \
>   -o /tmp/toolagent_trace.jsonl

Override the workload mappings while keeping the engine configuration fixed:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-synthetic.yaml \
>   --set 'traffic.source={type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}' \
>   --set 'traffic.load={type: trace_timestamps, speedup: 1.0}' \
>   --set 'traffic.stop={max_virtual_time_seconds: 3600}' \
>   --output-dir /tmp/dynosim-trace

traffic.source.block_size describes the trace hash granularity. The simulated KV-cache block size remains engine.workers.aggregated.kv_cache.block_size: 64.

Compare routing modes

Run the same trace through four workers and the KV router:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-synthetic.yaml \
>   --set 'traffic.source={type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}' \
>   --set 'traffic.load={type: trace_timestamps, speedup: 1.0}' \
>   --set 'traffic.stop={max_virtual_time_seconds: 3600}' \
>   --set router.policy=kv_router \
>   --set engine.workers.aggregated.parallelism.replicas=4 \
>   --output-dir /tmp/dynosim-kv-router

Compare /tmp/dynosim-trace/prediction.json with /tmp/dynosim-kv-router/prediction.json. Review throughput, Time to First Token (TTFT), Inter-Token Latency (ITL), and prefix-cache reuse.

Simulate disaggregated serving

Save a disaggregated configuration as /tmp/dynosim-disaggregated.yaml:

1 traffic:
2   source: {type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}
3   load: {type: trace_timestamps, speedup: 1.0}
4   stop: {max_virtual_time_seconds: 3600}
5 engine:
6   mode: disaggregated
7   model: meta-llama/Meta-Llama-3.1-8B-Instruct
8   hardware: h200_sxm
9   backend: vllm
10   context_length: 8192
11   kv_transfer: {bytes_per_token: 131072, bandwidth_gb_per_second: 400, timing_mode: destination_missing}
12   workers:
13     prefill:
14       parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
15       scheduler: {max_batched_tokens: 8192, max_sequences: 64}
16       kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
17       timing: {type: polynomial}
18     decode:
19       parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
20       scheduler: {max_batched_tokens: 8192, max_sequences: 256}
21       kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
22       timing: {type: polynomial}
23 router:
24   policy: kv_router
25   prefill_load_model: {type: none}
26 planner: {policy: disabled}

Run the disaggregated prediction:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-disaggregated.yaml \
>   --output-dir /tmp/dynosim-disaggregated

Compare the result with the aggregated baseline. To search topology and parallelism choices, use Sweep DynoSim Configurations.

Validate the result

DynoSim models scheduler, KV-cache, routing, and timing behavior, but it does not replace a real-hardware benchmark. Validate the candidate with AIPerf against either a Kubernetes deployment or a local deployment.