> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Run a DynoSim Simulation

A DynoSim prediction evaluates one workload against one simulated Dynamo configuration. AISimulate
drives the simulated engine cores directly without starting a frontend, registering workers, or
sending HTTP requests. It runs on CPUs and writes an AIPerf-style summary and JSON report.

For live simulation with registered Mocker workers, see
[Simulate a Local Deployment with Mocker](/dynamo/dev/knowledge-base/concepts/simulation/local-mocker). The former public Replay
online CLI remains unavailable, although the Python replay SDK retains online mode. For the
configuration and output contract, see the
[DynoSim Replay CLI Reference](/dynamo/dev/reference/components/dyno-sim-replay-cli-reference). For
the internal execution model, see
[DynoSim Architecture](/dynamo/dev/knowledge-base/concepts/simulation/dyno-sim-architecture).

## Prerequisites

Run the commands from the repository root. Build the runtime bindings and install Dynamo into the
project virtual environment:

```bash
.venv/bin/maturin develop --release -m lib/bindings/python/Cargo.toml
uv pip install -e .
```

Use a release build because simulation is CPU-bound.

#### Run a synthetic workload

Save this configuration as `/tmp/dynosim-synthetic.yaml`:

```yaml
traffic:
  source: {type: synthetic, input_tokens: 2048, output_tokens: 128}
  load: {type: concurrency, concurrency: 16}
  stop: {requests: 100}
engine:
  mode: aggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: 8192
  workers:
    aggregated:
      parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
      scheduler: {max_batched_tokens: 8192, max_sequences: 256}
      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
      timing: {type: polynomial}
router:
  policy: round_robin
  prefill_load_model: {type: none}
planner: {policy: disabled}
```

Run the prediction through the Dynamo stack:

```bash
aisimulate predict \
  --stack dynamo \
  --config /tmp/dynosim-synthetic.yaml \
  --output-dir /tmp/dynosim-synthetic
```

Confirm that all requests completed and that `/tmp/dynosim-synthetic/prediction.json` exists.

#### Add prefix reuse and multiple turns

Override the synthetic source with three-turn sessions. `--set` parses its right-hand side as YAML
and applies it after loading the file:

```bash
aisimulate predict \
  --stack dynamo \
  --config /tmp/dynosim-synthetic.yaml \
  --set 'traffic.source={type: synthetic-session, new_input_tokens_per_turn: 1024, output_tokens_per_turn: 128, session: {turns: 3, shared_prefix_ratio: 0.5, prefix_groups: 8, inter_turn_delay_ms: 250}}' \
  --set 'traffic.stop={sessions: 50}' \
  --output-dir /tmp/dynosim-prefix
```

Compare the prefix-cache reuse and latency metrics with the independent-request run.

#### Replay a saved trace

Download the public FAST'25 tool-agent trace:

```bash
curl -sL \
  https://raw.githubusercontent.com/kvcache-ai/Mooncake/refs/heads/main/FAST25-release/traces/toolagent_trace.jsonl \
  -o /tmp/toolagent_trace.jsonl
```

Override the workload mappings while keeping the engine configuration fixed:

```bash
aisimulate predict \
  --stack dynamo \
  --config /tmp/dynosim-synthetic.yaml \
  --set 'traffic.source={type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}' \
  --set 'traffic.load={type: trace_timestamps, speedup: 1.0}' \
  --set 'traffic.stop={max_virtual_time_seconds: 3600}' \
  --output-dir /tmp/dynosim-trace
```

`traffic.source.block_size` describes the trace hash granularity. The simulated KV-cache block size
remains `engine.workers.aggregated.kv_cache.block_size: 64`.

#### Compare routing modes

Run the same trace through four workers and the KV router:

```bash
aisimulate predict \
  --stack dynamo \
  --config /tmp/dynosim-synthetic.yaml \
  --set 'traffic.source={type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}' \
  --set 'traffic.load={type: trace_timestamps, speedup: 1.0}' \
  --set 'traffic.stop={max_virtual_time_seconds: 3600}' \
  --set router.policy=kv_router \
  --set engine.workers.aggregated.parallelism.replicas=4 \
  --output-dir /tmp/dynosim-kv-router
```

Compare `/tmp/dynosim-trace/prediction.json` with
`/tmp/dynosim-kv-router/prediction.json`. Review throughput, Time to First Token (TTFT), Inter-Token
Latency (ITL), and prefix-cache reuse.

#### Simulate disaggregated serving

Save a disaggregated configuration as `/tmp/dynosim-disaggregated.yaml`:

```yaml
traffic:
  source: {type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}
  load: {type: trace_timestamps, speedup: 1.0}
  stop: {max_virtual_time_seconds: 3600}
engine:
  mode: disaggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: 8192
  kv_transfer: {bytes_per_token: 131072, bandwidth_gb_per_second: 400, timing_mode: destination_missing}
  workers:
    prefill:
      parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
      scheduler: {max_batched_tokens: 8192, max_sequences: 64}
      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
      timing: {type: polynomial}
    decode:
      parallelism: {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
      scheduler: {max_batched_tokens: 8192, max_sequences: 256}
      kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
      timing: {type: polynomial}
router:
  policy: kv_router
  prefill_load_model: {type: none}
planner: {policy: disabled}
```

Run the disaggregated prediction:

```bash
aisimulate predict \
  --stack dynamo \
  --config /tmp/dynosim-disaggregated.yaml \
  --output-dir /tmp/dynosim-disaggregated
```

Compare the result with the aggregated baseline. To search topology and parallelism choices, use
[Sweep DynoSim Configurations](/dynamo/dev/knowledge-base/concepts/simulation/simulation-sweeps).

#### Validate the result

DynoSim models scheduler, KV-cache, routing, and timing behavior, but it does not replace a
real-hardware benchmark. Validate the candidate with AIPerf against either a
[Kubernetes deployment](/dynamo/dev/kubernetes/operations/benchmarking-with-ai-perf) or a
[local deployment](/dynamo/dev/cli/operations/benchmarking-with-ai-perf).