Sweep DynoSim Configurations | NVIDIA Dynamo Documentation

aisimulate recommend --stack dynamo searches simulated deployment configurations and writes each selected candidate as a concrete prediction YAML. The search runs offline on CPUs; the GPU count is a simulated constraint rather than a host requirement.

Use recommendation after a single DynoSim prediction works. For field and domain semantics, see the DynoSim Sweep Reference.

Prerequisites

Run from the repository root. Build the runtime bindings and install Dynamo, which installs the pinned AISimulate release:

$ .venv/bin/maturin develop --release -m lib/bindings/python/Cargo.toml
$ uv pip install -e .

Do not install the standalone aiconfigurator package. AISimulate includes the performance-model compatibility code used by the Dynamo stack.

Create a recommendation configuration

Save this configuration as /tmp/dynosim-recommend.yaml:

1 traffic:
2   source: {type: synthetic, input_tokens: 1024, output_tokens: 128}
3   load: {type: concurrency, concurrency: 8}
4   stop: {requests: 50}
5 engine:
6   mode: aggregated
7   model: Qwen/Qwen3-0.6B
8   hardware: h200_sxm
9   backend: vllm
10   context_length: 8192
11   workers:
12     aggregated:
13       parallelism:
14         preset:
15           - {replicas: 1, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
16           - {replicas: 2, tensor: 1, pipeline: 1, attention_data: 1, moe_tensor: 1, moe_expert: 1}
17       scheduler:
18         max_batched_tokens: {choices: [4096, 8192]}
19         max_sequences: 256
20       kv_cache: {block_size: 64, prefix_caching: true, capacity: {type: fixed, blocks: 32768}}
21       timing: {type: fixed, prefill_ms: 2, decode_ms: 0.5}
22 router:
23   policy: {choices: [round_robin, kv_router]}
24   prefill_load_model: {type: none}
25   overlap_score_credit: {choices: [0.5, 1.0]}
26   prefill_load_scale: {choices: [0.5, 1.0]}
27   temperature: {choices: [0.0, 0.2]}
28 optimization:
29   target: throughput
30   constraints: {max_candidate_gpus: 2}
31 optimizer:
32   algorithm: random
33   max_trials: 4
34   parallelism: 2
35   candidate_timeout_seconds: 30
36   seed: 42

Each parallelism preset is a complete mapping and becomes one categorical choice. Router and scheduler domains add independent search dimensions.

Run the recommendation

$ aisimulate recommend \
>   --stack dynamo \
>   --config /tmp/dynosim-recommend.yaml \
>   --output-dir /tmp/dynosim-recommendations

The command prints ranked candidates and writes concrete files under /tmp/dynosim-recommendations/recommendations/.

Predict the best candidate

Pass the highest-ranked recommendation directly to predict:

$ aisimulate predict \
>   --stack dynamo \
>   --config /tmp/dynosim-recommendations/recommendations/0001.yaml \
>   --output-dir /tmp/dynosim-best-prediction

Compare the prediction metrics with the baseline before deploying the candidate.

Search against a trace

Download the public FAST’25 tool-agent trace:

$ curl -sL \
>   https://raw.githubusercontent.com/kvcache-ai/Mooncake/refs/heads/main/FAST25-release/traces/toolagent_trace.jsonl \
>   -o /tmp/toolagent_trace.jsonl

Override the workload while retaining the engine and search domains:

$ aisimulate recommend \
>   --stack dynamo \
>   --config /tmp/dynosim-recommend.yaml \
>   --set 'traffic.source={type: trace, format: mooncake, paths: [/tmp/toolagent_trace.jsonl], block_size: 512}' \
>   --set 'traffic.load={type: trace_timestamps, speedup: 1.0}' \
>   --set 'traffic.stop={max_virtual_time_seconds: 3600}' \
>   --output-dir /tmp/dynosim-trace-recommendations

Use a shorter virtual-time cutoff or trial budget while iterating on large traces.

Customize the objective

Set optimization.target to throughput, throughput_per_gpu, throughput_per_user, goodput, goodput_per_gpu, ttft, e2e_latency, or pareto. Goodput targets require evaluation.sla. Pareto output contains the complete nondominated front rather than a scalar ranking.

Change one domain at a time. Use choices for categorical values, range for numeric domains, and complete preset mappings for correlated knobs such as parallelism.

Validate a candidate

A recommendation is a heuristic simulation result, not proof of optimality. Run the generated YAML through aisimulate predict, then deploy the candidate on its target hardware and benchmark it with AIPerf using either the Kubernetes workflow or the local workflow.