DynoSim Replay CLI Reference

aisimulate predict --stack dynamo evaluates one concrete workload and deployment configuration through the Dynamo simulation stack. AISimulate owns the traffic and engine schema. The ai-dynamo package supplies the Dynamo runner and the optional router and planner configuration adapters.

For an end-to-end workflow, see Run a DynoSim Simulation. To search configuration domains, see Sweep DynoSim Configurations.

Command

aisimulate predict --stack dynamo --config prediction.yaml

Configuration Sections

The YAML document uses these top-level sections:

Unknown fields are rejected. predict accepts only concrete values. Use aisimulate recommend for choices, range, or preset domains.

Dynamo Prediction Example

traffic:
  source:
    type: synthetic
    input_tokens: 1024
    output_tokens: 128
  load:
    type: constant_rate
    requests_per_second: 8
  stop:
    requests: 100
engine:
  mode: aggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: max
  workers:
    aggregated:
      parallelism:
        replicas: 2
        tensor: 1
        pipeline: 1
        attention_data: 1
        moe_tensor: 1
        moe_expert: 1
      scheduler:
        max_batched_tokens: 8192
        max_sequences: 256
      kv_cache:
        block_size: 64
        prefix_caching: true
        capacity: {type: default, memory_fraction: 0.9}
      timing: {type: default}
      startup_seconds: 0
router:
  policy: round_robin
  prefill_load_model: {type: none}
planner:
  policy: disabled
evaluation:
  sla: {ttft_ms: 500, itl_ms: 50}

Traffic Rules

traffic contains source, load, and stop mappings.

• Omitting the entire section creates 100 independent synthetic requests at concurrency 10, with 1,024 input tokens and 128 output tokens per request.
• source.type: synthetic creates independent requests with input_tokens and output_tokens.
• source.type: synthetic-session creates ordered multi-turn sessions.
• source.type: trace reads one or more paths. Supported formats include mooncake, mooncake-delta, agentic_mooncake, weka, applied_compute_agentic, and dynamo.
• load.type: concurrency keeps a fixed number of requests or sessions active.
• load.type: constant_rate schedules evenly spaced arrivals from requests_per_second or sessions_per_second.
• load.type: poisson uses the matching rate with exponential inter-arrival times and an optional seed, which defaults to 42.
• load.type: trace_timestamps preserves trace timing and accepts a positive speedup.
• stop.requests applies to independent requests. stop.sessions applies to session sources.
• stop.requests_per_load_unit and stop.sessions_per_load_unit derive the count from the concrete concurrency or arrival rate.
• stop.max_virtual_time_seconds is a soft virtual-time cutoff. Requests admitted before the cutoff may finish afterward.

For Dynamo request traces, omit traffic.source.block_size; Dynamo derives it from the source and rejects mixed block sizes across shards. For Weka corpora, AISimulate derives the block size from the source. An explicitly configured Weka block size is only an assertion against the published source metadata. Dynamo format accepts multiple trace shards. Weka accepts one file or directory; other formats accept exactly one file.

For Weka traces, traffic.source.nested_timestamp_basis controls how nested timestamps are interpreted:

• auto (the default) scans the entire corpus and selects one basis for all nested requests. If any replayable child timestamp precedes its parent marker, it selects relative; otherwise, it selects absolute. A corpus without replayable nested requests resolves to not_applicable. It does not choose a separate basis for each child or file.
• absolute uses timestamps directly and rejects a child timestamp before its parent marker, apart from the importer’s timestamp-rounding tolerance.
• relative adds the parent marker’s timestamp to each child timestamp.

Explicit absolute and relative overrides take precedence over inference. For example, a marker at 1 second with a child timestamp of 2 seconds is interpreted as 3 seconds with relative, and as 2 seconds with absolute. auto chooses 2 seconds for that corpus unless another child provides evidence for relative timestamps.

AISimulate’s Weka importer also changes two behaviors from the former Dynamo importer:

• A source request with out: 0 retains zero output tokens and runs as a prefill-only request. The former importer used max(1, out). Output-token counts, throughput, and timing metrics can therefore change for the same corpus.
• A missing or null api_time remains absent (None) in recorded_api_time_ms, preserving the distinction from an explicit zero. Dependency classification uses a zero-duration interval at the request timestamp when that duration is unknown. These requests were previously rejected. The fallback can affect inferred dependency edges and replay timing; it does not assert that the measured API duration was zero.

mooncake-delta requires aggregated engine mode. agentic_mooncake and weka require trace_timestamps. applied_compute_agentic requires concurrency load.

Canonical Python Engine Configuration

Pass an AISimulate engine launch mapping to extra_engine_args, prefill_engine_args, or decode_engine_args. Dynamo owns the optional dynamo runtime options; AISimulate owns the engine schema, topology, defaults, and validation.

engine_config = {
    "engine": {
        "backend": "sglang",
        "block_size": 16,
        "max_model_len": 4096,
        "sglang": {"chunked_prefill_size": 8192},
        "timing_model": {"type": "fixed", "prefill_ms": 2.0, "decode_ms": 1.0},
    },
    "dp_size": 1,
    "tensor_parallel_size": 1,
    "dynamo": {"enable_local_indexer": True},
}

The MockEngineArgs, SglangArgs, and TrtllmArgs Python classes have been removed. Replace constructor calls and flat engine JSON with mappings in this shape:

Use json.dumps and json.loads to save and restore configuration mappings. Keep num_gpu_blocks_is_explicit when saving a normalized configuration: false preserves automatic AISimulate capacity estimation. Providing engine.num_gpu_blocks without that marker selects an explicit capacity. Set engine.max_num_seqs or engine.max_num_batched_tokens to null (None in Python) for an unlimited scheduler limit; omit the field to use the engine default.

Typed agentic replay through the Dynamo API

AISimulate owns public Weka ingestion, validation, and lowering into the canonical ValidatedAgenticGraph. The lower-level dynamo.replay.run_trace_replay(...) API keeps trace_format="weka" as a compatibility adapter, but delegates graph construction directly to aisimulate-core before Dynamo composes its Router and Mocker runtime behavior.

Use weka_nested_timestamp_basis="absolute" or "relative" to override nested timestamp interpretation in run_trace_replay(...). Omitting it preserves the auto heuristic.

Set agentic_lanes to a positive integer to replay a fixed number of trajectories concurrently. Plays are stable-sorted and stride-assigned to lanes. A lane starts its next play only after its current play is quiescent, and lanes do not steal work. Agentic replay rejects replay_concurrency and Planner scaling. A Weka corpus may retain multiple source-model labels as provenance, but the current runtime projects every node onto the single configured execution_model. Per-node heterogeneous timing models are not yet supported.

from dynamo.replay import run_trace_replay
report = run_trace_replay(
    trace_files="traces/weka-agentx",
    trace_format="weka",
    execution_model="Qwen/Qwen3-32B-FP8",
    agentic_lanes=12,
    router_mode="kv_router",
    num_workers=4,
    extra_engine_args={"engine": {"backend": "vllm", "block_size": 64}},
)

Agentic reports retain the sorted source-model labels in agentic_graph.source_models and record the explicit projection policy and target in agentic_model_projection.

Weka reports also expose the resolved timestamp basis as the string weka_nested_timestamp_basis: absolute, relative, or not_applicable. For offline run_trace_replay(...), read report.summary["weka_nested_timestamp_basis"]; online replay includes the same field in its summary dictionary. The Dynamo runner includes it in report.metadata, including when raw-report capture is disabled. The requested basis remains in the workload configuration.

Agentic Mooncake v2 begins with a required versioned header:

{"schema":"dynamo.agentic_mooncake","version":2,"block_size":64,"hash_id_scope":"local","source":{"format":"weka","digest":"<corpus-digest>"}}

Each request row contains a globally unique request_id, nonempty play_id, session_id, model, exact input and output metadata, hash_ids, and not_before_ms. Optional source_play_ordinal and recorded_api_time_ms fields preserve source ordering and recorded timing evidence. The optional dependencies array contains typed incoming edges. Each edge identifies its predecessor, a dispatch or completion trigger, a nonnegative delay, and a sequence, spawn, join, or replay_barrier relation.

Agentic Mooncake v2 remains an optional materialized interchange format. Dynamo intentionally does not ship a Weka parser, lowering implementation, or Weka-to-v2 converter; those source semantics and any future materialization tooling belong to AISimulate.

AgentX host offload and lifecycle controls

Configure G2 through engine.workers.<role>.kv_cache.host_offload in CLI YAML, or engine.native_host_offload in the canonical engine configuration passed to the Python replay API. Both use the existing offline replay path for every trace format, including ordinary Dynamo, Mooncake, and Mooncake-delta traces. No connector configuration is required.

For AgentX graphs, G2 requires vLLM, one aggregated worker or one prefill plus one decode worker, attention DP=1 on every role, static worker pools, and no speculative decoding (engine.aic_nextn and engine.timing_model.config.speculation unset, engine.decode_speedup_ratio=1.0). Tensor parallelism may exceed one. These AgentX limits also apply when a Dynamo trace contains agent context but agentic_lanes is omitted. Ordinary traces retain their existing topology support; a standard Dynamo trace with two workers and attention DP=2 is covered by the G2 tests. G3 and online AgentX G2 replay are outside this support boundary.

Each role can omit offload or select dp_rank_local or cluster_shared. Shared roles must agree on layout, total capacity, and shared directional bandwidths. num_host_blocks is the capacity of one shared pool and is counted once. AISimulate’s public engine materializer derives the layout identity for CLI and ReplaySpec inputs; direct canonical configuration callers must supply a compatible kv_layout_id in engine.native_host_offload for a shared pool. Host-tier events remain distinct from GPU residency, while the KV router retains its configured host-tier weighting. P-to-D handoff uses GPU-resident hashes.

engine.kv_transfer_bytes_per_token describes P-to-D transfer size; engine.kv_cache_bytes_per_token describes the physical cache footprint used by G2. G2 requires a resolved, positive physical cache size. Set the transfer size separately when the geometries differ.

The Python run_trace_replay API also accepts agentic_snapshot (a mapping containing seed), agentic_warmup (default False), and agentic_profile (AISimulate’s duration, response-grace, and cancellation-drain options). These phase controls require offline replay. Snapshots require positive agentic_lanes; warmup and profiling require a snapshot. Profiling cannot be combined with max_sim_time_ms. Warmup retains G1/G2 state across the measurement barrier. Results retain agentic_snapshots, agentic_phases, and agentic_profile when those phases are enabled.

With per-request capture, inspect first_admission_g1_reused_input_tokens, first_admission_host_reused_input_tokens, and admission_history to distinguish G1 hits, completed G2 restores, and P/D admission. HBM-only reports may omit the host field. g2_domains reports shared capacity. This support is functional_only; it does not establish performance equivalence with a hardware offload recipe.

Engine and Adapter Rules

• engine.mode: aggregated requires engine.workers.aggregated.
• engine.mode: disaggregated requires engine.workers.prefill, engine.workers.decode, and engine.kv_transfer.
• aisimulate predict and aisimulate recommend currently support TensorRT-LLM only in aggregated mode. This is an offline simulation limitation; Dynamo runtime deployments support TensorRT-LLM disaggregated serving.
• Each parallelism mapping is concrete and contains replicas, tensor, pipeline, attention_data, moe_tensor, and moe_expert.
• engine.context_length defaults to max, which AISimulate resolves from the model’s Hugging Face configuration. Default KV block sizes are 64 for vLLM, 1 for SGLang, and 32 for TensorRT-LLM.
• Scheduler defaults are 8,192 batched tokens for every role and 256 sequences for aggregated and decode workers. Prefill workers default to one sequence.
• timing.type: default uses the AIConfigurator forward-pass model shipped in the aisimulate wheel. fixed requires both prefill_ms and decode_ms; polynomial selects the built-in polynomial model.
• router.policy: kv_router supports one or more routable workers. Set router.prefill_load_model.type to none or aic.
• planner.policy is disabled or enabled. Enabled settings use the production PlannerConfig defaults and normalization. The default target is throughput, which enables load scaling and disables throughput scaling. Throughput scaling with target: sla requires both TTFT and ITL thresholds.
• planner.max_num_gpus maps to Planner’s max_gpu_budget (default 8); concrete predictions also accept min_num_gpus for min_gpu_budget (default -1, no floor). Recommendations carry optimization.constraints.min_candidate_gpus into that runtime floor when set. Other candidate constraints govern the static search; planner.max_num_gpus governs runtime scaling.
• Omitted Planner adjustment intervals use production defaults. Explicit null uses the default only for an effectively disabled scaling mode. Exported enabled configurations contain concrete intervals, including the load interval used for performance-model updates.
• evaluation.sla accepts either e2e_ms alone or ttft_ms and itl_ms together. The two forms are mutually exclusive.

Overrides

--set accepts dot-separated, schema-valid paths. The field does not need to be present in the input YAML:

aisimulate predict \
  --stack dynamo \
  --config prediction.yaml \
  --set engine.workers.aggregated.parallelism.replicas=4 \
  --set router.policy=kv_router

An override cannot create an unknown field. Override a complete mapping when changing a tagged configuration shape, such as traffic.source.

Output

The output directory contains:

aisimulate-output/
├── prediction.json
└── requests.jsonl

prediction.json preserves the Dynamo runner report, including summary metrics and available Planner diagnostics. requests.jsonl is present only with --capture-per-request. --format changes standard output but not durable files.

The command exits with 0 on success, 1 for execution failure, 2 for CLI or configuration errors, and 130 when interrupted.