> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Spica Unrolled Samples

<Warning>
**Experimental.** Spica is intended for evaluation and feedback, not production capacity
planning. Its API, configuration schema, search results, and deployment output may change
without a standard deprecation period. Spica provides no SLA, accuracy, or configuration
optimality guarantees.
</Warning>

The [Spica search space](/dynamo/dev/knowledge-base/modular-components/ai-simulate/spica/search-space) defines what gets explored. A single *trial* picks
one concrete point in it: a **selection** (one value per searched knob, chosen by the
sampler), the one **`parallel_config`** the sampler selected or projected from the branch's valid pool, and
the **load-predictor winner** from the independent forecast sub-sweep.
`unroll_sample` (`aisimulate/src/aisimulate/spica/sample.py`) expands those into a **flat,
self-contained deployment-config dict** — the `config` of a `Candidate`
(`aisimulate/src/aisimulate/spica/config.py`). It
unrolls every composite knob (parallel shape, planner presets, load predictor) and folds
in the pinned scalars, so downstream DGD generation never re-decodes a preset string or a
structured config object.

The flat dict carries **only the knobs relevant to the chosen
`deployment_mode` / `router_mode` / planner state** — irrelevant dimensions are simply
absent (not set to a sentinel). What follows is what a sample contains and the three ways a
user pins/overrides what gets emitted.

## What the flat sample contains

`unroll_sample(search_space, selection, parallel_config, load_predictor)` builds the dict in
this order.

### Mode + backend

| key | source |
|---|---|
| `deployment_mode` | `selection["deployment_mode"]` (`agg` / `disagg`) |
| `backend` | `selection["backend"]` (`vllm` / `sglang` / `trtllm` enum values) |

### Pinned scalars (always present, never searched)

Folded straight off the `SearchSpace` (`_DEPLOYMENT_PINNED` + `_KV_MANAGER`):

- deployment/runtime context: `model_name`, `hardware_sku`, `context_length`,
  `startup_time`, `aic_nextn`
- planner limits: `gpu_budget`, `min_gpu_budget`, `min_endpoint`
- kv-manager offload: `num_g2_blocks`, `kv_bytes_per_token`, `bandwidth_g1_to_g2_gbps`,
  `bandwidth_g2_to_g1_gbps`, `offload_batch_size`

The GPU bounds and endpoint floor constrain candidate enumeration, and they also become
live runtime policy (`max_gpu_budget`, `min_gpu_budget`, `min_endpoint`) when the selected
candidate enables the planner. Keeping them in the flat sample makes replay and generated
deployment artifacts reproduce the search contract.

The current deployment generator applies these G2 host-offload fields to
aggregate and prefill workers. It intentionally omits them from disaggregated
decode workers, which already consume transferred KV and must not be scored with
a second local offload tier.

### Parallel fields (`_unroll_parallel`)

Unrolled from the chosen `parallel_config` via `_shape_fields`. The keys differ by mode:

- **agg** — flat: `tp`, `pp`, `attention_dp`, `moe_tp`, `moe_ep`, `strategy`, `replicas`,
  `used_gpus`.
- **disagg** — the same per-shape keys prefixed per role plus a per-role `replicas`:
  `prefill_tp` … `prefill_strategy`, `prefill_replicas`, `decode_tp` … `decode_strategy`,
  `decode_replicas`, and a single top-level `used_gpus`.

`strategy` (`tp` / `tep` / `dep` / `dtp`, with a `mixed` fallback the enumerator never emits) and
`used_gpus` (`gpus_per_worker × replicas`, summed across roles for disagg) are **derived
from the shape**, not settable. See [Search Space](/dynamo/dev/knowledge-base/modular-components/ai-simulate/spica/search-space).

### Engine knobs — the active branch only

The active mode's batching knobs come from the `selection` (`_*_SEARCHED`); the per-role
scalars are folded off the `SearchSpace` (`_*_PINNED`).

| branch | searched (from selection) | pinned (from search_space) |
|---|---|---|
| agg | `agg_max_num_batched_tokens`, `agg_max_num_seqs` | `agg_block_size`, `agg_gpu_memory_utilization`, `agg_enable_prefix_caching` |
| disagg | `prefill_max_num_batched_tokens`, `prefill_max_num_seqs`, `decode_max_num_batched_tokens`, `decode_max_num_seqs` | `prefill_block_size`, `prefill_gpu_memory_utilization`, `prefill_enable_prefix_caching`, `decode_block_size`, `decode_gpu_memory_utilization`, `decode_enable_prefix_caching` |

The inactive branch's engine keys are not emitted.

### Router knobs

`router_mode` is always present. The rest apply **only under `kv_router`**:

- `_KV_ROUTER_KNOBS` (from `selection`): `overlap_score_credit`, `prefill_load_scale`,
  `router_temperature`, and — only when present in the selection —
  `host_cache_hit_weight`, `disk_cache_hit_weight`. The two cache-hit weights are *gated
  out of the search* unless multi-tier offload is on (`num_g2_blocks > 0`; see
  `branch_knob_choices`), so the `if key in selection` guard simply skips them when absent.
- `_ROUTER_ADMISSION` (pinned, from `search_space`): `active_decode_blocks_threshold`,
  `active_prefill_tokens_threshold`, `active_prefill_tokens_threshold_frac`,
  `no_admission_control`. For searches including `kv_router`, these fields are currently
  reserved and rejected when set because Dynamo replay cannot model them yet.

Under `round_robin` none of these are emitted.

### Planner fields (`_unroll_planner`)

`planner_scaling_policy` decodes via `scaling_fields`
(`aisimulate/src/aisimulate/spica/planner.py`). (For a
non-goodput goal, `filter_scaling_policies` has already dropped any throughput-scaling
policy candidates before the sampler runs, so a selection can only carry a policy the goal
actually supports.) Always emitted:

- `planner_scaling_policy` — the **raw selection entry** (preset id *or* dict), kept for
  traceability
- `enable_throughput_scaling`, `enable_load_scaling`

**If both flags are false (`disabled`), unrolling stops here** — no intervals, fpm,
sensitivity, or predictor fields. Otherwise it also emits:

- `throughput_adjustment_interval_seconds`, `load_adjustment_interval_seconds`
- fpm fields from `planner_fpm_sampling` (`fpm_fields`): `max_num_fpm_samples`,
  `fpm_sample_bucket_size`
- sensitivity fields from `planner_load_sensitivity` (`load_sensitivity_fields`):
  `load_scaling_down_sensitivity`, `load_min_observations`

**Load-predictor fields are emitted only under throughput scaling** (`enable_throughput`
and a non-`None` `load_predictor`). The winner for *this policy's*
`throughput_adjustment_interval_seconds` is looked up in
`load_predictor.best_by_interval` and expanded by `predictor_fields`
(`aisimulate/src/aisimulate/spica/load_predictor_sweep.py`): always `load_predictor` (family) +
`load_predictor_log1p`; plus **only the family's own knobs** — `prophet_window_size` for
prophet, `kalman_q_level` / `kalman_q_trend` / `kalman_r` / `kalman_min_points` for kalman,
nothing extra for constant/arima. If the interval has no winner, no predictor fields are
added.

### Candidate-relative KV load

When `workload.kv_load_ratio` is set, a scalar ratio is pinned or a Pareto `[min, max]`
range becomes a continuous per-trial selection. After `unroll_sample`, the search evaluates
the selected backend, shape, and batching with AI Configurator and appends these derived fields:

- `kv_load_ratio` — the requested normalized load;
- `concurrency` — the concrete closed-loop in-flight cap;
- `kv_load_concurrency_capacity` — estimated concurrency at ratio `1`;
- `kv_load_capacity_tokens` — aggregate decode (disagg) or agg KV tokens;
- `prefill_kv_capacity_tokens` + `decode_kv_capacity_tokens` for disagg, or
  `agg_kv_capacity_tokens` for agg.

These fields are added by the search orchestration rather than `unroll_sample`, because
capacity depends on the fully selected candidate's backend, batching, shape, and replicas.
A fixed `workload.concurrency` is also copied into each candidate as `concurrency`.

## Overriding

A user controls what gets unrolled through **three mechanisms**, all in the
`search_space:` YAML. The first two are detailed in [Search Space](/dynamo/dev/knowledge-base/modular-components/ai-simulate/spica/search-space);
this section is about their effect on the *unrolled* sample.

### 1. Pin an atomic knob — single-element list

Any searched list knob with one entry pins it: that one value lands in every sample. E.g.
`router_mode: [round_robin]` forces `router_mode: round_robin` (and drops all kv-router /
admission keys); `agg_max_num_seqs: [512]` pins `agg_max_num_seqs: 512`. Multiple entries
leave it for the sampler.

### 2. Pin a composite's unrolled fields — a dict entry (escape hatch)

The four composite knobs (`planner_scaling_policy`, `planner_fpm_sampling`,
`planner_load_sensitivity`, `load_predictor_candidates`) accept a **dict** in place of a
preset id. The dict **pins the unrolled fields directly**, *replacing* the preset
expansion — there is no merge. `scaling_fields` / `fpm_fields` / `load_sensitivity_fields`
/ `predictor_fields` accept a dict identically to a preset, so the unrolled sample looks
the same as if a preset had produced it.

The dict's keys are drawn from that composite's allowed field names (`COMPOSITE_DICT_KEYS`;
unknown keys are rejected), and it must be **self-contained**: the three planner composites
require *all* their fields (allowed == required), while `load_predictor_candidates` needs at
least `load_predictor` and allows a wider set (`COMPOSITE_REQUIRED_KEYS` ⊊ `COMPOSITE_DICT_KEYS`)
— remaining family params default per family. Use this to pin a value no preset offers
(e.g. a 240 s throughput interval). Value legality is validated downstream by Dynamo's
`PlannerConfig`.

### 3. Pin the parallel shape — a `parallel_configs` dict entry

`parallel_configs` (empty by default — the shape is then enumerated by KV-feasibility)
takes a list of dicts. One entry **pins** the parallel shape that unrolls into the
`tp`/`pp`/`attention_dp`/`moe_tp`/`moe_ep`/`strategy`/`replicas`/`used_gpus` fields (or the
`prefill_*`/`decode_*` pair for disagg). **Pinning `parallel_configs` requires
`deployment_mode` to list exactly one mode.** Omitted shape dims default to `1`; `strategy`
and `used_gpus` remain derived. See [Search Space](/dynamo/dev/knowledge-base/modular-components/ai-simulate/spica/search-space) for the full shape
rules and legality checks.

## Never searched

These reach the sample only as **pinned scalars** (always one value, mechanism #1 does not
apply): `model_name`, `hardware_sku`, `aic_nextn`, the kv-manager offload fields, the
per-role `*_block_size` / `*_gpu_memory_utilization` / `*_enable_prefix_caching`, and the
admission-control fields. The parallel shape + replica count are *derived* (enumerated)
unless pinned via mechanism #3. See [Kinds of Knob](/dynamo/dev/knowledge-base/modular-components/ai-simulate/spica/search-space#kinds-of-knob).