Planner Examples

Planner-specific examples for advanced configuration and non-Kubernetes integrations. For DGDR manifests, see DGDR Templates. For the full configuration reference, see the Planner Guide.

Custom Load Predictors

Each YAML block in this section is a standalone PlannerConfig. Save the block as planner.yaml and pass it to python -m dynamo.planner --config planner.yaml. To use the same fields in a DGDR, nest them under spec.features.planner.

Warm-starting with Trace Data

Pre-load predictors with historical request patterns before live traffic:

1
optimization_target: sla
2
load_predictor: arima
3
load_predictor_warmup_trace: /data/trace.jsonl
4
load_predictor_log1p: true

The parser accepts per-request Mooncake JSONL records:

1
{"timestamp": 0, "input_length": 4096, "output_length": 512}

It also accepts dynamo.request.trace.v1 request_end records. The Planner groups requests into adjustment intervals and computes request count, average input sequence length (ISL), and average output sequence length (OSL).

Kalman Filter Tuning

For workloads with rapid changes, tune the Kalman filter:

1
optimization_target: sla
2
load_predictor: kalman
3
kalman_q_level: 2.0       # Higher = more responsive to level changes
4
kalman_q_trend: 0.5       # Higher = trend changes faster
5
kalman_r: 5.0             # Lower = trusts new measurements more
6
kalman_min_points: 3      # Fewer points before forecasting starts
7
load_predictor_log1p: true

Prophet for Seasonal Workloads

For workloads with daily/weekly patterns:

1
optimization_target: sla
2
load_predictor: prophet
3
prophet_window_size: 100   # Larger window for seasonal detection
4
load_predictor_log1p: true

Virtual Connector

For non-Kubernetes environments, use the VirtualConnector to communicate scaling decisions:

1
from dynamo._core import DistributedRuntime, VirtualConnectorClient
2
3
# Initialize client
4
client = VirtualConnectorClient(distributed_runtime, namespace)
5
6
# Main loop: watch for planner decisions and execute them
7
while True:
8
    # Block until the planner makes a new scaling decision
9
    await client.wait()
10
11
    # Read the decision
12
    decision = await client.get()
13
    print(f"Scale to: prefill={decision.num_prefill_workers}, "
14
          f"decode={decision.num_decode_workers}, "
15
          f"id={decision.decision_id}")
16
17
    # Execute scaling in your environment
18
    scale_prefill_workers(decision.num_prefill_workers)
19
    scale_decode_workers(decision.num_decode_workers)
20
21
    # Report completion
22
    await client.complete(decision)

See the VirtualConnector integration test for a complete example.

Related Documentation

• Planner Guide — Planner configuration reference
• DGDR Templates — DGDR YAML examples
• Profiler Guide — Profiling workflow