> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# dynamo.frontend

`dynamo.frontend` publishes 12 classes and 42 functions. Source: [`components/src/dynamo/frontend/__init__.py`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/__init__.py)

#### EngineFactory (class)

No summary available.

```python
from dynamo.frontend.vllm_processor import EngineFactory
```

```python
EngineFactory(config: FrontendConfig, flags: Namespace)
```

[`components/src/dynamo/frontend/vllm_processor.py#L1480`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L1480)

**Public methods**

#### **init**

```python
__init__(config: FrontendConfig, flags: Namespace)
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L1481)

#### chat\_engine\_factory

```python
chat_engine_factory(instance_id: ModelCardInstanceId, mdc: ModelDeploymentCard, routed_engine: RoutedEngine) -> PythonAsyncEngine
```

Called by Rust when a model is discovered.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L1500)

#### FrontendArgGroup (class)

Frontend configuration parameters.

```python
from dynamo.frontend.frontend_args import FrontendArgGroup
```

[`components/src/dynamo/frontend/frontend_args.py#L361`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L361)

**Public methods**

#### add\_arguments

```python
add_arguments(parser) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L364)

#### FrontendConfig (class)

Configuration for the Dynamo frontend.

```python
from dynamo.frontend.frontend_args import FrontendConfig
```

[`components/src/dynamo/frontend/frontend_args.py#L172`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L172)

**Public methods**

#### validate

```python
validate() -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L227)

#### PreprocessError (class)

Raised by preprocess workers for user-facing errors (e.g., n!=1).

```python
from dynamo.frontend.utils import PreprocessError
```

```python
PreprocessError(message: str)
```

Carries a plain message because the worker→main-process boundary
pickles the exception; the main process re-raises a Dynamo-typed
exception so PyO3 can route it through the proper backend-error path.

[`components/src/dynamo/frontend/utils.py#L131`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L131)

**Public methods**

#### **init**

```python
__init__(message: str)
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L139)

#### PreprocessResult (class)

No summary available.

```python
from dynamo.frontend.prepost import PreprocessResult
```

```python
PreprocessResult(request_for_sampling: ChatCompletionRequest, tool_parser: ToolParser | None, chat_template_kwargs: dict[str, Any], engine_prompt: dict[str, Any], prompt_token_ids: list[int], guided_decoding: dict[str, Any] | None = None, uses_dynamo_json_tool_call_fallback: bool = False) -> None
```

[`components/src/dynamo/frontend/prepost.py#L71`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L71)

**Public methods**

#### **init**

```python
__init__(request_for_sampling: ChatCompletionRequest, tool_parser: ToolParser | None, chat_template_kwargs: dict[str, Any], engine_prompt: dict[str, Any], prompt_token_ids: list[int], guided_decoding: dict[str, Any] | None = None, uses_dynamo_json_tool_call_fallback: bool = False) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py)

#### SglangEngineFactory (class)

No summary available.

```python
from dynamo.frontend.sglang_processor import SglangEngineFactory
```

```python
SglangEngineFactory(config: FrontendConfig, debug_perf: bool = False, tool_call_parser_name: str | None = None, reasoning_parser_name: str | None = None, chat_template: str | None = None)
```

[`components/src/dynamo/frontend/sglang_processor.py#L1022`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L1022)

**Public methods**

#### **init**

```python
__init__(config: FrontendConfig, debug_perf: bool = False, tool_call_parser_name: str | None = None, reasoning_parser_name: str | None = None, chat_template: str | None = None)
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L1023)

#### chat\_engine\_factory

```python
chat_engine_factory(instance_id: ModelCardInstanceId, mdc: ModelDeploymentCard, routed_engine: RoutedEngine) -> PythonAsyncEngine
```

Called by Rust when a model is discovered.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L1050)

#### SglangPreprocessResult (class)

Result of SGLang preprocessing.

```python
from dynamo.frontend.sglang_prepost import SglangPreprocessResult
```

```python
SglangPreprocessResult(prompt_token_ids: list[int], tool_call_parser: ToolCallParserType | None, reasoning_parser: ReasoningParser | None, guided_decoding: dict[str, Any] | None, request: dict[str, Any], force_reasoning: bool = False, named_zero_arg_tool: str | None = None) -> None
```

[`components/src/dynamo/frontend/sglang_prepost.py#L53`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L53)

**Public methods**

#### **init**

```python
__init__(prompt_token_ids: list[int], tool_call_parser: ToolCallParserType | None, reasoning_parser: ReasoningParser | None, guided_decoding: dict[str, Any] | None, request: dict[str, Any], force_reasoning: bool = False, named_zero_arg_tool: str | None = None) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py)

#### SglangPreprocessWorkerResult (class)

Picklable return value from the SGLang preprocess worker.

```python
from dynamo.frontend.sglang_processor import SglangPreprocessWorkerResult
```

```python
SglangPreprocessWorkerResult(prompt_token_ids: list[int], dynamo_preproc: dict[str, Any], request: dict[str, Any], force_reasoning: bool = False, named_zero_arg_tool: str | None = None, effective_reasoning_parser_name: str | None = None) -> None
```

[`components/src/dynamo/frontend/sglang_processor.py#L305`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L305)

**Public methods**

#### **init**

```python
__init__(prompt_token_ids: list[int], dynamo_preproc: dict[str, Any], request: dict[str, Any], force_reasoning: bool = False, named_zero_arg_tool: str | None = None, effective_reasoning_parser_name: str | None = None) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py)

#### SglangProcessor (class)

No summary available.

```python
from dynamo.frontend.sglang_processor import SglangProcessor
```

```python
SglangProcessor(tokenizer, routed_engine: RoutedEngine, tool_call_parser_name: str | None, reasoning_parser_name: str | None, eos_token_ids: list[int] | None, debug_perf: bool = False, preprocess_pool: ProcessPoolExecutor | None = None, preprocess_workers: int = 0, stream_interval: int = 1, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto')
```

[`components/src/dynamo/frontend/sglang_processor.py#L529`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L529)

**Public methods**

#### **init**

```python
__init__(tokenizer, routed_engine: RoutedEngine, tool_call_parser_name: str | None, reasoning_parser_name: str | None, eos_token_ids: list[int] | None, debug_perf: bool = False, preprocess_pool: ProcessPoolExecutor | None = None, preprocess_workers: int = 0, stream_interval: int = 1, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto')
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L530)

#### generator

```python
generator(request: dict[str, Any], context: Any | None = None) -> AsyncGenerator[dict[str, Any], None]
```

Main entry point: preprocess, route, post-process a chat request.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_processor.py#L580)

#### SglangStreamingPostProcessor (class)

Streaming post-processor using SGLang parsers and HF tokenizer detokenization.

```python
from dynamo.frontend.sglang_prepost import SglangStreamingPostProcessor
```

```python
SglangStreamingPostProcessor(*, tokenizer, tool_call_parser: ToolCallParserType | None, reasoning_parser: ReasoningParser | None, history_tool_calls_count: int = 0, sglang_tools: list[SglangTool] | None = None, tool_call_parser_name: str | None = None, named_zero_arg_tool: str | None = None, eos_token_ids: list[int] | None = None, prompt_token_ids: list[int] | None = None, stop_strings: set[str] | None = None, stop_token_ids: set[int] | None = None, skip_special_tokens: bool | None = None, guided_json_is_content: bool = False) -> None
```

Handles:

* Incremental detokenization across tokenizer-safe boundaries
* Reasoning content extraction via SGLang ReasoningParser
* Tool call parsing via SGLang FunctionCallParser or JsonArrayParser

[`components/src/dynamo/frontend/sglang_prepost.py#L1161`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L1161)

**Public methods**

#### **init**

```python
__init__(*, tokenizer, tool_call_parser: ToolCallParserType | None, reasoning_parser: ReasoningParser | None, history_tool_calls_count: int = 0, sglang_tools: list[SglangTool] | None = None, tool_call_parser_name: str | None = None, named_zero_arg_tool: str | None = None, eos_token_ids: list[int] | None = None, prompt_token_ids: list[int] | None = None, stop_strings: set[str] | None = None, stop_token_ids: set[int] | None = None, skip_special_tokens: bool | None = None, guided_json_is_content: bool = False) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L1170)

#### process\_output

```python
process_output(engine_response: dict[str, Any]) -> dict[str, Any] | None
```

Process a single engine response chunk into an OpenAI SSE choice dict.

**Parameters**

**`engine_response`** `dict[str, Any]`

Dict with `token_ids` and optional `finish_reason`.

---

**Returns**

* `dict[str, Any] | None` — OpenAI choice dict or `None` if nothing to emit yet.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L1615)

#### StreamingPostProcessor (class)

No summary available.

```python
from dynamo.frontend.prepost import StreamingPostProcessor
```

```python
StreamingPostProcessor(*, tokenizer: TokenizerLike, request_for_sampling: ChatCompletionRequest, sampling_params: SamplingParams, prompt_token_ids: Sequence[int], tool_parser: ToolParser | None, reasoning_parser_class: type[ReasoningParser] | None, chat_template_kwargs: dict[str, Any], model_config: ModelConfig | None = None, response_reasoning_ended: bool | None = None, stream_response: bool = True, uses_dynamo_json_tool_call_fallback: bool = False) -> None
```

[`components/src/dynamo/frontend/prepost.py#L936`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L936)

**Public methods**

#### **init**

```python
__init__(*, tokenizer: TokenizerLike, request_for_sampling: ChatCompletionRequest, sampling_params: SamplingParams, prompt_token_ids: Sequence[int], tool_parser: ToolParser | None, reasoning_parser_class: type[ReasoningParser] | None, chat_template_kwargs: dict[str, Any], model_config: ModelConfig | None = None, response_reasoning_ended: bool | None = None, stream_response: bool = True, uses_dynamo_json_tool_call_fallback: bool = False) -> None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L937)

#### process\_output

```python
process_output(output: Any) -> dict[str, Any] | None
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L1454)

#### VllmProcessor (class)

No summary available.

```python
from dynamo.frontend.vllm_processor import VllmProcessor
```

```python
VllmProcessor(tokenizer: TokenizerLike, input_processor: InputProcessor, output_processor: OutputProcessor, tool_parser_class: type[ToolParser] | None, reasoning_parser_class: type[ReasoningParser] | None, routed_engine: RoutedEngine, block_size: int = 16, enable_auto_tool_choice: bool = False, default_chat_template_kwargs: dict[str, Any] | None = None, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto', structural_tag_excludes_reasoning: bool = False)
```

[`components/src/dynamo/frontend/vllm_processor.py#L697`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L697)

**Public methods**

#### **init**

```python
__init__(tokenizer: TokenizerLike, input_processor: InputProcessor, output_processor: OutputProcessor, tool_parser_class: type[ToolParser] | None, reasoning_parser_class: type[ReasoningParser] | None, routed_engine: RoutedEngine, block_size: int = 16, enable_auto_tool_choice: bool = False, default_chat_template_kwargs: dict[str, Any] | None = None, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto', structural_tag_excludes_reasoning: bool = False)
```

No summary available.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L698)

#### generator

```python
generator(request: dict[str, Any], context: Any | None = None) -> AsyncGenerator[dict[str, Any], None]
```

Run a single request through the engine. Does pre and post processing on this machine, delegates model inference to a backend using the router.

[source](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L919)

#### apply\_default\_thinking\_mode\_to\_template\_kwargs (function)

Merge deployment thinking default unless the request already controls it.

```python
from dynamo.frontend.thinking import apply_default_thinking_mode_to_template_kwargs
```

```python
apply_default_thinking_mode_to_template_kwargs(chat_template_kwargs: dict[str, Any], default_thinking_mode: str | None, *, request_has_root_thinking: bool = False) -> dict[str, Any]
```

[`components/src/dynamo/frontend/thinking.py#L33`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/thinking.py#L33)

#### as\_error\_envelope (function)

Tag an error dict so the binding reads it as an error frame.

```python
from dynamo.frontend.utils import as_error_envelope
```

```python
as_error_envelope(error_payload: dict[str, Any]) -> dict[str, Any]
```

Without `_dynamo_annotated` the binding reads the dict as a completion
chunk and fails with ``missing field `id```. The client then gets a 500 and
never sees the message. The HTTP layer reads the text back off ``comment`.
See `depythonize\_annotated`in`lib/bindings/python/rust/engine.rs\`\`.

[`components/src/dynamo/frontend/utils.py#L230`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L230)

#### async\_main (function)

Main async entry point for the Dynamo frontend.

```python
from dynamo.frontend.main import async_main
```

```python
async_main()
```

Initializes the distributed runtime, configures routing, and starts
the HTTP server or interactive mode based on command-line arguments.

[`components/src/dynamo/frontend/main.py#L370`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L370)

#### backend\_invalid\_argument\_to\_http\_error (function)

Recover a worker's own HTTP status from a serialized backend error.

```python
from dynamo.frontend.utils import backend_invalid_argument_to_http_error
```

```python
backend_invalid_argument_to_http_error(exc: BaseException) -> HttpError | None
```

A worker that rejects a request -- an unsupported sampling parameter, an
unparseable grammar -- fails it with `{"message": ..., "code": 4xx}`.
Crossing the Rust boundary turns that into a plain Python exception whose
text is `BackendInvalidArgument: {json}`, so a generic `except
Exception` reports a 500 and discards both the status and the reason.

Rust's HTTP service already performs this recovery
(`extract_backend_error_if_present` in
`lib/llm/src/http/service/openai.rs`), but the Python chat processors do
not go through it, so they need their own. Mirror its status rules: honour
an explicit code, and otherwise fall back to 400 -- the prefix itself is
the discriminator proving the argument was invalid.

Returns None when `exc` is not that shape, so callers fall through to
their existing handling instead of inventing a status.

[`components/src/dynamo/frontend/utils.py#L245`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L245)

#### build\_response\_format\_guided\_decoding (function)

Build Dynamo guided decoding from OpenAI chat response\_format.

```python
from dynamo.frontend.sglang_prepost import build_response_format_guided_decoding
```

```python
build_response_format_guided_decoding(request: dict[str, Any]) -> dict[str, Any] | None
```

[`components/src/dynamo/frontend/sglang_prepost.py#L780`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L780)

#### build\_tool\_call\_guided\_decoding (function)

Build tool-call guidance through vLLM's configured tool parser.

```python
from dynamo.frontend.prepost import build_tool_call_guided_decoding
```

```python
build_tool_call_guided_decoding(request: ChatCompletionRequest, tool_parser: ToolParser | None, *, parser_guided_decoding: dict[str, Any] | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto', starts_in_reasoning: bool = False) -> dict[str, Any] | None
```

`starts_in_reasoning` is true when generation begins inside a reasoning
block the prompt opened and the engine does not hold the grammar back until
that block ends. The structural tag must then accept the reasoning text and
its end marker before the tool-call section.

[`components/src/dynamo/frontend/prepost.py#L270`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L270)

#### build\_tool\_call\_guided\_decoding (function)

Build native-SGLang-like tool call constraints for guided decoding.

```python
from dynamo.frontend.sglang_prepost import build_tool_call_guided_decoding
```

```python
build_tool_call_guided_decoding(request: dict[str, Any], *, tool_call_parser_name: str | None, tokenizer: Any | None = None, sglang_tools: list[SglangTool] | None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto') -> dict[str, Any] | None
```

[`components/src/dynamo/frontend/sglang_prepost.py#L637`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L637)

#### convert\_tools (function)

Convert OpenAI tool dicts to SGLang Tool objects.

```python
from dynamo.frontend.sglang_prepost import convert_tools
```

```python
convert_tools(tools: list[dict[str, Any]] | None, *, structural_tag_schema: str | None = None) -> list[SglangTool] | None
```

`structural_tag_schema` is set only for the guidance copy. Prompt and
parser copies retain SGLang's existing omitted-strict representation.

[`components/src/dynamo/frontend/sglang_prepost.py#L212`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L212)

#### create\_parsers (function)

Create tool call and reasoning parsers for a request.

```python
from dynamo.frontend.sglang_prepost import create_parsers
```

```python
create_parsers(request: dict[str, Any], *, tool_call_parser_name: str | None, reasoning_parser_name: str | None, sglang_tools: list[SglangTool] | None = None, force_reasoning: bool = False, guided_decoding: dict[str, Any] | None = None, tokenizer: Any | None = None) -> tuple[ToolCallParserType | None, ReasoningParser | None]
```

Shared by both the single-process preprocessing path and the pool path
(which must recreate non-picklable parsers in the main process).

If `sglang_tools` is provided, reuses them; otherwise converts from
the request's `tools` field.

Required and named choices use the model-specific `FunctionCallParser`
when the effective guidance is a structural tag. Their JSON fallback keeps
`JsonArrayParser`. Automatic choices use `FunctionCallParser`.

[`components/src/dynamo/frontend/sglang_prepost.py#L297`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L297)

#### detect\_force\_reasoning\_from\_template (function)

Return True if the chat template auto-opens a reasoning block.

```python
from dynamo.frontend.sglang_prepost import detect_force_reasoning_from_template
```

```python
detect_force_reasoning_from_template(chat_template: str | None) -> bool
```

Intended to be called once at processor startup with
`tokenizer.chat_template` and cached on the processor.

[`components/src/dynamo/frontend/sglang_prepost.py#L90`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L90)

#### effective\_tool\_strict (function)

Resolve request strictness for grammar construction.

```python
from dynamo.frontend.structural_tag_policy import effective_tool_strict
```

```python
effective_tool_strict(request_strict: bool | None, schema_mode: str) -> bool
```

In `auto` schema mode, omitted `strict` is schema-enforced and only an
explicit `strict: false` opts out. `strict` schema mode overrides that
request-level opt-out.

[`components/src/dynamo/frontend/structural_tag_policy.py#L95`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/structural_tag_policy.py#L95)

#### extract\_mm\_urls (function)

Extract media and vLLM processor-cache UUIDs from chat messages.

```python
from dynamo.frontend.utils import extract_mm_urls
```

```python
extract_mm_urls(messages: list[dict[str, Any]]) -> tuple[dict[str, list[dict[str, str]]] | None, dict[str, list[str | None]] | None]
```

URL-backed parts become `Url` variants. Image parts with no URL and an
opaque `uuid` become `UuidOnly` variants for vLLM's multimodal
processor cache. UUID lists preserve slot order for every media type

```
({"image_url": [{"Url": "https://..."}, {"UuidOnly": "image-1"}]},
 {"image_url": ["image-1", "image-1"]})

```

The UUID map is `None` when no user UUID is present. A media content part
with neither a URL nor UUID is rejected instead of being silently dropped.

[`components/src/dynamo/frontend/utils.py#L148`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L148)

#### graceful\_shutdown (function)

Handle graceful shutdown of the distributed runtime.

```python
from dynamo.frontend.main import graceful_shutdown
```

```python
graceful_shutdown(runtime: DistributedRuntime) -> None
```

**Parameters**

**`runtime`** `DistributedRuntime`

The DistributedRuntime instance to shut down.

---

[`components/src/dynamo/frontend/main.py#L505`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L505)

#### handle\_engine\_error (function)

Classify an invalid engine response and return an OpenAI-style error dict.

```python
from dynamo.frontend.utils import handle_engine_error
```

```python
handle_engine_error(engine_response: Any, request_id: str, logger: logging.Logger) -> dict[str, Any]
```

Called when engine\_response is None or missing 'token\_ids'.

[`components/src/dynamo/frontend/utils.py#L291`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L291)

#### legacy\_guided\_decoding (function)

Convert one legacy guided-decoding constraint and its modifier to a dict.

```python
from dynamo.frontend.utils import legacy_guided_decoding
```

```python
legacy_guided_decoding(request: dict[str, Any]) -> dict[str, Any] | None
```

[`components/src/dynamo/frontend/utils.py#L47`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L47)

#### load\_frontend\_route\_extensions (function)

Load trusted frontend route extensions.

```python
from dynamo.frontend.main import load_frontend_route_extensions
```

```python
load_frontend_route_extensions(extension_names: list[str]) -> list[FrontendRoute]
```

Each value is either a name registered under the `dynamo.frontend.routes`
entry-point group (preferred) or a direct `module:function` path.

[`components/src/dynamo/frontend/main.py#L217`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L217)

#### main (function)

Entry point for the Dynamo frontend CLI.

```python
from dynamo.frontend.main import main
```

```python
main() -> None
```

[`components/src/dynamo/frontend/main.py#L514`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L514)

#### make\_backend\_error (function)

Build an OpenAI-style error dict, guarding against None/missing message.

```python
from dynamo.frontend.utils import make_backend_error
```

```python
make_backend_error(engine_response: dict[str, Any]) -> dict[str, Any]
```

[`components/src/dynamo/frontend/utils.py#L208`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L208)

#### make\_internal\_error (function)

Build an OpenAI-style internal error dict with request-specific fallback.

```python
from dynamo.frontend.utils import make_internal_error
```

```python
make_internal_error(request_id: str, detail: str | None = None) -> dict[str, Any]
```

[`components/src/dynamo/frontend/utils.py#L219`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L219)

#### map\_finish\_reason (function)

No summary available.

```python
from dynamo.frontend.vllm_processor import map_finish_reason
```

```python
map_finish_reason(raw_reason: str | dict[str, Any] | None) -> FinishReason | None
```

[`components/src/dynamo/frontend/vllm_processor.py#L406`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/vllm_processor.py#L406)

#### named\_closed\_zero\_arg\_tool (function)

Return the named tool when its only valid argument value is `{}`.

```python
from dynamo.frontend.sglang_prepost import named_closed_zero_arg_tool
```

```python
named_closed_zero_arg_tool(request: dict[str, Any]) -> str | None
```

[`components/src/dynamo/frontend/sglang_prepost.py#L362`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L362)

#### nvext\_extra\_field\_requested (function)

Return whether a request opted into a response nvext field.

```python
from dynamo.frontend.utils import nvext_extra_field_requested
```

```python
nvext_extra_field_requested(request: dict[str, Any], field: str) -> bool
```

[`components/src/dynamo/frontend/utils.py#L117`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L117)

#### parse\_args (function)

Parse command-line arguments for the Dynamo frontend.

```python
from dynamo.frontend.main import parse_args
```

```python
parse_args() -> tuple[FrontendConfig, Optional[Namespace], Optional[Namespace]]
```

**Returns**

* `tuple[FrontendConfig, Optional[Namespace], Optional[Namespace]]` — Tuple of (FrontendConfig, vllm\_flags, sglang\_flags).

[`components/src/dynamo/frontend/main.py#L245`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L245)

#### preprocess\_chat\_request (function)

No summary available.

```python
from dynamo.frontend.prepost import preprocess_chat_request
```

```python
preprocess_chat_request(request: dict[str, Any] | ChatCompletionRequest, *, tokenizer: TokenizerLike, renderer: _Renderer, tool_parser_class: type[ToolParser] | None, reasoning_parser_class: type[ReasoningParser] | None = None, model_config: ModelConfig | None = None, exclude_tools_when_tool_choice_none: bool = True, enable_auto_tool_choice: bool = False, default_chat_template_kwargs: dict[str, Any] | None = None, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto', structural_tag_excludes_reasoning: bool = False) -> PreprocessResult
```

[`components/src/dynamo/frontend/prepost.py#L734`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/prepost.py#L734)

#### preprocess\_chat\_request (function)

Preprocess a chat request using SGLang tokenizer and parser APIs.

```python
from dynamo.frontend.sglang_prepost import preprocess_chat_request
```

```python
preprocess_chat_request(request: dict[str, Any], *, tokenizer, tool_call_parser_name: str | None, reasoning_parser_name: str | None, exclude_tools_when_tool_choice_none: bool = True, template_force_reasoning: bool = False, default_thinking_mode: str | None = None, structural_tag_mode: str = 'off', structural_tag_scope: str = 'auto', structural_tag_schema: str = 'auto') -> SglangPreprocessResult
```

`template_force_reasoning` is the static per-server flag derived from
the chat template (see `detect_force_reasoning_from_template`);
the effective per-request value combines it with the configured parser and
request-level thinking controls.

Synchronous -- suitable for both main-process and worker-process execution.

[`components/src/dynamo/frontend/sglang_prepost.py#L865`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L865)

#### random\_call\_id (function)

Generate a random tool call ID in OpenAI format.

```python
from dynamo.frontend.utils import random_call_id
```

```python
random_call_id() -> str
```

[`components/src/dynamo/frontend/utils.py#L112`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L112)

#### random\_uuid (function)

Generate a random 16-character hex UUID.

```python
from dynamo.frontend.utils import random_uuid
```

```python
random_uuid() -> str
```

[`components/src/dynamo/frontend/utils.py#L107`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L107)

#### read\_jinja\_chat\_template (function)

Read a Jinja chat template using backend-specific file semantics.

```python
from dynamo.frontend.utils import read_jinja_chat_template
```

```python
read_jinja_chat_template(template_path: str, *, backend: ChatProcessorBackend) -> str
```

[`components/src/dynamo/frontend/utils.py#L66`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L66)

#### resolve\_chat\_template (function)

Return a chat template stored beside the model, or None.

```python
from dynamo.frontend.utils import resolve_chat_template
```

```python
resolve_chat_template(source_path: str, *, backend: ChatProcessorBackend = 'vllm') -> str | None
```

Covers models (e.g. Qwen3-Omni) whose template lives in chat\_template.json
or chat\_template.jinja rather than tokenizer\_config.json, which the HF
tokenizer does not merge. The backend selects native .jinja file semantics.

[`components/src/dynamo/frontend/utils.py#L81`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L81)

#### resolve\_request\_force\_reasoning (function)

Resolve the effective force\_reasoning flag for a single request.

```python
from dynamo.frontend.sglang_prepost import resolve_request_force_reasoning
```

```python
resolve_request_force_reasoning(request: dict[str, Any], reasoning_parser_name: str | None, template_default: bool) -> bool
```

Mirrors sglang.srt.entrypoints.openai.serving\_chat.\_get\_reasoning\_from\_request
combined with template\_manager.force\_reasoning:

* opt-out families (`glm45`/`qwen3`/`kimi_k2`/`kimi_k3`/...): on by
  default, `chat_template_kwargs.enable_thinking=False` (or
  `thinking=False` for Kimi) disables it.
* MiniMax-M3 defaults to adaptive, but SGLang still enables the
  reasoning parser unless `chat_template_kwargs.thinking_mode` is
  explicitly `"disabled"`.
* Mistral is enabled only when `reasoning_effort` is present and not
  `"none"`.
* opt-in families (`deepseek-v3`/`gemma4`): off by default,
  enabled by `chat_template_kwargs.{thinking,enable_thinking}=True`.
* anything else: follow the statically-detected template default.

[`components/src/dynamo/frontend/sglang_prepost.py#L134`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L134)

#### resolve\_skip\_special\_tokens (function)

Honor explicit decoding options without hiding parser delimiters.

```python
from dynamo.frontend.sglang_prepost import resolve_skip_special_tokens
```

```python
resolve_skip_special_tokens(requested: bool | None, *, has_parser: bool) -> bool
```

[`components/src/dynamo/frontend/sglang_prepost.py#L1154`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/sglang_prepost.py#L1154)

#### runtime\_default\_thinking\_mode (function)

Read deployment-level default thinking mode from model runtime metadata.

```python
from dynamo.frontend.thinking import runtime_default_thinking_mode
```

```python
runtime_default_thinking_mode(runtime_config: dict[str, Any] | None) -> str | None
```

[`components/src/dynamo/frontend/thinking.py#L20`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/thinking.py#L20)

#### runtime\_structural\_tag\_excludes\_reasoning (function)

Whether tool-call structural tags must leave reasoning to the backend.

```python
from dynamo.frontend.structural_tag_policy import runtime_structural_tag_excludes_reasoning
```

```python
runtime_structural_tag_excludes_reasoning(runtime_config: Any) -> bool
```

Mirrors `resolve_reasoning_boundary` in the Rust preprocessor: the
canonical `structural_tag.reasoning_boundary` setting (`auto` by
default) decides, and `auto` follows the vLLM worker's published
`tool_call_structural_tag_excludes_reasoning` flag. Missing or invalid
worker metadata keeps the compatibility behavior (the tag models reasoning).

[`components/src/dynamo/frontend/structural_tag_policy.py#L41`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/structural_tag_policy.py#L41)

#### runtime\_structural\_tag\_options (function)

Read structural-tag policy from a model-card runtime config.

```python
from dynamo.frontend.structural_tag_policy import runtime_structural_tag_options
```

```python
runtime_structural_tag_options(runtime_config: Any) -> tuple[str, str, str]
```

[`components/src/dynamo/frontend/structural_tag_policy.py#L14`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/structural_tag_policy.py#L14)

#### setup\_engine\_factory (function)

When using vllm pre and post processor, create the EngineFactory that creates the engines that run requests.

```python
from dynamo.frontend.main import setup_engine_factory
```

```python
setup_engine_factory(config: FrontendConfig, vllm_flags: Namespace) -> EngineFactory
```

[`components/src/dynamo/frontend/main.py#L100`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L100)

#### setup\_sglang\_engine\_factory (function)

When using sglang pre and post processor, create the SglangEngineFactory that creates the engines that run requests.

```python
from dynamo.frontend.main import setup_sglang_engine_factory
```

```python
setup_sglang_engine_factory(config: FrontendConfig, sglang_flags: Optional[Namespace] = None)
```

[`components/src/dynamo/frontend/main.py#L113`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/main.py#L113)

#### should\_attempt\_structural\_tag (function)

Return whether frontend preprocessing should request a structural tag.

```python
from dynamo.frontend.structural_tag_policy import should_attempt_structural_tag
```

```python
should_attempt_structural_tag(*, mode: str, scope: str, tool_choice_kind: ToolChoiceKind, has_tools: bool, any_explicit_strict: bool, parallel_tool_calls_explicitly_false: bool) -> bool
```

[`components/src/dynamo/frontend/structural_tag_policy.py#L74`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/structural_tag_policy.py#L74)

#### validate\_legacy\_guided\_decoding\_constraints (function)

Reject malformed or multiple legacy guided-decoding constraints.

```python
from dynamo.frontend.utils import validate_legacy_guided_decoding_constraints
```

```python
validate_legacy_guided_decoding_constraints(request: dict[str, Any]) -> None
```

[`components/src/dynamo/frontend/utils.py#L20`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L20)

#### validate\_model\_name (function)

Validate that model-name is a non-empty string.

```python
from dynamo.frontend.frontend_args import validate_model_name
```

```python
validate_model_name(value: str) -> str
```

[`components/src/dynamo/frontend/frontend_args.py#L154`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L154)

#### validate\_model\_path (function)

Validate that model-path is a valid directory on disk.

```python
from dynamo.frontend.frontend_args import validate_model_path
```

```python
validate_model_path(value: str) -> str
```

[`components/src/dynamo/frontend/frontend_args.py#L163`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/frontend_args.py#L163)

#### warn\_if\_frontend\_cpu\_affinity\_spans\_numa\_nodes (function)

Log a warning if the current CPU affinity covers multiple NUMA nodes.

```python
from dynamo.frontend.cpu_affinity import warn_if_frontend_cpu_affinity_spans_numa_nodes
```

```python
warn_if_frontend_cpu_affinity_spans_numa_nodes(logger: logging.Logger) -> None
```

[`components/src/dynamo/frontend/cpu_affinity.py#L25`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/cpu_affinity.py#L25)

#### worker\_warmup (function)

Dummy task to ensure a ProcessPoolExecutor worker is fully initialized.

```python
from dynamo.frontend.utils import worker_warmup
```

```python
worker_warmup() -> bool
```

[`components/src/dynamo/frontend/utils.py#L126`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/frontend/utils.py#L126)