Skip to navigation
Tool Calling & Reasoning

Tool Call Parsing (Dynamo)

Connect Dynamo to external tools and services using Dynamo’s built-in tool call parsers

View as Markdown

Tool calling lets Dynamo connect models to external tools and services by returning function arguments for your application to execute. Requests use the tool_choice and tools parameters. This page covers Dynamo-native parsing; if your model is not listed, use engine fallback, which also explains valid combinations of --dyn-tool-call-parser, --dyn-chat-processor, and --dyn-reasoning-parser.

Configure Tool Calling

1

Launch Dynamo Backend

Select a --dyn-tool-call-parser that matches your model based on the list below. The backend can be SGLang, TensorRT-LLM, vLLM, or another installed backend.

To inspect the available worker flags, run:

python -m dynamo.<backend> --help

The following example starts an SGLang worker for Qwen3.5:

python -m dynamo.sglang \
  --model Qwen/Qwen3.5-4B \
  --dyn-tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --dyn-reasoning-parser qwen3

If the model supports tool calling but its default chat template does not, add --custom-jinja-template /path/to/template.jinja to the worker command. The template must be readable inside the container.

If the model emits reasoning content that should be separated from normal output, see Reasoning Parsing (Dynamo) for supported --dyn-reasoning-parser values.

2

Start the Frontend

Start the Dynamo frontend in another terminal:

python -m dynamo.frontend
3

Wait for Readiness

Wait for the frontend health endpoint to return a successful response:

curl --fail http://localhost:8000/health
4

Send a Tool-Calling Request

Send an OpenAI-compatible chat completion request with the available tools:

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3.5-4B",
    "messages": [
      {"role": "user", "content": "What is the weather in San Francisco and New York?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location.",
        "parameters": {
          "type": "object",
          "properties": {"location": {"type": "string"}},
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

Dynamo parses the model output and returns OpenAI-compatible tool_calls.

{
  "id": "chatcmpl-b415caad-9be0-4d9e-ac6d-9d23bfe57703",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "reasoning_content": "The user is asking about the weather in two cities: San Francisco and New York. I need to call the get_weather function for each city. I'll make two separate function calls to get the weather information for both locations.\n",
        "tool_calls": [
          {
            "id": "call-56223a95-3d14-4433-a94e-011f106c0e40",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"San Francisco\"}"
            }
          },
          {
            "id": "call-d5b5772b-6b0c-4120-ad01-623278a937fe",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"New York\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls",
      "logprobs": null
    }
  ],
  "created": 1778653281,
  "model": "Qwen/Qwen3.5-4B",
  "object": "chat.completion"
}

If a tool call is parsed incorrectly, add "logprobs": true to one reproduction request and share the response. See Troubleshooting Tool Calls for what to capture when reporting an issue.

Custom Freeform Tools

Chat Completions accepts custom tools whose input is freeform text, such as code or a patch. Use the existing tool parser for your model and declare a custom tool:

{
  "tools": [{
    "type": "custom",
    "custom": {
      "name": "run_python",
      "description": "Execute Python source code.",
      "format": {"type": "text"}
    }
  }],
  "tool_choice": {"type": "custom", "custom": {"name": "run_python"}}
}

Returned message.tool_calls entries use type: "custom" with custom.name and custom.input. Your application executes the tool and replays the assistant call followed by a role: "tool" result with its matching tool_call_id. Ordinary function tools can appear in the same request.

Responses supports the same capability using its own custom-tool API format. Custom grammar formats are rejected before generation. Streaming custom input is buffered until the call completes, then emitted as text in custom.input. This streamed custom delta is a Dynamo extension to the published OpenAI Chat streaming schema. The optional tool_call_dispatch events currently cover function tools only; custom calls arrive through ordinary data chunks.

Function-oriented engines generate a JSON wrapper with a string input field. If a completed call does not match that wrapper, Dynamo returns the raw argument text as custom.input, preserving other calls, content, and usage. Applications must validate tool input before executing it. If the token limit interrupts the wrapper, Dynamo preserves finish_reason: "length" without emitting the truncated input.

Supported Tool Call Parsers

Choose a model family, then expand the matching model option to see its parser name and configuration details.

The Upstream name column shows where the vLLM or SGLang parser name differs from Dynamo’s — relevant when using --dyn-chat-processor vllm or sglang (see Parser Engine Fallback). A blank upstream column means the same name works everywhere. Dynamo-only means no upstream parser exists for this format.

Parser: kimi_k2

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: Pair with --dyn-reasoning-parser kimi or kimi_k25. For Kimi K2.5 thinking models, use kimi_k25 so <think> blocks and tool calls are parsed from the same response. See Reasoning Parsing (Dynamo).

Parser: minimax_m2

Upstream name: vLLM: minimax_m2

Notes: XML <minimax:tool_call>

Parser: minimax_m3

Upstream name: vLLM: minimax_m3

Notes: MiniMax namespace-token XML

Parser: deepseek_v4

Upstream name: vLLM: deepseek_v4; SGLang: deepseekv4

Notes: DSML tags (<｜DSML｜tool_calls>...). Aliases: deepseek-v4, deepseekv4

Parser: deepseek_v3

Upstream name: SGLang: deepseekv3

Notes: Special Unicode markers

Parser: deepseek_v3_1

Upstream name: Dynamo-only

Notes: JSON separators

Parser: deepseek_v3_2

Upstream name: Dynamo-only

Notes: DSML tags (<｜DSML｜function_calls>...)

Parser: qwen3_coder

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: XML <tool_call><function=...>

Parser: hermes

Upstream name: vLLM: qwen2_5; SGLang: qwen25 (for Qwen models)

Notes: <tool_call> JSON

Parser: glm47

Upstream name: Dynamo-only

Notes: XML <arg_key>/<arg_value>

Parser: nemotron_deci

Upstream name: Dynamo-only

Notes: <TOOLCALL> JSON

Parser: nemotron_nano

Upstream name: Dynamo-only

Notes: Alias for qwen3_coder

Parser: gemma4

Upstream name: vLLM: gemma4

Notes: Custom non-JSON grammar with <\|"\|> string delimiters and <\|tool_call>...<tool_call\|> markers. Aliases: gemma-4. Pair with --dyn-reasoning-parser gemma4 and --custom-jinja-template examples/chat_templates/gemma4_tool.jinja.

Parser: beam

Upstream name: none (Dynamo-native, served by the dynamo-parsers-v2 unified family)

Notes: Channel grammar: reasoning in <\|channel\|>analysis<\|message\|>...<\|end\|>, the answer in <\|channel\|>final<\|message\|>...<\|return\|>, and one <\|start\|>assistant to=NAME<\|channel\|>analysis <\|constrain\|>xml<\|message\|>...<\|call\|> message per tool call with <rfl:parameter name="KEY">VALUE</rfl:parameter> arguments typed from the tool schema. Pair with --dyn-reasoning-parser beam; both flags are required and the pair routes to the unified parser without DYN_PARSER_VERSION=2. The route accepts the request field preserve_thinking (supplied prior assistant reasoning always stays in the rendered history; true and omitted are the same, false is accepted and has no effect — omit reasoning_content to drop it) and defaults allow_missing_thinking=true so history turns without reasoning render an empty analysis block instead of failing and chat_template_kwargs.builtin_tools (names resolved against tools for rendering only).

Per-turn grammar: on the Beam route the frontend holds every chat turn to this channel grammar with a structural tag (a non-empty analysis channel, then either tool calls whose rfl:parameter values are held to their JSON-schema types or the final answer), mirroring the model builder’s SGLang serving layer; required and named tool choices reuse the same tag, so they are enforced on the Harmony wire. A response_format (json_object or json_schema) is held inside the final channel behind the analysis, so the response keeps its reasoning and the channel structure the parser expects; a forced tool choice ignores response_format, and the nvext guided-decoding fields (guided_json, guided_regex, …) stay bare constraints from the first token. Set DYN_BEAM_TURN_GRAMMAR=0 on the frontend to disable it for automatic tool choice (forced choices keep theirs; response_format then falls back to the bare JSON constraint). A tool declared strict: true whose schema is outside OpenAI’s strict subset is served as non-strict on this route (logged as a warning) instead of the HTTP 400 other routes return; the grammar holds every typeable tool either way.

Parser: harmony

Upstream name: Dynamo-only

Notes: Harmony channel format

Parser: phi4

Upstream name: vLLM: phi4_mini_json

Notes: functools[...] JSON

Parser: pythonic

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: Python-list tool syntax

Parser: llama3_json

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: <\|python_tag\|> tool syntax

Parser: mistral

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: [TOOL_CALLS]...[/TOOL_CALLS]

Parser: jamba

Upstream name: Dynamo-only

Notes: <tool_calls> JSON

Parser: default

Upstream name: Dynamo-only

Notes: Empty JSON config (no start/end tokens). Prefer a model-specific parser for production use.

Structural tags are enabled by default

For parser and backend combinations that support them, Dynamo enables xgrammar structural tags by default so guided decoding matches the parser’s tool-call format at token granularity. See Structural Tag (Guided Decoding for Tool Calls) for the activation policy, compatibility fallbacks, and opt-out settings.