Troubleshooting Tool Calls

When a tool call comes back wrong (tool_calls is null, the arguments look malformed, raw <tool_call> markers appear in message.content, or finish_reason is "stop" when you expected "tool_calls"), the request and response alone usually do not say where the bug is. The model and the parser produce indistinguishable failures from the response side.

Adding "logprobs": true to a single repro request makes the engine’s raw token output visible in the response. That is enough for someone on the Dynamo team to identify whether the issue is in the model, the parser configuration, or the parser itself. This page shows the field to add and what the response will look like, so you can capture and share useful diagnostic info.

The request

Add "logprobs": true to your failing request. Port-forward the Frontend Service first (kubectl port-forward svc/<deployment-name>-frontend 8000:8000 -n ${NAMESPACE}):

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [
      {"role": "user", "content": "What is the weather in NYC?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string"},
            "unit": {"enum": ["celsius", "fahrenheit"]}
          },
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto",
    "temperature": 0.0,
    "logprobs": true
  }'

The response

You will get back the usual fields (message.tool_calls, message.content, finish_reason) plus a new choices[0].logprobs.content field carrying the engine’s raw token stream:

{
  "choices": [{
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}"
        }
      }]
    },
    "logprobs": {
      "content": [
        {"token": "<tool_call>", "bytes": [60, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]},
        {"token": "\n", "bytes": [10]},
        {"token": "{\"", "bytes": [123, 34]},
        {"token": "name", "bytes": [110, 97, 109, 101]},
        "...",
        {"token": "</tool_call>", "bytes": [60, 47, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}
      ]
    }
  }]
}

Each entry in logprobs.content is one generated token with its exact UTF-8 bytes. Concatenating those bytes in order reconstructs the raw model output, before any tool-call parser touched it. That is the key piece for triage: it tells us what the model actually produced, separately from what the parser made of it.

What to include when reporting an issue

Share these four things in the bug report or issue thread:

1. The full request body (model name, messages, tools, sampling params, and logprobs: true).
2. The full response body. Do not truncate logprobs.content — the per-token entries are the part that matters.
3. The Dynamo version and the backend (vLLM, SGLang, TRT-LLM, including versions if known).
4. The worker launch command, especially the --dyn-tool-call-parser value if set.

With those four pieces, the Dynamo team can usually localize the bug without standing up your model. The team will reconstruct the raw stream from the bytes arrays and compare it against message.content and message.tool_calls to decide whether the issue is in the model output, the parser configuration, or the parser logic.

Parser Generation Routing

To select a parser generation, set DYN_PARSER_VERSION=1 or DYN_PARSER_VERSION=2. Leave it unset or use auto to keep the existing family defaults. The aliases v1 and v2 are not accepted.

Unsupported values stop startup. Explicit selections validate the configured tool and reasoning parsers and reject combinations that the selected generation cannot support.

Version 2 uses a compatible unified parser for streaming and batch when either the tool-call parser or reasoning parser identifies a supported family. When both are configured, they must resolve to the same family. The DeepSeek V4.1 pair uses V2 by default for streaming and batch. With unset settings or auto, Muse streaming requests with named/required tool choices or structural tags retain their existing V1 handling; Muse batch responses use V2. Set DYN_PARSER_VERSION=2 to require V2 for supported request modes. Tool-only and reasoning-only V1 configurations keep their existing parsers under the original family defaults.

Explicit V1 and V2 apply the same generation selection to streaming and batch. Streaming passes output chunks to the selected parser as they arrive; partial tool-call deltas depend on the family and output shape. Batch parsing waits for the complete response before constructing message.content, message.reasoning_content, and message.tool_calls. tool_choice: none suppresses tool calls while the selected content decoder removes recognized native markup from visible content.

Forced tool choices can install a guided JSON constraint, and structural-tag requests keep that constraint when the configured family has a compatible unified parser. A request mode that requires the V1 tool-call jail cannot silently fall back when explicit V2 is selected; Dynamo rejects that unsupported parser configuration before serving requests.

On the default Kimi K3 route, a prompt ending with the XTML reasoning opener <|open|>think<|sep|> starts parsing inside the reasoning channel; whitespace between the markers is ignored. This also applies when DYN_PARSER_VERSION is unset.

Known limitations and follow-up

This revision pins the latest published dynamo-parsers-v2 crate, version 0.7.17. It includes the merged quoted-control and streaming fixes from frontend-crates #326, along with the Kimi tool-ID and call-boundary fixes from frontend-crates #332. The five Dynamo regression tests that were ignored for 0.7.13 are enabled and pass against 0.7.17. The targeted cargo test --locked -p dynamo-llm quoted_ -- --nocapture run passed 13 unit tests, one aggregator test, and two integration tests. The every-split Muse structured-response case also passes in cargo test --locked -p dynamo-llm response_format_explicit_reasoning_without_prompt_prefill -- --nocapture. These checks do not cover every parser path.

A separate quoted native tool-call opener case can still cut off the rest of an answer under V2, in streaming and batch requests. The exact regression remains follow-up work tracked with related native Qwen3 and Kimi streaming changes in Dynamo #15577. With generic <think> prompt prefill, splitting thought</think> after thought< can also leave the remaining tool JSON in reasoning; this behavior is present on the existing V1 route as well. Both cases still need qualification before they can be claimed as fixed.

For configurations supported by V1, select DYN_PARSER_VERSION=1 to use the legacy parser.

See also

• Tool Call Parsing (Dynamo) — Dynamo-native parser names and request examples
• Chat Processors — --dyn-chat-processor and the engine-fallback path to vLLM and SGLang parsers
• Frontend Configuration Reference — full CLI flag reference for the frontend and worker