Skip to content

feat(chat): Laguna reasoning + tool-call parser detector (CoT leaks into content)#4

Open
zentradev-rabih wants to merge 2 commits into
poolsideai:lagunafrom
zentradev-rabih:laguna-reasoning-detector
Open

feat(chat): Laguna reasoning + tool-call parser detector (CoT leaks into content)#4
zentradev-rabih wants to merge 2 commits into
poolsideai:lagunafrom
zentradev-rabih:laguna-reasoning-detector

Conversation

@zentradev-rabih

Copy link
Copy Markdown

Problem

Serving Laguna-S-2.1 GGUFs via this fork, chain-of-thought is not separated
it leaks into message.content instead of message.reasoning_content, even with
--reasoning-format deepseek --jinja.

Root cause: Laguna's laguna_glm_thinking chat template matches none of the
common_chat_params_init_* detectors in common/chat.cpp
(common_chat_try_specialized_template returns std::nullopt), so it falls back
to the generic format, which has no thinking-tag handling. --reasoning-format
sets the extraction mode, but the <think>/</think> tags are assigned per
template by a detector that doesn't exist for Laguna.

A subtlety that rules out reusing an existing detector: the assistant turn is
generated already inside a reasoning block. The template's reasoning directive
is prose ("Put your reasoning inside … tags"), so the model emits its thoughts and
then a literal </think> before the answer, with no emitted opening <think>.
Detectors that gate on tmpl.source().find("<think>") (e.g. LFM2) therefore never
enable extraction for Laguna.

Fix

Add common_chat_params_init_laguna:

  • Force-opened reasoning: content up to the first </think> is the thought;
    the remainder is the answer. Gated on reasoning_format != NONE (not on a literal
    <think> in the template).
  • <system> / <user> / <assistant> roles, generation prompt <assistant>\n.
  • <tool_call>{"name":…,"arguments":…}</tool_call> tool calls via
    standard_json_tools.
  • Dispatch keyed on two template-unique strings (render_assistant_messages_raw
    and <available_tools>) so it never shadows another family.

Also includes a one-line build fix: common/speculative.cpp uses std::isfinite
without #include <cmath>, which fails to compile on GCC 13.

Verification (Laguna-S-2.1-Q4_K_M, sm_121 / GB10 CUDA build)

  • Reasoning prompt → reasoning_content populated, content is the clean answer
    (no CoT, no </think>), finish_reason: stop.
  • Tool prompt → tool_calls: [get_weather({"city":"Paris"})], reasoning still
    separated, finish_reason: tool_calls.

Happy to adjust the dispatch markers or tool-call keys if you have canonical ones —
this is the interim serving path for us and we wanted to surface the gap so it can
be fixed properly upstream.

🤖 This fix was developed with the assistance of Claude Code.

zentradev-rabih added 2 commits July 23, 2026 09:12
Building the laguna branch with GCC 13 fails: common/speculative.cpp uses
std::isfinite (DFlash non-finite feature guard) without including <cmath>.
Laguna GGUFs (laguna_glm_thinking template) match none of the
common_chat_params_init_* detectors, so they fall to the generic format and
lose the reasoning/answer split -- chain-of-thought leaks into message.content.

The assistant turn is generated already inside a reasoning block: the template's
reasoning directive is prose, so the model emits its thoughts then a literal
</think> before the answer, with NO emitted opening <think>. Add
common_chat_params_init_laguna: force-opened </think> reasoning extraction
(gated on reasoning_format, NOT on a literal <think> in the template),
<system>/<user>/<assistant> roles, and <tool_call>{"name","arguments"}</tool_call>
tool calls via standard_json_tools. Dispatch keys on two template-unique strings.

Verified on Laguna-S-2.1-Q4_K_M: reasoning_content populated, content clean,
tool_calls parsed (get_weather({"city":"Paris"})).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant