Skip to content

[bug] OmniAgent drops MagenticBrain tool calls when the server peels them into message.tool_calls #573

Description

@zboyles

Describe the bug

OmniAgent only parses MagenticBrain tool use from message.content (<tool_call>{...}</tool_call> XML). Some OpenAI-compatible servers detect those markers, move them into native message.tool_calls, and leave only the model's prose in content.

OmniAgent then treats the turn as bare text: one nudge, then a "final answer" that narrates using the tool instead of calling it. Sub-agents (e.g. Fara via delegate_cua) never start.

This is not MagenticBrain failing to emit tool calls — the server already has them in tool_calls. The client just never reads that field.

The documented vLLM / Hugging Face / Foundry path is likely unaffected (those servers typically leave the XML in content). The bug shows up with servers that implement native tool-call extraction, including mlx_lm.server.

Steps to reproduce

  1. Serve MagenticBrain through an OpenAI-compatible endpoint that extracts <tool_call> into message.tool_calls (reproduced with mlx_lm.server --model mlx-community/MagenticBrain-8bit).
  2. Point MagenticLite's orchestrator base_url at that /v1 endpoint.
  3. Ask a task that should dispatch the web agent, e.g. "Go to https://example.com and describe the homepage."
  4. Optionally confirm the server did emit native tool calls:
from openai import OpenAI
c = OpenAI(base_url="http://127.0.0.1:8100/v1", api_key="not-needed")
r = c.chat.completions.create(
    model="mlx-community/MagenticBrain-8bit",
    messages=[
        {"role": "system", "content": "...MagenticLite tool prompt with delegate_cua..."},
        {"role": "user", "content": "Go to https://example.com and describe the homepage."},
    ],
)
print(r.choices[0].message.content)
print(r.choices[0].message.tool_calls)
print(r.choices[0].finish_reason)  # often "tool_calls"

### Expected behavior

OmniAgent parses a delegate_cua (or other) tool call and runs it. The UI shows Using tool: delegate_cua, and Fara starts.

### Actual behavior

### **UI loop:****Reasoning**: "This is a web browsing task, so I should use the delegate_cua function..."**Nudge**: response did not contain <tool_call> or <answer>**Final answer**: "I need to delegate the web browsing task to the web agent. Let me use the delegate_cua function with the appropriate parameters."

Fara never runs. Trace shows content is the prose only; finish_reason on the raw completion is tool_calls.

### MagenticLite version

0.2.2

### Operating system

macOS

### Python version

3.12

### Model client / provider

`mlx-community/MagenticBrain-8bit` via `mlx_lm.server` & `mlx-community/Fara1.5-9B-8bit` via `mlx_vlm.server`

### Additional context

**Root cause**: `OmniResponses._call_api` takes `response.choices[0].message.content or ""` and `parse_response()` only looks for `<tool_call>` in that string. Native `message.tool_calls` is ignored.

**Suggested fix**: If `tool_calls` is present and `content` does not already contain `<tool_call>` tags, reconstruct the XML MagenticLite already understands, then parse as today. If content already has the tags (vLLM path), leave it alone.

I prototyped this in `zboyles/magentic-ui` branch `fix/rehydrate-native-tool-calls` (commit [ece97a2](https://github.com/zboyles/magentic-ui/commit/ece97a2)): `rehydrate_native_tool_calls()` in `_parse.py`, used from `_responses.py`, plus tests in `tests/agents/omni/test_parse.py`. Ran several live checks successfully, the same browsing task emitted `delegate_cua` and started Fara.

I can adapt that patch if you want to take it in-tree; I created this issue first per `CONTRIBUTING.md`.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions