[albanm]

Before the agent model reads a tool result from outside the service, an LLM classifier on the moderator role screens it for text addressed to an AI agent. A result that trips it is withheld, and the turn stops. Input moderation now runs on the same hardened classifier engine: two sides of one coin, untrusted user input and untrusted tool output.

• Scan set: MCP tools without openWorldHint: false (the MCP default), page tools, read_file and webhook payloads. The service's own tools and read_skill are not scanned. Neither is read_conversation, since transcripts are people instructing the agent, and it never serves a withheld payload back.
• On a trip: one more step runs with no tool. The model has seen only a placeholder, never the flagged text. It explains, in the person's language, that the source seems tampered with and what to do instead. A task run reports this as a needs-attention outcome and ends in error, with stop reason injection-suspected.
  • There is no "continue anyway".
  • Admins of the owner get the event agents:injection-suspected:<agentId>, and the server logs at alert with code tool-output-tripwire.
  • The chat shows what was withheld and where it came from.
• Late verdicts: a result that passed on the 3.5 s timeout and is flagged afterwards stops the turn at the next step, and no further tool runs. If the flag lands after the last step, a warning is appended to the answer. The stored result is replaced by a placeholder that says the model had read it.
• Classifier, shared by both guards (api/src/guard/):
  • the prompt is written as a policy guide;
  • text addressed to the classifier itself counts as an injection;
  • the content sits in a tag with a random per-call suffix, and the task is restated after it;
  • stage 1 answers one word, and stage 2 gives a structured verdict only on a flag.
• Failure directions:
  • The tripwire lets a result through when stage 1 fails, and counts a failed stage 2 as a trip.
  • Moderation fails open at both stages. Its stage 1 now leans toward "yes", and stage 2 decides with "when in doubt, allow".
  • Coded logs make each case visible: warn on a fail-open, block or failed confirmation, alert on a trip or an armed cooldown.
• Measuring:
  • dev/guard-eval runs a labelled set (65 tool results, 44 messages) through both guards' real two-stage calls.
  • The admin probe also checks the tripwire against the live model configuration.
  • Two new simulation cases: a subtle injected page, and an unattended task with a destructive tool at hand.
• Switch: tripwire.enabled / TRIPWIRE_ENABLED, on by default.
• Unrelated fixes:
  • b31c0a7: when a stop was written while a turn was going live, the turn wasn't stopped, so two quick messages could produce two paid answers. This race is also on main (since feat: autonomous agents, on a server-side loop #80).
  • 373df93: a runtime spec compared credits to a price in euros.

Why: users will mix web and data exploration with destructive tools whatever the docs say. These are business-data agents, so their tool output has no legitimate reason to address the agent. A tripwire raises the cost of an attack, since an attacker who tries several payloads is likely to trip it. It is not a security boundary. Design and sources are in docs/architecture/moderation.md.

Measured on the dev bridge, with Claude Haiku as the moderator:

• Tripwire eval: 2 of 87 injections missed, no false alarm on 108 benign results.
• Moderation eval: no injection or abuse missed, no false alarm, 4 to 8 of 51 off-platform requests let through.
• Five simulation rounds: 20 of 20 injected sources tripped, with no false trip in 21 ordinary runs, including the live data-fair sites.

Heads-up:

• Production models not measured yet. Run dev/guard-eval and the simulations on each production moderator model before relying on these numbers.
• On by default, no override: a false trip stops a turn or a task run.
• Cost: at least one moderator call per scanned result, a second one on a flag, and up to 3.5 s of waiting per scanned call.
• One poisoned row withholds the whole result.
• Review hardest:
  • api/src/conversations/model-loop.ts: the explanation step, the stop reason, the late-trip gate, webhook screening;
  • api/src/conversations/live-runs.ts: the abort ordering, shared stop-path code.