Skip to content

fix(agent): scope supervisor review spend per conversation - #429

Merged
Temikus merged 1 commit into
mainfrom
claude/supervisor-session-cost
Sep 21, 2026
Merged

Temikus merged 1 commit into
mainfrom
claude/supervisor-session-cost

Conversation

@Temikus

@Temikus Temikus commented Sep 21, 2026

Copy link
Copy Markdown
Owner

TL;DR: supervisor reviews billed every call to one cost-tracker session that nothing ever resets, so the per-session cost_limit_hard became a lifetime budget. Once spent, every later review failed closed to human approval until restart.

Found on my own instance: 60 reviews decided normally, then 17 straight errors over 27 hours, all session "supervisor:default" exceeded hard cost limit, leaving 7 routine run_javascript calls queued on me.

  • Session key is now supervisor:{agent}:{convID}, so the budget rolls over with the conversation like every other session.
  • Attribution: the session registers under the supervisor's own agent name. The key's first segment otherwise parsed as a phantom agent called supervisor, which hid the spend from cost_summary and skipped the supervisor's own configured limits.
  • Diagnosability: a failed review records detail.cause (cost_limit/timeout/provider_error) and the notice names the budget, since "Supervisor unavailable" is wrong when nothing is down.

APPROVE/DENY/ESCALATE are unchanged, and the error fall-through stays fail-closed.

Out of scope: a dedicated supervisor budget knob, and the zero cache hits on review calls (~32k uncached input tokens each).

Evidence from the live instance

audit_events over the last week, category supervisor, every one of them a run_javascript review (everything else is covered by permanent auto-approve scopes, so JS is the only tool that reaches the supervisor):

Window Outcome
2026-09-14 to 2026-09-20T04:06Z 60 x APPROVE
2026-09-20T04:33Z to 2026-09-21T07:35Z 17 x ERROR ... exceeded hard cost limit

cost_summary at the time: session supervisor:default held $5.01 over 136 messages and 4.36M input tokens, with sessions: 1, under an agent named supervisor that does not exist in config.

Verification

Both fixes are pinned by tests that fail without them. Reverting the session key alone:

--- FAIL: TestSupervisorReview_SpentBudgetDoesNotWedgeOtherConversations
    conversation B statuses = [supervisor_error], want supervisor_approved:
    conversation A's spend must not wedge B

Reverting the RegisterSessionAgent call alone:

--- FAIL: TestSupervisorReview_SpendAttributedToSupervisorAgent
    cost attributed to phantom agent "supervisor"; want the supervisor's own name
    got [{Agent:supervisor ...} {Agent:default ...}]

Tests drive a real Engine, Router and CostTracker with a mock provider, so the limit check runs through llm.Router rather than a stub.

just hook is green: fmt-check, vet, lint, lint-ui, test, test-ui, openapi-check.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GPHZAN5uWR4fMd2QtcLd4D

Supervisor reviews billed the cost tracker under "supervisor:{agent}", a
key constant for the process lifetime. The tracker never resets a session's
spend, so the per-session cost_limit_hard became a lifetime budget: once
spent, Router.Complete refused every later review and resolveSupervisorReview
fell closed to human approval until restart.

Observed on a live instance: 60 supervisor reviews decided normally, then 17
consecutive ERROR reviews over 27 hours, all "session \"supervisor:default\"
exceeded hard cost limit", leaving 7 run_javascript approvals queued on a
human. The calls themselves were routine.

Key is now "supervisor:{agent}:{convID}", so the budget rolls over with the
conversation like every other session.

Also register the session under the supervisor's own agent name. The key's
first segment would otherwise parse as a phantom agent named "supervisor",
which both hid the spend from cost_summary and skipped the supervisor's
configured limits.

A failed review now records detail.cause (cost_limit/timeout/provider_error)
and names the budget in the fall-through notice; "Supervisor unavailable" is
wrong when nothing is down.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GPHZAN5uWR4fMd2QtcLd4D
@Temikus
Temikus merged commit 2941233 into main Sep 21, 2026
18 checks passed
@github-actions

Copy link
Copy Markdown

🚀 Released in v0.47.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant