fix(agent): scope supervisor review spend per conversation - #429
Merged
Merged
Conversation
Supervisor reviews billed the cost tracker under "supervisor:{agent}", a
key constant for the process lifetime. The tracker never resets a session's
spend, so the per-session cost_limit_hard became a lifetime budget: once
spent, Router.Complete refused every later review and resolveSupervisorReview
fell closed to human approval until restart.
Observed on a live instance: 60 supervisor reviews decided normally, then 17
consecutive ERROR reviews over 27 hours, all "session \"supervisor:default\"
exceeded hard cost limit", leaving 7 run_javascript approvals queued on a
human. The calls themselves were routine.
Key is now "supervisor:{agent}:{convID}", so the budget rolls over with the
conversation like every other session.
Also register the session under the supervisor's own agent name. The key's
first segment would otherwise parse as a phantom agent named "supervisor",
which both hid the spend from cost_summary and skipped the supervisor's
configured limits.
A failed review now records detail.cause (cost_limit/timeout/provider_error)
and names the budget in the fall-through notice; "Supervisor unavailable" is
wrong when nothing is down.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GPHZAN5uWR4fMd2QtcLd4D
|
🚀 Released in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: supervisor reviews billed every call to one cost-tracker session that nothing ever resets, so the per-session
cost_limit_hardbecame a lifetime budget. Once spent, every later review failed closed to human approval until restart.Found on my own instance: 60 reviews decided normally, then 17 straight errors over 27 hours, all
session "supervisor:default" exceeded hard cost limit, leaving 7 routinerun_javascriptcalls queued on me.supervisor:{agent}:{convID}, so the budget rolls over with the conversation like every other session.supervisor, which hid the spend fromcost_summaryand skipped the supervisor's own configured limits.detail.cause(cost_limit/timeout/provider_error) and the notice names the budget, since "Supervisor unavailable" is wrong when nothing is down.APPROVE/DENY/ESCALATE are unchanged, and the error fall-through stays fail-closed.
Out of scope: a dedicated supervisor budget knob, and the zero cache hits on review calls (~32k uncached input tokens each).
Evidence from the live instance
audit_eventsover the last week, categorysupervisor, every one of them arun_javascriptreview (everything else is covered by permanent auto-approve scopes, so JS is the only tool that reaches the supervisor):APPROVEERROR ... exceeded hard cost limitcost_summaryat the time: sessionsupervisor:defaultheld $5.01 over 136 messages and 4.36M input tokens, withsessions: 1, under an agent namedsupervisorthat does not exist in config.Verification
Both fixes are pinned by tests that fail without them. Reverting the session key alone:
Reverting the
RegisterSessionAgentcall alone:Tests drive a real
Engine,RouterandCostTrackerwith a mock provider, so the limit check runs throughllm.Routerrather than a stub.just hookis green: fmt-check, vet, lint, lint-ui, test, test-ui, openapi-check.🤖 Generated with Claude Code
https://claude.ai/code/session_01GPHZAN5uWR4fMd2QtcLd4D