fix(codex): stop double-billing reasoning output, price cache writes at the explicit rate only - #1078
Merged
Merged
Conversation
…at the explicit rate only Reasoning tokens are a subset of output_tokens for OpenAI models, not an extra bucket: on a 1,396-rollout corpus all 134,316 token_count events carrying a total satisfy input + output == total. CodeBurn added reasoning_output_tokens on top when pricing a codex call, in the cache-rehydration re-price, and in the models/audit display sums. That overstated codex cost by $166.03 (3.5%) and displayed output tokens by 34.6% on that corpus. Both cost sites and the display sums now go through one shared billableOutputTokens() so a cold parse and a warm read cannot drift apart. cache_write_input_tokens (codex PR #33454) was never read and cacheCreationInputTokens was hardcoded to 0. It is now carved out of the uncached-input bucket and clamped to it, but routed to the cache-write bucket ONLY when the pricing source publishes an explicit cache-write rate. buildCosts fabricates 1.25x input when a source omits one, which is correct for Anthropic and would have invented a surcharge OpenAI never charged on gpt-5.5 / 5.4 / 5.3-codex / gpt-5. ModelCosts now carries cacheWriteCostIsExplicit so that distinction survives getModelCosts. A cost change invalidates persisted output: codex-results.json v10 -> v11 (stores costUSD verbatim), the codex parse version moves (the token-bucket change does not self-heal on read), and the daily cache goes 20 -> 23 (21 is claimed by the #946 landing branch and 22 by PR #1056). The upgrade-path corpus asserts codex tokens and calls exactly and reports the repricing. Closes #1075
This was referenced Aug 21, 2026
Closed
iamtoruk
added a commit
that referenced
this pull request
Aug 21, 2026
fix(pricing): harden the #1078 follow-ups — cache-flag survival, credits trap, harness cost bound
pull Bot
pushed a commit
to TheTechOddBug/codeburn
that referenced
this pull request
Aug 21, 2026
…reasoning param The pricing cache written to disk had no schema version, so a cache written by a pre-getagentseal#1078 binary lacked cacheWriteCostIsExplicit on every entry. Reading it back resolved the missing key to undefined (falsy), silently reintroducing the surcharge-fabrication bug getagentseal#1078 killed for up to CACHE_TTL_MS after an upgrade. loadCachedPricing now rejects any cache whose version doesn't match the current schema instead of reading it verbatim. codexCredits() still accepted an optional reasoningTokens param that added it to output - the exact double-count getagentseal#1078 removed from every real caller. The only caller never passed it; deleted it so it can't be reintroduced by accident. parser.ts's activeGeneratedTokens fallback went through billableOutputTokens in getagentseal#1078, but codex is the only caller of activeDurationMs/activeGeneratedTokens and always sets both together, so the fallback branch is unreachable for it. Reverted to reduce diff noise.
pull Bot
pushed a commit
to TheTechOddBug/codeburn
that referenced
this pull request
Aug 21, 2026
… stale comments The COST_CHANGED_BY_DESIGN carve-out in compare.mjs left codex cost entirely unasserted after getagentseal#1075/getagentseal#1078. It now requires the upgraded cost to be strictly lower than baseline and within 25% of it, and the row verdict says "repriced" instead of the misleading "identical (cost N% drift)". grok.ts's comment on the reasoning/output split still claimed provider-side splitting was the repo's only mechanism; it now also names billableOutputTokens/REASONING_INCLUDED_IN_OUTPUT (models.ts), which is the other half since getagentseal#1078. usage-aggregator.ts's "folds reasoning into output" comment was true pre-getagentseal#1078 but is backwards for codex now (reasoning is already inside output, not added to it). Test exemplars for the "reasoning is additive" case used hermes, whose upstream is OpenAI-shaped and may not stay a safe example; swapped to gemini, which documents "thoughts" as genuinely separate output. CHANGELOG's getagentseal#1075 entry gets one line noting days whose codex transcripts have aged out keep their pre-fix totals via the daily-cache never-lose guard, matching the disclosure already given for getagentseal#1040.
iamtoruk
added a commit
that referenced
this pull request
Aug 21, 2026
iamtoruk
added a commit
that referenced
this pull request
Aug 21, 2026
fix(codex): port the #1078 pricing fixes (reasoning double-count + cache-write bucketing) to feat/core-extraction
avs-io
pushed a commit
to avs-io/codeburn
that referenced
this pull request
Aug 21, 2026
The same reasoning-inclusion bug getagentseal#1075/getagentseal#1078 fixed for cost also affected the throughput display: activeGeneratedTokens/taskGeneratedTokens in the Codex parser and generatedTokens in the live codex-tps reader summed outputTokens + reasoningTokens, but reasoning is already inside output. All three sites now route through the billableOutputTokens('codex', ...) helper getagentseal#1078 introduced, so the throughput numerator can never drift from the billed one. Cost and every token count are unchanged (verified byte-identical on a real 51,753-call corpus); Tok/s drops 20-42% depending on how reasoning-heavy the model is. activeGeneratedTokens/activeDurationMs/toolWaitMs are stored verbatim in both the Codex result cache and the session cache rather than re-derived on read, so neither self-heals: CODEX_CACHE_VERSION moves 11 -> 13 (12 is claimed by feat/core-extraction's own port of this feature) and PROVIDER_PARSE_VERSIONS.codex gains a codex-tps-v1 suffix, forcing Codex sessions to re-parse once. Closes getagentseal#1079
iamtoruk
added a commit
to avs-io/codeburn
that referenced
this pull request
Aug 22, 2026
Resolves conflicts from main's getagentseal#1078 (billableOutputTokens for the output bucket) and getagentseal#1084 (models-report test exemplar swap) against this branch's canonical-id row merging. - src/audit-report.ts: import-line collision only. Union both sides' imports (billableOutputTokens + fallbackRawModelDisplayName/ getShortModelName); both are used elsewhere in the file and neither side's logic needed further changes. - src/models-report.ts: import-line collision resolved the same way. The row-construction conflict was structural, not a data conflict: main's side pushed one row per raw bucket (a simpler variant this branch's canonical-id folding already made obsolete downstream -- rowsByKey/foldedCategoryCost/foldKey are used unconditionally past this point). Kept this branch's rowsByKey merge-by-canonical-id structure, which already computes credits from bucket.outputTokens (already billable-output-summed at the accumulation stage, main's change, untouched by this conflict) using the identical formula main used; folded main's explanatory comment about billable output into the kept credits block.
iamtoruk
added a commit
to avs-io/codeburn
that referenced
this pull request
Aug 22, 2026
Resolves conflicts against main after getagentseal#1078/getagentseal#1084/getagentseal#1088/getagentseal#1090/getagentseal#1092/ getagentseal#1053/getagentseal#1056 landed since this branch's last upstream merge. - CHANGELOG.md: kept both entries (this PR's getagentseal#968/getagentseal#1050 note plus main's getagentseal#1079/getagentseal#1088, getagentseal#1082, getagentseal#1075 notes that had moved into the same "### Fixed" slot). - Everything else (src/models.ts, main.ts, usage-aggregator.ts, daily-cache.ts, tests/*) merged cleanly with no conflict markers; git's recursive merge combined getagentseal#1050's flat-rate classifier changes with getagentseal#1056's codex-auto-review -> gpt-5.5 alias without overlap. Verified the codex-auto-review / getagentseal#1056 interaction post-merge: isBuiltInFlatRateModel no longer matches codex-auto-review (dropped per getagentseal#1050), while MODEL_ALIASES still aliases it to gpt-5.5 (getagentseal#1056), so it prices at GPT-5.5 rates rather than $0. Covered by the existing tests/models.test.ts "Codex activity ids (getagentseal#1047)" describe block and the "does not treat a priced sibling as expected-free" case. getFlatRateModelsConfigHash's output is folded into getDailyCacheConfigHash's template literal unconditionally (flatRateModels=<hash>), so the flat-rate section always participates in the daily-cache invalidation hash even when empty -- no DAILY_CACHE_VERSION bump needed for this change.
iamtoruk
added a commit
to kelchm/codeburn
that referenced
this pull request
Aug 22, 2026
Resolves the conflicts getagentseal#946 accumulated while it was in validation. Eight files conflicted; the session-store accounting is unchanged. src/daily-cache.ts — version collision. This PR minted 25 when main was at 24; getagentseal#1056 (`codex-auto-review` pricing) then spent 25 on main. The bump moves to 26/MIN 26 and daily-cache.v26.json, with main's full comment ladder kept as the foundation and this PR's paragraph rewritten to name 26 and record the collision. PENDING_REDERIVE_PROVIDERS and the B1 migration semantics from b6481c1 carry over intact, retargeted at 26. src/models.ts, src/parser.ts, src/audit-report.ts, src/models-report.ts — getagentseal#1075/getagentseal#1078 replaced the per-site "reasoning is already inside output" tests with billableOutputTokens() and REASONING_INCLUDED_IN_OUTPUT. This PR had added copilot to that case at three sites independently. Union: all three sites take main's helper call verbatim, and copilot joins claude and codex in the set — same accounting this PR shipped, now through main's single source of truth. It also reaches parser.ts activeGeneratedTokens (a fourth site, from getagentseal#1079), which is the same correction: a copilot supplementary call carries reasoning with output 0, so counting it as generated repeats the per-turn output. The audit legend already said so on this side. src/providers/copilot.ts — comment-only. getagentseal#1054's lastEventTimestamp-first shutdown fallback was derived from this branch, so the code was already identical on both sides: the `shutdownTimestamp` expression and the `copilot:<sid>:shutdown:<model>:<n>` key are byte-for-byte main's. Both rationales are kept (leg-collapse on date, and residual anchoring). src/session-cache.ts — PROVIDER_PARSE_VERSIONS.copilot takes this PR's `-session-store-v3` suffix; main's getagentseal#1051 note about why a fingerprint change is expensive is kept above it. Codex keeps main's getagentseal#1092 suffix chain untouched. src/main.ts — getagentseal#1067 deleted the unreachable live dailyMap fallback that this PR had taught behavioral weight. Main's deletion wins; the now-unused isBehavioralTurn import goes with it. tests/parser.test.ts — import union. Also: scripts/upgrade-path/run.mjs NEW_DAILY_CACHE -> daily-cache.v26.json, CODEBURN_COPILOT_SESSION_STORE_DB added to the getagentseal#1064 env-isolation CLEARED list, and the CHANGELOG entry's stale "v21" corrected to v26. Verified: tsc clean; 3132 tests pass across 223 files; test:locks 26/26; verify:upgrade PASSED, re-deriving daily-cache.v26.json and holding durable copilot history across the bump. getagentseal#1054's regression ("keeps three stampless shutdown legs as :n keys with lastEventTimestamp") passes on the merged tree. Real-corpus A/B against origin/main over 2026-07-01..2026-08-22: codex, grok, kimicode and opencode byte-identical in export, audit and models; claude drifts only monotonically with run order (a live session writing transcripts, confirmed by interleaving four runs). This machine has no copilot data, so the copilot recovery semantics rest on the suites and the upgrade-path corpus.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes claims 2 and 1 from @chr-evensen's report, in the form the verification comment on #1075 landed on. Claim 3 (long-context tiers) is deferred to #1076 at the real 272k threshold; the missing
gpt-5.6-codexsnapshot rows are #1077.Fix A — reasoning was billed twice
OpenAI bills reasoning tokens as part of
output_tokens. On a 1,396-rollout corpus, all 134,316token_countevents carrying a total satisfyinput + output == total; zero satisfy the exclusive reading. CodeBurn addedreasoning_output_tokenson top in four places:src/providers/codex.ts(fresh parse)output + reasoningsrc/parser.tscachedCallToApiCall(cache rehydration)claudeonlysrc/models-report.ts,src/audit-report.tsoutput + reasoningThe two cost sites are twins — codex is not on the reported-cost pass-through allowlist, so a warm run re-prices from the stored token buckets while a cold run uses the parser's number. If only one had been fixed, a user's total would change between a cold and a warm run. Both now call one helper:
so the two sites cannot drift, and each carries a comment pointing at the other.
claudeis in the set to preserve the carve-outparser.tsalready had; the claude parser always emitsreasoningTokens: 0, so no claude number moves. The rawreasoningTokensfield is untouched and still reported on its own — only the double-count is gone.Fix B —
cache_write_input_tokens, guardedThe field (codex PR #33454) was never read and
cacheCreationInputTokenswas hardcoded to0. It is now read on both decode paths, clamped tomax(0, min(cache_write, input − cached)), carved out of the uncached-input bucket, and passed ascacheCreationInputTokens.The guard.
buildCostsinsrc/models.tsfabricatescacheWriteCostPerToken = 1.25 × inputwhen the pricing source omits one. That default is right for Anthropic-style pricing, where a cache write genuinely costs more. OpenAI charges nothing extra for a cache write before gpt-5.6, and the snapshot reflects that:gpt-5.6-terra,-sol,-luna,gpt-5.6gpt-5.5,gpt-5.4,gpt-5.3-codex,gpt-5null→ fabricatedRouting tokens through the fabricated rate would have invented a surcharge OpenAI never billed on every pre-5.6 model. So
ModelCostsnow carriescacheWriteCostIsExplicit(set once, inbuildCosts, from whether the raw tuple's cache-write slot was non-null), and the codex path only moves tokens into the cache-write bucket when it is true. Otherwise they stay in plain input and the price is unchanged to the cent.This is deliberately a decision about which bucket codex tokens land in.
buildCosts' defaults are not changed, and no other provider's behaviour moves.Corpus A/B
Read-only,
CODEBURN_PRICING_SNAPSHOT_ONLY=1, scratch cache dirs,models -p lifetime --provider all --format jsononorigin/mainvs this branch:Every other provider is byte-identical row for row, with one exception: the two
clauderows move, and they move base-vs-base too (I was using Claude Code while the runs were in flight —claude-opus-5calls32243 -> 32244between two consecutiveorigin/mainruns). No non-claude, non-codex row differs at all.The upgrade-path corpus reaches the same conclusion independently, and asserts the stronger half: codex tokens
476,815 -> 476,815and calls123 -> 123identical, cost1.451527 -> 1.207871.Cache versions
A cost change invalidates persisted output, so three layers move:
CODEX_CACHE_VERSION10 → 11 —codex-results.jsonstores each call'scostUSDand token buckets verbatim.PROVIDER_PARSE_VERSIONS.codexgains-codex-pricing-v1— the session cache serves unchanged files without invoking the provider parser. Fix A self-heals through it (cost is re-derived on read), but Fix B's bucket move does not: cached entries store the buckets, not the raw event. Without this bump the cache-write carve-out would never reach an existing user.DAILY_CACHE_VERSION/MIN_SUPPORTED_VERSION20 → 23. Not 21: v21 is claimed by the feat(copilot): read per-request input/cache from session-store.db #946 landing branch and v22 by PR fix(models): price Codex activity ids via the official underlying model #1056, so reusing either would let two incompatible schemas share a filename.feat/core-extractionsits at 26 and reconciles at its final merge by keeping the max.scripts/upgrade-path/run.mjstracks the bump (daily-cache.v23.json), which the upgrade-path CI asserts.compare.mjsgains aCOST_CHANGED_BY_DESIGNlist holdingcodex: it keeps the full exact treatment for the call count and every token field, and only lifts the 0.5% cost tolerance, reporting the delta instead. Dropcodexfrom that list once a published CLI carries this fix.Verification
npm test: 2970 passed, 5 skipped, 0 failed (219 files).npx tsc --noEmit: clean.npm run build: clean.npm run verify:upgrade: PASSED, includingdaily-cache.v23.json re-derived.codex.tscost site →prices a fresh codex parse from output_tokens aloneparser.tscost site →codex-pricing-1075-rehydrate.test.ts(drives the full pipeline cold then warm)THE GUARD: leaves cache writes in the input bucket…(gpt-5.5 with a nonzerocache_write; cost identical to before the fix)CODEX_CACHE_VERSION, andDAILY_CACHE_VERSION/MIN_SUPPORTED_VERSIONeach have their own failing-on-revert test.tests/audit-report.test.tsandtests/models-report.test.tseach asserted the old additive behaviour for a codex/claude call. The additive branch is still covered — the models-report case moved tohermes, and the new suite asserts codex1000vs hermes1400side by side.Known adjacent, deliberately not in this PR
Codex throughput double-counts reasoning the same way (
src/providers/codex.tsactiveGeneratedTokens/taskGeneratedTokens, andsrc/codex-throughput.ts'sgeneratedTokens), so tok/s reads high. That is a display metric with its own test surface and no bearing on cost; it deserves its own PR rather than being smuggled into a pricing fix.Closes #1075