fix(models): price Codex activity ids via the official underlying model - #1056
Merged
Merged
Conversation
Extra High MERGE AFTER FIX on getagentseal#1056: bump Codex/session/daily cache so the 659M-token corpus reprices; do not invent codex-code-review.
Member
|
Reviewed in depth with real-data runs — this is the factually supported direction: OpenAI's own tracker says auto review consumes normal model usage (openai/codex#32224), so pricing it is right and #1050's
The three cache bumps (CODEX_CACHE_VERSION 9→10, parse |
This was referenced Aug 21, 2026
…view pricing getagentseal#1055 already took CODEX_CACHE_VERSION 10 and session-meta-fields-v1. Keep both parse-version tokens and bump Codex results to 11 so a take-ours merge cannot drop either invalidation.
Maintainer review on getagentseal#1056: pin the activity id to the recommended review model (existing bundled row, not an invented rate), resolve the same alias for credits, and take daily-cache 22 so this PR does not collide with getagentseal#946 claiming v21.
iamtoruk
added a commit
to ozymandiashh/codeburn
that referenced
this pull request
Aug 21, 2026
Base has moved since this branch opened: main shipped 17 in v0.9.20 and now sits at 20, with 21 (getagentseal#946) and 22 (getagentseal#1056) claimed on the main-side pipeline. Bumping to only 16 would let a main-built cache pass the version check unchanged, so the widened Codex discovery re-derivation this PR depends on would never fire for those users. Take 23 to stay above every value a real cache file can carry on either line of history, and update the test literal and MIN_SUPPORTED_VERSION to match. Also documents, next to CODEX_CACHE_VERSION, why that constant is deliberately not bumped alongside it: the guarded record shapes were measured at 0 occurrences across 136k real events, so forcing a full re-parse of multi-GB rollout corpora for them is a bad trade. The daily-cache bump alone already propagates the discovery widening.
ozymandiashh
previously requested changes
Aug 21, 2026
ozymandiashh
left a comment
ozymandiashh
left a comment
Collaborator
There was a problem hiding this comment.
This needs a rebase plus fresh cache generations; resolving the conflicts mechanically would create real stale-cache collisions. The branch uses Codex results v11 and the old unsuffixed file while current main already uses v11 for a different invalidation and now has version-suffixed cache files. It also uses daily cache v22 while main is v23, and drops codex-pricing-v1 instead of composing it with activity-price-v1 in the session parse token. Please rebase, allocate the next non-colliding versions (Codex v12 / daily v24 on the current main), preserve both invalidations, and rerun upgrade-path/cache migration coverage.
pull Bot
pushed a commit
to TheTechOddBug/codeburn
that referenced
this pull request
Aug 21, 2026
…at the explicit rate only Reasoning tokens are a subset of output_tokens for OpenAI models, not an extra bucket: on a 1,396-rollout corpus all 134,316 token_count events carrying a total satisfy input + output == total. CodeBurn added reasoning_output_tokens on top when pricing a codex call, in the cache-rehydration re-price, and in the models/audit display sums. That overstated codex cost by $166.03 (3.5%) and displayed output tokens by 34.6% on that corpus. Both cost sites and the display sums now go through one shared billableOutputTokens() so a cold parse and a warm read cannot drift apart. cache_write_input_tokens (codex PR #33454) was never read and cacheCreationInputTokens was hardcoded to 0. It is now carved out of the uncached-input bucket and clamped to it, but routed to the cache-write bucket ONLY when the pricing source publishes an explicit cache-write rate. buildCosts fabricates 1.25x input when a source omits one, which is correct for Anthropic and would have invented a surcharge OpenAI never charged on gpt-5.5 / 5.4 / 5.3-codex / gpt-5. ModelCosts now carries cacheWriteCostIsExplicit so that distinction survives getModelCosts. A cost change invalidates persisted output: codex-results.json v10 -> v11 (stores costUSD verbatim), the codex parse version moves (the token-bucket change does not self-heal on read), and the daily cache goes 20 -> 23 (21 is claimed by the getagentseal#946 landing branch and 22 by PR getagentseal#1056). The upgrade-path corpus asserts codex tokens and calls exactly and reports the repricing. Closes getagentseal#1075
Main already spent Codex results v11 and daily v23 on getagentseal#1075, and result caches are now version-suffixed. Keep both parse-version tokens (codex-pricing-v1 + activity-price-v1) so a take-ours merge cannot drop the pricing invalidation. Do not Extra High again.
Main already spent Codex results v13 (getagentseal#1079) and daily v24 (getagentseal#1090). Keep all three parse-version tokens (codex-pricing-v1 + codex-tps-v1 + activity-price-v1) so a take-ours merge cannot drop the pricing or throughput invalidation. Do not Extra High again.
avs-io
pushed a commit
to avs-io/codeburn
that referenced
this pull request
Aug 21, 2026
Neither id is in LiteLLM yet, so litellm-snapshot.json had no row for them (285 sessions / 5,446 calls priced at $0 on one real corpus, flagged during getagentseal#1075 verification). Add explicit snapshot rows sourced from the pattern every prior Codex-suffixed id LiteLLM does carry follows: it bills identically to its bare-model sibling of the same generation (gpt-5-codex == gpt-5, gpt-5.1-codex == gpt-5.1-codex-max == gpt-5.1, gpt-5.2-codex == gpt-5.2, gpt-5.3-codex == gpt-5.3, verified against the live model_prices_and_context_window.json) - both new ids get the exact gpt-5.6 tuple rather than an invented rate. getModelCosts already resolved both ids to the correct rate through the `gpt-5.6` prefix fallback before this, so a fresh parse or a warm session-cache read (codeburn models included) was already pricing these calls correctly. The daily cache is the one place that does not self-heal: it has no per-provider invalidation, so a day finalized before a `gpt-5.6` snapshot row existed at all keeps that $0 forever. Raising MIN_SUPPORTED_VERSION (v23 -> v24) forces the one-time re-derivation, a lossless no-op for days already correct - the mechanism itself is generic and already covered by daily-cache-version-rederivation.test.ts. getagentseal#1056 also claims 24 on its own branch; whichever lands second takes the next number. Closes getagentseal#1077
Main already spent Codex results v14 (getagentseal#1092). Daily stays 25 (getagentseal#1090 still the last daily bump). Keep all four parse-version tokens (codex-pricing-v1 + codex-tps-v1 + codex-mcp-skills-v1 + activity-price-v1) so a take-ours merge cannot drop the pricing, throughput, or MCP/skills invalidation. Do not Extra High again.
iamtoruk
dismissed
ozymandiashh’s stale review
August 22, 2026 11:14
Dismissing per maintainer decision: this review was filed against pre-rebase commit 61e76b7 and the three rebase commits since (33d344a..9e24d09) address it point-for-point — codex cache 15 (14 spent by #1092), daily 25 (24 by #1090), and the parse-version string now composes -codex-pricing-v1 with -activity-price-v1 instead of dropping it. Independently verified file-by-file against current main: models.ts/codex-credits.ts/snapshot are clean unions, providers/codex.ts is byte-identical to main, full suite green, and a real-corpus spot-check shows only the codex-auto-review row changing (+$775.43, zero collateral drift). Thanks for the review — it flagged exactly the right risks and they got fixed.
iamtoruk
added a commit
to avs-io/codeburn
that referenced
this pull request
Aug 22, 2026
Resolves conflicts against main after getagentseal#1078/getagentseal#1084/getagentseal#1088/getagentseal#1090/getagentseal#1092/ getagentseal#1053/getagentseal#1056 landed since this branch's last upstream merge. - CHANGELOG.md: kept both entries (this PR's getagentseal#968/getagentseal#1050 note plus main's getagentseal#1079/getagentseal#1088, getagentseal#1082, getagentseal#1075 notes that had moved into the same "### Fixed" slot). - Everything else (src/models.ts, main.ts, usage-aggregator.ts, daily-cache.ts, tests/*) merged cleanly with no conflict markers; git's recursive merge combined getagentseal#1050's flat-rate classifier changes with getagentseal#1056's codex-auto-review -> gpt-5.5 alias without overlap. Verified the codex-auto-review / getagentseal#1056 interaction post-merge: isBuiltInFlatRateModel no longer matches codex-auto-review (dropped per getagentseal#1050), while MODEL_ALIASES still aliases it to gpt-5.5 (getagentseal#1056), so it prices at GPT-5.5 rates rather than $0. Covered by the existing tests/models.test.ts "Codex activity ids (getagentseal#1047)" describe block and the "does not treat a priced sibling as expected-free" case. getFlatRateModelsConfigHash's output is folded into getDailyCacheConfigHash's template literal unconditionally (flatRateModels=<hash>), so the flat-rate section always participates in the daily-cache invalidation hash even when empty -- no DAILY_CACHE_VERSION bump needed for this change.
iamtoruk
added a commit
to avs-io/codeburn
that referenced
this pull request
Aug 22, 2026
The merge with main pulled in getagentseal#1056's codex-auto-review -> gpt-5.5 alias, which findUnpricedModels' pre-existing hasBillableRate check now resolves for the raw id, so a synthetic $0 row for it is no longer reported as unpriced (it correctly has a billable rate). Removed it from the expected findUnpricedModels() output in the "skips subscription / flat-rate product SKUs" test; the display-name variant ('Codex Auto Review') is unaffected since getModelCosts does not resolve display names, so it stays in the expected list. The codex-auto-review / getagentseal#1056 pricing interaction itself is already covered by the "Codex activity ids (getagentseal#1047)" describe block.
iamtoruk
added a commit
to kelchm/codeburn
that referenced
this pull request
Aug 22, 2026
Resolves the conflicts getagentseal#946 accumulated while it was in validation. Eight files conflicted; the session-store accounting is unchanged. src/daily-cache.ts — version collision. This PR minted 25 when main was at 24; getagentseal#1056 (`codex-auto-review` pricing) then spent 25 on main. The bump moves to 26/MIN 26 and daily-cache.v26.json, with main's full comment ladder kept as the foundation and this PR's paragraph rewritten to name 26 and record the collision. PENDING_REDERIVE_PROVIDERS and the B1 migration semantics from b6481c1 carry over intact, retargeted at 26. src/models.ts, src/parser.ts, src/audit-report.ts, src/models-report.ts — getagentseal#1075/getagentseal#1078 replaced the per-site "reasoning is already inside output" tests with billableOutputTokens() and REASONING_INCLUDED_IN_OUTPUT. This PR had added copilot to that case at three sites independently. Union: all three sites take main's helper call verbatim, and copilot joins claude and codex in the set — same accounting this PR shipped, now through main's single source of truth. It also reaches parser.ts activeGeneratedTokens (a fourth site, from getagentseal#1079), which is the same correction: a copilot supplementary call carries reasoning with output 0, so counting it as generated repeats the per-turn output. The audit legend already said so on this side. src/providers/copilot.ts — comment-only. getagentseal#1054's lastEventTimestamp-first shutdown fallback was derived from this branch, so the code was already identical on both sides: the `shutdownTimestamp` expression and the `copilot:<sid>:shutdown:<model>:<n>` key are byte-for-byte main's. Both rationales are kept (leg-collapse on date, and residual anchoring). src/session-cache.ts — PROVIDER_PARSE_VERSIONS.copilot takes this PR's `-session-store-v3` suffix; main's getagentseal#1051 note about why a fingerprint change is expensive is kept above it. Codex keeps main's getagentseal#1092 suffix chain untouched. src/main.ts — getagentseal#1067 deleted the unreachable live dailyMap fallback that this PR had taught behavioral weight. Main's deletion wins; the now-unused isBehavioralTurn import goes with it. tests/parser.test.ts — import union. Also: scripts/upgrade-path/run.mjs NEW_DAILY_CACHE -> daily-cache.v26.json, CODEBURN_COPILOT_SESSION_STORE_DB added to the getagentseal#1064 env-isolation CLEARED list, and the CHANGELOG entry's stale "v21" corrected to v26. Verified: tsc clean; 3132 tests pass across 223 files; test:locks 26/26; verify:upgrade PASSED, re-deriving daily-cache.v26.json and holding durable copilot history across the bump. getagentseal#1054's regression ("keeps three stampless shutdown legs as :n keys with lastEventTimestamp") passes on the merged tree. Real-corpus A/B against origin/main over 2026-07-01..2026-08-22: codex, grok, kimicode and opencode byte-identical in export, audit and models; claude drifts only monotonically with run order (a live session writing transcripts, confirmed by interleaving four runs). This machine has no copilot data, so the copilot recovery semantics rest on the suites and the upgrade-path corpus.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
codeburn models/ the unpriced warning reportsCodex Auto Reviewat $0. On one real corpus that was 659M tokens (#1047). The id is a Codex activity surface, not a missing LiteLLM row and not a subscription SKU.Closes #1047.
Root cause
Rollouts store
codex-auto-review. That string has a display name but no catalog row, socalculateCostreturns 0. Official rate card (help.openai.com/articles/20001106): auto review uses the recommended review model and draws from the same Codex credit pool as other work. $0 is the wrong honesty.Builtin aliases are not in
getModelAliasesConfigHash(), so a warm Codex result cache exact hit and a finalized daily-cache day keep the pre-fix $0 forever.Change
Same class as
cursor-auto/copilot-openai-auto:codex-auto-reviewto the existinggpt-5.5LiteLLM rowautoModelNamesCODEX_CACHE_VERSION15 (main's v14 is the fix(codex): attribute MCP + skills from the exec custom tool and the item model (#478) #1092 MCP/skills fix; v13 is codex tok/s double-counts reasoning tokens in generated-token throughput #1079 tok/s; v12 is feat/core-extraction feat(cli): rehome cline-cli and codex throughput (landing #940) #1086),PROVIDER_PARSE_VERSIONS.codexkeepscodex-pricing-v1+codex-tps-v1+codex-mcp-skills-v1and appendsactivity-price-v1,DAILY_CACHE_VERSION/MIN_SUPPORTED_VERSION25 (v21 feat(copilot): read per-request input/cache from session-store.db #946, v22 this PR's earlier claim, v23 Codex: price cache_write_input_tokens, avoid reasoning-output double count, and apply long-context tiers #1075, v24 fix(models): price gpt-5.6-codex and gpt-5.6-codex-max #1090; fix(codex): attribute MCP + skills from the exec custom tool and the item model (#478) #1092 left daily alone)codex-code-review(not in Codex source or real rollouts)User impact
Auto-review tokens price at GPT-5.5. The models table still says "Codex Auto Review". Existing caches re-derive; this is a lossless daily bump (v24 kept as baseline).
Preservation
No new LiteLLM rates. No
codex-*prefix collapse. No date-invented Terra mapping. No Keychain / Buzz / #868 / #916 / #998. No second Extra High.Note for #1050: that PR lists
codex-auto-reviewas a built-in flat-rate SKU. This PR says the opposite. When they meet, drop the id from the #1050 classifier — do not Extra High #1050 again.Testing
getModelCosts('codex-auto-review') === getModelCosts('gpt-5.5')codex-code-review, barecode-review) stay unpriced/tmp/codeburn-xhigh-review/VERDICT-1047.md→ MERGE AFTER FIX. Named class closed. Do not Extra High again.