Skip to content

Always offer BB's curated Claude models and preload the picker#863

Merged
SawyerHood merged 1 commit into
mainfrom
bb/check-recent-claude-model-commit-thr_bktriwey9d
Jul 23, 2026
Merged

Always offer BB's curated Claude models and preload the picker#863
SawyerHood merged 1 commit into
mainfrom
bb/check-recent-claude-model-commit-thr_bktriwey9d

Conversation

@SawyerHood

Copy link
Copy Markdown
Collaborator

Summary

  • always offer BB's curated, version-pinned Claude models and make account-scoped discovery purely additive, so a narrow probe result can never strand the model picker
  • preload the picker from the last catalog this account reported, so it is populated instantly instead of waiting ~2-3s on a CLI process spawn, and a selection made during that window does not snap back
  • substitute the curated rows server-side when the probe fails transiently
  • remove Mythos from the curated catalog
  • bump HOST_DAEMON_PROTOCOL_VERSION because provider.list_models semantics changed

Problem

Claude's model probe spawns a CLI process on the host. Measured on this branch: provider.list_models takes 2.1-2.6s and /api/v1/system/execution-options 2.2-2.3s. Until it returned, the picker was empty. A probe that reported a narrow list could strand it entirely.

#835 had made the catalog curated ∩ discovered to fix Mythos being offered to accounts that could not run it. That fixed phantoms but left the picker fully dependent on a slow, failable probe.

Approach

Catalog rule inverted. buildClaudeCodeModels now emits CLAUDE_CODE_ACTIVE_CATALOG unconditionally and appends only discovered ids BB has no curated entry for. Selected-only rows stay discovery-gated — they exist to label a selection the user already has, so there is nothing to keep stable. Mythos is deleted outright; the discovered-passthrough path already surfaces it with Anthropic's own label for entitled accounts.

Catalog moved to @bb/agent-providers, since the daemon (filtering), server (fallback), and app (preload) all need the same rows. DEFAULT_CLAUDE_CODE_MODEL and the xhigh reasoning ladder are single-sourced there too.

Preload. useSystemExecutionOptions sets react-query placeholderData for claude-code from the last catalog the account reported, cached in localStorage keyed by host/environment. Because those ids match what the fresh probe returns, a pick during the preload window survives. Curated rows are the cold-cache path only.

Failure fallback. The server substitutes the curated rows in loadSystemProviderModels's catch, gated to timeout/failed — not missing_executable/auth_required, which the app routes to an install/auth prompt where offering models would only defer the failure to submit time.

The invariant that keeps this safe. A provisional catalog must never look authoritative. Recovery is now gated on modelCatalogIsUnverified, covering both a load error and placeholder data, so neither can retire a stored selection — only a successful probe can. Caching is likewise gated on modelLoadError === null, so the failure fallback cannot later masquerade as the account's real models.

Trade-off

Deliberate and reviewer-visible: a curated row can name a model an account cannot run — an entitlement it lacks, or a CLI older than Fable 5's v2.1.170 requirement. Verified live on this account, whose probe does not report claude-opus-4-7[1m] yet is now offered it. That failure surfaces from Claude Code at turn time rather than as a clean 400 from BB, because absence from the list is no longer evidence of unavailability. In exchange the picker is always usable. Both sites carry comments saying so.

Protocol bump

provider.list_models results change meaning from "account-verified" to "curated ∪ discovered". An enrolled daemon on the old build would keep returning a discovery-filtered list, so the bump is what forces it to update. The contract test pins 65 and records why.

Validation

Verified against a live dev instance — the curated four are present including claude-opus-4-7[1m] (not reported by this account's probe), plus additive claude-haiku-4-5-20251001 and claude-horchata-eap[1m].

  • @bb/app 2077/2077
  • @bb/agent-runtime 795/795
  • @bb/agent-providers 9/9, @bb/host-daemon-contract 50/50
  • @bb/server 1253/1255 — both failures pre-existing and environment-dependent: a umask-sensitive skill-tree fixture (asserts 0644, local umask 0002 creates 0664) and plugin-update.test.ts, which passes 13/13 in isolation
  • turbo run build typecheck lint passes on all five affected packages

model-list.test.ts's old "returns an empty catalog when the provider reports no models" asserted exactly the behavior this PR removes; it is rewritten as "still offers the curated catalog when the provider reports no models" so the suite guards the new contract instead of losing the coverage.

🤖 Generated with Claude Code

Claude's model probe spawns a CLI process on the host and takes 2-3s, so
the model picker sat empty until it returned, and a probe that reported a
narrow list could strand it entirely.

Invert the catalog rule: BB's curated, version-pinned Claude models are
always offered and account-scoped discovery is purely additive, appending
only ids BB has no curated entry for. Selected-only rows stay
discovery-gated since they exist to label an already-stored selection.
Drop Mythos, which discovery surfaces with Anthropic's own label for
entitled accounts.

Preload the picker so it is never empty: the app renders placeholder data
from the last catalog this account reported (cached in localStorage,
keyed by host/environment), falling back to the curated rows on a cold
cache. Because the cached ids match what the fresh probe returns, a
selection made during the preload window survives instead of snapping
back to a default. The server substitutes the same curated rows when the
probe fails transiently, excluding missing_executable and auth_required
because the app routes those to an install/auth prompt.

Provisional catalogs must never look authoritative: only a successful
probe may retire a stored selection, so recovery is now gated on
modelCatalogIsUnverified, which covers both a load error and placeholder
data. Caching is likewise gated on modelLoadError being null so the
failure fallback cannot masquerade as this account's real models.

The trade-off is deliberate: a curated row can name a model an account
cannot run (an entitlement it lacks, or a CLI too old for Fable), and
that now surfaces at turn time rather than at selection time. In
exchange, the picker is always usable.

Bump HOST_DAEMON_PROTOCOL_VERSION because provider.list_models semantics
changed; an older daemon would keep returning a discovery-filtered list.

Tests: @bb/app 2077, @bb/agent-runtime 795, @bb/agent-providers 9,
@bb/host-daemon-contract 50. @bb/server 1253/1255, with both failures
pre-existing and environment-dependent (a umask-sensitive skill-tree
fixture asserting 0644, and plugin-update.test.ts, which passes 13/13 in
isolation). build, typecheck, and lint pass on all affected packages.

Co-Authored-By: Claude <noreply@anthropic.com>
@SawyerHood
SawyerHood merged commit a18a996 into main Jul 23, 2026
10 checks passed
@SawyerHood
SawyerHood deleted the bb/check-recent-claude-model-commit-thr_bktriwey9d branch July 23, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant