Skip to content

Latest commit

 

History

History
2005 lines (1713 loc) · 145 KB

cli-reference.md

File metadata and controls

2005 lines (1713 loc) · 145 KB

Provider CLI reference

Last updated: 2026-10-27

Reference for the darkbloom command-line tool: every subcommand and flag, the files and identifiers it creates, the provider.toml keys it reads with their defaults, the environment variables it forwards to the daemon, and its runtime constants, as declared in provider-swift/Sources/darkbloom/ (Darkbloom, version ProviderCore.version in provider-swift/Sources/ProviderCore/ProviderCore.swift). For operators; types and defaults are the ArgumentParser declarations; — means required.

waiting_inventory means a selected model's ordinary update introduced an ID outside the Autopilot set. Enrollment remains saved, ordinary serving continues, and the update's verified prefetch or darkbloom autopilot models adds it to the set before control can resume. Waiting and shadow Autopilot preserve the normal picker and --model serving selection. Other verified cached models are reported separately for planning and cannot be loaded by ordinary routing. See Autopilot architecture.

Local GLM qualification

darkbloom start --local --model EigenLabs/GLM-5.3-Flash-MLX-NVFP4-mtp can include the experimental GLM root only when DARKBLOOM_EXPERIMENTAL_GLM5_NEXT names an absolute qualification JSON file. Unset, empty, 1, malformed selections and mismatched identities refuse GLM. The file pins the exact catalog record, manifest and expected aggregate; see configuration for its schema. GLM always uses its native paged KV backend: automatic selection resolves paged, and an explicit contiguous selection or the DARKBLOOM_CBV2_PAGED_KV kill switch refuses the load instead of falling back. GLM serves only through its embedded-MTP engine, which is on by default: the default mtp_mode = "auto" enables it without any setting (MTPMode.enablesMTP). mtp_mode = "off" or the DARKBLOOM_CBV2_MTP=0 kill switch skips the model at startup with a message, because GLM cannot serve target-only. This opt-in does not select a performance profile, bypass memory admission or qualify the model. It is not forwarded into launchd. Stop the isolated local process to disable it. There is no target-only GLM serving engine; target-only construction is limited to benchmarks.

Every GLM load uses the native GLM5NextExecutionPolicy.serving arithmetic (absorbed sparse attention with native decode selection and sparse prefill kernels, parallel-threshold selection, native KDA recurrence, fused hyper-connections and routed-expert combine), recorded in the encrypted checkpoint identity. The encrypted SSD prefix store defaults on for EigenLabs/GLM-5.3-Flash-MLX-NVFP4-mtp; DARKBLOOM_PREFIX_CACHE=0 opts out. GLM MTP verifies up to three drafted tokens per round in one target forward with an adaptive depth; its greedy output is not token-identical to target-only decode. DARKBLOOM_GLM_RECTANGULAR_VERIFY=0 selects serial single-proposal verification instead. Prefill chunks and steps are each at most 2,048 tokens, and solo prefill stripes are off, including DARKBLOOM_CBV2_SOLO_PREFILL_STRIPE. While other requests decode, each prompt chunk is followed by decode-only steps; DARKBLOOM_GLM_DECODE_STEPS_BETWEEN_CHUNKS sets how many (default and range), and 0 restores a chunk in every step. Requests that omit sampling fields use the artifact's generation_config.json (temperature 1.0, top_p 0.95). Image input runs through the native GLM media path; the embedded-MTP serving engine accepts prepared causal media, so image rows prefill and decode target-only (no MTP speculation, no prefix reuse). Video input (video_url with an inline data:video/mp4 or data:video/quicktime URL, constant or variable frame rate) is sampled at 2 FPS and decoded target-only. This variable never advertises the local identity to the coordinator (provider-swift/Sources/ProviderCore/Inference/Models/GLM5Next/GLM5NextLocalQualification.swift, GLM5NextLocalQualification). Coordinator-connected darkbloom start serves only the catalog build glm-5.3-flash, downloaded by the normal revision path, on Apple M5 Max or M5 Ultra with at least 256 GB; other Macs report the exact refusal (chip and memory) in the load error and doctor (provider-swift/Sources/ProviderCore/Inference/Models/GLM5Next/GLM5NextCoordinatorServing.swift, GLM5NextCoordinatorServing; provider-swift/Sources/ProviderCore/Models/GLM5NextHardwareRequirement.swift). --local does not serve the catalog build; use the selection file above.

darkbloom cache

Persistent settings for encrypted inference caches; downloaded model weights use darkbloom models location separately. Changes apply when serving starts again. Commands use --config and never migrate, erase, format or mount a disk (provider-swift/Sources/darkbloom/Cache/CacheCommand.swift, Cache; Cache/CacheConfiguration.swift, updateCacheSettings).

Command / flag Behavior
cache status Read saved settings and inspect the selected volume without loading keys or creating cache directories
cache status --json Emit directory, saved daily byte limit and unlimited flag (omitted when unset), limit source, selected volume details, storage problem and saved_settings scope
cache set --daily-write-gb <number> Save the rolling-day write ceiling in decimal GB; 0 explicitly selects unlimited writes
cache set --directory <absolute-path> Select an existing private directory and pin its volume UUID. Payloads live in its darkbloom/kv3 subtree (a dev build: its own namespace)
cache set --reset-directory Return to the built-in directory; preserve the daily write choice and existing cache files

Both setting flags can be combined. The saved daily choice takes precedence over DARKBLOOM_PREFIX_CACHE_SSD_MAX_WRITE_GB_PER_DAY, including an older value baked into launchd. When no daily choice is saved, the environment/default rules in configuration still apply. Invalid, missing or changed selected storage disables SSD caching; inference can continue cold. See selecting cache storage for disk preparation, limits and verification. These checks do not certify disk firmware.

Desktop frontend control

darkbloom desktop serve --config <path> runs the authenticated loopback API. darkbloom desktop ensure --config <path> installs/starts its user LaunchAgent, reinstalling it when the plist changed and restarting it once when the running API reports a different version than the invoking CLI. The desktop frontend sends validated actions to existing CLI operations and keeps the provider independent of its window lifetime. Internal desktop start-local --model <id>, stop-local, and restart-local commands (each accepting --config) retain the native provider owner and termination handler. Cooling uses desktop configure-cooling --enabled <bool> --speed <percent> --temperature <celsius> to request macOS administrator authorization for the existing signed helper. darkbloom desktop login [--config <path>] signs into the account workspace through browser approval; darkbloom desktop logout [--config <path>] signs out without changing provider linkage or serving state. Both use the same CLI and native account store as Electron (Desktop.Login, Desktop.Logout). It prefers a scoped account session and supports existing coordinator device-code grants when the desktop scope is not reported. Dashboard credentials stay separate from the running provider (DeviceLoginScope, DesktopAccountCredential). See desktop control for the exact routes, credentials, outcomes, and limits (Desktop, DesktopBackend).

Global options

The hidden --native-serve-qualification-v1 <ws-url> <private-directory> entry is an explicitly enabled physical diagnostic, not an installed-service command. NativeServeQualificationCommand in provider-swift/Sources/darkbloom/NativeServeQualificationCommand.swift runs a separate AppKit host without Start, APNs registration or daemon replacement. It requires DARKBLOOM_NATIVE_SERVE_QUALIFICATION=1, a signed release app, literal 127.0.0.1 WebSocket transport and owner-only inputs. See native serving qualification for isolation, evidence and untested-scope limits.

Every darkbloom invocation on macOS below 27 prints an informational upgrade warning to stderr before command parsing or AppKit hosting, including help, version and background commands. MacOSUpgradeNotice.emit in provider-swift/Sources/darkbloom/MacOSUpgradeNotice.swift names the local OS, upcoming Darkbloom MDM deactivation, continued legacy verification during the transition and the need to retain the profile until App Attest migration is approved. The warning performs no network/config/profile operations and does not change command execution, exit codes or stdout/JSON. macOS 27+ prints no upgrade warning. It is independent of DARKBLOOM_NO_UPDATE_CHECK.

Option Type Default Effect Source
-c, --config <path> String? ~/.config/darkbloom/provider.toml Provider TOML path. Accepted by the commands marked ✓ below provider-swift/Sources/darkbloom/Darkbloom.swift (ConfigOptions); provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift (defaultConfigPath)
--version flag — Prints ProviderCore.version Darkbloom.configuration
-h, --help flag — Help; darkbloom with no subcommand prints help Darkbloom.run

Before most subcommands run, runUpdateBannerIfEnabled (provider-swift/Sources/darkbloom/Darkbloom.swift) checks for a newer release with a 2 s hard timeout and prints a one-line banner; DARKBLOOM_NO_UPDATE_CHECK set to any value skips it. Logging goes to stderr so launchd captures it in ~/.darkbloom/provider.log.

Subcommands

Subcommands declared by Darkbloom.configuration.subcommands:

Command Purpose --config Source (provider-swift/Sources/darkbloom/…)
cache Inspect or set encrypted-cache storage and daily write limits ✓ Cache/CacheCommand.swift (Cache)
start Serve. Default: install and start the LaunchAgent; --local for a coordinator-less server ✓ StartCommand.swift (Start)
schedule Edit, show or disable saved weekly availability and startup loading; never start/stop the service ✓ Scheduling/ScheduleCommand.swift (AvailabilitySchedule)
switch Gracefully replace hosted models in the running coordinator-connected provider, without restart or reconnect SwitchCommand.swift (Switch)
addon Debug add-on holds on the running provider: status (default), hold, release AddonCommand.swift (Addon)
stop Drain accepted requests, then stop the LaunchAgent; --uninstall removes the provider, watchdog and desktop API agents StopCommand.swift (Stop)
restart Drain, restart with recorded configuration, and confirm fresh authorization ✓ RestartCommand.swift (Restart)
status Config, hardware, schedule, live daemon state (including the coordinator's last Trust: <level> / <status> message), per-slot KV/MTP posture ✓ StatusCommand.swift (Status)
doctor Diagnostics (see troubleshooting) ✓ DoctorCommand.swift (Doctor)
models list, catalog, download, remove ✓ ModelsCommand.swift (Models)
local Print the direct-mode endpoint and API key LocalCommand.swift (Local)
login Link the machine to an account (RFC 8628 device code) ✓ LoginCommand.swift (Login)
logout Stop the provider, revoke its current saved token, then remove the local credential ✓ LogoutCommand.swift (Logout)
benchmark Inference benchmarks and harnesses ✓ BenchmarkCommand.swift (Benchmark)
update Self-update ✓ UpdateCommand.swift (Update)
verify doctor --strict ✓ VerifyCommand.swift (Verify)
enroll Show App Attest setup guidance or check frozen legacy eligibility before profile setup ✓ EnrollCommand.swift (Enroll)
unenroll Choose full exit or MDM removal with App Attest UnenrollCommand.swift (Unenroll)
logs Unified logs for subsystem dev.darkbloom.provider LogsCommand.swift (Logs)
report Upload recent unified logs to the coordinator ✓ ReportCommand.swift (Report)
autoupdate Toggle provider.auto_update ✓ AutoUpdateCommand.swift (AutoUpdate)
beta list, status, enable, disable beta features ✓ BetaCommand.swift (Beta)
idle Configure the saved idle-memory policy ✓ IdleCommand.swift (Idle)
autopilot Experimental enrollment/status and residency policy; shadow by default ✓ AutopilotCommand.swift (Autopilot)
fan Experimental fan control (status, diagnose, enable, configure, disable, uninstall) Fan/FanCommand.swift (Fan)
watchdog Internal, hidden: crash-recovery watchdog process ✓ WatchdogCommand.swift (Watchdog)
runtime-smoke Internal, hidden: load packaged Metal runtime and exit RuntimeSmokeCommand.swift (RuntimeSmoke)

darkbloom start

Flag Type Default Effect
--coordinator-url <url> String? coordinator.url (wss://api.darkbloom.dev/ws/provider) Override the coordinator WebSocket URL
--model <id> [String], repeatable [] Select these startup models and skip the picker; compatible with Autopilot enrollment
--all flag false Ordinary serving of every runtime-supported local model; skips the picker. Compatible with --autopilot, where only verified eligible downloaded network builds enroll
--idle-timeout <mins> UInt64? backend.idle_timeout_mins (60) Override the idle unload timeout for this run
--schedule flag false Open the interactive availability/loading wizard before background startup; rejects --foreground and standalone --local (Start.run, StartCommand.swift)
--autopilot flag false Keep the normal startup selector, verify cached network inventory and save experimental enrollment; verification adds no downloads and the default rollout is shadow
--no-autopilot flag false Save opt-out and retain ordinary idle-policy mode
--foreground / --no-foreground flag, hidden false Serve in this process instead of installing the LaunchAgent; launchd passes it
--local flag false Coordinator-less OpenAI-compatible server (direct mode)
--local-endpoint flag false Local endpoint alongside the coordinator; mutually exclusive with --local
--port <n> UInt16 8000 Local server port
--bind <addr> String 127.0.0.1 Local server bind address
--no-auth flag false Disable the local bearer-token check
--timeout <seconds> integer, 0–3600 600 Drain a running provider before replacing its process/configuration
--force flag false Explicitly permit cancellation if the old provider cannot drain

Exit 1 (ExitCode.failure) when --local and --local-endpoint are combined, a debugger is attached, RAM is below 8 GB, Metal is unavailable, hardware detection fails, no model is selected, or the local server does not bind within 5 s (StartCommand+Preflight.swift, StartCommand+Modes.swift). A selection that would serve GLM-5.3-Flash below the macOS GPU wired-memory limits prints a warning: with the fix and continues (GLM-5.3-Flash GPU wired-memory limit).

The ordinary catalog picker uses the selected models' shared activation reserve when checking each model's load estimate. Adding an unmeasured model can raise that reserve: the TUI refuses a toggle that would invalidate an existing choice, and numbered input validates the complete requested set. all first excludes models that cannot fit alone, then filters the remaining rows under their shared reserve. Total selected weights may exceed the budget because models can take turns. These are nominal estimates; live admission still applies the configured reserve and current available memory. A resolved Autopilot opt-in retains the conservative flat picker reserve used by its separate inventory check (Start.PickerReserveMode, Start.resolveFallbackSelection in provider-swift/Sources/darkbloom/Start/StartCommand+Picker.swift).

Autopilot inventory verification names each cached model before checking its bytes. If another process holds that model’s update/verification lock, the check reports the busy model immediately. An ordinary start with saved consent then fails before changing enrollment or draining the running provider; retry after the update finishes. status and autopilot status read the same daemon snapshot, including nested Autopilot residents and load history. Enrollment does not make a live daemon disappear from status. autopilot status prints Adopted (serving outside the selection) for models an operation adopted and Held: <model> until <UTC> (<time> left) for each placement hold (provider-swift/Sources/darkbloom/Autopilot/AutopilotStatusCommand.swift).

A replacement start, with or without Autopilot, completes the picker/preflight and saves the selected IDs under backend.enabled_models while holding the lifecycle lease, before it disables recovery or drains/stops the current provider. Failure of this initial selection write leaves the current service unchanged. Autopilot additionally verifies cached inventory before persistence or drain; selected startup models must be in that verified inventory. Enrollment itself preserves other preferences. After drain acknowledgement, Autopilot consent is saved before stopping the daemon and installing the chosen configuration. If this later write fails, a gracefully drained daemon is left running and drained, with recovery disabled; correct the configuration and retry start. No replacement is installed (Start.completeDaemonReplacement). Foreground/local starts also require a drained handoff; the process-lifetime kernel lock never silently sends SIGKILL after a short grace period. On ordinary non-enrolled launchd-managed foreground starts (including restart and watchdog recovery), an explicitly pinned enabled_models takes precedence over old --model plist arguments. A directly invoked foreground --model still overrides config (Start.usesPinnedModelSelection, Start.launchDaemon).

The config sidecar lock spans persistence and synchronous drain setup. If disabling recovery or publishing the request fails, the exact previous TOML bytes (or original file absence) are restored, including whether the model key was pinned. Restoration failures are reported. Once publication succeeds, a later drain timeout retains the replacement intent; no config lock is held while waiting (ProviderModelSelection.withReplacement, provider-swift/Sources/ProviderCore/Service/ProviderModelSelection.swift). Missing custom files are seeded from this invocation's resolved configuration, not from the separate canonical config file.

After a successful provider bootout, the lifecycle waits for launchctl print to confirm that the exact service label is absent before replacing its plist or bootstrapping it. It polls at 100 ms intervals with a 10-second monotonic budget and at most 101 probes. Unknown or permission errors fail the operation; an unconfirmed removal leaves replacement unstarted. This also applies to stop, uninstall and restart after drain. The budget includes returning probe calls; the shared launchctl subprocess has no command timeout, so a hung subprocess can exceed that wall time. Bootstrap error 37 (operation in progress) is a failure, not confirmation that the replacement started. Source: provider-swift/Sources/ProviderCore/Service/LaunchAgent.swift (unloadService, waitForServiceRemoval, loadService).

start --schedule requires interactive stdin and stdout. Its confirmed draft stays in memory until model selection succeeds; cancellation or an empty/failed selection does not save the schedule or replace the running service (Start.run, Start.launchDaemon in provider-swift/Sources/darkbloom/Start/StartCommand+Daemon.swift). Ordinary start does not add a schedule prompt; it uses saved settings. Saved availability also controls an attached --local-endpoint, but standalone --local ignores it.

darkbloom schedule

Flag Type Default Effect Source
No flags interactive editor saved settings Edit weekly availability and startup loading; requires interactive stdin/stdout provider-swift/Sources/darkbloom/Scheduling/ScheduleCommand.swift (AvailabilitySchedule.run, requireInteractiveTerminal)
--show flag false Read saved settings and local time zone, not live daemon state; no edits Same file (AvailabilitySchedule.run)
--disable flag false Disable scheduling while retaining windows and preload policy; mutually exclusive with --show Same file (AvailabilitySchedule.disabled, validate)
-c, --config <path> String? default provider TOML path Read/write this configuration; use the same path when starting it Same file (AvailabilitySchedule.configOptions, run)

The wizard offers these modes in provider-swift/Sources/darkbloom/Scheduling/ScheduleWizard.swift (ScheduleWizard.run):

Mode Saved availability
Use saved windows Re-enable existing windows; requires a saved window
Always available / disable scheduling Serve whenever the provider runs; retain existing windows
Overnight Mon-Fri starts, 22:00 to 08:00 the following day
Weekends Full Saturday and Sunday, 00:00 to 00:00 the following day
Custom weekly windows Start from saved windows and add, edit or remove windows

Enabled modes open the add/edit/remove editor before confirmation. Days accept comma-separated names, weekdays, weekends, daily, or ranges such as fri-mon; days identify the window's start day. Enter keeps the displayed default; q, cancel, EOF or declining confirmation discards the draft (ScheduleWizard.editWindows, editWindow, parseDays, ask).

The loading choice edits only backend.startup_preload: preload at startup/window opening, or skip preload and load on demand. Coordinator load commands can still load models in on-demand mode. The editor preserves backend.preload_models, backend.enabled_models and backend.idle_timeout_mins (ScheduleSettings.apply, save in provider-swift/Sources/darkbloom/Scheduling/ScheduleSettings.swift). It reloads under the config lock and refuses to overwrite concurrent schedule/preload edits.

schedule never starts or stops the provider. Saved edits apply on the next start/restart, not live; use darkbloom restart for a running provider. The Mac must stay awake. See availability configuration and calendar/window semantics.

darkbloom switch

Flag Type Default Effect
--model <id> [String], repeatable [] Replace the complete hosted selection with these local IDs; any invalid ID rejects the whole selection
--all flag false Select all eligible local models; mutually exclusive with --model
--timeout <seconds> integer, 0–3600 600 Graceful drain deadline; 0 refuses unfinished work immediately but still acknowledges an already-settled drain

With neither selection flag, this command reuses the start catalog picker and downloader. It checks fresh daemon identity and switch capability before opening the picker, and uses the running daemon's runtime capabilities rather than initializing a second inference runtime. There is no --force or --config: the daemon's own resolved config path, including a custom path, is authoritative.

The provider validates local artifacts before fencing new coordinator and unified-local admissions, finishes accepted requests and terminal usage, and asks the coordinator to validate the complete selection before unloading anything. It then replaces its full model inventory on the existing coordinator session. It does not restart the process, reconnect, re-attest, install a service, or change watchdog/login recovery settings. Coordinator URL, authentication and local endpoint settings remain unchanged. Selected resident models can stay loaded; new models load on demand under the existing memory safeguards. The coordinator must support models_replace; deploy the coordinator upgrade before enabling this command on providers. Unsupported or missing receipts fail closed rather than forcing a reconnect.

With --timeout 0, an already-idle provider still waits up to 30 seconds for the coordinator's terminal barrier; zero never skips usage settlement. Nonzero deadlines retain their full drain-and-barrier budget. Artifact validation occurs before this deadline and can be preempted by stop/restart or OS shutdown; a late hash result cannot change provider state. Each switch request is atomically consumed, so a later scheduled window in the same process cannot replay it. Sources: provider-swift/Sources/ProviderCore/ProviderLoop+ModelSwitch.swift (drainForModelSwitch, cancelModelSwitchAndWait), provider-swift/Sources/ProviderCore/Service/LifecycleMailbox.swift (claimSwitchRequest).

Validation carries each snapshot's pre-hash fingerprint into the live model state. Where load policy permits hash reuse, unchanged snapshots avoid a second full weight read; metadata changes and mandatory fresh/SSD checks still rehash. Rollback restores the previous hash/fingerprint pair (ProviderModelSwitchValidation, provider-swift/Sources/ProviderCore/Service/ProviderModelSwitchValidation.swift).

Eligible local off-catalog models can remain in the selection for owner-only inference, just as at registration. They do not become publicly routable; tracked models still need their pinned catalog hashes and required runtime capabilities (coordinator/registry/provider_models_replace.go, ReplaceProviderModels).

Success requires a matching completion receipt from the running provider, not just mailbox publication. The provider requires a matching models_replace_resumed receipt from the coordinator before writing that completion receipt. A successful selection is persisted for later restart, watchdog recovery and scheduled serving windows. Stale/missing/older daemons and standalone --local servers fail without launching anything. A drain timeout returns failure and leaves admission closed while accepted work continues; retry switch after the outstanding work finishes. Rejected selections are not partially applied. Live rollback restores an originally absent model key/file when no other edit intervened. Concurrent unrelated config changes are retained; a newer model selection is never silently overwritten. An unavailable completion receipt is reported as unconfirmed, never success, and coordinator routing stays fenced when the committing models_replace_ack cannot be written. If the final models_replace_resumed receipt is lost, routing may already have resumed; switch reports the outcome as unconfirmed. status displays the latest switch outcome, request ID, unfinished-request count, message and selection; stale daemon snapshots are explicitly marked (Status.printDaemonStatus in provider-swift/Sources/darkbloom/StatusCommand.swift). Sources: provider-swift/Sources/darkbloom/SwitchCommand.swift (Switch), provider-swift/Sources/ProviderCore/Service/ProviderModelSelection.swift (ProviderModelSelection.stageReplacement, ProviderModelSelection.restore). After provider readiness, the coordinator refreshes desired alias builds for the current inventory; the provider preserves snapshots received during the commit wait. It restores prefetching before sending readiness, so the refreshed snapshot can converge even if it arrives before the routing receipt. The provider publishes refreshed capacity immediately after reopening local admission; the ready frame names that heartbeat's capacity_seq. The coordinator waits for the matching sequence and ready frame before routing queued work.

Scheduled serving keeps the initial foreground selection, including manual --model overrides, until a live switch or a change to backend.enabled_models on disk. Each later window reads that selection from the same resolved config path. It also reads the full current backend.model_autopilot settings and consent. Disabling Autopilot between windows restores ordinary saved enabled_models selection even when that list has not changed; an enrolled window uses its current recorded inventory instead of stale startup consent. An empty saved list selects all eligible local models found for that window; explicit IDs select only those models. The provider validates and hashes the result before reopening; an invalid selection fails instead of reverting to startup models. The scheduled loop keeps the original window end while hashing and skips startup if that window has closed. A late start serves only for the remaining window time. Other provider settings outside model selection and Autopilot settings, runtime identity/capabilities and local endpoint options remain frozen for the process (ScheduledWindowSelection in provider-swift/Sources/darkbloom/ScheduledWindowSelection.swift; Start.runScheduled in provider-swift/Sources/darkbloom/Start/StartCommand+Modes.swift).

Graceful stop and restart

darkbloom stop --timeout 600
darkbloom restart --timeout 600 --startup-timeout 180
darkbloom stop --force             # explicit interruption, including stalled work

Use the stop flags and restart flags below for deadline recovery. Commands do not initiate graceful shutdown by killing the serve task. SIGTERM, SIGINT and AppKit termination enter the same drain; standalone local mode also waits for active HTTP response bodies. SIGHUP enters it too for a provider started by hand (start --local, or start --foreground typed into a terminal), so closing its terminal window drains it instead of ending the process at once. SIGHUP is not a stop signal in two cases: the provider's launchd job does not take it, so it has whatever action launchd started the job with, and a start that arrived with SIGHUP ignored, for example under nohup, keeps ignoring it. A provider that re-executes itself in place for a self-update carries the same choice into the new process (ProviderStopSignals in provider-swift/Sources/ProviderCore/Service/ProviderStopSignals.swift). The signal deadline is configurable with DARKBLOOM_DRAIN_TIMEOUT_SECONDS. Once local HTTP admission closes, new POST requests receive HTTP 503 with the OpenAI JSON error message Provider capacity is temporarily unavailable. and the usual CORS header. Health reads and OPTIONS preflights remain available; already admitted response bodies retain their drain ownership. Signal-only shutdown disarms current watchdog recovery but preserves configured login startup; use darkbloom stop for a persistent stop. With no accepted work left, a graceful drain retries native MiMo owner retirement until the owners retire or its deadline passes: after served requests, the first attempt only starts joining their finished consumers (provider-swift/Sources/ProviderCore/ProviderLoop+Lifecycle.swift, performLifecycleDrain). Newly installed or CLI-restarted jobs have launchd ExitTimeOut = 3660; an existing job must be restarted to load that allowance. OS logout/shutdown may impose its own limit. Crashes, power loss, SIGKILL and explicit force can interrupt responses. The protocol barrier requires the updated coordinator before normal provider shutdown can be confirmed.

darkbloom status

Only --config. Read-only. Prints the daemon snapshot (refresh cadence under Runtime constants) and the last trust message the coordinator sent; what the levels mean is in architecture/security/attestation.md#trust-levels, and how to read the line in attestation → Verify.

darkbloom doctor

Flag Type Default Effect
--strict flag false Exit 1 on any WARN as well as FAIL
--coordinator <url> String? config URL Coordinator for the network checks
--support flag false Append coordinator URL, token presence, MDM state, PID-file path
--clear-backend-guard flag false Delete ~/.darkbloom/kv-backend-guard.json, reset the crash-loop counter in watchdog-state.json, exit
--hardware flag false Print the desktop hardware-load document (topology, one sample, counter capabilities) as JSON, exit

Clearing restores model-aware auto on the next model load, not guaranteed paged service. It preserves explicit settings, the kill switch and capability vetoes (provider-swift/Sources/darkbloom/DoctorCommand.swift, runClearBackendGuard). See guard recovery.

Exit 1 when any detailed check or diagnosis line is FAIL (or WARN with --strict). On macOS 27 or later the diagnosis includes an APP ATTEST section. It shows the daemon's local key state and launch session, and the diagnostics from the provider's last ready: how the process started, how the previous one exited, SIP and authenticated root, signing preflight, key history, the last native Apple error chain and APNs push history. The section also reads local devicecheckd log evidence; macOS lets only administrator accounts read the system log. The check names are listed in troubleshooting. app-attest-shadow.md defines the fields.

darkbloom verify

Flag Type Default Effect
--coordinator <url> String? config URL Coordinator for the network checks

Same checks as doctor; any WARN or FAIL exits 1.

darkbloom models

Subcommand Flag / positional Type Default Effect
list --json flag false Raw output
list --all flag false Include models filtered out by backend.enabled_models
list --hash <model-id> String? nil Compute the aggregate SHA-256 of one model
catalog --coordinator <url> String? config URL Catalog source
catalog --json flag false Raw output
catalog --type <t> String? nil Filter by model_type (e.g. text)
download <modelID> String — Catalog id (or S3 name)
download --coordinator <url> String? config URL Resolve the catalog entry
download --r2-cdn <url> String? DARKBLOOM_R2_CDN_URL, else the build default https://models.darkbloom.ai (provider-swift/Sources/ProviderCore/Models/ModelDownloader.swift, resolveCDNURL) Mirror base URL
remove <modelID> String — Model to delete from the effective model cache
remove --force flag false Skip confirmation
location [PATH] String? status/menu Select an existing readable, writable cache directory; interactive changes require yes
location --check flag false Inspect PATH, or the effective cache, without changing config or weights
location --from-env flag false Explicitly import the current Hugging Face environment cache once and save its absolute path
location --reset flag false Clear the saved location and restore the legacy home cache, regardless of ambient variables

All model subcommands accept --config <path>. Location behavior is implemented by Models.Location in provider-swift/Sources/darkbloom/ModelsLocationCommand.swift; cache precedence is specified in model cache configuration.

darkbloom local

Flag Type Default Effect
--json flag false Print the raw ~/.darkbloom/local.json record

Exit 1 (and {} in JSON mode) when no live local server is recorded (LocalEndpoint.readLiveInfo, provider-swift/Sources/ProviderCore/Server/LocalEndpoint.swift).

Provider-local Chat Completions, Completions and Responses reject negative output token limits with HTTP 400 before model invocation or streaming headers. Explicit zero, positive and omitted limits retain their existing semantics. This local SDK validation does not change coordinator normalization or model numerics (libs/mlx-swift-lm/Libraries/MLXLMServer/Runtime/OpenAIRequestValidation.swift, OpenAIRequestValidation.preparedRequest).

darkbloom login / darkbloom logout

login takes --config only and runs performDeviceCodeLogin (provider-swift/Sources/ProviderCore/Auth/DeviceAuth.swift): POST /v1/device/code, print URL and code, poll POST /v1/device/token, write ~/.darkbloom/auth_token. It also records the successful token response's issuer in owner-only auth_token.issuer.json, bound to a hash of that token. The provider token is not a consumer API key. A failed save removes newly written credential files so login can be retried instead of leaving a partial local login.

logout uses Logout.run in provider-swift/Sources/darkbloom/LogoutCommand.swift and ProviderLogout.perform in provider-swift/Sources/ProviderCore/Auth/ProviderLogout.swift:

Flag Type Default Effect / source
--config <path> path Global config default Accepted for CLI compatibility; never overrides the saved token's issuer (Logout.configOptions)
--timeout <seconds> integer, 0-3600 600 Wait for accepted work and the coordinator drain acknowledgement (ServiceDrain.swift, DrainOptions)
--force flag false Explicitly permit interruption of unfinished work; process exit must still be confirmed (DrainOptions, Logout.requireExited)

The command disables watchdog/login recovery, drains and stops the provider, confirms the captured process identities have exited, and acquires the single-instance lock. It holds that lock and the lifecycle/update lease through revocation and local token deletion, preventing start/recovery from reusing the credential in between. A missing token file does not skip process shutdown. Unconfirmed exit, lock failure or a normal drain timeout refuses logout without removing the credential; a published drain remains closed with recovery disabled.

After shutdown, it sends the current saved token itself, not its hash, as Authorization: Bearer <token> to POST /v1/device/revoke at its recorded issuer. Changed configuration or a different running provider cannot retarget that credential. Legacy credentials without matching issuer metadata are removed locally with remote revocation explicitly unconfirmed; no URL is guessed. Unreadable or malformed configuration does not prevent local logout. ProviderLogout.revokeToken uses a ten-second request/resource timeout, follows no redirects, and confirms remote revocation only for HTTP 204. Offline errors, cancellation, unsupported endpoints and every other status leave remote invalidation unconfirmed but still permit local deletion. Output distinguishes confirmed revocation, local-only logout with a warning, and no saved credential. Tokens and response bodies are not logged. This requires a matching platform endpoint; local completion does not establish remote support. The request URL keeps the original coordinator's host and port while changing only the WebSocket scheme and endpoint path; path text cannot become an authority.

This revokes only the current token, not every machine/account credential. It does not remove App Attest/SE keys, model downloads or management profiles. Automatic restart stays disabled until darkbloom start; link again with darkbloom login when needed. The backend contract belongs in the sibling platform API reference, not this repository; consult the platform's latest default branch for current backend behavior. Immutable source links elsewhere are evidence snapshots.

darkbloom benchmark

The throughput sweep installs the same MLXMemoryGuard allocator limits as serving before it loads weights. Its progress log separates active allocations, reusable cache bytes and the active-allocation peak at each decode cell and shutdown (provider-swift/Sources/ProviderBenchmark/ThroughputSweep.swift, run and runDecodeBatch). These counters are not OS process footprint.

Group Flags (type = default)
Throughput --model <id> (String?), --prompt <text> (ModelBenchmark.defaultPrompt), --iterations <n> (ModelBenchmark.defaultIterations), --max-tokens <n> (ModelBenchmark.defaultMaxTokens)
Ordinary token scores --teacher-forced-input <json> (String?, unset), explicit --model <id> and --kv-backend contiguous|paged (BenchmarkCommand.swift, teacherForcedOptionError)
Runtime generation --runtime-generation, optional --runtime-prompt-file <utf8>, --runtime-prompt-date YYYY-MM-DD, --runtime-mtp, --runtime-assistant-directory <path>, --runtime-concurrency <sizes>, --runtime-stagger; production slot output and memory as JSON, prefix cache off (RuntimeGenerationBenchmark; provider-swift/Sources/ProviderBenchmark/RuntimeConcurrencyBenchmark.swift, RuntimeConcurrencyBenchmark)
Live KV precision --kv-quantization balanced|k8v4|k8v8|native; overrides live config/environment for production benchmark engines; default resolves to native. Use --runtime-generation, teacher forcing, sweep or scheduler modes to exercise serving KV. Ordinary scalar MLX.generate benchmarks do not measure this representation. MiMo stays native.
Checkpoint KV precision --kv-checkpoint-quantization balanced|k8v4|k8v8|native; independently overrides checkpoint config/environment; default requests balanced. Applies only to eligible complete stores and never forces the live backend. Cache-disabled modes do not measure checkpoint writes or restores.
Scheduler prefill decision --scheduler-prefill-decision, --expected-model-aggregate-sha256, --expected-registered-binary-sha256, --expected-version, --source-sha, --decision-iterations (SchedulerPrefillDecisionReport.minimumLiveIterations), --output <path> (BenchmarkCommand+SchedulerPrefillDecision.swift)
Sweep --sweep, --prefill-lengths ("128,512,2048"), --max-batch (6), --batch-sizes (String?), --decode-tokens, --decode-prompt-tokens, --decode-iterations (ThroughputSweep defaults), --kv-backend ("auto") (BenchmarkCommand+Sweep.swift)
Scheduler prefill --scheduler-prefill, --prefill-iterations (2), --prefill-fixed-prompt (false; scheduler-prefill only) (BenchmarkCommand.swift, BenchmarkCommand+Sweep.swift)
Arrival invariance --arrival-invariance, --arrival-width (4, range 1...16), --arrival-prompt-tokens (512), --arrival-prompt-lengths (String?; exactly one integer ≥2 per row, overrides uniform length), --arrival-decode-tokens (64), --arrival-iterations (3) (BenchmarkCommand.swift, Benchmark.arrivalPromptLengths); requested width alone is not measured forward-width evidence
Backend parity --parity, --assistant-model <id> (String?), --parity-max-tokens (48), --parity-prefix-tokens (28672) (BenchmarkCommand+Parity.swift)
Prompt preparation --prompt-prep, --prompt-prep-lengths ("2048,8192,32768"), --iterations; CPU only, loads only the tokenizer, times uncached, first-sighting, next-turn and repeated prompt preparation and the body decode for chat, agent and tool-loop conversations with the prompt token cache's background comparisons off, prints JSON and exits 2 on any token-ID or decoded-field difference (provider-swift/Sources/ProviderBenchmark/PromptPreparationBenchmark.swift)

Scheduler prefill defaults to rotating the seed tokens by iteration * 17 for measured iterations; both bootstrap and full-context warmups use offset 0. --prefill-fixed-prompt keeps offset 0 for every request, repeating identical input tokens at each prompt length for sustained repeated-input measurements. It preserves fresh engines, disabled prefix caching, chunk policy and timing. The JSON report records promptMode ("rotating" or "fixed"), warmupPromptOffset, and each sample's promptOffset before wrapping by the seed length. These optional fields extend schema 4; absent metadata in older reports remains unspecified. The flag is rejected outside --scheduler-prefill before runtime loading (provider-swift/Sources/ProviderBenchmark/SchedulerPrefillBenchmark.swift, run, makePrompt, SchedulerPrefillBenchmarkReport). Schema 5 adds prefillChunkTokens and maxBatchedTokensPerStep; with soloPrefillStripeTokens they record the scheduler the production build handed the warm-up engine, after serving, calibration and GLM policy (EngineV2Factory.ProductionBuild.prefillGeometry), and a measured engine built with different geometry fails the run.

Scheduler prefill installs MLXMemoryGuard.configureOnce before loading weights, matching serving and the throughput sweep. Stderr records the selected allocator limits and effective cache limit. Earlier scheduler-prefill runs omitted this setup; account for that difference when comparing saved results. This does not change memory reserves or establish wired weight residency.

Ordinary DiffusionGemma generation uses its production native block engine and honors --kv-quantization. Its NATIVE_BLOCK_BENCHMARK records include resolved precision, prepared prompt tokens/date, generated text/tokens and observed cache memory. Runtime generation emits one autoregressive request; it does not use --iterations. Cache capacity peaks sample the host before submission and on delta/terminal events, so they do not capture every prefill transient.

With balanced live quantization, --kv-backend auto requires paged storage. Missing required sweep cells return a nonzero exit status for balanced, k8v4 and k8v8, including auto. Scheduler-prefill lengths must exceed one token; its producer and arrival invariance already reject incomplete measurements. With --kv-quantization native, auto uses the exact qualified-artifact allowlist and retains its observable contiguous fallback. Explicit --kv-backend paged refuses construction failures rather than measuring a fallback; kill-switch and capability/span-mask vetoes require native precision to serve contiguous. Inspect the measured engine's resolvedKVBackend and report kvBackend block (provider-swift/Sources/darkbloom/BenchmarkCommand.swift, Benchmark.kvBackend). The five-artifact rollout is not yet validated; benchmark selection alone is not release evidence.

Environment inputs for the harnesses are in reference/configuration.md. Model verification I/O uses reusable-buffer reads and up to four independent file readers by default, retaining complete integrity checks. It affects load/verification work, not ordinary resident decode; explicit overrides provide the original serial path. For a pinned GPT-OSS matrix with aggregate B=2/B=4 decode, raw token timing, and mixed prompt arrivals, see the profiling workflow.

darkbloom update

Flag Type Default Effect
--coordinator <url> String? config URL Release source
--check-only flag false Report; do not install
--override-quarantine flag false Reinstall a version quarantined after 3 failed starts
--timeout <seconds> integer, 0–3600 600 Drain the running service before activating the installed update
--force flag false Explicitly permit interruption during update activation

Exit 1 on quarantined, busy, cancelled, downloadFailed, hashMismatch, replaceFailed, or a failed check (UpdateResult, provider-swift/Sources/ProviderCore/Update/SelfUpdater.swift). See installation → Update.

darkbloom enroll / darkbloom unenroll

EnrollmentService.enroll in provider-swift/Sources/ProviderCore/Auth/Enrollment.swift returns App Attest setup guidance on macOS 27 or later before checking profiles, contacting the enrollment endpoint or opening Settings. This guidance asks the operator to verify current status; it is not an App Attest grant. Older macOS uses the legacy profile flow only for eligible frozen identities. The linked provider token and fresh SE-key proof must pass POST /v1/enroll even when a local Darkbloom profile exists. Only then does the CLI return "Already enrolled", without saving or reinstalling the profile. New identities require macOS 27 or later and qualified App Attest, not a copied profile or OS-only grant. ProviderOnboardingPolicy in provider-swift/Sources/ProviderCore/Auth/ProviderOnboardingPolicy.swift owns the OS choice and the upgrade/upcoming MDM deactivation notice. The OS choice never grants serving authorization or removes an existing profile.

Command Flag Type Default Effect
enroll --coordinator <url> String? config URL Coordinator to request the profile from
enroll --no-open flag false Save the .mobileconfig; do not open System Settings
unenroll --force flag false Select full exit, stop the service and confirm local-data cleanup without prompting
unenroll --no-open flag false Do not open System Settings
unenroll --keep-serving flag false Require fresh coordinator App Attest removal readiness, preserve identity/account data and guide removal of only Darkbloom enrollment

darkbloom logs

Flag Type Default Effect
--file flag false Tail ~/.darkbloom/provider.log instead of unified logging
-f, --follow flag false Stream new lines
--last <duration> String? nil log show --last <duration>; with --follow, history first then live stream
--debug flag false Include debug-level entries (unified logging only)
-l, --lines <n> Int 50 Lines to show; only with --file

Without flags: log stream --predicate 'subsystem == "dev.darkbloom.provider"' --level info.

Flag Description
--coordinator-url <url> Override the coordinator WebSocket URL
--model <id> Model to serve; repeatable (skips the interactive picker)
--all Serve all downloaded models
--idle-timeout <mins> Idle-memory policy, saved to [backend] idle_timeout_mins (0 = always ready); skips the memory prompt
--foreground Run in the foreground (used by launchd; normally implicit)
--local Run a local OpenAI server only; do not connect to the coordinator
--local-endpoint Serve a local OpenAI endpoint alongside the coordinator
--port <port> Port for --local / --local-endpoint (default 8000)
--bind <addr> Bind address for local modes (default 127.0.0.1)
--no-auth Disable local API-key auth (trusted/airgapped only)
Flag Type Default Effect
--last <duration> String 24h Window of unified logs to collect
--dry-run flag false Print the report; do not upload

After the model picker, an interactive darkbloom start asks how the machine should treat its memory when nobody is sending requests:

Memory when idle
  1) Always ready    Keep models loaded (~18 GB while idle). Instant responses,
                     full base rewards.
  2) Free when idle  Unload after 60 min without requests; reload on demand
                     (~10-30 s cold start). Your Mac gets its memory back.
  3) Custom          Choose the number of idle minutes.
  Choice [2]:

Enter keeps the policy already in force (Free when idle on a fresh install). The answer is written to [backend] idle_timeout_mins for ordinary coordinator-connected idle unloading; --model/--all, --idle-timeout, non-interactive runs and the launchd relaunch never prompt. Autopilot enrollment also skips this prompt and preserves the saved policy. See darkbloom idle. Every darkbloom start mode preloads selected models with the default startup_preload = true, regardless of the idle-memory policy. A scheduled provider begins this loading at the window opening, not beforehand. A coordinator-connected provider prioritizes previously loaded models and defers registration for up to startup_preload_timeout_secs; standalone --local finishes preloading before it listens. An explicit [backend] preload_models list takes precedence. The slot limit and available memory can leave models to load on a later request. startup_preload = false disables preloading in either mode. The one-token startup_selftest and startup_selftest_fail_closed settings apply only to coordinator-connected startup; --local does not run a synthetic decode.

Enrolled Autopilot can advertise more cached models than the saved serving selection. startupPreloadPlan preserves saved explicit or implicit preferences; that larger inventory does not replace enabled_models/preload_models or ask startup to load every advertised model (provider-swift/Sources/ProviderCore/ProviderLoop+StartupPreload.swift, startupPreloadPlan).

Examples:

darkbloom autoupdate <action>

Positional Type Default Effect
action String — (required) enable/on/true, disable/off/false, or status; anything else exits 1

Writes provider.auto_update to the config file under the shared config lock, reloading the file before saving so a concurrent live switch's model selection is retained.

darkbloom autopilot

Experimental enrollment for the verified downloaded network inventory; off by default. Opt-in records interest/consent for the default shadow rollout, not active memory-residency control. Source: provider-swift/Sources/darkbloom/Autopilot/AutopilotCommand.swift (Autopilot) and provider-swift/Sources/darkbloom/Start/StartCommand+Autopilot.swift (saveAutopilotEnrollment). Every subcommand accepts --config.

Command / option Effect
status, status --json Configured consent and fresh daemon state, including selected/ready models and transition result. Also downloaded lists the verified downloaded catalog builds the running provider adds to the Autopilot set; Downloads shows whether Autopilot downloads are on, their limits, the models and GiB Autopilot downloaded that are still on disk, and free disk (downloads and download_usage in JSON)
enable Discover/verify all eligible downloaded network builds and safely save experimental enrollment; normal startup selector, verification adds no downloads, default shadow mode is inactive, and an already-enrolled provider's pause is preserved
models Explicitly refresh the recorded cached network inventory through discovery, verification and safe drain/restart; normal startup selector and no implicit resume
pause, resume Runtime participation update; pause preserves ready models and blocks new demand-based changes. Resume follows the coordinator's current mode and never promotes shadow to live. Retired, unadvertised residents may still unload when unpinned and unused
pin MODEL_ID..., unpin MODEL_ID... Live unload protection during active control, explicit pause or an accepted operation; pins must belong to the selected set
downloads enable [--max-storage-gb N] [--max-models N] Turn Autopilot downloads of catalog models on your behalf back on and set limits (on by default while enrolled). Each run replaces both limits: an omitted flag means no limit; on an interactive terminal without flags it asks for both, and a blank answer means no limit. Writes [backend.autopilot_downloads] and never changes consent (configuration)
downloads disable Stop Autopilot downloads; models it downloaded stay on disk and in the Autopilot set
disable Revoke new commands and restore the saved idle policy after any accepted operation finishes
start --autopilot, start --autopilot --all Explicit scripted enrollment of all verified eligible downloaded network builds; --all cannot include arbitrary local/off-catalog models
start --no-autopilot Explicitly save the ordinary idle-policy mode

Every normal interactive start asks about experimental Autopilot again, explicitly naming shadow mode as not activated and a later live rollout. The first default is No ([y/N]); later prompts use the saved choice ([Y/n] when enrolled). Enter keeps that default. Explicit enrollment flags and automatic restarts do not prompt. Both answers follow the ordinary model picker and idle-policy prompt. Yes records consent, not activation; the normal selector may download models the operator explicitly chooses. It discovers all already-downloaded active network-supported catalog models and verifies their manifest or matching registry weight hash. Arbitrary local, off-catalog, retired, ineligible, malformed, stale or unverified builds are excluded; verification failures warn and skip the build. Empty eligible inventory fails before persistence, drain or restart; verification adds no downloads. The normal selector saves the chosen startup models and idle policy. Enrollment itself leaves other preferences unchanged; explicit --idle-timeout remains a requested override. Autopilot does not change residency in the current shadow rollout. Live rollout can later choose cached models for utilization, not an earnings guarantee.

Discovery filters empty or over-256-UTF-8-byte IDs before verifying builds. Inventory validation deduplicates and sorts the verified result, then requires 1...256 approved IDs through the shared ModelAutopilotSettings.hasConsent predicate before persistence or drain (validatedAutopilotSelection, verifiedAutopilotInventory in provider-swift/Sources/darkbloom/Start/StartCommand+Autopilot.swift). Invalid bounds fail without stopping the existing daemon or installing a replacement.

The recorded inventory is an exact-build allowlist. Ordinary starts with saved enrollment validate all recorded IDs or fail before persistence or drain: a missing, ineligible or unverified build, including a transient manifest error, cannot silently shrink consent. The saved selection never grows by itself, but with saved consent the running provider adds every verified catalog build in the cache, including builds Autopilot downloads on the operator's behalf and verified coordinator-desired replacements, to the Autopilot set it reports (Autopilot set). Explicit --autopilot, enable or models refreshes eligible cached inventory without downloading and may exclude/prune failed or ineligible builds. saveAutopilotEnrollment retains paused when consent already exists, including ordinary starts and explicit refresh/enable; only resume resumes that enrollment. New or previously disabled enrollment starts unpaused. While enrolled, darkbloom switch returns a busy receipt with that guidance; disable Autopilot to use manual switching. Optional MTP may fall back to target-only serving without an Autopilot download. Files stay on disk.

A valid shadow lease produces shadow, explicitly not activated, with active=false and observe_only=true. waiting means no valid lease is acknowledged. Consent and shadow control retain ordinary loading and idle behavior. Only a matching live lease after an operator permits live rollout and selects the verified machine can produce active; providers have no shadow/live mode command. The coordinator-issued Machine ID in darkbloom status is the cohort selector, not a provider connection ID or serial. An unselected machine stays shadow even when other machines are live. paused retains ready models; recovering means an accepted transition is still settling. Policy changes are consumed at the next capacity poll. models uses the existing safe restart. See architecture and operator procedures.

darkbloom beta

Subcommand Flag / positional Type Default Effect
list (default) --json flag false Table or JSON of every feature with on/off/auto
status [feature] String? all Details for one or all features
enable <feature> String — Write the feature's config key on
disable <feature> String — Write it off

Feature ids and semantics: beta features.

darkbloom fan

Subcommand Flag Type Default Effect Needs sudo
status (default) --json flag false Helper install/load state, policy, temperatures no
diagnose --json flag false Fans and GPU sensors detected no
enable --speed <pct> Double 80 Target, % of each fan's maximum; 60–90 accepted yes
enable --temperature <C> Double 45 Engage threshold; release is --temperature − 5 yes
configure --speed <pct> Double? nil Change speed only yes
configure --temperature <C> Double? nil Change threshold only; at least one of the two is required yes
disable — Restore automatic control; keep the helper installed yes
uninstall — Restore automatic control; remove helper and LaunchDaemon yes
test-lease (debug builds only, hidden) --seconds <n> Int 30 Hold a provider activity lease for 1–300 s no

darkbloom idle

Manage the idle-memory policy: whether a model stays loaded while the machine receives no requests, or is unloaded to give the memory back and reloaded on demand. The single source of truth is [backend] idle_timeout_mins in provider.toml (0 = always ready); darkbloom start's memory prompt and --idle-timeout write the same key, and the launchd plist never carries it.

darkbloom idle status                 # current policy and where it is set (default)
darkbloom idle keep-loaded            # always ready: models stay loaded
darkbloom idle unload-after <minutes> # free when idle: unload after N idle minutes (1..10080)
Policy idle_timeout_mins Effect
Always ready 0 Models stay resident; instant responses; the machine stays eligible for base rewards the whole time
Free when idle (default) 60 A model with no requests for 60 min is unloaded; the next request reloads it (~10-30 s cold start)
Custom N Same as above with an N-minute window

Changes are saved with the same locked read-modify-write as darkbloom beta and take effect after darkbloom restart. The provider reports the policy in its heartbeat (idle_unload_mins) so the dashboard shows an empty slot as "sleeping, wakes on demand" rather than as a fault. --json prints the policy machine-readably.

darkbloom addon

Operator debugging of add-on holds: inspect, place or release the hold a local add-on uses while its VMs run. The commands reach the running provider through its owner-only lifecycle mailbox, bound to the live process identity; add-ons themselves use the local socket.

darkbloom addon [status]
darkbloom addon hold [--lease-seconds <30...600>]
darkbloom addon release [--force]
Subcommand / flag Default Effect
status default subcommand Active holds (add-on, draining or held, start time, lease) and whether an add-on holds the machine lock
hold --lease-seconds <n> 600 Hold as add-on operator, exactly like an add-on: admission closes, accepted work finishes (up to 120 s), every model unloads. Repeating renews the lease
release — Release every active hold; the provider serves again at once on the same connection
release --force false Release even while an add-on holds the machine lock, so its VMs may still be using memory

Without --force, release refuses while the machine lock is held. A provider without add-on support (older, or a standalone --local server) is refused before anything is sent. The command waits up to 10 seconds for the provider's answer. darkbloom status prints Not serving: held by add-on … while held. Code: provider-swift/Sources/darkbloom/AddonCommand.swift (Addon), provider-swift/Sources/ProviderCore/Addon/ProviderLoop+AddonHold.swift (handleAddonControl).

darkbloom stop

Drain accepted requests and their terminal usage before persistently stopping the launchd service.

darkbloom stop [--timeout <seconds>] [--force] [--uninstall]
Flag Type / valid range Default Effect
--timeout <seconds> integer, 0–3600 600 Wait for accepted coordinator requests, local response writes and the coordinator acknowledgement
--force flag false Explicitly permit bounded cancellation and termination of unfinished work
--uninstall flag false After draining or explicit force, remove the provider and watchdog plists and boot out and remove the desktop API agent (io.darkbloom.desktop-api, best effort)

The command disarms the watchdog and disables login/reboot startup before requesting the drain. If setup fails before the mailbox request is published, it restores the prior launchd/watchdog recovery state and retains its history. A normal drain timeout returns non-success and leaves the process draining with automatic restart disabled. Repeat stop or restart with a new deadline, or explicitly pass --force. Interrupting the CLI does not cancel accepted inference or reopen admission. A running provider without the drain control protocol requires an upgrade or an explicit forced interruption.

Code: provider-swift/Sources/darkbloom/ServiceDrain.swift (DrainOptions, ServiceDrain.prepare) and provider-swift/Sources/darkbloom/StopCommand.swift (Stop). See graceful lifecycle behavior for signal handling and launchd's termination allowance.

darkbloom restart

Drain accepted work, reload the recorded launchd configuration, and confirm a new provider process with fresh serving authorization. The coordinator URL, model selection and provider config arguments are preserved.

darkbloom restart [--timeout <seconds>] [--force] [--startup-timeout <seconds>] [--config <path>]
Flag Type / valid range Default Effect
--timeout <seconds> integer, 0–3600 600 Wait for accepted work and the coordinator acknowledgement before restarting
--force flag false Explicitly permit interruption of unfinished work before restarting
--startup-timeout <seconds> integer, 1–3600 180 Wait for a new process identity and fresh App Attest, legacy, or owner self-route authorization
--config <path> path unset Override the config used to re-arm the watchdog; the provider keeps its recorded config arguments

A drain timeout returns non-success and leaves the old process draining with automatic restart disabled; repeat the command with a new deadline or explicitly choose --force. A startup timeout returns non-success while the new service keeps starting, without issuing another restart. Inspect darkbloom status to check its progress. An installed, stopped service is started; a missing service returns non-success. The watchdog is re-armed according to provider.auto_restart. A foreground/local process is explicitly terminated after draining before the saved launchd configuration starts. If no launchd configuration exists, restart refuses before disturbing that process; use start to choose a replacement. Owner-only/preferred-owner connections may confirm self_route authorization; this does not claim public-fleet eligibility.

Code: provider-swift/Sources/darkbloom/ServiceDrain.swift (DrainOptions, ServiceDrain.waitForRestart) and provider-swift/Sources/darkbloom/RestartCommand.swift (Restart).

darkbloom status

Show local configuration, hardware, schedule, and live daemon state.

darkbloom status

Output includes:

  • Provider version and config path.
  • Coordinator URL and backend settings.
  • Startup preload on/off, whether the explicit list or selected models drive it, and the registration timeout.
  • Detected hardware (chip, RAM, GPU cores).
  • Inference memory is the nominal hardware budget, not live free RAM.
  • Schedule state (active/inactive).
  • Live daemon PID, uptime, trust verdict, and last model-load error with how long ago it was recorded (Recorded … ago); the readiness lines above it are live. A refusal that could not evict a resident model ends with Not evictable: …, naming each such model with its resident GB and every reason (load gate). A GLM-5.3-Flash error that reports a Metal command buffer out of memory adds the → fix: line from GLM-5.3-Flash GPU wired-memory limit.
  • Enabled model filter is the saved config; a running daemon applies a changed enabled_models only at its next start. Serving outside the enabled model filter names each warm or advertised model the filter omits and why it serves: an Autopilot operation's adoption, the running daemon's selection, or the live Autopilot lease. Autopilot placements lists adoptions and placement holds with each hold's UTC expiry and time left. Not evictable for a load now lists the resident models no load can evict at the snapshot, with the same reasons a refusal gives (provider-swift/Sources/darkbloom/StatusCommand+Residency.swift).
  • Not serving: when the daemon is alive but a graceful drain has closed admission — draining, a drain that did not finish, or a drain whose relaunch never happened — with the commands that finish or interrupt it (provider-swift/Sources/darkbloom/StatusCommand+LifecycleDrain.swift, Status.lifecycleDrainLine).
  • Memory when idle: the idle-memory policy in force (always ready or free after N idle). Advertised models without a resident engine are separated into Startup preload pending, Not loaded (loads on request), Preload skipped (no eviction), and Cold load blocked (memory). A fresh daemon snapshot reports the no-eviction usable load memory beside a blocked model's scanner estimate, activation + minimum-KV serving reserve, required total and no-eviction shortfall. When request-time eviction still cannot fit the model, the cold-load shortfall (the amount to free) is shown separately. Older or stale snapshots and snapshots taken during active or queued requests, a model load or a reload withhold a definitive verdict. The daemon writes the load transition during startup preload even before its first backend-capacity snapshot. always ready retains loaded models but does not override the memory load gate. An eviction-aware allowance distinguishes a preload that preserves resident models from a cold request that can evict idle slots; only the latter earns the Cold load blocked label when it still cannot fit. A memory skip also writes a fixed public category to darkbloom logs; model loads refused at final admission, allocation recheck, or measured post-load KV headroom, or fleet KV re-slice serviceability use the same warning. Model names and exact load figures remain private there and appear in the owner's live status and doctor output instead.
  • Per-slot posture: the KV backend each loaded model actually resolved to (paged / contiguous), the selection the config asked for, whether MTP is enabled, active, or enabled-but-inert, and the SSD prefix cache's status (prefix_cache=).

Slot posture

Slot posture: state written 2s ago
  google/gemma-4-26b: kv=paged (requested paged) | mtp=enabled, active | … | prefix_cache=ready
  openai/gpt-oss-20b: kv=contiguous (requested auto) | mtp=enabled but INERT (inert_kv_unsupported)
  qwen3.5-35b-a3b: kv=paged (requested auto) | mtp=enabled, active | … | prefix_cache=FAILED (cache_init_failed: key_unavailable)
    → prefix cache: its encryption key could not be loaded from the Secure Enclave and Keychain
    → fix: run the signed release (`darkbloom update`); …
  big/model-70b: kv=NOT SERVING (requested paged) — load failed: …

requested is what engine_v2_kv_backend (or a per-model override in engine_v2_kv_backend_by_model) asked for; the kv= value is what the engine was actually built with. They differ when a request was vetoed, degraded, or refused — an explicitly requested paged backend that cannot be built REFUSES the load rather than serving contiguous, so that model shows kv=NOT SERVING.

mtp=enabled but INERT means a drafter is resident and charging memory while producing no drafts. It is not the same state as mtp=enabled, active, and the reason is always named.

prefix_cache= is the slot's SSD prefix-cache status as the heartbeat reports it: ready, starting, off (<reason>) for a model or slot that does not cache by design, closed (unloading), or FAILED (<reason>: <cause>) with the cause and fix on the next lines. A failed cache serves the model uncached and is built again when the model next loads. doctor reports the same failure as a prefix cache check; the desktop app's diagnostics run doctor (PrefixCacheDiagnosis, provider-swift/Sources/ProviderCore/Diagnostics/PrefixCacheDiagnosis.swift). Causes are listed in the SSD KV cache reference.

These values come from the daemon's state file (~/.darkbloom/daemon-state.json, override with DARKBLOOM_STATE_FILE), which the running daemon rewrites every heartbeat_interval_secs / 2 seconds — about every 2 s at the default. The header carries the snapshot's age, and the block is prefixed STALE once it has gone unrefreshed for four write cycles: a value from before a reload is worse than no value.

darkbloom doctor

Run local diagnostics and fetch the coordinator's trust view.

darkbloom doctor [--strict] [--coordinator <url>] [--support] [--clear-backend-guard] [--hardware]
Flag Description
--strict Treat warnings as failures
--coordinator <url> Override coordinator URL for remote checks
--support Print local identifiers useful for support
--clear-backend-guard Remove the crash-loop KV guard, reset its restart chain and exit; normal selection resumes on the next load
--hardware Print one whole-machine load sample and which counters this Mac exposes, as JSON (format), and exit

The report header prints Build (prod or dev, fixed at compile time), Coordinator (the --coordinator value, else [coordinator] url) and Model CDN (DARKBLOOM_R2_CDN_URL from the shell, else the build default). Model CDN does not read the LaunchAgent plist.

darkbloom doctor is read-only except for the subprocess calls used by public ProviderCore checks and the explicit --clear-backend-guard action (provider-swift/Sources/darkbloom/DoctorCommand.swift, runClearBackendGuard). The operator report begins with a readiness summary and the first concrete action. A failed model-fit check names the live usable memory, the required load budget and their shortfall; it tells the operator to free memory, rerun doctor and restart to retry preload when enabled. Interactive terminals color section headings and PASS/WARN/FAIL markers. Every advertised cold model is checked, largest first, so a small fit cannot hide a larger model's failure. Pipes, NO_COLOR, CLICOLOR=0, and TERM=dumb retain plain text. When the daemon's capacity snapshot is fresh, doctor uses its paired no-eviction usable memory and serving headroom sample; otherwise it falls back to a local read-only memory sample and does not claim to know the earlier startup decision. An already resident target is reported as resident without pretending it needs another cold load. When the fresh daemon reports that idle eviction could fit a cold model, doctor warns about no-eviction preload instead of claiming request-time loading is impossible. On a multi-model Mac, a selected cold model with a recent load failure is diagnosed before an unrelated recently used resident model, so the model-fit line explains the failure the operator came to check.

Two of the detailed checks cover the KV-backend rollout:

Check Fails when
daemon state freshness The daemon is running but has not rewritten its state file for eight write periods — it is wedged, and every live value below it is a guess. The bar is derived from heartbeat_interval_secs (the daemon writes every half-heartbeat) with a 90 s floor, so raising the heartbeat does not make a healthy daemon look wedged.
kv backend posture An EXPLICIT paged or contiguous request was not honoured: refused (no engine built, the box serves nothing for that model) or silently degraded to another backend.

The posture check compares explicit live-backend requests; an auto slot does not fail this check. Native live precision is the default and keeps the model-specific automatic backend policy. Explicit non-MiMo live balanced, k8v4 and k8v8 auto require paged even outside the exact qualified-artifact cohort, and refuse construction if a veto or failure resolves contiguous. Native auto retains its observable contiguous fallback; native precision can also use explicit paged. Checkpoint compression never selects the live backend. MiMo resolves both precision choices to native before overrides and keeps its native backend path (EngineV2KVQuantizationPolicy.resolve, provider-swift/Sources/ProviderCore/Inference/Engine/EngineV2KVQuantizationPolicy.swift).

Current slot diagnostics add live_kv, checkpoint_kv and, for an active complete store, layout. These are effective bridge/store values, not requested settings. checkpoint_kv=off means no complete store is active in a current snapshot; an older snapshot with no live precision field makes no precision claim (DaemonState.SlotPosture, KVBackendPosture.precisionPhrase). The same posture is shown by status and doctor; it is not a new heartbeat field.

For a guarded automatic AR paged retry, use darkbloom doctor --clear-backend-guard before retrying construction; direct Diffusion assembly does not consult this crash-loop guard. For explicit contiguous recovery, resolve native precision before selecting contiguous or disabling paged storage with DARKBLOOM_CBV2_PAGED_KV=0; see the precision settings. DARKBLOOM_CBV2_PAGED_KV=0 also fails the Nemotron Super load, which serves paged KV only and ignores engine_v2_kv_backend (configuration reference). A posture pass alone does not qualify precision or the runtime. Explicit paged construction failures refuse the load; a native policy veto that serves contiguous still fails the explicit-request check (KVBackendPosture.backendCheck, provider-swift/Sources/darkbloom/Diagnostics/KVBackendPosture.swift). When the state file is past the wedge bar the backend verdict is WITHHELD rather than asserted from a snapshot that may predate a reload.

An explicit engine_v2_kv_backend with no slot behind it — startup preload off, or every slot idle-unloaded — WARNs rather than passes: nothing on the box has loaded, let alone proved, the backend it was configured for. Under --strict (and therefore darkbloom verify) that warning exits non-zero, which is the point: an unproven paged rollout must not certify.

GLM-5.3-Flash GPU wired-memory limit

On a 256 GB M5 Mac where GLM-5.3-Flash is downloaded, named in [backend] model, enabled_models or preload_models, or served by the daemon, doctor adds the glm wired limit detailed check. It compares iogpu.wired_limit_mb (Metal's recommended working set while it is 0, the macOS default), vm.global_user_wire_limit and vm.user_wire_limit with the production values. Below them it is ⚠, never ✗: the detail gives the planned full-KV GPU memory and the sudo sysctl -w … fix, and a GLM-5.3-FLASH GPU WIRED-MEMORY LIMIT — ACTION RECOMMENDED block after the detailed checks prints the current values, the fix, darkbloom restart and the copy-paste LaunchDaemon install (sudo tee heredoc, then sudo launchctl bootstrap system …). At or above them it is ✓ with the current values. Other Macs, sizes and model sets show no line, and darkbloom verify does not run the check (provider-swift/Sources/darkbloom/Diagnostics/GLMWiredLimitAdvice.swift, GLMWiredLimitAdvice.doctorCheck).

The same one-line fix also appears:

Where When Source
darkbloom start stderr, as warning: … The selection would serve GLM-5.3-Flash below the values; start continues GLMWiredLimitAdvice.startNotice
darkbloom status, as → fix: … under Last model-load error The recorded GLM-5.3-Flash load error reports a Metal command buffer out of memory and the values are still low GLMWiredLimitAdvice.statusNotice
Daemon stderr (~/.darkbloom/provider.log; the terminal of start --local or --foreground), plus the fixed darkbloom logs line GLM-5.3-Flash hit the macOS GPU wired-memory limit. Run darkbloom doctor for the fix. Once per process, when a GLM-5.3-Flash load, startup preload, self-test or coordinator request fails with that error and the values are still low provider-swift/Sources/ProviderCore/Diagnostics/GLM5NextWiredLimitNotice.swift (GLM5NextWiredLimitNotice)

The error text is only matched, never logged. Failures the native engine reports without the MLX error text cannot be matched; doctor and start still flag the limit. None of these change admission, KV grants or capacity. Steps: troubleshooting.

darkbloom verify

Equivalent to a strict doctor run. Any warning or failure exits non-zero.

darkbloom verify [--coordinator <url>]

darkbloom models

Manage locally cached MLX models.

darkbloom models catalog

Show the coordinator's supported-model catalog.

darkbloom models catalog [--coordinator <url>] [--json] [--type <type>]

darkbloom models list

List local models.

darkbloom models list [--json] [--all] [--hash <model-id>]
Flag Description
--all Show every discovered model, ignoring enabled_models
--hash <model-id> Compute an on-demand integrity hash for one model

darkbloom models download <id>

Download a model from the coordinator catalog.

darkbloom models download <id> [--coordinator <url>] [--r2-cdn <url>]

If the model's models--<id> cache entry is a dangling symlink (for example, to an unavailable external drive), downloading preserves it as a hidden sibling .models--<id>.unavailable-link-<UUID> and creates a real model directory in the selected cache. This also applies to downloads from darkbloom start and background prefetch. Reconnect the drive before downloading if you want to keep using its existing model directory. Valid directory symlinks are followed; regular files and symlinks to files cause an error and are left intact. Code: provider-swift/Sources/ProviderCore/Models/ModelDownloader+Cache.swift (prepareModelCacheDirectory).

darkbloom models remove <id>

Delete a downloaded model. The command returns a busy error if a download or revision update currently owns that model's writer lease; retry after it finishes. --force skips confirmation and does not bypass the lease. Code: provider-swift/Sources/ProviderCore/Models/ModelDownloader.swift (remove), provider-swift/Sources/ProviderCore/Models/ModelArtifactWriteLease.swift (acquireIfAvailable).

darkbloom models remove <id> [--force]

darkbloom models location

Inspect a cache or explicitly save its location in provider.toml (Models.Location, provider-swift/Sources/darkbloom/ModelsLocationCommand.swift). No beta flag is involved. Without a saved location, existing providers continue using the legacy cache even if Hugging Face/XDG variables are exported.

darkbloom models location                              # terminal menu; status otherwise
darkbloom models location /Volumes/Models/hub           # explicitly save an existing hub root
darkbloom models location --from-env                    # explicitly import and pin the current HF cache
darkbloom models location --check /Volumes/Models/hub   # inspect only; never opts in
darkbloom models location --reset                       # restore the legacy default; retain all weights

The menu offers keeping the current location, restoring the default, choosing a custom path, or importing the detected environment cache. Enter/EOF cancels; terminal changes require yes. An explicit PATH or --from-env also works noninteractively. --from-env cannot be combined with PATH, --check, or --reset. An import pins the resolved absolute directory; later environment changes cannot switch it. No valid cache variable means no import and no saved change. See the one-time import precedence.

Empty writable directories are valid for future downloads. Missing directories are not created: mount the external volume and create the intended directory explicitly first. Nothing moves or deletes existing weights, downloads models, or restarts a running provider. --reset returns to the legacy home cache even when cache environment variables remain set.

Before applying a selection, inspect the chosen directory and confirm the expected model IDs. --check can succeed for an empty writable directory; it is discovery, not weight-integrity verification or network eligibility. Use models list --hash <model-id> for an on-demand aggregate hash and doctor for serving diagnostics.

After saving, apply the configuration with darkbloom restart, or darkbloom start if stopped. Use the intended --config <path> on the location command and start for a custom config; restart retains the installed job's config argument. The CLI reports the selected config, not a running daemon's already-loaded state.

darkbloom benchmark

Run a standardized local inference benchmark.

darkbloom benchmark [--model <id>] [--prompt <text>] [--iterations <n>] [--max-tokens <n>]
Flag Description
--model <id> Model to benchmark (defaults to the largest model that fits)
--prompt <text> Prompt text
--iterations <n> Number of iterations (default from ModelBenchmark)
--max-tokens <n> Maximum tokens to generate per iteration
--runtime-generation Generate through the production CBv2 slot and emit tokens/text, resolved precision/backend, model/runtime identity, timings and memory as JSON
--runtime-assistant-directory <path> Offline local Gemma assistant for --runtime-generation --runtime-mtp; normal artifact and target checks apply, with assistant hashes in the report
--runtime-concurrency <sizes> Comma-separated batch sizes from 1 to 64 (for example 1,2,4,8) for --runtime-generation; an empty, non-numeric or out-of-range entry refuses the whole list (RuntimeConcurrencyBenchmark.batchSizes). One production slot per size, warmed by one identical batch, then distinct greedy rows; a slot whose model policy lowers its width below the size is listed in refusedCells instead of measured. JSON carries the verified model hash, prompt hash, render date and per-row prompt-token hashes; per cell, the resolved backend, KV precisions, MTP state and assistant identity; per-row tokens, arrival times of the first, third and first eight tokens, decode rates, chained steps (measured batch) and MTP metrics (warmup included). Exit status 2 when a cell is refused or a row does not end with length at its quota or a configured stop token within it (provider-swift/Sources/ProviderBenchmark/RuntimeConcurrencyBenchmarkReport.swift, RuntimeConcurrencyBenchmark.rowFailure)
--runtime-stagger With --runtime-concurrency: row i of a B-row batch stops after max-tokens*(B-i)/B tokens, so the decode cohort shrinks to one row (speculation resuming as concurrency drops). --max-tokens must be at least 3×B for every size, keeping the quotas distinct and at least 3 (RuntimeConcurrencyBenchmark.rowMaxTokens); a row that stops before its quota fails the cell
--kv-quantization <precision> Live balanced, k8v4, k8v8 or native; overrides config/environment, otherwise the provider default is native
--kv-checkpoint-quantization <precision> Independent complete-checkpoint request with the same values; overrides config/environment, otherwise defaults to balanced; does not enable a cache or change the live backend

For paired live-cache comparisons, use the same artifact, public prompt and --runtime-prompt-date with --runtime-generation --kv-backend paged, once with --kv-quantization native and once with balanced. Target-only generation is the default in this diagnostic; --runtime-mtp explicitly includes the configured artifact assistant. Inline Qwen heads need no separate directory; Gemma requires --runtime-assistant-directory pointing to a local assistant. The diagnostic never downloads an assistant. A generation limit is reported as length, not a correctness pass. Prefix caches remain off in these runs: emitted kvCheckpointQuantization and a missing checkpointBackendLayout do not prove checkpoint compression. Use the separate checkpoint-storage qualification for encrypted write/restore comparisons.

For native Qwen4 model types, the ordinary command uses the production CBv2 model/factory path with MTP and prefix caching off. It preserves model/tokenizer EOS, checks complete weight integrity before and after load, and releases the session between independent runs. Non-native model types keep their generic path and JSON5 configuration support (ModelBenchmark.run, provider-swift/Sources/ProviderBenchmark/ModelBenchmarkNativeQwen4.swift). Iteration/output counts must be positive. The prefill column measures time to the first generated token, including prompt preparation; model loading and integrity hashing are outside the reported iteration time. An eight-token smoke proves entry-point operation, not sustained decode performance.

For diffusion_gemma, the ordinary command uses the native block benchmark. --kv-backend auto|contiguous|paged selects live storage. Default native precision leaves auto contiguous on this path. Explicit live balanced, k8v4 or k8v8 makes auto paged. Explicit contiguous requires native live precision, and an unavailable paged route refuses quantized construction (provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2SlotFactory+Native.swift, EngineV2SlotFactory.makeProductionBundle). The independent checkpoint request retains native encoding for native-block diffusion; these cache-disabled benchmark runs do not exercise a durable codec. The prefill column is encoder prefill, not time to first output. Each NATIVE_BLOCK_BENCHMARK JSON row separately reports first committed output, generation including first-block work, completion usage including EOS, committed tokens excluding EOS, resolved backend and the production KV grant. Native framing can be included in committed tokens; the row does not certify a visible-token performance target. Loading and its integrity hashes have a separate clock. The numeric promptTokens usage count remains separate from the exact promptTokenIDs array (provider-swift/Sources/ProviderBenchmark/DiffusionGemmaBenchmarkRow.swift, DiffusionGemmaBenchmarkRow). The row also exposes existing native execution/prefill quantum counts, post-first-block commit count and quantum wall-time sum/maximum. Those are work diagnostics, not output tokens or GPU-only timing. Prefill, refinement and committed-block re-encoding all contribute native work; do not count the execution-quanta total as refinement passes without separating those phases. Emitting these counters adds no sampling or GPU evaluation step. AR sweep, scheduler-prefill, arrival, teacher-forcing and parity modes reject this architecture instead of substituting an autoregressive iterator.

Eligible native inference uses the SDK's ordered expert-output reduction. That reference defines DARKBLOOM_DIFFUSION_EXPERT_UNSORT and its explicit rollback. Compare warmed original/optimized runs with unchanged artifact, prompt, seed and denoising controls; cold JIT timings and isolated kernel timings do not establish a request-throughput improvement.

For an exclusive native benchmark, the descriptor-route diagnostic adds DIFFUSION_PROVIDER_ROUTE rows. It observes the first iteration and checks that counters stay disarmed for subsequent iterations; it does not remove the first sample from the ordinary output. Configure expert routing through [gemma_optimizations].weighted_r1; the benchmark refuses conflicting low-level environment overrides. DiffusionGemma's separate expert reduction control remains independent of the Gemma-specific weighted-unsort setting. The same reference documents opt-in soft-conditioning and native compiled-sampler candidates. softEmbeddingCalls and compiledSamplerCalls prove dispatch in the observed first iteration; subsequent iterations must retain the disarmed counts. Report first-use compilation separately from warmed results. An environment value alone does not prove shape eligibility or a request-throughput gain. Keep native weights, sampling, canvas and output-count oracles identical when comparing either route.

Teacher-forced scores

--teacher-forced-input <json> selects bounded ordinary target scoring with an explicit model and backend. The UTF-8 JSON input is at most 1 MiB; fields are defined by provider-swift/Sources/ProviderBenchmark/TeacherForcedBenchmarkInput.swift (TeacherForcedBenchmarkInput):

Field Required value
modelID Exact selected model ID
expectedModelAggregateSHA256 Verified model aggregate hash, 64 lowercase hexadecimal characters
promptTokens Exact nonempty token-ID array, at most 32,768 IDs
continuation Exact nonempty token-ID array, at most 256 IDs

IDs must fit the model's declared vocabulary; the native request bounds and actual logit geometry are checked by libs/mlx-swift-lm/Libraries/MLXLMCommon/ContinuousBatchingV2/CBv2TeacherForcedScores.swift. This mode disables prefix caching and MTP, uses a production single-slot KV grant, and refuses backend fallback. It is mutually exclusive with the other benchmark modes and rejects --assistant-model and --output; JSON goes to stdout (BenchmarkCommand.swift, teacherForcedOptionError).

The report includes plain top-1 IDs, two diagnostic observations, forward counts, runtime/input/model hashes and the production grant. observed means finite scores and matching ordinary-forward controls; inconclusive preserves evidence and exits 2. Neither status certifies free generation, model quality or speculative verification (provider-swift/Sources/ProviderBenchmark/TeacherForcedBenchmark.swift). See the developer test procedure.

darkbloom update

Check for and apply provider updates.

darkbloom update [--check-only] [--coordinator <url>] [--timeout <seconds>] [--force]
Flag Description
--check-only Report whether an update is available without installing
--coordinator <url> Override coordinator URL
--timeout <seconds> Drain deadline: default 600, valid 0–3600 seconds
--force Explicitly permit interruption during activation; default false

The update path verifies bundle, binary, and mlx.metallib hashes before replacing the running binary (provider-swift/Sources/ProviderCore/Update/SelfUpdater.swift).

darkbloom autoupdate

Enable or disable automatic update checks at startup.

darkbloom autoupdate <enable|disable|status>

This toggles provider.auto_update in provider.toml. It reloads the file under the same sidecar lock as darkbloom switch, preserving a selection saved by a concurrent switch.

darkbloom beta

Manage configurable beta features. Defaults are feature-specific: the selected Gemma optimizations default on, while reserved/opt-in features default off. Provider TOML is authoritative for every serve mode. The Gemma defaults and missing-key decode are defined in provider-swift/Sources/ProviderCore/Config/GemmaOptimizationSettings.swift:16-34, with the missing-section fallback in provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift:397-400. The shared pre-Metal projection is provider-swift/Sources/darkbloom/ServeRuntimePreparer.swift:24-35.

darkbloom beta list                 # all features + on/off (default subcommand)
darkbloom beta status [feature]     # details for all features, or one
darkbloom beta enable <feature>     # turn on (then: darkbloom restart)
darkbloom beta disable <feature>    # turn off
Feature Effect
gemma-prefill-layer18 Default-on layer-18 prefill submission; disable and restart for legacy submission behavior
gemma-weighted-r1 Default-on atomic weighted-unsort + safe-R1 pair; disable and restart to roll back both
mtp MTP policy. Default auto requests validated embedded Qwen-family, native Qwen4, Nemotron Lightning and native MiMo heads, and the catalog spec_dec assistant for exact gemma-4-26b-qat-4bit and the exact Prism/Eigen Bonsai 9B identities under prism_hadamard_qwen35. Actual artifact/owner/budget checks remain required; explicit off is the rollback

enable/disable read-modify-write the TOML config and report whether a restart is required. Restart is the activation boundary for process-wide optimization state. The durable locked write and restart instruction are implemented in provider-swift/Sources/darkbloom/BetaCommand.swift:201-235. See Beta Features for the full guide. darkbloom beta list also accepts --json. Under the default auto mode a served checkpoint that embeds its MTP head drafts without any beta toggle. Exact gemma-4-26b-qat-4bit also resolves its external assistant automatically; other checkpoints without an embedded declaration stay target-only. Missing or invalid assistants fall back to ordinary decode. Standalone serving also downloads the verified assistant in the background, using the configured coordinator.url catalog, and activates it only when the current engine is idle. Existing requests keep their engine; insufficient memory or preparation failure preserves target-only serving. Local parity results are not a blanket M1-M3/unknown-chip certification. The published assistant metadata is visible in the public production catalog under gemma-4-26b-qat-4bit.metadata.spec_dec. kv-quant was removed in v0.8.0 and is no longer a valid feature id.

darkbloom fan (experimental)

Inspect or opt into provider-only temperature-based fan control.

darkbloom fan status [--json]
darkbloom fan diagnose [--json]
sudo darkbloom fan enable [--speed 80] [--temperature 45]
sudo darkbloom fan configure [--speed 60...90] [--temperature C]
sudo darkbloom fan disable
sudo darkbloom fan uninstall

Ordinary Darkbloom installation leaves the bundled helper dormant. State changes require explicit sudo; read-only status and diagnostics do not. The helper applies a target only while a signed provider holds an activity lease and a validated GPU sensor exceeds the threshold. Defaults are 80% of each fan's reported maximum, engage at 45 C, and release below 40 C. See Experimental Fan Control for hardware gates and recovery behavior.

darkbloom login

Link this machine to a Darkbloom account via RFC 8628 device-code flow.

darkbloom login

darkbloom logout

Stop the provider and remove its saved account credential. See the logout flags and confirmation rules; local completion is not confirmation of remote invalidation.

darkbloom logout

darkbloom enroll

Show App Attest setup guidance or check frozen legacy eligibility before MDM profile setup; see the enrollment behavior.

darkbloom enroll [--coordinator <url>] [--no-open]
Flag Description
--coordinator <url> Override coordinator URL
--no-open Download the profile but do not open System Settings

darkbloom unenroll

Without a flag, ask whether to fully exit Darkbloom or remove only MDM and keep serving with App Attest. Enter or closed input cancels without changing anything. The App Attest option requires macOS 27 or later and fresh coordinator removal approval; an unsupported/unqualified choice never falls back to cleanup.

Full exit stops the launchd provider and disables its automatic restart before profile-removal guidance and a separate local cleanup confirmation. If a foreground provider is still running, cleanup is refused. The cleanup removes the current (v2) Secure Enclave signing key; a leftover v1 keychain item is neither read nor removed. Model downloads and server-side account history remain intact.

If profile inventory needs administrator access, run this command in the foreground of an interactive terminal. sudo prompts there with terminal echo disabled; only the fixed, read-only profile inventory command is elevated. A denied prompt, noninteractive session, or background terminal job withholds profile-removal guidance. The command does not remove a profile itself; confirm the exact Darkbloom profile in System Settings. See attestation.md.

Code: provider-swift/Sources/darkbloom/UnenrollCommand+Choice.swift (chooseUnenrollmentMode, performUnenrollment); provider-swift/Sources/darkbloom/UnenrollCommand.swift (performFullUnenrollment). Noninteractive use requires an explicit mode flag.

darkbloom unenroll [--force] [--no-open]
darkbloom unenroll --keep-serving [--no-open]
Flag Description
--force Select full exit and confirm local cleanup; cannot combine with --keep-serving
--no-open Do not open System Settings
--keep-serving Select macOS 27+ App Attest migration directly, retaining account/keys/data

darkbloom local

Print the local (direct-mode) OpenAI endpoint URL and API key.

darkbloom local [--json]

darkbloom local reads ~/.darkbloom/local.json, but only advertises it if the recorded server process is still alive.

darkbloom logs

Show provider logs from macOS unified logging or the legacy log file.

darkbloom logs [--file] [--follow] [--last <duration>] [--debug] [--lines <n>]
Flag Description
--file Read from the legacy log file instead of unified logging
--follow, -f Stream new lines
--last <duration> Historical window, e.g. 1h, 30m, 24h
--debug Include debug-level messages
--lines <n> Number of lines (only with --file)

Refusal lines

A request refused as a client error, before its inference task or while its prompt is prepared, writes one fixed public line: the failure category, then a reason from the closed InferenceRefusalReason set (provider-swift/Sources/ProviderCore/Inference/Engine/InferenceRefusalReason.swift). A private line pairs the request ID with the same reason; neither contains request text. The inference_error frame sent to the coordinator is the same as without the line. A prompt-time refusal takes its reason from the typed error (InferenceRefusalReason.init(promptRefusal:)): the refusal a prompt policy attaches to MultiModelBatchSchedulerEngineError.invalidToolPayload, or the error case itself. Other failures, and a client refusal that names no reason, keep the bare category, such as Inference failure: invalid request (provider-swift/Sources/ProviderCore/Inference/Engine/InferenceFailure.swift, InferenceFailureLogger).

Line Refused because Source
Inference failure: invalid request (missing sender key) The sealed request has no 32-byte sender key provider-swift/Sources/ProviderCore/ProviderLoop+InferenceHandler.swift (handleInferenceRequest)
Inference failure: invalid request (undecryptable request) The body does not decrypt with this provider's key same
Inference failure: invalid request (reserved tool schema metadata) Tools carry coordinator-only schema metadata without its authenticated protocol, or GLM-5.3-Flash tools carry it at all same; GLM5NextPromptPolicy.RawControls
Inference failure: invalid request (malformed body) The body is not a JSON chat request object InferenceRefusalReason.init(chatDecodeFailure:)
Inference failure: invalid request (invalid messages) A message does not decode, such as a content part without its URL same
Inference failure: invalid request (unsupported message role) A role other than system, user, assistant or tool and their aliases developer and function same; InboundChatNormalization
Inference failure: invalid request (invalid tool history) A history tool call or result does not decode, such as a call without type or a non-object legacy function_call same
Inference failure: invalid request (invalid tool arguments) A history call's arguments is not a string same
Inference failure: invalid request (invalid tool declaration) A declared tool does not decode, such as a function without a name same; GLM5NextPromptPolicy.RawControls
Inference failure: invalid request (invalid tool choice) tool_choice does not decode, such as a named choice without a name same
Inference failure: invalid request (unsupported response format) response_format does not decode, such as json_schema without schema same
Inference failure: invalid request (invalid request field) Another field has the wrong type, such as a string temperature same
Inference failure: invalid request (unsupported reasoning control) A reasoning or template control the model refuses: an unknown effort, a non-positive budget, preserve_thinking, an unknown reasoning member, thinking key or chat_template_kwargs key, or a non-Boolean switch; also a reasoning field that does not decode, such as a string reasoning.enabled ProviderPromptContractPipeline.nativeControlsRefusal; InferenceRefusalReason.init(chatDecodeFailure:)
Inference failure: invalid request (contradictory reasoning controls) Thinking-off beside a thinking-on switch, effort or budget, or a reasoning.context that contradicts clear_thinking ProviderPromptContractPipeline.nativeControlsRefusal
Inference failure: invalid request (unsupported tool type) A GLM-5.3-Flash tool declaration whose type is not function same
Inference failure: invalid request (invalid tool schema) A GLM-5.3-Flash tool schema holding a number it cannot represent exactly, or nested too deeply same
Inference failure: invalid media (unsupported audio) Audio input to a model without native audio MediaIngest.rejectUnsupportedAudio
Inference failure: invalid request (unanswered tool call) A GLM-5.3-Flash history call has no result before the next non-tool message or the end of the conversation GLM5NextPromptPolicy.validateHistory
Inference failure: invalid request (unmatched tool result) A tool result names no pending call of the turn before it, repeats one, or carries tool_calls same
Inference failure: invalid request (native delimiter in history) A history argument name or string value contains GLM tool-call delimiter text GLM5NextPromptPolicy.normalizedTextInput
Inference failure: invalid request (non-text content) Image, video or other non-text content on GLM-5.3-Flash's text path same
Inference failure: invalid request (unsupported forced tool choice) A required or named tool_choice the model cannot enforce: a schema outside the enforceable subset, or GLM-5.3-Flash image or video input ToolChoicePromptPolicy; MultiModelBatchSchedulerEngine
Inference failure: invalid request (request exceeds context) The prompt plus max_tokens exceeds the model's context window EngineV2Bridge (advertisedContextExceeded)
Inference failure: invalid request (unsupported output control) An output control the model's engine cannot honor, such as logprobs on an engine without them EngineV2Bridge.nativeOutputControlRefusal

Prompt-time refusals also use the earlier reasons: a GLM-5.3-Flash history call with an empty or repeated ID, a bad function name or a non-assistant role is invalid tool history; history arguments that are not one exactly representable JSON object (malformed, duplicate keys, a number that does not survive the round trip) are invalid tool arguments; an invalid or repeated tool name is invalid tool declaration; a required choice without tools or a named choice of an undeclared function is invalid tool choice; a JSON response_format with image or video input is unsupported response format; and render-time control rechecks keep unsupported reasoning control and contradictory reasoning controls.

Boot-security diagnostics pass only when SIP and authenticated root are both positively enabled. A missing reading produces a warning, and failed diagnostic commands time out with unknown fields. APNs history is rendered under APNs code-identity readiness on all supported macOS versions, even without an App Attest snapshot. An absent history file is omitted. Zero observed pushes is [INFO] with an indeterminate delivery result, not a warning: cached code identity may avoid APNs entirely. Informational results do not fail --strict. A recorded App Attest environment_mismatch fails the signing diagnostic even when the entitlement is a known production/development value. APNs token presence follows late callbacks; a recorded reply means the local WebSocket write completed, not that the coordinator verified it.

darkbloom report

Collect recent Darkbloom provider unified logs and explicitly upload them to the coordinator for troubleshooting.

darkbloom report [--last <duration>] [--dry-run]
Flag Description
--last <duration> Time window, e.g. 1h, 6h, 24h
--dry-run Print the exact report locally without uploading

The assembled upload reserves room for App Attest evidence within the coordinator's 10 MiB raw-body limit; older provider-log lines are trimmed as needed. --dry-run prints this same bounded payload.

The command runs only when invoked by the provider operator. It collects the dev.darkbloom.provider subsystem, preserves macOS unified-log privacy redaction, and does not include debug-level messages. Automatic report upload is disabled.

The device-wide log collector allows 30 seconds, then a 250 ms termination grace before killing an unresponsive child. A timeout returns unavailable evidence without an unbounded wait (DeviceCheckEvidence.runLog). Reads retain at most the newest 8 MiB of stdout and 4 KiB of stderr. Parsing skips lines over 64 KiB and retains only the newest 200 matching events in a ring; the output describes the collected log tail rather than claiming complete two-hour coverage.

It also appends App Attest evidence as extra NDJSON lines:

  • the daemon's local App Attest snapshot (source darkbloom.app_attest_state), with key history refreshed after each proof, ages advanced to report time, process start and the native error chain;
  • the APNs push receipt/reply summary;
  • device-wide devicecheckd / com.apple.appattest observations from the last 2 h (source darkbloom.devicecheck_evidence). Every outcome carries scope=device_wide and attribution=not_attributable_to_darkbloom: other apps can cause these events, so they do not establish this provider's key state. Only timestamp, category, message type and closed-pattern numeric matches remain, such as SecKeyCreateSignature failed, CryptoTokenKit Code, AKSError, Should fetch CD hash, invalidKey and unknownSystemFailure. Message text, key identifiers and paths are dropped.

macOS lets only administrator accounts read the system log. From a standard account, macOS answers Operation not permitted. The command reports this and still uploads the App Attest snapshot. To include the logs, run it from an administrator account, or run sudo darkbloom report if this account is allowed to use sudo. Under sudo it reads the invoking user's daemon state and provider config (unless --config is given), plus that user's ~/.darkbloom/auth_token through AuthTokenStore.loadReadOnly. It writes no config or token file as root. An explicit nonempty DARKBLOOM_AUTH_TOKEN_PATH replaces that token path. See ReportAppAttestEvidence in provider-swift/Sources/darkbloom/Diagnostics/. --dry-run prints every appended line before anything is uploaded.

darkbloom watchdog

Internal command used by the launchd crash-recovery watchdog. Not intended for manual use.

Exit codes

Code Meaning
0 Success, --help, --version
1 ExitCode.failure — every runtime error listed above
64 swift-argument-parser validation error (unknown flag, missing positional, fan configure with no option)

Paths and identifiers

Item Value Source
Install root ~/.darkbloom/ scripts/install.sh (INSTALL_DIR)
App bundle ~/.darkbloom/Darkbloom.app; swapped atomically, backup in .install-backup-* during the swap scripts/install.sh (commit_staged_app)
CLI symlinks ~/.darkbloom/bin/darkbloom, darkbloom-enclave, mlx.metallib → ../Darkbloom.app/Contents/MacOS/*; best-effort /usr/local/bin/darkbloom scripts/install.sh
Capability markers Darkbloom.app/Contents/Resources/darkbloom-runtime-capabilities/{paged-kernel-v1,fan-helper-v1} scripts/install.sh (verify_staged_app, verify_fan_helper_capability)
Config ~/.config/darkbloom/provider.toml (or --config); retired legacy locations are not read provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift (defaultConfigPath)
Device token ~/.darkbloom/auth_token (DARKBLOOM_AUTH_TOKEN_PATH) provider-swift/Sources/ProviderCore/Auth/DeviceAuth.swift
Local-mode token / discovery ~/.darkbloom/local_token, ~/.darkbloom/local.json (DARKBLOOM_LOCAL_DIR), both 0600 provider-swift/Sources/ProviderCore/Server/LocalEndpoint.swift
Daemon state ~/.darkbloom/daemon-state.json (DARKBLOOM_STATE_FILE) provider-swift/Sources/ProviderCore/Service/DaemonStateFile.swift
Add-on runtime ~/.darkbloom/runtime/ (0700): addon.sock (0600, test override DARKBLOOM_ADDON_SOCKET), addon-hold.json (0600, active holds) and the add-on's ownership.lock (IPC reference) provider-swift/Sources/ProviderCore/Addon/AddonHoldPaths.swift (AddonHoldPaths)
PID file ~/.darkbloom/provider.pid (DARKBLOOM_PID_FILE) provider-swift/Sources/ProviderCore/Service/ProcessLifecycle.swift
Warm-model journal ~/.darkbloom/loaded-models.json (DARKBLOOM_LOADED_MODELS_FILE) provider-swift/Sources/ProviderCore/Service/LoadedModelsStore.swift
Verified weight hashes ~/.darkbloom/verified-weights.json (DARKBLOOM_VERIFIED_WEIGHTS_FILE), 0600; managed GLM load hashes bound to file identities, reused for startup registration provider-swift/Sources/ProviderCore/Service/VerifiedWeightHashStore.swift
Watchdog state ~/.darkbloom/watchdog-state.json (DARKBLOOM_WATCHDOG_STATE) provider-swift/Sources/ProviderCore/Service/WatchdogState.swift
KV-backend crash-loop guard ~/.darkbloom/kv-backend-guard.json (DARKBLOOM_KV_BACKEND_GUARD) provider-swift/Sources/ProviderCore/Service/KVBackendGuard.swift
App Attest stall restart marker app-attest-stall-restart.json beside the daemon state file, 0600; time of the last automatic restart for a stalled DeviceCheck call (limits) provider-swift/Sources/ProviderAppAttest/AppAttestStallRestart.swift (AppAttestStallRestartMarker)
Provider run marker provider-run.json beside the daemon state file, 0600. Set to running when a serve process starts and to clean (with cause) after its drain or before an update/stall relaunch. Explicit update/stall relaunch causes survive later generic termination callbacks; a failed hand-off restores running state and clears the cause. The next process derives previous_exit and start_reason from it provider-swift/Sources/ProviderCore/Service/ProviderRunMarker.swift, ProviderProcessRun.swift
APNs push history apns-push-history.json beside the daemon state file, 0600. The last 50 code-identity push receipt times and reply times, plus whether a device token was present. No token, nonce or payload provider-swift/Sources/ProviderCore/Apns/APNsPushHistory.swift
Provider LaunchAgent label io.darkbloom.provider; ~/Library/LaunchAgents/io.darkbloom.provider.plist; RunAtLoad = true, KeepAlive = false; stdout/stderr → ~/.darkbloom/provider.log provider-swift/Sources/ProviderCore/Service/LaunchAgent.swift (label, plistPath, logPath)
Watchdog LaunchAgent label io.darkbloom.watchdog; ~/Library/LaunchAgents/io.darkbloom.watchdog.plist; log ~/.darkbloom/watchdog.log provider-swift/Sources/ProviderCore/Service/WatchdogAgent.swift
Unified-log subsystem dev.darkbloom.provider provider-swift/Sources/darkbloom/LogsCommand.swift (Logs.subsystem)
Model cache Hugging Face hub layout under the resolved model cache provider-swift/Sources/ProviderCoreFoundation/ModelScanner+CacheDirectory.swift (ModelScanner.resolveCache)
Keychain KEK item service io.darkbloom.kv.kek.v1; access group SLDQ2GJ6TL.io.darkbloom.provider (DARKBLOOM_KEYCHAIN_ACCESS_GROUP) provider-swift/Sources/ProviderCore/KVCache/WrappedKEKStorage.swift (defaultService); provider-swift/Sources/ProviderCore/Security/PersistentEnclaveKey.swift (defaultAccessGroup)
Secure Enclave key label io.darkbloom.provider.attestation-signing.v2; a leftover retired …v1 item is never read provider-swift/Sources/ProviderCore/Security/PersistentEnclaveKey.swift (defaultLabel)
Apple Team ID SLDQ2GJ6TL (pinned in installer requirements and fan IPC) scripts/install.sh; provider-swift/Sources/DarkbloomFanProtocol/FanIPC.swift (teamID)
Fan helper files /Library/PrivilegedHelperTools/io.darkbloom.fan-helper, /Library/LaunchDaemons/io.darkbloom.fan.plist, /Library/Application Support/Darkbloom/fan-policy.json, …/fan-session.json provider-swift/Sources/DarkbloomFanService/FanServiceConfiguration.swift

provider.toml keys read by the CLI

Defaults are the ProviderConfig initialisers (provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift); a missing key decodes to its default, and that file is the complete schema (this table lists the keys an operator is likely to set). Environment variables, which override provider.toml for one process, are in reference/configuration.md.

Key Default Effect
`[provider] memory_reserve_gb 4 GiB withheld from the selected OS-available/MLX load budget; the larger unified-cap reserve still applies. By default OS availability includes inactive pages or, when larger, the reclaimable file page cache, so this is not a free-page floor. Set process-start DARKBLOOM_MEMORY_AVAILABILITY=free-only for shared hosts; see memory admission.
[provider] auto_update true Startup + periodic self-update
[provider] auto_restart true Arm the watchdog LaunchAgent
[provider] update_jitter_seconds 300 Max random delay before an automatic install or a network provider drains a model for a prepared MTP replacement; serving continues during the delay. 0 disables jitter; capped at 3600. Standalone MTP upgrades skip this delay. Random staggering provides no fleet availability guarantee (provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift, updateJitterSeconds; provider-swift/Sources/ProviderCore/Update/UpdateJitter.swift, delay; provider-swift/Sources/ProviderCore/ProviderLoop+MTPDrain.swift, waitBeforeMTPUpgradeDrain)
[backend] enabled_models [] Advertise only these ids; empty = all serveable
[backend] model_cache_directory unset Explicit saved hub directory; set or import once with models location, clear with --reset. Ambient cache variables never override it; hand-written relative paths are anchored to the config file (provider-swift/Sources/ProviderCore/Config/ModelCacheConfiguration.swift, ConfigManager.modelCacheDirectory)
[backend] idle_timeout_mins 60 Unload a model idle this long; 0 disables; paused during active or explicitly paused Autopilot
[backend] max_model_slots 3 Resident models
[backend] engine_v2_max_concurrent Absent: automatic, legacy 4 (8 on M5 Pro, Max and Ultra for models on the native auto paged allowlist); explicit values preserved Concurrent requests per engine, bounded by exact reviewed profile or legacy [1, 8], architecture and memory. Only an automatic setting may inherit a reviewed or M5 paged higher default; a width above the configured one returns to it under Low Power Mode or thermal pressure. ServingPerformanceProfiles, BackendSettings
[backend] engine_v2_max_concurrent_by_model {} Exact model ID → operator cap; overrides the default for that model under the same qualification, architecture and memory bounds. status and doctor show the default policy and all configured model overrides, with unknown-profile bounds when different from the requested cap (provider-swift/Sources/ProviderCore/Inference/Performance/ServingPerformanceProfile.swift, ServingPerformanceProfiles.summary)
[backend] engine_v2_kv_backend "auto" auto / paged / contiguous; per-model table engine_v2_kv_backend_by_model takes precedence. Only live packed precision requires paged. Native auto follows the exact qualified-artifact allowlist and retains observable contiguous fallback; checkpoint compression does not change it (EngineV2KVBackendPolicy.parseSelection, preferredBackend)
[backend] engine_v2_kv_quantization "native" Live attention precision: balanced K4/V4, k8v4 K8/V4, k8v8 K8/V8 or native. Explicit live packed auto selects paged; unavailable paging refuses rather than silently serving native. Its durable reuse remains the existing complete packed contract; legacy resident/tensor prefixes stay disabled. MiMo remains native (EngineV2KVQuantizationPolicy.resolve).
[backend] engine_v2_kv_quantization_by_model {} Exact model-ID live precision overrides the global setting; DARKBLOOM_CBV2_KV_QUANTIZATION overrides both. Existing explicit live balanced settings remain live packed. MiMo exclusion precedes parsing.
[backend] engine_v2_kv_checkpoint_quantization "balanced" Independent complete-checkpoint encoding request with the same four values. Eligible native paged and contiguous targets encode older attention rows and restore to native state. Unsupported layouts retain native encoding; live-packed targets keep their existing packed format. This does not enable caching, force paging or reduce native memory admission. See numerical limits and fallback.
[backend] engine_v2_kv_checkpoint_quantization_by_model {} Exact model-ID checkpoint request overrides the global setting; DARKBLOOM_CBV2_CHECKPOINT_QUANTIZATION overrides both. MiMo remains native before parsing.
[backend] mtp_mode auto Written by `darkbloom beta enable
[backend] mtp_acceptance unset (resolves to typical, delta 0.2) exact / typical draft acceptance for eligible sampled target-prefix requests. typical keeps a draft when the target's filtered probability for it is above min(1, 0.2 * exp(-H)) (H = the target row's entropy in nats); sampled output is approximate, not distribution-exact. Greedy requests never change. Native MiMo remains exact and does not apply this preference. This setting does not enable disabled MTP or widen eligibility. Parsing ignores case and surrounding whitespace; unknown values warn and resolve to exact (provider-swift/Sources/ProviderCore/Inference/MTP/MTPAcceptancePolicy.swift, resolve)
[backend] mtp_acceptance_by_model {} Exact model ID to acceptance string; overrides mtp_acceptance for that model. An unknown override resolves to exact, not the global value. Same eligibility and native MiMo exclusion as above (MTPAcceptancePolicy.resolve)
[backend.model_autopilot] enabled false Experimental cached-inventory enrollment/consent, not activation; nonempty verified inventory is required, and only a live lease enables residency control (provider-swift/Sources/ProviderCore/Autopilot/ModelAutopilotSettings.swift)
[backend.model_autopilot] min_dwell_seconds 1800 Minimum residence before Autopilot replacement; runtime clamps to 60...86400 (ModelAutopilotSettings.effectiveMinDwellSeconds)
[backend.model_autopilot] pinned_models [] Models autopilot must retain; configured [backend] model is additionally pinned (provider-swift/Sources/ProviderCore/Autopilot/ProviderLoop+Autopilot.swift, autopilotPinnedModels)
[backend] startup_preload true Preload preload_models when set, otherwise selected models (previously loaded first on coordinator starts), within slot and memory limits
[coordinator] url Build default: "wss://api.darkbloom.dev/ws/provider" for prod builds, "wss://api.dev.darkbloom.dev/ws/provider" for dev release builds (provider-swift/Sources/ProviderBuildEnvironment/BuildEnvironment.swift) The installer binds it to the coordinator that served it: another coordinator, such as dev, writes its URL; the production installer removes the line so this default applies (scripts/install.sh, bind_provider_coordinator)
[coordinator] heartbeat_interval_secs 5 Heartbeat; state file refresh is half of it
[coordinator] private_only false Serve only the owner's self-route traffic
[gemma_optimizations] prefill_layer18, weighted_r1 true See beta features
config_version retired Ignored top-level key left by releases up to v0.9.9; no longer written
[backend] continuous_batching, adaptive_prefill, engine_v2, legacy_compiled_decode, kv_quant, mtp retired Parsed for presence only; one startup WARN each (RetiredCodingKeys). The boolean mtp is superseded by mtp_mode

Both precision parsers ignore case and surrounding whitespace. k4v4, int4 and on mean balanced; int8 means k8v8; off and 0 mean native. An empty value selects that control's default. Invalid selected values fail parsing except on the early-excluded MiMo path. Native live execution does not make a lossy checkpoint exact; choose native checkpoint storage as well when an exact-native replay control is required.

To restore exact acceptance, set mtp_acceptance = "exact" under [backend] in provider.toml and restart the provider. Per-model entries take precedence: remove any "typical" overrides from [backend.mtp_acceptance_by_model], or set them to "exact" too. To opt out for only one model, set its exact model ID to "exact" in that table. Removing both settings restores the typical default, not exact acceptance. The benchmark default --mtp-acceptance exact is unchanged; see the benchmark-only environment rule.

For foreground/local mixed-prefill tuning, DARKBLOOM_CBV2_MIXED_PREFILL_CAP sets a process-wide token cap and DARKBLOOM_CBV2_MIXED_PREFILL_CAP_BY_MODEL accepts comma-separated exact overrides such as gemma-4-26b-qat-4bit=128,gpt-oss-20b=256. Per-model values precede the global value, then a reviewed performance profile. Positive Gemma caps retain the 128-token floor; 0 defers prefill while decoding. Pure-prefill stripes are unchanged. These variables are not forwarded to a LaunchAgent; see the scheduler environment reference.

LaunchAgent environment passthrough

For sandboxed foreground/local startup, TMPDIR selects the anonymous runtime metallib snapshot directory. It is not forwarded to the background provider; see runtime metallib snapshots for accepted paths, failure behavior, and serving-process scope.

The MiMo candidate controls are process-scoped settings, not new CLI subcommands or release switches. Native attention and admitted grouping default on; other experimental kernels remain opt-in. The native factory requests larger eligible solo-text chunks without changing explicit stripe overrides or memory safeguards. Only the standing-residency and complete-prefix controls are included in LaunchAgent.inferencePassthroughEnvKeys; other MiMo overrides require foreground/local serving. The native benchmark uses the managed load/retirement route. Exact native MiMo ordinary dispatch is implemented; benchmark success alone does not qualify API or catalog availability. mtp_mode = "auto" requests genuine inspected MiMo embedded heads by default; mtp_mode = "off" or DARKBLOOM_CBV2_MTP=0 disables speculation. Actual native assembly and proposal/acceptance metrics—not the setting—prove activation. Serial-target verification remains default; the rectangular flag does not itself enable MTP. No documented flag enables missing paging, media-prefix or coordinator audio capabilities.

Model-cache locations are read from provider.toml; Hugging Face/XDG cache variables are neither forwarded nor runtime overrides. The optional --from-env import saves an absolute path once, so the foreground CLI and daemon use the same explicitly selected directory.

The Bonsai performance profile uses source-default-on eligible paths in foreground and daemon processes. It leaves model bytes, native precision, context limits and MTP capabilities unchanged. Explicit 0 restores the prior path; other explicit values except 1 also disable it. These names are not daemon shell-environment passthrough entries: foreground overrides work, but do not assume a shell setting reaches an installed LaunchAgent. The generic constant-cache kill switch remains effective.

For native Flash-Next foreground/local serving, the lower-only DARKBLOOM_QWEN4_LISTING_CONTEXT control bounds the complete request envelope. Its parsing, default and mandatory PLE acceptance setting are in the candidate configuration reference. The same reference describes the default Qwen4 full-KV/PV32/layer-submission profile and its explicit 0 rollback controls. It primarily affects decode and short MTP verification, not larger prefill chunks. These Qwen-specific controls are not in the daemon passthrough list below: source defaults apply there, while shell overrides require foreground/local serving. This candidate adds no release or catalog command.

darkbloom start copies only these variables from the invoking shell into the provider plist's EnvironmentVariables (provider-swift/Sources/ProviderCore/Service/LaunchAgent.swift, passthroughEnvKeys + inferencePassthroughEnvKeys, passthroughEnvironment). Every other variable — including PATH and all the media, remaining SSD-prefix and memory-cap tunables — reaches the engine only under darkbloom start --foreground or --local. The DARKBLOOM_PREFIX_CACHE switch defaults to enabled for the exact Qwen, Nemotron Lightning and Bonsai 2 artifacts, Gemma 4 26B QAT (gemma-4-26b-qat-4bit), GPT-OSS 20B (gpt-oss-20b) and the exact native MiMo identities; see prefix-cache defaults. Other models need an explicit affirmative value for SSD caching. Resident payload retention requires DARKBLOOM_PREFIX_CACHE_MEMORY=1; both switches are forwarded to the daemon, and the global disable wins (PrefixCachePolicy.isEnabled, isMemoryEnabled). Coordinator cache preference separately requires EIGENINFERENCE_CACHE_ROUTING_MODE=on; its default is off, and no provider CLI cache setting enables it (coordinator/registry/config.go, ReadConfig). Resident routing also requires the separate live capability described in cache-aware-routing.md. Effects and defaults are specified once in reference/configuration.md.

DARKBLOOM_PREFIX_CACHE_SSD_MAX_WRITE_GB_PER_DAY overrides the compiled SSD write budget when [cache].daily_write_gb is absent. A saved daily choice wins over the environment, so darkbloom cache set also works with an older plist. It and DARKBLOOM_PREFIX_CACHE_DISK_GB are also forwarded when darkbloom start installs the launchd job. An ordinary darkbloom restart reuses the saved plist and does not import newly exported shell variables; stop and start with the intended environment to update them. See the SSD cache limits.

DARKBLOOM_CBV2_HYBRID_PREFIX_CACHE and DARKBLOOM_CBV2_HYBRID_PREFIX_BYTES control the explicitly opted-in recurrent checkpoint bank in foreground/local processes; they are not forwarded into the LaunchAgent. Their defaults and budget semantics are listed in the resident cache configuration table (provider-swift/Sources/ProviderCore/Inference/PrefixCache/PrefixCachePolicy+Hybrid.swift, hybridConfig). They do not change mtp_mode; eligible persistent assistants must support the checkpoint contract described in prefix caching.

Variable Read by
DARKBLOOM_PREFIX_CACHE_MEMORY provider-swift/Sources/ProviderCore/Inference/PrefixCache/PrefixCachePolicy+Activation.swift (memoryEnvironmentFlag)
DARKBLOOM_PREFIX_CACHE provider-swift/Sources/ProviderCore/Inference/PrefixCache/PrefixCachePolicy+Activation.swift (environmentFlag)
DARKBLOOM_MIMO_COMPLETE_PREFIX provider-swift/Sources/ProviderCore/Inference/PrefixCache/PrefixCachePolicy+Activation.swift (isMiMoCompletePrefixEnabled); unset/empty uses the model default; 0 disables native MiMo text-prefix caching
DARKBLOOM_MIMO_PERSISTENT_WIRED_RESIDENCY provider-swift/Sources/ProviderCore/Inference/Engine/Factory/MiMo/MiMoV26WiredResidency.swift (isEnabled)
DARKBLOOM_GLM_PERSISTENT_WIRED_RESIDENCY provider-swift/Sources/ProviderCore/Inference/Models/GLM5Next/GLM5NextWiredResidency.swift (isEnabled); on by default, 0/false/no/off rolls back to per-command-buffer residency
DARKBLOOM_MLX_RESOURCE_DEBUG forwarded to mlx-swift-lm
DARKBLOOM_TTFT_GDN_BLOCKED libs/mlx-swift-lm/Libraries/MLXLMCommon/GatedDelta.swift (GatedDeltaPrefill.mode); 0 keeps the stock Qwen GDN prompt kernel
DARKBLOOM_TTFT_SHAPE_WARMUP provider-swift/Sources/ProviderCore/Inference/Performance/ModelCalibrationPolicy.swift (shapeWarmupEnvironmentKey); 0 disables the load-time shape warmup, always skips its shader-cache probe
DARKBLOOM_CBV2_PAGED_KV provider-swift/Sources/ProviderCore/Inference/Engine/EngineV2KVBackendPolicy.swift
DARKBLOOM_CBV2_KV_QUANTIZATION provider-swift/Sources/ProviderCore/Inference/Engine/EngineV2KVQuantizationPolicy.swift (resolve, live)
DARKBLOOM_CBV2_CHECKPOINT_QUANTIZATION provider-swift/Sources/ProviderCore/Inference/Engine/EngineV2KVQuantizationPolicy.swift (resolve, checkpoint)
DARKBLOOM_CBV2_MTP provider-swift/Sources/ProviderCore/SpecDec/SpecDecArtifactFunnel.swift
DARKBLOOM_MTP_MAX_RECTANGULAR_TOKENS MTP verification policy (tighten-only cap)
DARKBLOOM_KV_BACKEND_GUARD provider-swift/Sources/ProviderCore/Service/KVBackendGuard.swift
DARKBLOOM_MLX_CACHE_LIMIT_GB provider-swift/Sources/ProviderCore/Inference/Memory/MLXMemoryGuard.swift (defaultCacheLimitGB)
DARKBLOOM_MLX_MEMORY_RESERVE_GB provider-swift/Sources/ProviderCore/Inference/Memory/MLXMemoryGuard.swift
DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2Factory+Configuration.swift (maxPartialPrefillsKey)
DARKBLOOM_CBV2_PREFILL_CHUNK, DARKBLOOM_CBV2_STEP_TOKENS provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2Factory+Configuration.swift (prefillChunkKey, stepTokenBudgetKey)
DARKBLOOM_PREFILL_DEADLINE_MODE provider-swift/Sources/ProviderCore/Inference/Engine/PrefillDeadlineMode.swift (environmentKey)
DARKBLOOM_TTFT_ADMISSION_FIXES provider-swift/Sources/ProviderCore/Inference/Performance/Deadline/AdmissionRateEstimators.swift (DeadlineAdmissionFixes.environmentKey)
DARKBLOOM_TTFT_IDLE_CACHE_SWEEP provider-swift/Sources/ProviderCore/Inference/Memory/GlobalKVCacheBudget.swift (idleCacheSweepEnvironmentKey)
DARKBLOOM_TTFT_SEGMENT_RETENTION provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2Factory+SegmentedBackend.swift (segmentRetentionEnvironmentKey)
DARKBLOOM_TTFT_SEGMENT_HOST_ZEROS provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2Factory+SegmentedBackend.swift (segmentHostZerosEnvironmentKey)
MLX_GATHER_QMM_EXPERT_SLICES only when the shell value is exactly 1 (GemmaOptimizationEnvironment.daemonDrainPassthrough, provider-swift/Sources/ProviderCore/Config/GemmaOptimizationEnvironment.swift)
DARKBLOOM_APP_ATTEST_WORKER_QUALIFICATION only when the shell value is exactly 1; selects the opt-in worker candidate (AppAttestWorkerSelection, provider-swift/Sources/ProviderCore/Service/LaunchAgent.swift)

To opt a background provider into the worker candidate, stop it and run DARKBLOOM_APP_ATTEST_WORKER_QUALIFICATION=1 darkbloom start. Restart and watchdog recovery reuse that saved choice; stop/start without the flag restores the default. Native diagnostic/fault-injection flags are never forwarded to the daemon.

The watchdog plist carries its own list: DARKBLOOM_NO_UPDATE_CHECK, DARKBLOOM_STATE_FILE, DARKBLOOM_WATCHDOG_STATE, DARKBLOOM_KV_BACKEND_GUARD (provider-swift/Sources/ProviderCore/Service/WatchdogAgent.swift). DARKBLOOM_NO_UPDATE_CHECK is not forwarded to the provider daemon; disable automatic updates with darkbloom autoupdate disable.

Runtime constants

Constant Value Source
Coordinator reconnect backoff ExponentialBackoff(base: 1.0, max: 30.0) s provider-swift/Sources/ProviderCore/Coordinator/CoordinatorClient+Connection.swift
WebSocket ping interval / pong timeout pingInterval = 10.0 s / pongTimeout = 30.0 s same
State-file and capacity refresh every max(1, heartbeat_interval_secs / 2) s; the heartbeat default is in the provider.toml table provider-swift/Sources/ProviderCore/ProviderLoop+Capacity.swift
State-file stale threshold isStale(maxAge: 90) s; doctor calls the daemon wedged after max(8 × refresh period, 90) s provider-swift/Sources/ProviderCore/Service/DaemonStateFile.swift; provider-swift/Sources/darkbloom/Diagnostics/KVBackendPosture.swift (wedgedAfterSeconds)
Idle unload idle_timeout_mins (provider.toml table); polled every 60 s; unloads the model, the daemon keeps running provider-swift/Sources/ProviderCore/ProviderLoop+IdleTimeout.swift
Watchdog check interval checkIntervalSeconds = 60 provider-swift/Sources/ProviderCore/Service/WatchdogAgent.swift
Crash-loop guard trip crashLoopTripThreshold = 3 restarts provider-swift/Sources/ProviderCore/Service/WatchdogDecision.swift
Auto-update first check / interval / drain 300 s / 1800 s / 120 s provider-swift/Sources/ProviderCore/ProviderLoop+AutoUpdate.swift (autoUpdateInitialDelay, autoUpdateInterval, updateDrainTimeout)
Update quarantine rollbackThreshold = 3; defaultStabilizationSeconds = 600 provider-swift/Sources/ProviderCore/Update/UpdateRecoveryState.swift
Release endpoint GET /v1/releases/latest?platform=macos-arm64 provider-swift/Sources/ProviderCore/Update/SelfUpdater.swift
Update banner timeout 2 s provider-swift/Sources/ProviderCore/Update/UpdateBanner.swift
Local chat body cap localInferenceMaxUploadBytes = 32 * 1024 * 1024 provider-swift/Sources/ProviderCore/Server/LocalChatUploadResponder.swift
Local bind wait 5 s provider-swift/Sources/darkbloom/Start/StartCommand+Modes.swift (waitUntilBound)
Fan lease / renewal leaseDurationSeconds = 15 / renewalIntervalSeconds = 5 provider-swift/Sources/DarkbloomFanProtocol/FanIPC.swift
Fan policy defaults trigger 45 °C, release 40 °C, speed 80 %, engage after 3 samples, release after 30; speed range 60–90 provider-swift/Sources/DarkbloomFanCore/FanPolicy.swift
Minimum RAM to serve hardware.memoryGb floor — ../architecture/hardware-support.md#context provider-swift/Sources/darkbloom/Start/StartCommand+Preflight.swift

Related

GPT-OSS benchmark and foreground execution supports the performance controls. The full-projection and kernel rollback modes support paired comparisons with identical request inputs.

Autopilot enrollment fields

Source: provider-swift/Sources/ProviderCore/Autopilot/ModelAutopilotSettings.swift (ModelAutopilotSettings).

[backend.model_autopilot] key Default Meaning
consent_recorded false An explicit startup decision was saved
selected_models [] The saved selection: exact approved cached network build IDs; explicit inventory refresh updates it, ordinary restarts do not; empty cannot enroll. With saved consent the running provider adds every verified downloaded catalog build to the Autopilot set it reports (Autopilot set)
revision empty string CLI-generated identity for the approved configuration
paused false Suspend new automatic changes while retaining ready models
min_idle_seconds 60 Inactivity guard, independent from minimum residence