This file is a portable context for AI coding agents (Claude Code, etc.) and the humans driving them. Its job is to let you — or an agent helping you — install, configure, run, and debug WeightsLab and Weights Studio without having to reverse-engineer the system first.
It deliberately covers only the two shipped repositories:
- weightslab — the Python backend / core (training instrumentation, data ledger, gRPC service, the shared proto).
- weights_studio — the browser frontend (the studio UI that inspects and edits a running experiment).
File/line references drift as the code evolves — treat them as starting points and verify against the current source before relying on them. Environment variable names and defaults are the most stable thing here; when in doubt the authoritative reference is
weightslab/docs/configuration.rst.
So an agent actually has this context when you ask it for help:
-
Working inside a checkout of the repo (
git clone): this guide is committed asAGENTS.md; the repo keeps a gitignoredCLAUDE.mdcopy of it at the root so Claude Code auto-loads it every session. Nothing to do. (Claude Code also loads~/.claude/CLAUDE.mdglobal memory and any parent-dirCLAUDE.md.) -
You only ran
pip install weightslab(no checkout — the package lives insite-packages): absolute@importpaths are fragile because the path changes per venv/OS. The robust pattern is a small skill that locates the installed file at runtime. Create~/.claude/skills/weightslab/SKILL.md:--- name: weightslab description: Load the WeightsLab debugging & configuration guide when helping with weightslab or weights_studio problems (connection, TLS, env vars, training hangs, rendering). --- !`python -c "import weightslab, os; print(open(os.path.join(os.path.dirname(weightslab.__file__), 'AGENTS.md')).read())"` Use the guide above to diagnose the user's weightslab / weights_studio issue.
Then run
/weightslab(or let Claude auto-invoke it). This requires the guide to be shipped as package data inside the installed package (see §7); the copy at the repo root is for contributors working in a checkout. -
Quick-and-dirty: copy this file to
~/.claude/WEIGHTSLAB.mdand add@~/.claude/WEIGHTSLAB.mdto your~/.claude/CLAUDE.md.
A user wraps their own PyTorch training script with WeightsLab so a running experiment becomes inspectable/editable; Weights Studio is the UI for that.
Wire path (the thing that breaks most often):
Browser → Weights Studio :8080 (grpc-web → grpc proxy) → Python gRPC servicer → training loop
- The Weights Studio server is a pure-Python HTTP server that serves the bundled
SPA and translates grpc-web (browser) to raw gRPC (backend). No Docker, no
Envoy.
wl.serve()runs it in a background thread of the training process (it starts with the backend, opens the browser, and stops when the process exits);weightslab startruns the same server as its own process (a UI that outlives runs, a chosen experiment dir, remote/tunnel setups).wl.serve()skips its own UI when aweightslab startalready serves its gRPC port, in Google Colab, withserving_ui=False, or withWEIGHTSLAB_SERVE_UI=0. If neither server is running, the browser has no UI to load. - The gRPC servicer and the training loop run in the same process, different
threads, coordinated by locks in
weightslab/weightslab/components/global_monitoring.py. - One proto is the single source of truth:
weightslab/weightslab/proto/experiment_service.proto.
pip install weightslab # or: uv add weightslab / uv pip install weightslabWith uv, pick the PyTorch build with --torch-backend (uv pip install weightslab --torch-backend=auto), never --extra-index-url (see §5). For the
CLI alone (e.g. only the Studio UI): uvx --torch-backend=auto weightslab start
or uv tool install weightslab --torch-backend=auto; a training script still
needs weightslab in its own environment.
In your training script:
import weightslab as wl
# wrap your objects so the studio can see/edit them (see §3), then:
wl.serve(serving_grpc=True, serving_cli=True) # background threads, same process
# ... your training loop ...
wl.keep_serving() # keep the process alive for the UIRunning the script opens Weights Studio in the browser: wl.serve() serves it
at http://localhost:8080 by default (a busy port falls back to a free one; the
URL is printed). No second terminal is needed. weightslab start stays
available for a UI that outlives the run, and when one is already running for
the same gRPC port, wl.serve() uses it instead of starting its own:
weightslab start # optional; serves at http://localhost:8080 by defaultWorking starting points live in
weightslab/weightslab/examples/{PyTorch,Lightning,Usecases}/<usecase>/
(each is a main.py + config.yaml) — find the closest example and mirror it.
UI deployment details (port, TLS, certs) are documented in
weightslab/docs/weights_studio.rst. TLS turns on once certs exist: run
weightslab se once, and weightslab start and the backend then use them
automatically (weightslab start --no-certs forces HTTP); the UI wl.serve()
serves always follows the backend's TLS setting. On Windows se uses the PowerShell script and
the Windows openssl; weightslab se --force-ubuntu uses the bash script
through WSL instead.
How a user's script plugs in. Wrap each training object with
wl.watch_or_edit(obj, flag=...); the returned tracked proxy is registered in
the global ledger (weightslab/weightslab/backend/ledgers.py,
GLOBAL_LEDGER — the hub everything reads/mutates through).
flag="hyperparameters"(dict),flag="model"(nn.Module,device=…),flag="optimizer",flag="data"(Dataset → tracked DataLoader:loader_name,batch_size,is_training,collate_fn, …),flag="loss"(areduction="none"criterion, called with(preds_raw, targets, batch_ids=ids, preds=preds)),flag="metric".
Conventions that matter for correctness:
- Wrap the train step in
with wl.guard_training_context:and eval inwith wl.guard_testing_context:(both re-exported at package level; no deep import). This is how pause/resume and train/test separation work — skip it and pause/resume or stats will misbehave. - Use
model.get_age()(steps actually trained; survives checkpoint reloads), not the raw loop counter. - Pause from code with
wl.pause_training()(the counterpart ofwl.start_training()): the next guarded step waits for Play. Running a cell in the Studio notebook pauses training the same way, and it stays paused. task_typeon the dataset/model selects rendering:classification,segmentation,detection,detection_pointcloud.- Hyperparameter handle access: the registered hyperparameters proxy
supports both
hp.get("lr")andhp["lr"](subscript ==.get), and stays live — reads reflect in-place updates and re-registration.
Recording values (pick by what the value is about, not by convenience):
| Value describes | Verb | Example |
|---|---|---|
| one sample | wl.save_signals(signals={...}, batch_ids=ids, ...) |
an image's loss |
| one annotation | wl.save_instance_signals(...) |
one box's IoU |
| a group of samples | wl.save_group_signals(signals={...}, group_ids=[...]) |
a pair's contrastive loss |
| one training step | wl.save_model_signals(signals={...}) |
a gradient norm |
| a sample's place in a 2-D/3-D projection | wl.save_projection_coords(coords, batch_ids=ids, prefix="tsne") |
your own t-SNE / umap-learn layout |
The first three write dataframe rows (sortable/filterable in the grid); the
fourth only plots a curve. Never fake a step-level value by broadcasting it
across batch_ids — that writes a number onto samples it was never about.
- Training-dynamics signals come free with
wl.watch_or_edit(model, flag="model", track_model_signals=True, model_signals_every_n_steps=N). That emits, per step and with no call in the training loop:metrics/global/{grad_norm,weights_norm}andmetrics/layer/<layer_id>/{grad_norm,weights_norm,activation_mean, activation_std,activation_max,activation_min}.<layer_id>is the same id architecture ops (freeze/reset) address, so a bad curve names the layer to act on. Only collects insideguard_training_context, so eval never contaminates it. Implementation:weightslab/weightslab/components/model_signals.py; example:examples/Usecases/wl-fashion-mnist-signals.- Diagnosing from these: early-layer
grad_norm→ 0 with healthy late layers = vanishing gradient;grad_normspiking orders of magnitude = exploding;activation_std→ 0 on a layer = that layer went constant (dead ReLU/saturated BN);weights_normgrowing while loss flattens = needs decay. - Per-layer curves need per-layer modules: an
nn.Sequentialblock resolves to ONE layer id, so a model built out of Sequentials gets one curve for the whole block.
- Diagnosing from these: early-layer
WeightsLab and Weights Studio are configured almost entirely through env vars.
Authoritative reference: weightslab/docs/configuration.rst. The high-signal
ones when debugging:
Backend (Python):
| Variable | Default | Why you touch it |
|---|---|---|
WEIGHTSLAB_LOG_LEVEL |
INFO |
Terminal only. Set DEBUG to see what's happening. (WATCHDOG level sits between WARNING/ERROR.) |
WEIGHTSLAB_LOG_FILE_LEVEL |
(unset = all) | The session log file keeps every record whatever the terminal shows; set this to cap the file too. Log lives in <root_log_dir>/weightslab_logs/. |
WEIGHTSLAB_TQDM_LOG_INTERVAL |
30 |
Seconds between snapshots of live tqdm bars into the log (0 disables). tqdm never goes through logging, so without this the file has no record of training progress. |
GRPC_BACKEND_HOST / GRPC_BACKEND_PORT |
0.0.0.0 / 50051 |
Backend gRPC bind address. |
WEIGHTSLAB_SERVE_UI |
1 |
0/false stops wl.serve() from serving (and opening) Weights Studio itself, without a code change (CI, sweeps). weightslab start still works. |
GRPC_TLS_ENABLED |
0 |
TLS on the gRPC socket. Set to 1 automatically when certs are found; 0/false forces plaintext (also for weightslab start). |
GRPC_TLS_REQUIRE_CLIENT_AUTH |
0 |
mTLS. Must match what weightslab start presents when it uses certs. |
WEIGHTSLAB_CERTS_DIR |
~/.weightslab-certs |
Where cert files are looked up (single source of truth). Falls back to ~/.weightslab-certs when unset, not an absolute path, or holding no certs. |
GRPC_AUTH_TOKEN |
(unset) | Optional metadata-token auth on top of mTLS. |
GRPC_MAX_MESSAGE_BYTES |
268435456 (256 MB) |
Raise it if large tensors/image batches fail. |
WEIGHTSLAB_DISABLE_WATCHDOGS |
0 |
Set 1 when debugging with breakpoints (see §5). |
GRPC_WATCHDOG_STUCK_SECONDS |
60 |
Lock/RPC stuck threshold + lock-acquire timeout. |
WEIGHTSLAB_PROJECTION |
on | Live 3-D parametric-UMAP projection (beta). 0 removes it; per model: watch_or_edit(model, flag="model", projection=False). Tuning: _EVERY, _DIM, _NEIGHBORS, _GRAPH — see docs/projection.rst. |
Frontend (Weights Studio) — runtime-injected window.* globals:
| Variable | Default | Why you touch it |
|---|---|---|
WS_SERVER_HOST / WS_SERVER_PORT / WS_SERVER_PROTOCOL |
localhost / 8080 / http |
How the browser reaches the UI server (weightslab start, or the one wl.serve() runs). The #1 connection-issue knob. |
WS_HISTOGRAM_MAX_BINS |
512 |
Cap on metadata histogram bars. |
BB_THUMB_RENDER |
10 |
Max bounding boxes drawn per thumbnail, per overlay (GT and PRED capped independently). |
BB_MODAL_RENDER |
100 |
Max bounding boxes drawn per modal image, per overlay. A ? button in the modal shows the active limit. |
ENABLE_PLOTS |
1 |
0/false removes the plots board + Signals card and stops plot auto-refresh. |
ENABLE_DATA_EXPLORATION |
1 |
0/false removes the data grid + metadata/details panel and stops the data/metadata auto-refresh. |
ENABLE_HYPERPARAMETERS_OPTIMIZATION |
1 |
0/false removes the Hyperparameters section, makes HP inputs read-only, and stops the HP poll. |
ENABLE_AGENT |
1 |
0/false removes the agent chat bar + history panel and stops the agent health poll. |
ENABLE_NOTEBOOK |
1 |
0/false removes the notebook button (left of the logo) + notebook window. The notebook runs Python in a shared in-process kernel against the live experiment (df, model, checkpoints), persisted as notebook.ipynb under root_log_dir; >-prefixed cells ask the agent to propose code. |
ENABLE_TUTORIAL |
1 |
0/false removes the new-user tour: the "New to WeightsLab?" notice (shown when no wl_studio_seen cookie and no saved Studio settings), its video popup ending on the MNIST Colab notebook, and the header Tour button. TUTORIAL_VIDEO_URL / TUTORIAL_COLAB_URL replace the video / notebook. |
VITE_ vs WS_/BB_/ENABLE_:
VITE_*variables are baked at build time (changing them needs a frontend rebuild).WS_*/BB_*/ENABLE_*are injected intoconfig.jswhen the UI server starts (weightslab start, orwl.serve()'s own UI) and read aswindow.*globals — changing them needs only a restart + browser reload. EachENABLE_*defaults to on; set it to0/false/no/offto disable. Full reference:weightslab/docs/configuration.rst(“Feature toggles”).
This is the core of the guide. Each entry is a real failure mode (several are distilled from issues hit in development).
UI loads but the sample grid is empty / "failed to fetch" / gRPC errors.
The wire path (§1) is broken somewhere. Check in order: (1) backend actually
serving on 0.0.0.0:50051; (2) the UI server (wl.serve()'s own, or
weightslab start) is running and the browser can reach it on :8080 (or the
port it printed); (3) TLS mismatch — a separate weightslab start and the
backend each turn TLS on when they find certs, so both must see the same
WEIGHTSLAB_CERTS_DIR (the browser console prints TLS: ENABLED/DISABLED;
the UI wl.serve() serves itself always matches the backend).
For local debugging drop TLS on both sides (weightslab start --no-certs;
GRPC_TLS_ENABLED=0 for the backend).
weightslab se hangs with no output (Windows).
Only the WSL path can do this: --force-ubuntu, or the fallback after the
PowerShell script fails. There bash is the WSL launcher, the script's output
is captured, and there is no timeout, so a stuck WSL distro blocks forever.
Confirm with wsl -e echo ok (it hangs too). Fix with wsl --shutdown, or drop
--force-ubuntu so the PowerShell script runs.
Changed an env var, restarted, but the UI still uses the old value.
VITE_*is build-time → you must rebuild the frontend, not just restart.WS_*/BB_*/ENABLE_*are injected when the UI server starts → you must restartweightslab start(or the training script, for the UIwl.serve()serves) then reload the tab.
Sample grid flashes empty cells when auto-refresh fires.
An auto-refresh (timer or manual) that lands while a GetDataSamples grid fetch
is still in flight used to clear the cache mid-render. The fix in
weights_studio/src/grid_data/gridDataManager.ts is isFetchInProgress():
refreshes are skipped while a grid fetch is ongoing. If you see this, confirm
you're on a build that has that guard.
Detection overlays are slow or unreadably cluttered.
Dense detection samples can carry hundreds of boxes. Cap rendering with
BB_THUMB_RENDER (thumbnails) and BB_MODAL_RENDER (modal); each is applied
separately to GT and to predictions. Render-only — no sample data is dropped.
Training appears hung; RPCs return RESOURCE_EXHAUSTED; server "restarts".
A watchdog monitors the global rlock and in-flight RPCs. If a lock/RPC is held
longer than GRPC_WATCHDOG_STUCK_SECONDS (60s) it's flagged; locks get
interrupted, and after GRPC_WATCHDOG_RESTART_THRESHOLD unhealthy polls the
gRPC server restarts. When debugging with breakpoints that intentionally
pause longer than that, set WEIGHTSLAB_DISABLE_WATCHDOGS=1. If RPCs fail with
RESOURCE_EXHAUSTED, a handler couldn't acquire the lock within the window —
something else is holding it; check for a long/blocking train or eval step.
Pause/resume doesn't work, or train vs test stats are mixed up.
The train step isn't wrapped in guard_training_context (or eval in
guard_testing_context). See §3 — these context managers are how the system
gates and separates phases.
Projection board stays empty, or stopped updating. wl.projection.get_tracker()
is None when it is off (WEIGHTSLAB_PROJECTION=0 or projection=False).
Otherwise .stats() says why: fits == 0 means the hooked layer is not on the
forward path (pass projection={"layer": "<named_modules() name>"});
disabled_reason is set when 5 fits in a row failed and it turned itself off;
skipped_nonfinite > 0 means the model's features went NaN. Training is never
affected by any of these. Full page: weightslab/docs/projection.rst.
Large weights/images fail to transfer. Raise GRPC_MAX_MESSAGE_BYTES.
uv: "No solution found … only torchmetrics==1.0.3 is available". The
PyTorch index was passed with --extra-index-url (the pip habit). uv takes each
package from the first index that has it, and that index carries an old
torchmetrics (and numpy, pillow…). Use --torch-backend=auto|cpu|cu126 with
uv pip install / uvx; in a uv project, an explicit = true index plus
[tool.uv.sources] for torch/torchvision, which must be direct dependencies
(sources are ignored for transitive ones). Full recipe: docs/quickstart.rst.
pip install -e . / uv sync fails: "The 2.0.3.dev5 can't be bumped". A
checkout older than the only-version setuptools_scm scheme in pyproject.toml
(guess-next-dev refused to bump a .devN tag). Update the checkout, or set
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_WEIGHTSLAB=<version> for that build.
The agent bar says it's unconfigured. The LLM agent needs a provider: a
local Ollama server (provider: ollama, available immediately) or cloud
OpenRouter initialized from the UI via /init (then /model to switch,
/reset to clear). See weightslab/docs/weights_studio.rst.
Backend (weightslab/weightslab/):
src.py— the public verbs (watch_or_edit,serve,keep_serving,tag_samples,query_*, decorators) re-exported from__init__.py.trainer/services/—experiment_service.py(gRPC servicer) delegating to{model,data,agent}_service.py;data_image_utils.py(preview/mask encoding).components/—global_monitoring.py(locks,guard_*contexts, pause),checkpoint_manager.py,evaluation_controller.py.data/—dataframe_manager.py,data_samples_with_ops.py,sample_stats.py, H5 storage (h5_dataframe_store.py,h5_array_store.py,array_proxy.py).backend/—ledgers.py(GLOBAL_LEDGER),logger.py,audit_logger.py,cli.py.security/(CertAuthManager),proto/,examples/,docs/.
Frontend (weights_studio/src/):
main.ts— bootstrap; builds the grpc-web transport fromWS_SERVER_*.experiment_service.client.ts/experiment_service.ts— generated client (regenerate withnpm run generate-proto:data; do not hand-edit).grid_data/— grid + modal rendering (GridCell.ts,DataImageService.ts,gridDataManager.ts,BboxRenderer.ts,SegmentationRenderer.ts,PointCloudViewer.ts).ui/—server.py(pure-Python HTTP + gRPC-Web proxy),static/(bundled SPA),utils/(cert-generation scripts, sync-frontend helper).
Docs: weightslab/docs/ (Sphinx) — configuration.rst (all env vars),
weights_studio.rst (studio deploy + agent), quickstart.rst, grpc/.
- The two repos must sit side by side (
…/weightslab,…/weights_studio); proto codegen scripts reach across by relative path. - Editing the proto is cross-repo — do all of: edit
experiment_service.proto; regenerate Python stubs from the repo root; runnpm run generate-proto:datain weights_studio. Editing one side only leaves the build broken. - uv:
uv sync --extra utestmakes an editable env with thedevgroup (pytest, grpcio-tools, …); thenuv run pytest tests/.... The version comes from the nearest git tag;uv.lockis gitignored. - Tests: backend
python -m pytest weightslab/tests/...; frontend unitnpm run test(vitest); E2E/user-simulation Playwright lives in weights_studio (test:realtime:*,test:e2e:*), not here. - CI on a custom branch: pushes to non-
main/devbranches only run CI when the commit message contains[force ci](both repos). - TLS/auth in the bundled UI is decided by cert presence under
WEIGHTSLAB_CERTS_DIR(single source of truth) — don't hardcode secure/insecure. - To make this guide available to pip users, ship it as package data inside
the installed package (e.g. as
weightslab/weightslab/AGENTS.md) so the §0 skill can locate it; keep the rootAGENTS.md(mirrored as the gitignoredCLAUDE.md) as the contributor-facing source.