Skip to content

[DRAFT] Add how-to for training a VLA model - #5412

Draft
Eliza Farley (elizafarley) wants to merge 3 commits into
mainfrom
claude/train-vla-on-custom-containers
Draft

Eliza Farley (elizafarley) wants to merge 3 commits into
mainfrom
claude/train-vla-on-custom-containers

Conversation

@elizafarley

Copy link
Copy Markdown
Contributor

Summary

Adds docs/train/train-a-vla.md, a how-to for training a vision-language-action (VLA) model with Viam, so a docs search for "VLA" finds a real path to follow. It works with any arm and builds on the custom training containers from #5407.

  1. Capture camera GetImages and arm JointPositions at one rate.
  2. Record each demonstration as a sequence with capture-control, tagged cmd:<instruction>.
  3. Collect the sequences into a sequence dataset. Export and convert locally with viam-sequence-to-lerobot to catch bad sequences before training.
  4. Build a custom training container on huggingface/lerobot-gpu and register it.
  5. Training script: convert the dataset to LeRobot format, run lerobot-train from lerobot/smolvla_base, and save the checkpoint to --model_output_directory.
  6. Submit the job with --container-id.
  7. Run the policy with the viam-labs:vla registry module (vla:policy and vla:controller), loading the checkpoint from ${packages.ml_model.<name>}.
  8. Record the policy's runs and retrain.

Also adds a "Next steps" card to custom-containers.md linking to the new page.

Evidence and what's untested

Steps 1–3 and 7 match what the BoxBot team ran for the IROS 2026 demo: the datasets and models are under the viamrobotics Hugging Face org. BoxBot trained on Hugging Face Jobs, not Viam. The Viam training steps (4–6) haven't been run. Before merging, someone should run one job and check:

  • The Dockerfile adds pip, because lerobot-gpu's uv-created venv has none. I haven't built the image.
  • The image runs as a non-root user (user_lerobot). Unknown whether Viam's job launcher cares.
  • The script downloads lerobot/smolvla_base from Hugging Face at runtime, so this needs outbound internet from training jobs. If jobs don't have it, bake the weights into the image.
  • The script symlinks the job's three Parquet files into the export layout the converter reads. That the absolute /gcs/ image paths resolve was checked in the converter's source, not by running it.
  • Job time and GPU limits versus SmolVLA fine-tuning. BoxBot measured about 30 minutes on an A100 at batch 64.

The page relies on two viam-labs repos. Confirm the team is comfortable with docs pointing to them.

Checks

  • prettier 3.2.5
  • markdownlint
  • vale 3.12.0 (CI-pinned)
  • scripts/frontmatter_rules.py
  • make build-prod

🤖 Generated with Claude Code

New docs/train/train-a-vla.md: record teleoperated demonstrations as
sequences, convert them with viam-sequence-to-lerobot, fine-tune SmolVLA
in a custom training container, and run the policy with the
viam-labs:vla module. Link it from the custom containers page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@netlify

netlify Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for viam-docs ready!

Name Link
🔨 Latest commit 9d40c71
🔍 Latest deploy log https://app.netlify.com/projects/viam-docs/deploys/6acee468f59d6c11c48d6402
😎 Deploy Preview https://deploy-preview-5412--viam-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 36 (🔴 down 3 from production)
Accessibility: 100 (no change from production)
Best Practices: 100 (no change from production)
SEO: 92 (no change from production)
PWA: 60 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

@viambot viambot added the safe to build This pull request is marked safe to build from a trusted zone label Oct 14, 2026
Cite LeRobot's ~50-episode recommendation, explain which demonstrations
to delete, introduce viam-sequence-to-lerobot before its first use,
reword the web UI submission path, explain why to monitor the job, and
rename "Next steps" to "Learn more".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to build This pull request is marked safe to build from a trusted zone

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants