diff --git a/docs/cli/datasets-and-training.md b/docs/cli/datasets-and-training.md index b42785c6ce..8ae37d360d 100644 --- a/docs/cli/datasets-and-training.md +++ b/docs/cli/datasets-and-training.md @@ -36,6 +36,9 @@ To list training containers and find container IDs, including your organization' viam train containers list --org-id= ``` +The `--org-id` flag is optional. +Without it, the command lists only Viam-managed containers. + ## Manage datasets ### Create a dataset @@ -270,7 +273,9 @@ On ARM Macs, Docker runs them through Rosetta emulation, which may be slower. viam training-script test-local \ --dataset-root=./my-dataset/ \ --training-script-directory=./my-training-script/ \ - --container-id= + --dataset-file=dataset.jsonl \ + --container-id= \ + --model-output-directory=./output/ ``` ## Manage training containers diff --git a/docs/cli/reference.md b/docs/cli/reference.md index 057c498624..456fc73aa1 100644 --- a/docs/cli/reference.md +++ b/docs/cli/reference.md @@ -3144,7 +3144,7 @@ Manage training scripts for [custom ML training](/train/custom-training-scripts/ ```sh {class="command-line" data-prompt="$"} viam training-script upload --framework= --org-id= --path= --script-name= --type= viam training-script update --org-id= --script-name= --visibility= -viam training-script test-local --training-script-directory= --dataset-root= --container-id= [...named args] +viam training-script test-local --training-script-directory= --dataset-root= --container-id= [--dataset-file=] [--model-output-directory=] [--custom-args=] ``` ### `training-script upload` @@ -3193,7 +3193,7 @@ Run an ML training script locally with Docker, in the same container a cloud tra See [Test locally with Docker](/train/custom-training-scripts/#test-locally-with-docker). ```sh {class="command-line" data-prompt="$"} -viam training-script test-local --training-script-directory=my-training/ --dataset-root=./my-dataset/ --container-id=abc123 --model-output-directory=./output/ +viam training-script test-local --training-script-directory=my-training/ --dataset-root=./my-dataset/ --dataset-file=dataset.jsonl --container-id=abc123 --model-output-directory=./output/ ``` diff --git a/docs/train/custom-containers.md b/docs/train/custom-containers.md index 95aef1eb3e..23dd68b539 100644 --- a/docs/train/custom-containers.md +++ b/docs/train/custom-containers.md @@ -20,7 +20,8 @@ When you start a [custom training job](/train/custom-training-scripts/), you can Viam-managed containers come with Python and TensorFlow. Any other dependency your training script lists in `setup.py` gets installed with pip at the start of every job, on billed GPU time. For large dependency stacks, such as the libraries for fine-tuning a vision-language-action (VLA) model, that install can take a long time. -A custom container has those dependencies built in, so each job starts training right away. +A custom container has those dependencies built in. +As long as the container includes every dependency your training script needs, jobs don't spend time installing dependencies with pip and start training right away. The container holds only the environment. Your training script stays in the registry and works without changes: the same `setup.py` and `model/training.py` structure runs in a custom container just as it does in a Viam-managed one. @@ -48,20 +49,8 @@ Keep your training code in your training script, which stays private to your org ## Build and push an image -Start from an NVIDIA CUDA base image so your dependencies can use the GPU that training jobs run on. -Then install Python, pip, and the packages your training script needs. - -For example, this `Dockerfile` installs PyTorch and Hugging Face Transformers: - -```dockerfile {class="line-numbers linkable-line-numbers"} -FROM nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 - -RUN apt-get update && \ - apt-get install -y --no-install-recommends python3 python3-pip && \ - rm -rf /var/lib/apt/lists/* - -RUN pip3 install --no-cache-dir torch transformers -``` +Build an image that meets the [requirements](#requirements) and includes the packages your training script needs. +You can start from any base image. Build the image for `linux/amd64` and push it to Docker Hub: @@ -73,7 +62,6 @@ docker buildx build --platform linux/amd64 \ Make sure the repository is public on Docker Hub. You can use any tool that produces a standard Docker image. -For example, [Cog](https://cog.run/) can generate a GPU-ready image from a short configuration file. ## Register the container @@ -139,6 +127,7 @@ See [Test locally with Docker](/train/custom-training-scripts/#test-locally-with ```sh {class="command-line" data-prompt="$"} viam training-script test-local \ --training-script-directory=my-training/ \ + --dataset-file=dataset.jsonl \ --dataset-root= \ --container-id= \ --model-output-directory=