tart-executor

tart-executor is an app to launch ephemeral virtual machines on demand from Github action jobs using Tart.

tart-router is an optional app to handle routing Github actions to multiple macOS computers running tart-executor.

🚀 Getting Started

Tart

First install Tart if not already installed on all macOS computers running VMs:

brew install cirruslabs/cli/tart

Tart Executor

Install tart-executor on all macOS computers that will run VMs:

brew tap bludesign/tart
brew install bludesign/tart/tart-executor

Create config file in home directory named tart-executor.yaml

# Host name for metrics and logs (requried)
hostname: server
# Total CPU capacity used for aggregate VM scheduling (required)
cpuLimit: 10
# Total memory capacity used for aggregate VM scheduling, in binary MB (required)
totalMemory: 16384
# Optional logging endpoint URL to send NDJSON via HTTP POST (optional)
loggingEndpoint: http://10.0.1.50:8080/logs
# Optional bearer token protecting the management API at /api/v1/* (optional)
# When set, /api/v1/* (except /health, /openapi.yaml, /docs) requires the header
# Authorization: Bearer <token>. When omitted the management API is open.
apiToken: change-me-secret
tart:
  # Custom home folder to use for Tart if you use an external drive you will need to allow access when the dialog shows up (optional)
  homeFolder: /Volumes/Files/tart
  # Run VMs in headless mode (optional)
  isHeadless: true
  # Array of OCI registries that use insecure pulls (optional)
  insecureDomains:
    - 10.0.1.100
  # Max VMs to run at a time. This will not work higher then 2 because of Apple's EULA and Virtualization.Framework limit (required)
  numberOfVirtualMachines: 2
  # SSH username and password for Tart's images this is admin for username and password
  ssh:
    username: admin
    password: admin
  # Default CPU count to use per VM recommend setting this to machine CPU count/2. If not set will use jobs label if set or images default (optional)
  defaultCpu: 5
  # Default memory to use per VM. If not set will use jobs label if set or images default (optional)
  defaultMemory: 6144
github:
  # organization or repo (required)
  runnerScope: organization
  # Required for organization runner scope leave out for repo scope
  organizationName: bludesign
  # Required for repo runner scope leave out for organization scope
  ownerName: bludesign
  # Required for repo runner scope leave out for organization scope
  repositoryName: tart
  # Github app ID (required)
  appId: 1234567
  # Path to private key for github app (required)
  privateKey: /Users/server/private-key.pem
  # Scanning GitHub for jobs that are queued but have no runner (optional, on by default).
  # Powers GET /api/v1/github/queued-jobs. Requires the GitHub App to have Actions: read.
  scan:
    # Set to false to disable scanning entirely; the endpoint then answers 503 (optional)
    enabled: true
    # How long a scan result is served before refreshing in the background (optional)
    cacheSeconds: 30
    # How long the repository list is reused before being re-fetched (optional)
    repositoryCacheSeconds: 600
    # Most repositories to visit in one scan, most-recently-pushed first (optional)
    maxRepositories: 200
    # Requests in flight at once. Raising this risks GitHub's secondary rate limits (optional)
    maxConcurrentRequests: 4
    # Workflow runs fetched per repository, newest first (optional)
    runsPerRepository: 50
    # Wall-clock budget for one scan in seconds (optional)
    timeoutSeconds: 20
    # Abort the scan when fewer than this many GitHub API requests remain (optional)
    rateLimitFloor: 200
    # Only scan these owner/repo names. Leave out to scan every installed repository (optional)
    includeRepositories:
      - bludesign/tart
    # Never scan these owner/repo names (optional)
    excludeRepositories:
      - bludesign/archive
runner:
  # Runner label to look for in Github action job labels to create VM (required)
  labels: tartelet
webhook:
  # Port to listen for webhook calls (required)
  port: 3250
  # URL for tart-router no trailing slash only needed if using tart-router
  routerUrl: http://10.0.4.1:3251

Run by typing tart-executor into terminal.

Tart Router

Optionally if running VMs on multiple macOS computers install tart-router on one computer. Make sure you are aware of the licensing limits for Tart. This can be run on a computer that is also running tart-executor as long as the webhook port is different or a seperate computer:

brew tap bludesign/tart
brew install bludesign/tart/tart-router

Create config file in home directory named tart-router.yaml

# Host name for metrics and logs (requried)
hostname: macbook-router
# Runner label to look for in Github action job labels should match the tart-executors (required)
labels: tartelet
# Port to listen on for Github webhook calls (required)
port: 3251
# Optional logging endpoint URL to send NDJSON via HTTP POST (optional)
loggingEndpoint: http://10.0.1.50:8080/logs
# Optional bearer token protecting the management API at /api/v1/* (optional)
# When set, /api/v1/* (except /health, /openapi.yaml, /docs) requires the header
# Authorization: Bearer <token>. When omitted the management API is open.
apiToken: change-me-secret
# Array of tart-executor hosts to forward Github action jobs to (required)
hosts:
    # Hostname set in config for tart-executor (required)
  - hostname: server-1
    # URL for tart-executor no trailing slash (required)
    url: http://127.0.0.1:3250
    # Priority to send jobs to host (required)
    priority: 0
    # Per-job CPU maximum. Aggregate capacity comes from the executor's cpuLimit (optional)
    cpuLimit: 5
    # Per-job memory maximum. Aggregate capacity comes from the executor's totalMemory (optional)
    memoryLimit: 8192
  - hostname: server-2
    url: http://10.0.4.2:3250
    priority: 1
  - hostname: server-3
    url: http://10.0.4.3:3250
    priority: 2

Run by typing tart-router into terminal.

The router treats each host's cpuLimit and memoryLimit as per-job maximums. Before dispatching, it also checks the executor's live aggregate CPU and memory usage and its virtual machine count. A job without an explicit cpu: or memory: label uses the executor's configured default for scheduling when available. Unknown resource values retain the previous behavior and do not block dispatch. For example, a 10-CPU / 28672-MB executor can use router per-job limits of cpuLimit: 10 and memoryLimit: 28672: a 7-CPU / 24576-MB job can share the host with a 2-CPU / 4096-MB job, while another 7-CPU / 24576-MB job waits for resources.

Github App

tart-executor will automatically register the virtual machine as a runner on a GitHub organization or on a specified repository. In order to do this it must be configured with the relevant credentials. Follow the steps below to add the credentials to tart-executor. You can reuse the credentials from the GitHub App on all host machines running tart-executor.

1. Create a GitHub App on your organization or account. This can be done by under "Developer settings" in your organization's or your personal settings or by following this link when creating the GitHub app on a personal account or the following link when creating the GitHub app on an organization: https://github.com/organizations/{YOUR_ORGANIZATION_NAME}/settings/apps. Remember to change the link to include the name of your organization if needed.
2. When creating the GitHub App, the required permissions depend on whether you are creating the app for an organization or a personal account. 3. 3. Organizations should enable the Organization: Actions and Self-hosted runners (Read and write) permission and personal accounts should enable the Repository: Actions, Administration and Metadata (Read and write) permissions.
3. Under subscribe to events check Workflow job.
4. Set the Webhook URL to the URL that is reachable by Github that goes to the webhook server on tart-executor or tart-router if using the router.
5. After creating the app install the GitHub app on your organization.
6. Select "Generate a private key". tart-executor will use this to send authorized requests to the API. The generated key should automatically be downloaded.
7. Transfer the generated private key to your tart-executor machines and set the path in the config.

Actions: read (required for queued-job scanning)

GET /api/v1/github/queued-jobs additionally needs the repository permission Actions (Read), on top of the permissions above. Metadata read is already implicit and covers the repository listing.

If you are adding this to an App that already exists, GitHub treats it as a permission update: request it in the App's settings, then an organization owner has to approve it on the installation. Until they do, every scan is denied with Resource not accessible by integration, and the endpoint reports that in warnings (it returns 200 with an empty jobs array, so read the warnings rather than concluding the queue is empty).

Recovering a job that is stuck without a runner

Occasionally GitHub reports a job as queued but no VM ever starts for it — a webhook delivery was missed, or the executor lost the job. The management API can find and fix these without any GitHub credentials of your own:

# What is GitHub waiting on? `matching=all` includes jobs this executor cannot run, each
# annotated with matchesRunnerLabels and why.
curl -s -H "Authorization: Bearer $TOKEN" \
  'http://127.0.0.1:3250/api/v1/github/queued-jobs?matching=all' | jq

# Start a VM for one. Pass the job's labels exactly as the scan reported them.
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"id":123456,"labels":["tartelet","ghcr.io/cirruslabs/macos-sequoia-xcode:latest"]}' \
  http://127.0.0.1:3250/api/v1/jobs/dispatch | jq

Send it to the router instead (http://127.0.0.1:3251/api/v1/jobs/dispatch) to have it placed on whichever executor has capacity, optionally pinning one with "host": "server-1".

Two things to know about the labels: they are passed to the runner's --labels verbatim, so reordering or dropping any of them (including cpu:/memory:) registers a runner GitHub is not waiting on — the VM boots, idles, and the job stays queued. And force bypasses the already-tracked and capacity guards (deliberately over-committing past numberOfVirtualMachines), but never the guard against a VM that already exists for that job id; cancel the job first if you really want to replace its machine.

Brew Service

You can use a brew service to run the executor and router in the background and on startup:

brew services start tart-executor

brew services start tart-router

MacOS Local Network Access

The first time you run tart-executor on a host machine, macOS shows a popup asking you to allow local network access (and, when a job attaches a disk, removable-volume access). Grant it once: you will probably need to cancel the in-flight job and check GitHub settings to make sure there are no leftover runners, then restart tart-executor and rerun the job. On startup tart-executor automatically removes leftover temporary VMs (named tart-executor-*) from a previous run.

Release binaries are signed with a stable Developer ID (com.BluDesign.TartExecutor / com.BluDesign.TartRouter, team B88BRQ88KH) and notarized by Apple, so this grant is tied to a code-signing identity that stays constant across releases. You only need to allow local network / removable storage access once — the permission now persists through brew upgrade instead of being requested again on every update.

🔌 Management & Debugging API

Both tart-executor and tart-router expose a JSON management/debugging API under /api/v1 on the same port they already listen on (the executor's webhook.port, e.g. 3250, and the router's port, e.g. 3251). Use it to inspect and control a running instance: list and stop jobs, manage Tart VMs and images, and view effective settings, status, and health.

Authentication

Set apiToken in the config to require a bearer token. When set, every /api/v1/* endpoint except /health, /openapi.yaml, and /docs requires the header Authorization: Bearer <token>. When apiToken is omitted the API is open, matching the existing endpoints which rely on private-network placement.

Interactive docs

Each service serves its own OpenAPI 3.1 spec and a browsable Redoc page:

• Spec: GET /api/v1/openapi.yaml
• Docs: open http://<host>:<port>/api/v1/docs in a browser

tart-executor endpoints

Method	Path	Description
GET	/api/v1/health	Liveness and version
GET	/api/v1/status	Job counts and capacity
GET	/api/v1/settings	Effective settings (secrets redacted)
GET	/api/v1/jobs	List jobs (optional state query: pending, in_progress, active)
GET	/api/v1/jobs/{id}	Job detail
POST	/api/v1/jobs/{id}/cancel	Cancel one job by id
POST	/api/v1/jobs/cancel	Cancel by labels (body {"labels":["..."]})
POST	/api/v1/jobs/cancel-all	Cancel all active jobs
POST	/api/v1/jobs/dispatch	Start a VM for a job (body {"id":123,"labels":["..."],"force":false})
GET	/api/v1/github/queued-jobs	GitHub Actions jobs waiting for a runner (?matching=all, ?refresh=true)
GET	/api/v1/vms	List local Tart VMs / images
GET	/api/v1/vms/{name}	VM detail including IP
DELETE	/api/v1/vms/{name}	Delete a VM
POST	/api/v1/images/pull	Pull an image (body {"name":"...","isInsecure":false})

tart-router endpoints

Method	Path	Description
GET	/api/v1/health	Liveness and version
GET	/api/v1/status	Router queue and capacity
GET	/api/v1/settings	Effective settings
GET	/api/v1/jobs	List tracked jobs
GET	/api/v1/jobs/{id}	Job detail
POST	/api/v1/jobs/{id}/cancel	Cancel one job (and on its host)
POST	/api/v1/jobs/cancel	Cancel by labels across all hosts
POST	/api/v1/jobs/dispatch	Queue and place a job (body {"id":123,"labels":["..."],"host":null,"force":false})
GET	/api/v1/hosts	List executors and their last-polled status
GET	/api/v1/hosts/{hostname}	Executor detail
POST	/api/v1/hosts/refresh	Force an immediate status re-poll

Example:

curl -H "Authorization: Bearer <token>" http://127.0.0.1:3250/api/v1/jobs
curl -X POST -H "Authorization: Bearer <token>" http://127.0.0.1:3250/api/v1/jobs/42/cancel

👨‍🔧 How does it work?

For jobs that you want tart-executor to handle set the labels on a job as the label you have set in the config for example tartelet and then the image that you would like to use for example ghcr.io/cirruslabs/macos-sequoia-xcode:latest so in the action config you would have: runs-on: [tartelet, "ghcr.io/cirruslabs/macos-sequoia-xcode:latest"] when a new job runs tart-executor will get a callback and check if the labels in the job. If the labels contains the label set in config it handles creating a VM for that job. The lifecycle of a GitHub Actions runner managed by tart-executor is as follows:

1. tart-executor uses Tart to pull a virtual machine which will update the image if needed.
2. tart-executor uses Tart to clone a virtual machin to a temporary image.
3. The virtual machine is booted.
4. After the machine is booted, a setup script is being run. The script downloads the newest version of GitHub's runner application and registers the runner on the GitHub organization.
5. The runner listens for a job and executes it.
6. After executing the job, the runner automatically removes itself from the GitHub organization.
7. The virtual machine is shutdown.
8. tart-executor uses Tart to delete the temporary virtual machine.

You can also set memory and CPU labels on jobs to override the number of CPUs or memory the VM will have that is created for that job. If you don't set these tart-executor will use the default set in the config or if no default is set the amount set on the VM image when it was created. For example:

runs-on: [tartelet, "ghcr.io/cirruslabs/macos-sequoia-xcode:latest", "memory:24576", "cpu:6"]

🏎 How is the performance?

From the time a job is started to a VM being booted and ready to start that job is around 25 seconds. The performace of the Tart VM depends on the CPUs and memory set for that VM.

👩‍💻 How can I contribute?

Pull requests with bugfixes and new features are much appreciated. We are happy to review PRs and merge them once they are ready.

Linting the Codebase with SwiftLint

We use SwiftLint to ensure uniformity in the code. Install SwiftLint using Homebrew by running the following command in your terminal.

brew install swiftlint

🙏 Acknowledgements

• Forked from Tartelet which has a GUI instead of running from CLI.
• Tart does all the heavy-lifting of creating, cloning, and running virtual machines.
• Tartelet is heavily inspired by Cilicon.