Skip to content

feat(api): add POST /snapshots/{snapshotID}/fork — fork N sandboxes from a named snapshot - #3441

Open
AdaAibaby wants to merge 3 commits into
e2b-dev:mainfrom
AdaAibaby:feat/fork-from-named-snapshot
Open

feat(api): add POST /snapshots/{snapshotID}/fork — fork N sandboxes from a named snapshot#3441
AdaAibaby wants to merge 3 commits into
e2b-dev:mainfrom
AdaAibaby:feat/fork-from-named-snapshot

Conversation

@AdaAibaby

@AdaAibaby AdaAibaby commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Problem

RL/agent training workloads need to run many evaluation batches from a single checkpoint, but there was no way to do this efficiently.

The typical pattern (from issue #3424):

1. Create sandbox → install deps, load model weights  (expensive, done once)
2. POST /sandboxes/{id}/snapshots  →  "team/rl-checkpoint-v1"
3. DELETE /sandboxes/{id}          →  original no longer needed
4. For each training batch:
     POST /snapshots/team%2Frl-checkpoint-v1/fork?count=32
     ... run rollouts ...
     DELETE each episode sandbox
5. Repeat step 4 indefinitely from the same checkpoint

Before this PR, step 4 failed after step 3. The only alternatives were:

  • Keep the source alive — wastes resources for the entire training run
  • Fork from running sandbox (POST /sandboxes/{id}/fork) — requires source to still be running
  • Create N fresh sandboxes — no shared checkpoint state; model weights must be loaded again every batch

Solution

New endpoint: POST /snapshots/{snapshotID}/fork

POST /snapshots/team%2Frl-checkpoint-v1/fork
X-API-Key: ...
Content-Type: application/json

{ "count": 32, "timeout": 300 }

Response (201) — one entry per fork, each independently succeeded or failed:

[
  { "sandbox": { "sandboxID": "iabc...", "templateID": "xyz...", ... } },
  { "sandbox": { "sandboxID": "idef...", "templateID": "xyz...", ... } },
  ...
]

Partial batches are still useful — a single failed fork doesn't abort the others.

Key implementation detail: why templateCache not snapshotCache

Ephemeral pause snapshots (source = 'snapshot' in the envs table) are soft-deleted when the sandbox is killeddeleted_at is set, active_envs JOIN returns nothing, and snapshotCache.Get returns 404.

Named snapshot templates (source = 'snapshot_template') are persistent — they survive sandbox deletion and are never touched by the kill path. This handler resolves the alias via templateCache.ResolveAlias then fetches build data from env_build_assignments for the persistent template env. This is exactly the same code path POST /sandboxes uses to boot from a template, so cluster routing, envd-version checks, and placement logic are all inherited automatically.

Verified end-to-end on dev

Full RL training simulation: create sandbox → snapshot → kill original → fork 3 batches (3 + 3 + 5 forks) from the same checkpoint:

Step 1: Create sandbox                     HTTP 201  sandboxID: i47hue0e3nf7gvp8m44bw
Step 2: Snapshot → 972963285/rl-checkpoint HTTP 201
Kill original sandbox                      HTTP 204

Batch 1: fork 3 sandboxes                 HTTP 201  forks=3 errors=0
  [kill 3]                                all 204
Batch 2: fork 3 sandboxes                 HTTP 201  forks=3 errors=0
  [kill 3]                                all 204
Batch 3: fork 5 sandboxes                 HTTP 201  forks=5 errors=0
  [kill 5]                                all 204

Checkpoint still listed after all batches: True

Fixes #3424 (Proposal B).

/cc @jakubno @dobrac @ValentaTomas @arkamar @tvi Looking forward to your code review.

Implements Proposal B from issue e2b-dev#3424: fork N sandboxes from a named
snapshot template without requiring the original sandbox to still be running.

Why

RL/agent training workloads need to checkpoint once then run many
evaluation batches from that checkpoint:

  1. Create sandbox, load model weights / install deps (expensive)
  2. Snapshot the sandbox to a named checkpoint (e.g. "team/rl-v1")
  3. Kill original sandbox -- no longer needed
  4. Per training batch: fork N episode sandboxes, run rollouts, kill
  5. Repeat step 4 indefinitely from the same checkpoint

Before this change step 4 was impossible after step 3. The only
alternatives were fork-from-running-sandbox (keeps the source alive,
wastes resources) or N fresh sandboxes (no shared checkpoint, slow).

What

New endpoint: POST /snapshots/{snapshotID}/fork

Request body (same schema as POST /sandboxes/{id}/fork):
  count    int -- number of forks (default 1, max 100)
  timeout  int -- TTL for each fork in seconds

Returns 201 with a list of SandboxForkResult entries, one per requested
fork. Each entry carries either {sandbox: ...} (success) or {error: ...}
(failure), so a partial batch is still useful.

Implementation note: templateCache vs snapshotCache

Ephemeral pause snapshots (source=snapshot) are soft-deleted when the
sandbox is killed -- their deleted_at is set and active_envs returns
nothing, so snapshotCache.Get returns 404 for killed sandboxes.

Named snapshot templates (source=snapshot_template) are persistent: they
survive sandbox deletion and are never touched by the kill path. This
handler resolves the alias via templateCache.ResolveAlias, then fetches
build data from env_build_assignments for the persistent template env --
the same path POST /sandboxes uses to start from a template.
AdaAibaby and others added 2 commits July 30, 2026 14:33
Unlike /sandboxes/{id}/fork where the source sandbox occupies a slot,
/snapshots/{id}/fork has no running source sandbox. Using >= rejected
count == limit even when all slots were free. Changed to > so users can
fork up to their full concurrency limit in one request.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC: Allow fork from paused sandbox and named snapshots for RL/agent training workloads

2 participants