Skip to content

feat(k8s): Kubernetes OpenShellSandbox API backed by the gateway #5062

Open

Description

@JimBugwadia

User Story

I use OpenShell on Kubernetes to run agents securely.

  • As a developer, I manage my Kubernetes workloads with kubectl and native tooling. I want to create, update, stop, and delete OpenShell sandboxes the same way, without switching to the openshell CLI, so sandboxes fit the workflows and RBAC I already use.
  • As a platform engineer, I want GitOps tools such as Argo CD or Flux to manage OpenShell sandboxes from Git, so sandbox definitions get the same review, drift detection, and audit trail as every other workload.

Problem Statement

OpenShell has no Kubernetes-native API for sandboxes. Users create them through the gateway (CLI, SDK, or gRPC). The agents.x-k8s.io/Sandbox objects the Kubernetes driver creates are an implementation detail and do not carry the OpenShell contract (policy, providers, templates). Templates and providers exist only in the gateway, so kubectl users cannot discover them or diagnose a sandbox.

Issue #1719 asks for Kubernetes-native management of OpenShell. This issue covers one piece: a namespaced sandbox resource, plus read-only views of the templates and providers it references. Gateway installation (the operator) stays as a separate effort.

Impact / Why This Matters

Without it, users run imperative CLI or SDK calls from CI, manage access in the gateway instead of with namespace RBAC, and lose GitOps drift detection, kubectl wait, admission policy, and Kubernetes audit for sandboxes.

A CRD and controller would fix the UX but copy sandbox state into etcd while the gateway database stays authoritative. That creates drift, split-brain failures on create and delete, conflicts with CLI changes, and etcd overhead when the agent density increases. The API should keep the gateway as the single source of truth.

Proposed Design

An optional Deployment serves openshell.ai/v1alpha1 through Kubernetes API aggregation translating requests to the gateway gRPC.

Aggregate Resources

Kind Gateway object v1alpha1 access
OpenShellSandbox Sandbox Full CRUD, watch, log subresource
OpenShellSandboxTemplate SandboxWorkloadTemplate Read-only
OpenShellProvider Provider Read-only; credentials never returned

The OpenShell prefix avoids confusion with agents.x-k8s.io/Sandbox, as recommended in #1719.

apiVersion: openshell.ai/v1alpha1
kind: OpenShellSandbox
metadata:
  name: research-agent
  namespace: team-a            # bound to an OpenShell workspace
spec:
  workloadTemplate: claude-code
  providers: [anthropic]
  policy:                      # inline SandboxPolicy
    version: 1
    filesystem: {}
    landlock: {}
    process: {}
    networkPolicies:
      anthropic-api:
        endpoints:
          - host: api.anthropic.com
            port: 443
        binaries:
          - path: /usr/local/bin/claude
  command: ["claude"]
  restartPolicy: OnFailure     # Never | OnFailure | Always
  desiredState: Running        # Running | Stopped | Suspended
status:
  phase: Ready
  conditions:                  # Ready, PolicyLoaded, ProvidersReady
    - type: Ready
      status: "True"
  observedGeneration: 1

Key behaviors:

  1. Gateway is authoritative. Sandboxes created or changed through the CLI appear in kubectl get and watch events.
  2. Updates follow the gateway. Providers, network policy, settings, and desiredState can change in place. Fields applied at startup (template, command, Landlock, process) are rejected with a "delete and recreate" error.
  3. Standard Kubernetes semantics. kubectl wait, watch across namespaces with 410 Gone, server-side apply, --dry-run=server, and label selectors work, so Argo CD and Flux can manage sandboxes without perpetual drift.
  4. Two-layer auth. Kubernetes RBAC authorizes users per namespace; the deployment uses its own narrowly scoped gateway identity.
  5. Safe when down. Discovery stays local, so a gateway outage does not block deletion of unrelated namespaces.

Exec, SSH, port forwarding, and draft approval stay in the openshell CLI.

Acceptance Criteria

  • kubectl apply creates exactly one gateway sandbox, and kubectl wait --for=condition=Ready succeeds.
  • Templates and providers in the bound workspace are listable; provider credentials are never returned.
  • No OpenShell objects are stored in etcd; restarting the component loses nothing.
  • kubectl get -A -w shows ordered events for changes from both Kubernetes and the CLI.
  • Supported fields update in place; fixed fields are rejected; desiredState stops, suspends, and resumes.
  • Argo CD and Flux sync, diff, prune, and assess health without perpetual drift.
  • RBAC is enforced per namespace, resource, and log subresource.
  • A gateway outage does not block discovery or deletion of unbound namespaces.

Alternatives Considered

  • CRD plus controller: two sources of truth (etcd and the gateway database). Rejected.
  • Expose agents.x-k8s.io/Sandbox: bypasses gateway policy and auth, and exists only for the Kubernetes driver.
  • CRUD-only facade: simpler, but breaks watch, kubectl wait, server-side apply, and GitOps.
  • Gateway interceptors, middleware, or providers: cannot serve a Kubernetes API group.

Agent Investigation

The gateway needs these changes to support Kubernetes semantics:

  • an atomic UpdateSandbox RPC (only targeted update RPCs exist today);
  • a collection-wide change journal for list and watch (watch is per sandbox today);
  • delete preconditions;
  • side-effect-free dry-run;
  • set-based label selectors;
  • a dedicated gateway identity.
  • New persistent state lives in the gateway database: namespace bindings, Kubernetes-only metadata (such as generation, managedFields, and desiredState), and the change journal.

Open questions for the RFC: network-policy ownership when drafts are approved on a GitOps-managed sandbox, and gateway audit attribution of the Kubernetes user.

Request: If the overall approach looks good, please assign an RFC number and apply the needs-rfc label. A detailed design (field mapping, gateway changes, phased plan) and prototype implementation can be provided with the RFC.

Checklist

  • I've reviewed existing issues and the published docs
  • This is a design proposal, not a "please build this" request

Activity

  1. changed the title [-]Kubernetes `OpenShellSandbox` API backed by the gateway[/-] [+]feat(k8s): Kubernetes `OpenShellSandbox` API backed by the gateway[/+] on Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions