Repository navigation
feat(k8s): Kubernetes OpenShellSandbox API backed by the gateway #5062
Description
User Story
I use OpenShell on Kubernetes to run agents securely.
- As a developer, I manage my Kubernetes workloads with kubectl and native tooling. I want to create, update, stop, and delete OpenShell sandboxes the same way, without switching to the openshell CLI, so sandboxes fit the workflows and RBAC I already use.
- As a platform engineer, I want GitOps tools such as Argo CD or Flux to manage OpenShell sandboxes from Git, so sandbox definitions get the same review, drift detection, and audit trail as every other workload.
Problem Statement
OpenShell has no Kubernetes-native API for sandboxes. Users create them through the gateway (CLI, SDK, or gRPC). The agents.x-k8s.io/Sandbox objects the Kubernetes driver creates are an implementation detail and do not carry the OpenShell contract (policy, providers, templates). Templates and providers exist only in the gateway, so kubectl users cannot discover them or diagnose a sandbox.
Issue #1719 asks for Kubernetes-native management of OpenShell. This issue covers one piece: a namespaced sandbox resource, plus read-only views of the templates and providers it references. Gateway installation (the operator) stays as a separate effort.
Impact / Why This Matters
Without it, users run imperative CLI or SDK calls from CI, manage access in the gateway instead of with namespace RBAC, and lose GitOps drift detection, kubectl wait, admission policy, and Kubernetes audit for sandboxes.
A CRD and controller would fix the UX but copy sandbox state into etcd while the gateway database stays authoritative. That creates drift, split-brain failures on create and delete, conflicts with CLI changes, and etcd overhead when the agent density increases. The API should keep the gateway as the single source of truth.
Proposed Design
An optional Deployment serves openshell.ai/v1alpha1 through Kubernetes API aggregation translating requests to the gateway gRPC.
Aggregate Resources
| Kind | Gateway object | v1alpha1 access |
|---|---|---|
| OpenShellSandbox | Sandbox | Full CRUD, watch, log subresource |
| OpenShellSandboxTemplate | SandboxWorkloadTemplate | Read-only |
| OpenShellProvider | Provider | Read-only; credentials never returned |
The OpenShell prefix avoids confusion with agents.x-k8s.io/Sandbox, as recommended in #1719.
apiVersion: openshell.ai/v1alpha1
kind: OpenShellSandbox
metadata:
name: research-agent
namespace: team-a # bound to an OpenShell workspace
spec:
workloadTemplate: claude-code
providers: [anthropic]
policy: # inline SandboxPolicy
version: 1
filesystem: {}
landlock: {}
process: {}
networkPolicies:
anthropic-api:
endpoints:
- host: api.anthropic.com
port: 443
binaries:
- path: /usr/local/bin/claude
command: ["claude"]
restartPolicy: OnFailure # Never | OnFailure | Always
desiredState: Running # Running | Stopped | Suspended
status:
phase: Ready
conditions: # Ready, PolicyLoaded, ProvidersReady
- type: Ready
status: "True"
observedGeneration: 1Key behaviors:
- Gateway is authoritative. Sandboxes created or changed through the CLI appear in kubectl get and watch events.
- Updates follow the gateway. Providers, network policy, settings, and desiredState can change in place. Fields applied at startup (template, command, Landlock, process) are rejected with a "delete and recreate" error.
- Standard Kubernetes semantics. kubectl wait, watch across namespaces with 410 Gone, server-side apply,
--dry-run=server, and label selectors work, so Argo CD and Flux can manage sandboxes without perpetual drift. - Two-layer auth. Kubernetes RBAC authorizes users per namespace; the deployment uses its own narrowly scoped gateway identity.
- Safe when down. Discovery stays local, so a gateway outage does not block deletion of unrelated namespaces.
Exec, SSH, port forwarding, and draft approval stay in the openshell CLI.
Acceptance Criteria
-
kubectl applycreates exactly one gateway sandbox, andkubectl wait --for=condition=Readysucceeds. - Templates and providers in the bound workspace are listable; provider credentials are never returned.
- No OpenShell objects are stored in etcd; restarting the component loses nothing.
-
kubectl get -A -wshows ordered events for changes from both Kubernetes and the CLI. - Supported fields update in place; fixed fields are rejected; desiredState stops, suspends, and resumes.
- Argo CD and Flux sync, diff, prune, and assess health without perpetual drift.
- RBAC is enforced per namespace, resource, and log subresource.
- A gateway outage does not block discovery or deletion of unbound namespaces.
Alternatives Considered
- CRD plus controller: two sources of truth (etcd and the gateway database). Rejected.
- Expose
agents.x-k8s.io/Sandbox: bypasses gateway policy and auth, and exists only for the Kubernetes driver. - CRUD-only facade: simpler, but breaks watch, kubectl wait, server-side apply, and GitOps.
- Gateway interceptors, middleware, or providers: cannot serve a Kubernetes API group.
Agent Investigation
The gateway needs these changes to support Kubernetes semantics:
- an atomic UpdateSandbox RPC (only targeted update RPCs exist today);
- a collection-wide change journal for list and watch (watch is per sandbox today);
- delete preconditions;
- side-effect-free dry-run;
- set-based label selectors;
- a dedicated gateway identity.
- New persistent state lives in the gateway database: namespace bindings, Kubernetes-only metadata (such as generation, managedFields, and desiredState), and the change journal.
Open questions for the RFC: network-policy ownership when drafts are approved on a GitOps-managed sandbox, and gateway audit attribution of the Kubernetes user.
Request: If the overall approach looks good, please assign an RFC number and apply the needs-rfc label. A detailed design (field mapping, gateway changes, phased plan) and prototype implementation can be provided with the RFC.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
Activity
[-]Kubernetes `OpenShellSandbox` API backed by the gateway[/-][+]feat(k8s): Kubernetes `OpenShellSandbox` API backed by the gateway[/+]