Skip to content

[WIP] ci: add an IPv6-only kind e2e job for both sandbox classes - #939

Draft
Yuan Gao (ygao-g) wants to merge 2 commits into
agent-substrate:mainfrom
ygao-g:ci-kind-ipv6
Draft

[WIP] ci: add an IPv6-only kind e2e job for both sandbox classes#939
Yuan Gao (ygao-g) wants to merge 2 commits into
agent-substrate:mainfrom
ygao-g:ci-kind-ipv6

Conversation

@ygao-g

@ygao-g Yuan Gao (ygao-g) commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Do not merge until #958 and #1275 land. The micro-VM leg also needs #1122.

Part of #246.

No CI job exercises IPv6-only, so nothing catches a change that breaks it, and
the IPv6 work is a set of changes that do little apart. This job builds a
single-stack IPv6 kind cluster, installs the full system, and runs the
networking suite — where every test that can tell one address family from
another lives — on both sandbox classes off the one cluster, asserting the
cluster really is single-stack before anything is installed so a green run
cannot be vacuous. It carries no copy of the code it exercises: dispatch it
with a list of pull requests and it merges them in that order, and the stack
lives only for the length of the run.

Both classes, because that is where address-family support diverges: runsc
adopts the interior netns wholesale and picks the actor's addresses up for
free, while a micro-VM's eth0 is a tap cross-connected to that netns at L2,
so the guest has to be told over the kata-agent channel. A lane that ran gVisor
alone could not tell a working micro-VM guest from one that never got an
address. The second commit adds that leg, under always() so a gVisor failure
cannot hide the micro-VM result. It stays red until the guest is given an IPv6
address, which is the defect it exists to catch.

A fresh IPv6-only cluster resolves nothing from a pod, and a runner has no IPv6
egress of its own, so both the cluster's DNS (#958) and the translator it needs
(#1275) come from hack/; this job asks for them and reads back what it got.
Setting that up here instead would have meant the job supplying the very thing
it exists to gate. The cost is that against main alone it stops at the NAT64
step, including on the pull_request path that lets this file test itself,
until those land. The job stays out of the e2e-test gate either way, so it
never blocks a pull request.

Demonstrated on a run of this exact file with the IPv6 stack and a not-yet-filed
micro-VM guest fix merged in — 20 passed, 0 skipped on each class
(run). Its
control is the same lane without that fix: gVisor 19/0, micro-VM 15/0 with the
four egress tests red
(run).

🤖 This PR was developed with AI assistance. I have reviewed and tested all changes.

@ygao-g
Yuan Gao (ygao-g) force-pushed the ci-kind-ipv6 branch 5 times, most recently from 72285e4 to 68972e5 Compare August 14, 2026 18:13
@ygao-g Yuan Gao (ygao-g) changed the title ci: add a label-gated IPv6-only kind e2e job ci: add an IPv6-only kind e2e job [DO NOT MERGE — signal only] Aug 14, 2026
@ygao-g Yuan Gao (ygao-g) added the DO NOT MERGE This PR must not be merged yet. label Aug 19, 2026
@ygao-g
Yuan Gao (ygao-g) force-pushed the ci-kind-ipv6 branch 5 times, most recently from ba8d79e to af37e8b Compare August 20, 2026 16:43
@ygao-g Yuan Gao (ygao-g) changed the title ci: add an IPv6-only kind e2e job [DO NOT MERGE — signal only] ci: add an IPv6-only kind e2e job [DO NOT MERGE — needs #958 and #911] Aug 20, 2026
@ygao-g
Yuan Gao (ygao-g) force-pushed the ci-kind-ipv6 branch 3 times, most recently from 39c862c to fa8afe0 Compare August 27, 2026 19:17
@ygao-g Yuan Gao (ygao-g) changed the title ci: add an IPv6-only kind e2e job [DO NOT MERGE — needs #958 and #911] ci: add an IPv6-only kind e2e job Aug 27, 2026
@ygao-g
Yuan Gao (ygao-g) force-pushed the ci-kind-ipv6 branch 2 times, most recently from 8347786 to 65c684c Compare August 28, 2026 15:27
@ygao-g

Copy link
Copy Markdown
Collaborator Author

Dispatch record for this lane. The runs are on the fork; a fork pull request cannot dispatch a workflow upstream.

prs outcome
958 753 1080 979 1060 1116 1057 1275 11 passed, 0 skipped — 33132033499, and again on a later main in 33187917892
1275 7 passed; TestActorEgress, TestActorEgressHTTPS, TestActorEgressGRPC and TestActorEgressNonStandardPort fail — 33129489452

#1275 on its own brings the cluster up and leaves exactly those four failing, so everything else in the stack is what those four measure. Nothing bisects the eight, so the minimal green set is unmeasured.

The two green runs used a byte-identical e2e-ipv6.yaml on different bases, so the pass is not an artifact of one main.

@ygao-g
Yuan Gao (ygao-g) force-pushed the ci-kind-ipv6 branch 6 times, most recently from 6b783e0 to da8b373 Compare August 31, 2026 22:14
@ygao-g Yuan Gao (ygao-g) changed the title ci: add an IPv6-only kind e2e job ci: add an IPv6-only kind e2e job for both sandbox classes Sep 2, 2026
@ygao-g Yuan Gao (ygao-g) changed the title ci: add an IPv6-only kind e2e job for both sandbox classes [WIP] ci: add an IPv6-only kind e2e job for both sandbox classes Sep 3, 2026
No CI job exercises IPv6-only, so nothing catches a change that breaks
it. This builds a single-stack IPv6 kind cluster, installs the full
system and runs the networking suite, where every test that can tell
one address family from another lives. It asserts the cluster really
is single-stack, and that cluster DNS is the shape an IPv6-only
install needs, before installing anything, so a green run cannot be
vacuous.

The lane carries no copy of what it exercises: dispatch it with a list
of pull requests and it merges them onto main for the length of the
run, because the IPv6 changes do nothing apart. Editing this file also
runs it, which stays red until those changes are on main. Either way
it stays out of the e2e-test gate, so it never blocks a pull request.
The lane only ever ran gVisor, so nothing in CI booted a micro-VM
guest on a cluster that has IPv6 -- the one configuration where the
two classes diverge. gVisor's runsc adopts the interior netns
wholesale and picks the actor's addresses up for free, while a
micro-VM's eth0 is a tap cross-connected to that netns at L2 and the
guest has to be told over the kata-agent channel, so a gVisor-only run
cannot tell a working guest from one that never got an address. The
suite now runs a second time under E2E_SANDBOX_CLASS=microvm, and the
vacuous-green guard covers both logs.

A runner without KVM, or a node whose device plugin never advertised
it, would otherwise degrade to gVisor in disguise and still report
success, and asset staging on a v6-only cluster fails ten minutes in
as an unexplained endpoint error; both now fail early and by name. A
job cancelled at its timeout no longer starts the second suite in the
cancellation grace window.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/network DO NOT MERGE This PR must not be merged yet.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants