From 71f40fe0ac6fb1d53821a3acf09612ad1ba01440 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Tue, 6 Oct 2026 10:58:10 -0700 Subject: [PATCH 01/19] Document NVX integration design Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 431 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 431 insertions(+) create mode 100644 docs/nvx-integration.md diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md new file mode 100644 index 000000000..189de076b --- /dev/null +++ b/docs/nvx-integration.md @@ -0,0 +1,431 @@ +# NVX integration in MXC + +**Status:** Integration proposal. + +Future +tense describes work proposed for the MXC integration. + +## 1. Introduction + +[NVX][nvx-readme] is an ultra-light microVM sandbox for running untrusted +workloads with hardware-enforced isolation. It is built on OpenVMM and runs +Linux as its guest. It is a cross-platform microVM sandbox for agentic workloads. + +NVX offers hardware-enforced isolation, warm +repeated execution, and direct Rust integration, with rich filesystem and +network controls compared to the existing Nanvix implementation. It will replace +Nanvix as the backend for the `microvm` option. + +## 2. NVX distribution and MXC consumption + +### 2.1 NVX shipping + +NVX will ship: + +- A`aci_edge_sandboxes` Rust crate, which will provide the host API; +- A zip containing signed binaries containing OpenVMM, the Linux kernel, and guest + images; and +- manifests, checksums, provenance, licences, and package inventories. + +### 2.2 How MXC will consume NVX + +MXC will use a pinned NVX crate version and a matching pinned runtime bundle. +The production ZIP will be made consumable through Rust artifact crates so +Cargo builds can acquire, verify, unpack, and stage it consistently. MXC will +use the interface from the Rust crate and the signed DLL implementation from +the ZIP. + +MXC will replace its current MicroVM artifact acquisition path with the NVX +artifact crates while preserving: + +- build-time download and staging; +- checksum verification; +- offline builds using pre-fetched artifacts; and +- packaging for the executor, Node SDK, and .NET SDK. +It will also add signature verification for the downloaded binaries. + +### 2.3 Key NVX files that MXC will use + +| File | Approximate size | Contents | +| --- | ---: | --- | +| NVX implementation DLL | Not yet published | Will provide the signed implementation of the Rust crate interface | +| `openvmm[.exe]` | 22 MB Windows;
461-475 MB Linux | Platform OpenVMM executable | +| `vmlinux` | 24 MB | NVX Linux kernel | +| `initramfs.cpio.gz` | 7.4 MB | Alpine userspace and NVX guest agent | + +The release also includes the supporting checksum, manifest, provenance, +licence, and package-inventory files. + +| Runtime bundle | Compressed | Expanded | +| --- | ---: | ---: | +| Windows WHP | 139 MB | 187 MB | +| Linux KVM | 247 MB | 626 MB | +| Linux MSHV | 247 MB | 641 MB | + +These are the current development-bundle sizes and can change. + +The current NVX binaries are not signed yet. The production ZIP will contain +the signed implementation DLL, OpenVMM executable, and image tool. +MXC will validate their signatures and the published checksums before staging +or using the files. + +### 2.4 Developer packaging and usage + +The NVX runtime will be distributed with the SDK for each ecosystem. +Developers will not separately install or invoke OpenVMM. + +| Ecosystem | Developer dependency | Packaging behaviour | +| --- | --- | --- | +| Rust | `mxc-sdk` with the `microvm` feature | The feature will include the NVX runtime artifact crate and stage the production ZIP contents | +| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The runtime package will supply the matching production ZIP contents | +| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The runtime NuGet package will supply the matching RID-specific production ZIP contents | + +#### Rust + +```toml +mxc-sdk = { version = "...", features = ["microvm"] } +``` + +#### Node + +```bash +npm install @microsoft/mxc-sdk @microsoft/mxc-nvx-runtime +``` + +#### .NET + +```xml + + +``` + +The developer experience will be the same in all three ecosystems: + +1. Developers will add the MXC SDK and its NVX runtime dependency. +2. They will select `microvm` in the MXC request. +3. The SDK will locate the packaged NVX runtime for the current platform. +4. MXC will validate the signatures and checksums, then launch OpenVMM through + the NVX Rust integration. + +Developers will not provide paths to `openvmm[.exe]`, `vmlinux`, or +`initramfs.cpio.gz`, and will not communicate with OpenVMM directly. + +## 3. Architecture + +### 3.1 NVX, OpenVMM, and Alpine Linux + +MXC will use the NVX Rust interface and its signed DLL implementation. The DLL +will launch OpenVMM, which will boot the NVX kernel and unpack the Alpine +initramfs into the VM's in-memory root filesystem. + +The initramfs contains two relevant agent components: + +| Guest path | Role | +| --- | --- | +| `/init` | Runs first, mounts the guest pseudo-filesystems, reads the kernel command line, prepares networking and host filesystem mappings, and resolves the workload identity | +| `/sbin/nvx-managed-agent` | Replaces `/init` for the managed lifecycle and remains running as PID 1 while workloads execute | + +The managed agent stays in the initramfs root filesystem and is not exposed +inside the workload's execution context. + +```mermaid +flowchart LR + MXC["MXC"] --> API["NVX Rust interface"] + API --> DLL["Signed NVX implementation DLL"] + DLL --> OpenVMM["Signed OpenVMM executable"] + OpenVMM --> Init["Alpine initramfs: /init"] + Init --> Alpine["/sbin/nvx-managed-agent (PID 1)"] + Alpine --> Workload["Non-root workload"] +``` + +### 3.2 MXC integration + +The integration will keep `containment: "microvm"` and will route it to NVX +through `mxc_engine`. + +| Surface | New NVX integration work | +| --- | --- | +| Rust SDK | Will enable the NVX backend in the SDK and engine build | +| .NET SDK | Will add the `microvm` choice through the existing `mxc_ffi` boundary and package the NVX-enabled native runtime | +| Node SDK | Will add the `microvm` choice to typed configuration and package the NVX-enabled native runtime | + +Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will +be linked on the Rust side; no separate NVX FFI library will be required. + +## 4. Filesystem, lifecycle, and network support + +The integration will target the MXC `1.1.0-alpha` development schema. + +### 4.1 Schema example + +```json +{ + "version": "1.1.0-alpha", + "containment": "microvm", + "process": { + "commandLine": "cat /mnt/c/input/message.txt > /mnt/c/output/result.txt", + "cwd": "/", + "timeout": 30000 + }, + "filesystem": { + "readonlyPaths": ["C:\\input"], + "readwritePaths": ["C:\\output"], + "deniedPaths": ["C:\\input\\private"] + }, + "network": { + "egress": { + "default": "deny", + "allow": [{ + "to": [{ "cidr": "203.0.113.0/24" }], + "ports": [{ "protocol": "tcp", "port": 443 }] + }] + }, + "ingress": { + "default": "deny", + "hostLoopback": "deny" + } + } +} +``` + +### 4.2 Filesystem + +MXC will pass the configured host paths to NVX. NVX currently asks OpenVMM for +one virtio-fs export rooted at the common host directory. The Alpine guest +agent then bind-mounts each requested path separately as read-only or +read-write: + +```mermaid +flowchart LR + Policy["MXC filesystem policy"] --> NVX["NVX mapping plan"] + NVX --> OpenVMM["OpenVMM: one virtio-fs export"] + OpenVMM --> Alpine["Alpine: per-path RO/RW bind mounts"] + Alpine --> Workload["Workload paths"] +``` + +Read-write mappings are +live: a guest write changes the mapped host file immediately. Denied entries +are hidden by OpenVMM within the exported host tree. Linux host permissions +continue to apply to the mapped paths. + +| MXC Schema field | Works today | +| --- | --- | +| `filesystem.readonlyPaths` | Yes | +| `filesystem.readwritePaths` | Yes | +| `filesystem.deniedPaths` | Yes | + +### 4.3 Network + +MXC will pass the directional network policy to NVX. NVX currently converts +the supported rules into OpenVMM network options. OpenVMM attaches a virtual +network device to the guest and uses its portable profile as the host-side, +in-process NAT and filtering data plane. + +```mermaid +flowchart LR + Policy["MXC network policy"] --> NVX["NVX rule conversion"] + NVX --> OpenVMM["OpenVMM virtual NIC + portable profile"] + OpenVMM --> Alpine["Alpine virtual network device"] +``` + + +| MXC Schema field or rule | Works today | +| --- | --- | +| `network.egress.default: allow` | Yes | +| `network.egress.default: deny` | Yes | +| IPv4/CIDR allow and deny rules | Yes | +| TCP and UDP ports and ranges | Yes | +| CIDR `except` entries | Yes | +| `protocol: any` with ports | Yes | +| `network.ingress.default: deny` | Yes | +| `network.ingress.hostLoopback: deny` | Yes | +| IPv6 rules | No | +| ICMP-only rules | No | +| `network.ingress.default: allow` | No | +| `network.ingress.hostLoopback: allow` | No | +| `runtimeConfig.networkProxy` | No | + +Unsupported network forms are rejected before the VM starts. + +### 4.4 Lifecycle + +State-aware MicroVM provision types are not currently registered in the MXC +`1.1.0-alpha` contract. The integration will add them before publishing the +NVX lifecycle through the SDKs. + +The proposed provision request will use the same `version`, `containment`, +`filesystem`, and `network` fields as the one-shot example in section 4.1. +The following shows only the state-aware difference: + +```json +{ + "phase": "provision", + "microvm": { + "provision": { + "memoryMib": 256, + "image": "my-python-filesystem:latest", + "imageTarPath": "C:\\images\\my-python-filesystem.tar" + } + } +} +``` + +The `process` section from the one-shot example will be omitted during +provision. The image fields will reuse the WSLC provision schema under the +`microvm` top-level element. A later `exec` request will supply the process +configuration. + +| MXC phase | How NVX handles it | +| --- | --- | +| `provision` | Stores the configuration and returns an NVX sandbox ID | +| `start` | Launches OpenVMM and waits for the guest agent | +| `exec` | Runs a workload in the running VM | +| `stop` | Stops the VM while retaining provisioned state | +| `deprovision` | Removes the provisioned state | +| One-shot | MXC will compose provision, start, exec, stop, and deprovision | + +The same running VM can serve repeated `exec` calls. Only one workload runs at +a time; another `exec` waits up to the configured control timeout. Guest-memory +state does not survive `stop`, while changes to mapped host files do. + +## 5. UI and other support + +| Schema field or capability | Works today | Notes | +| --- | --- | --- | +| `ui` | No | Not supported by design, reject when supplied | +| `process.commandLine` | Yes | Runs as a Linux shell command | +| `process.cwd` | Yes | Must be an absolute guest path | +| `process.env` / `inheritDefaultEnv` | Yes | | +| `process.timeout` | Yes | Maximum one hour; omitted or `0` means no execution deadline | +| Separate stdout and stderr | Yes | Combined output is limited to 1 MB | +| Live stdin | No | Future work if needed. Workload receives EOF | +| PTY | No | Future work if needed | +| Guest memory override | Yes | Default is 256 MB | +| `fallback` | No | Not supported by design, NVX does not select another backend | + +NVX lifecycle errors align with the existing +[MXC SDK error classifications](reference/rust/v1/types.md). The integration +will map the additional NVX execution outcomes as described in +[Appendix B](#appendix-b-nvx-execution-outcome-mapping). + +## 6. Image delivery models + +The implementation will support two image-delivery models. Both models will ultimately provide MXC with a tar file that contains the inputs needed to run the +workload. + +### 6.1 Default NVX image + +The default model will use a pre-built static tar published by +[`microsoft/nvx`][nvx-readme]. MXC will pin, download or stage, validate, and +import this tar directly. The tar will match the pinned NVX runtime version +and will provide the default workload filesystem. +**TBD on the workload it will support.** + + +### 6.2 Custom image + +The custom-image model will reuse the existing WSLC image handling code and +schema. The `image` and `imageTarPath` fields will keep the same meaning; only +the top-level element will change from `wslc` to `microvm`. + +```json +{ + "microvm": { + "image": "my-python-filesystem:latest", + "imageTarPath": "C:\\images\\my-python-filesystem.tar" + } +} +``` + +`image` will identify the workload image. `imageTarPath` will identify the +local tar that MXC imports and passes to NVX. The integration will reuse the +existing WSLC image download, conversion, validation, and tar-handling +implementation rather than duplicate it for NVX. + +The signed image tool from the production NVX ZIP will download the standard +image, combine it with the workload input, and produce the tar at +`imageTarPath`. The resulting tar will include the workload and image metadata +required by NVX. + +MXC will validate the declared checksums for both models. It will also validate +the signatures of the signed implementation DLL, OpenVMM executable, and image +tool before use. + +## 7. Host support + +| Backend | Requirements | Coverage today | +| --- | --- | --- | +| Windows WHP | Windows x64 with WHP enabled | Live-tested on the selected baseline | +| Linux KVM | Linux x64 with access to `/dev/kvm` | Supported by NVX; not live-tested in the supplied campaign | +| Linux MSHV | Linux x64 with access to `/dev/mshv` | Supported by NVX; not live-tested in the supplied campaign | +| macOS | Not supported | No NVX backend | +| ARM64 | Not supported by the selected runtime | No selected runtime bundle | + +Moving to another NVX version will require renewed host-compatibility testing. + +## 8. Requirements and end-to-end tests + +The NVX backend will be complete when the following areas pass through the +packaged MXC executor and all three SDKs on each supported host backend. + +| Area | Required coverage | +| --- | --- | +| Integration | `microvm` routes to NVX for one-shot and state-aware execution | +| Lifecycle | Provision, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, and stale IDs | +| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct | +| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, and Linux permissions | +| Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | +| Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | +| PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | +| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; tar conversion; static NVX tar staging; automatic runtime discovery; and missing/corrupt artifacts | +| Signing | Validate signatures and checksums and reject unapproved or tampered artifacts | +| Hosts | Real WHP and KVM runs; MSHV if shipped; clear rejection on macOS and unsupported architectures | +| Image models | Verify the default static `microsoft/nvx` tar and the custom-image WSLC code/schema reuse, conversion, validation, and import path | + +Negative filesystem and network tests must include a working positive control +so infrastructure failures are not mistaken for policy enforcement. + +## Appendix A: Planned MXC to OpenVMM communication + +| Connection | Mechanism | Purpose | +| --- | --- | --- | +| MXC to NVX Rust interface | In-process Rust API calls | Will invoke provision, start, execute, stop, and deprovision | +| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation supplied in the production ZIP | +| NVX implementation DLL to `openvmm.exe` | Process launch with CLI arguments | Will supply the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | +| NVX implementation DLL to `openvmm.exe`, during startup only | OpenVMM stdin | Will pass a one-time 32-byte authentication capability; stdin will not be the ongoing command channel | +| NVX implementation DLL to `openvmm.exe` | Windows named pipe or Linux Unix domain socket | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | +| OpenVMM to Alpine guest agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | + + +## Appendix B: NVX execution outcome mapping + +| NVX outcome | MXC result | +| --- | --- | +| `Exited(code)` | Will return the workload exit code | +| `Signaled(signal)` | Will return `128 + signal` through the existing integer exit result | +| `TimedOut` | Will return the existing MXC timed-out result | +| `Cancelled` | Will return exit code `137` | +| `Failed(WorkingDirectory)` | Will return `backend_error` with the working-directory failure | +| Other `Failed(...)` outcomes | Will return `backend_error` with the NVX failure reason | +| No outcome available | Will return `backend_error`; it will not invent a workload exit code | + +## References and open decisions + +### Open decisions + +- Future PTY support. + +### References + +- [NVX README][nvx-readme] +- [NVX Rust API overview][nvx-rust] +- [NVX setup guide][nvx-setup] +- [Selected NVX release][nvx-release] +- [MXC schema](schema.md) +- [MXC architecture](architecture.md) +- [MXC SDK error classifications](reference/rust/v1/types.md) + +[nvx-readme]: https://github.com/microsoft/nvx +[nvx-rust]: https://github.com/microsoft/nvx/tree/dev/aci_edge_sandboxes +[nvx-setup]: https://github.com/microsoft/nvx/blob/dev/doc/setup.md +[nvx-release]: https://github.com/microsoft/nvx/releases From bc3a1f48558abb5425d00af26a28e4c255c07036 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Tue, 6 Oct 2026 10:58:24 -0700 Subject: [PATCH 02/19] Fix NVX document whitespace Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 189de076b..2918b16d3 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -318,7 +318,7 @@ workload. The default model will use a pre-built static tar published by [`microsoft/nvx`][nvx-readme]. MXC will pin, download or stage, validate, and import this tar directly. The tar will match the pinned NVX runtime version -and will provide the default workload filesystem. +and will provide the default workload filesystem. **TBD on the workload it will support.** From e1162a8c1eb7a30c59f008ec2eafd119931774ec Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Tue, 6 Oct 2026 11:15:34 -0700 Subject: [PATCH 03/19] Refine NVX integration proposal Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 2918b16d3..06af07aa7 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -49,7 +49,7 @@ It will also add signature verification for the downloaded binaries. | File | Approximate size | Contents | | --- | ---: | --- | | NVX implementation DLL | Not yet published | Will provide the signed implementation of the Rust crate interface | -| `openvmm[.exe]` | 22 MB Windows;
461-475 MB Linux | Platform OpenVMM executable | +| `openvmm.exe` | 22 MB | Windows OpenVMM executable | | `vmlinux` | 24 MB | NVX Linux kernel | | `initramfs.cpio.gz` | 7.4 MB | Alpine userspace and NVX guest agent | @@ -59,8 +59,6 @@ licence, and package-inventory files. | Runtime bundle | Compressed | Expanded | | --- | ---: | ---: | | Windows WHP | 139 MB | 187 MB | -| Linux KVM | 247 MB | 626 MB | -| Linux MSHV | 247 MB | 641 MB | These are the current development-bundle sizes and can change. @@ -107,7 +105,7 @@ The developer experience will be the same in all three ecosystems: 4. MXC will validate the signatures and checksums, then launch OpenVMM through the NVX Rust integration. -Developers will not provide paths to `openvmm[.exe]`, `vmlinux`, or +Developers will not provide paths to `openvmm.exe`, `vmlinux`, or `initramfs.cpio.gz`, and will not communicate with OpenVMM directly. ## 3. Architecture @@ -205,8 +203,7 @@ flowchart LR Read-write mappings are live: a guest write changes the mapped host file immediately. Denied entries -are hidden by OpenVMM within the exported host tree. Linux host permissions -continue to apply to the mapped paths. +are hidden by OpenVMM within the exported host tree. | MXC Schema field | Works today | | --- | --- | @@ -351,35 +348,28 @@ MXC will validate the declared checksums for both models. It will also validate the signatures of the signed implementation DLL, OpenVMM executable, and image tool before use. -## 7. Host support +## 7. Windows requirement -| Backend | Requirements | Coverage today | -| --- | --- | --- | -| Windows WHP | Windows x64 with WHP enabled | Live-tested on the selected baseline | -| Linux KVM | Linux x64 with access to `/dev/kvm` | Supported by NVX; not live-tested in the supplied campaign | -| Linux MSHV | Linux x64 with access to `/dev/mshv` | Supported by NVX; not live-tested in the supplied campaign | -| macOS | Not supported | No NVX backend | -| ARM64 | Not supported by the selected runtime | No selected runtime bundle | - -Moving to another NVX version will require renewed host-compatibility testing. +The implementation will support Windows x64 and ARM only. Windows Hypervisor Platform +(WHP) must already be installed and enabled on the system. ## 8. Requirements and end-to-end tests The NVX backend will be complete when the following areas pass through the -packaged MXC executor and all three SDKs on each supported host backend. +packaged MXC executor and all three SDKs on Windows x64 and ARM with WHP. | Area | Required coverage | | --- | --- | | Integration | `microvm` routes to NVX for one-shot and state-aware execution | | Lifecycle | Provision, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, and stale IDs | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct | -| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, and Linux permissions | +| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | | Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; tar conversion; static NVX tar staging; automatic runtime discovery; and missing/corrupt artifacts | | Signing | Validate signatures and checksums and reject unapproved or tampered artifacts | -| Hosts | Real WHP and KVM runs; MSHV if shipped; clear rejection on macOS and unsupported architectures | +| Host | Real execution on Windows x64 and ARM with WHP installed and enabled | | Image models | Verify the default static `microsoft/nvx` tar and the custom-image WSLC code/schema reuse, conversion, validation, and import path | Negative filesystem and network tests must include a working positive control @@ -393,7 +383,7 @@ so infrastructure failures are not mistaken for policy enforcement. | NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation supplied in the production ZIP | | NVX implementation DLL to `openvmm.exe` | Process launch with CLI arguments | Will supply the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | | NVX implementation DLL to `openvmm.exe`, during startup only | OpenVMM stdin | Will pass a one-time 32-byte authentication capability; stdin will not be the ongoing command channel | -| NVX implementation DLL to `openvmm.exe` | Windows named pipe or Linux Unix domain socket | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | +| NVX implementation DLL to `openvmm.exe` | Windows named pipe | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | | OpenVMM to Alpine guest agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | From cef0da9159e1b0a0514322cd1f3de4805ef82885 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Tue, 6 Oct 2026 14:08:37 -0700 Subject: [PATCH 04/19] Expand NVX image integration design Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 99 ++++++++++++++++++++++++++++++++--------- 1 file changed, 78 insertions(+), 21 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 06af07aa7..da66dff59 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -11,6 +11,12 @@ tense describes work proposed for the MXC integration. workloads with hardware-enforced isolation. It is built on OpenVMM and runs Linux as its guest. It is a cross-platform microVM sandbox for agentic workloads. +Customers will use NVX through the MXC APIs and will not need to understand +OpenVMM, the guest agent, or image conversion. + +The implementation may initially remain experimental or preview while the +integration is validated end to end. + NVX offers hardware-enforced isolation, warm repeated execution, and direct Rust integration, with rich filesystem and network controls compared to the existing Nanvix implementation. It will replace @@ -160,6 +166,9 @@ The integration will target the MXC `1.1.0-alpha` development schema. { "version": "1.1.0-alpha", "containment": "microvm", + "microvm": { + "image": "alpine:latest" + }, "process": { "commandLine": "cat /mnt/c/input/message.txt > /mnt/c/output/result.txt", "cwd": "/", @@ -305,25 +314,53 @@ NVX lifecycle errors align with the existing will map the additional NVX execution outcomes as described in [Appendix B](#appendix-b-nvx-execution-outcome-mapping). -## 6. Image delivery models +## 6. Image model support -The implementation will support two image-delivery models. Both models will ultimately provide MXC with a tar file that contains the inputs needed to run the +The implementation will support two image-delivery models. Both models will +ultimately provide MXC with a tar file containing the filesystem used by the workload. -### 6.1 Default NVX image +### 6.1 Standard image and workload + +The developer will provide the workload and select a supported OCI image +through `image`. The image will default to `alpine:latest` when it is omitted. + +```json +{ + "microvm": { + "image": "alpine:latest" + } +} +``` + +The complete request in section 4.1 is an example of this model: + +- `microvm.image` selects the standard `alpine:latest` image. +- `imageTarPath` is omitted because the developer is not providing an image + tar. +- `process.commandLine` defines the workload to execute. +- The filesystem policy makes the workload's input available read-only and + its output location available read-write. +- The network policy limits the workload to the requested destination and + port. + +The integration will reuse the existing WSLC cache and registry handling: -The default model will use a pre-built static tar published by -[`microsoft/nvx`][nvx-readme]. MXC will pin, download or stage, validate, and -import this tar directly. The tar will match the pinned NVX runtime version -and will provide the default workload filesystem. -**TBD on the workload it will support.** +1. Use the image from the local cache when it is already available. +2. Otherwise, pull the image from its registry and cache it. +3. Use the signed NVX image tool to combine the standard image with the + workload and produce the NVX-compatible tar. +Image references without an explicit registry will resolve against Docker Hub. +Explicitly named permitted registries such as MCR or GHCR will also be +supported. -### 6.2 Custom image +### 6.2 Bring Your Own Image -The custom-image model will reuse the existing WSLC image handling code and -schema. The `image` and `imageTarPath` fields will keep the same meaning; only -the top-level element will change from `wslc` to `microvm`. +The developer will provide a custom local image tar through `imageTarPath`. +The integration will reuse the existing WSLC image handling code and schema. +The `image` and `imageTarPath` fields will keep the same meaning; only the +top-level element will change from `wslc` to `microvm`. ```json { @@ -334,15 +371,24 @@ the top-level element will change from `wslc` to `microvm`. } ``` -`image` will identify the workload image. `imageTarPath` will identify the -local tar that MXC imports and passes to NVX. The integration will reuse the -existing WSLC image download, conversion, validation, and tar-handling -implementation rather than duplicate it for NVX. +`image` will identify the custom image after import. `imageTarPath` will point +to the local tar containing that image. When the named image already exists in +the local cache, MXC will use the cached image and will not re-import the tar. -The signed image tool from the production NVX ZIP will download the standard -image, combine it with the workload input, and produce the tar at -`imageTarPath`. The resulting tar will include the workload and image metadata -required by NVX. +| Image content source | Configuration | Supported input | +| --- | --- | --- | +| Docker image archive | `imageTarPath` points to a local tar created by `docker save` | Archive containing a root-level `manifest.json` | +| Root filesystem tar | `imageTarPath` points to a local tar created by `docker export` | Root filesystem containing directories such as `bin`, `etc`, `usr`, `lib`, `sbin`, or `var` | +| NVX conversion tool output | `imageTarPath` points to the generated local tar | A supported Docker archive or root filesystem tar | + +`imageTarPath` will be a path to a local tar file accessible to MXC. It will +not be a registry reference, URL, directory, named pipe, or input stream. The +tar format will be detected automatically. Unreadable files and unrecognised +tar formats will be rejected. + +The integration will reuse the existing WSLC image download, conversion, +validation, cache, and tar-handling implementation rather than duplicate it +for NVX. MXC will validate the declared checksums for both models. It will also validate the signatures of the signed implementation DLL, OpenVMM executable, and image @@ -370,11 +416,22 @@ packaged MXC executor and all three SDKs on Windows x64 and ARM with WHP. | Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; tar conversion; static NVX tar staging; automatic runtime discovery; and missing/corrupt artifacts | | Signing | Validate signatures and checksums and reject unapproved or tampered artifacts | | Host | Real execution on Windows x64 and ARM with WHP installed and enabled | -| Image models | Verify the default static `microsoft/nvx` tar and the custom-image WSLC code/schema reuse, conversion, validation, and import path | +| Image models | Verify standard-image cache and registry resolution, workload conversion, BYOI `docker save` archives, `docker export` rootfs tars, conversion-tool output, invalid tar rejection, and WSLC code/schema reuse | Negative filesystem and network tests must include a working positive control so infrastructure failures are not mistaken for policy enforcement. +## 9. Long-term plan + +- Converge the NVX MicroVM integration under the broader WSL platform. +- Reuse and align session, image, SDK, and runtime concepts with WSLC. +- Allow MXC to replace its direct NVX integration without changing the + developer-facing contract. +- Preserve NVX policy capabilities while the common API evolves across the + WSL runtime options. +- Resolve long-term branding and component ownership as part of the WSL + integration. + ## Appendix A: Planned MXC to OpenVMM communication | Connection | Mechanism | Purpose | From a74c7b4590bb4e11e109883e0a4ac3f2a93bbc8c Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 12:20:44 -0700 Subject: [PATCH 05/19] Complete NVX developer integration proposal Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 318 +++++++++++++++++++++++++++------------- 1 file changed, 220 insertions(+), 98 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index da66dff59..ae82a1f67 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -13,8 +13,7 @@ Linux as its guest. It is a cross-platform microVM sandbox for agentic workloads Customers will use NVX through the MXC APIs and will not need to understand OpenVMM, the guest agent, or image conversion. - -The implementation may initially remain experimental or preview while the +The implementation will initially remain experimental or preview while the integration is validated end to end. NVX offers hardware-enforced isolation, warm @@ -26,29 +25,28 @@ Nanvix as the backend for the `microvm` option. ### 2.1 NVX shipping -NVX will ship: +NVX will ship a Rust crate containing: -- A`aci_edge_sandboxes` Rust crate, which will provide the host API; -- A zip containing signed binaries containing OpenVMM, the Linux kernel, and guest - images; and +- the Rust host API; +- the signed NVX implementation DLL; +- the signed OpenVMM executable; +- the signed image download and conversion tool; +- the Linux kernel and initramfs with the NVX init agent; and - manifests, checksums, provenance, licences, and package inventories. ### 2.2 How MXC will consume NVX -MXC will use a pinned NVX crate version and a matching pinned runtime bundle. -The production ZIP will be made consumable through Rust artifact crates so -Cargo builds can acquire, verify, unpack, and stage it consistently. MXC will -use the interface from the Rust crate and the signed DLL implementation from -the ZIP. +MXC will use a pinned NVX crate version. The crate will expose the Rust +interface and stage its matching signed runtime assets during the Cargo build. +MXC will use the interface and signed DLL implementation from the same crate, +so the API and runtime files remain versioned together. -MXC will replace its current MicroVM artifact acquisition path with the NVX -artifact crates while preserving: +MXC will be implementing: -- build-time download and staging; -- checksum verification; -- offline builds using pre-fetched artifacts; and +- build-time staging of the runtime assets from the NVX crate; +- checksum and signature verification; +- offline builds using a pre-fetched crate and dependencies; and - packaging for the executor, Node SDK, and .NET SDK. -It will also add signature verification for the downloaded binaries. ### 2.3 Key NVX files that MXC will use @@ -56,20 +54,17 @@ It will also add signature verification for the downloaded binaries. | --- | ---: | --- | | NVX implementation DLL | Not yet published | Will provide the signed implementation of the Rust crate interface | | `openvmm.exe` | 22 MB | Windows OpenVMM executable | +| Image download and conversion tool | Not yet published | Will download and convert standard OCI images | | `vmlinux` | 24 MB | NVX Linux kernel | | `initramfs.cpio.gz` | 7.4 MB | Alpine userspace and NVX guest agent | The release also includes the supporting checksum, manifest, provenance, licence, and package-inventory files. +These are the current development-bundle sizes and can change. The final Rust +crate size will also include the implementation DLL and image tool. -| Runtime bundle | Compressed | Expanded | -| --- | ---: | ---: | -| Windows WHP | 139 MB | 187 MB | - -These are the current development-bundle sizes and can change. - -The current NVX binaries are not signed yet. The production ZIP will contain -the signed implementation DLL, OpenVMM executable, and image tool. +The current NVX binaries are not signed yet. The production Rust crate will +contain the signed implementation DLL, OpenVMM executable, and image tool. MXC will validate their signatures and the published checksums before staging or using the files. @@ -80,9 +75,9 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | -| Rust | `mxc-sdk` with the `microvm` feature | The feature will include the NVX runtime artifact crate and stage the production ZIP contents | -| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The runtime package will supply the matching production ZIP contents | -| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The runtime NuGet package will supply the matching RID-specific production ZIP contents | +| Rust | `mxc-sdk` with the `microvm` feature | The feature will include the NVX crate and stage its signed runtime assets | +| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The MXC build will package the NVX crate assets beside `mxc_ffi.dll` | +| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The MXC build will package the RID-specific NVX crate assets beside `mxc_ffi.dll` | #### Rust @@ -114,6 +109,46 @@ The developer experience will be the same in all three ecosystems: Developers will not provide paths to `openvmm.exe`, `vmlinux`, or `initramfs.cpio.gz`, and will not communicate with OpenVMM directly. +For Node and .NET, the published application output will use a co-located +native layout: + +```text +\ + mxc_ffi.dll + nvx\ + .dll + openvmm.exe + vmlinux + initramfs.cpio.gz + + +``` + +The native MXC layer will resolve the `nvx` directory relative to the loaded +`mxc_ffi.dll`. This avoids searching beside `node.exe` or `dotnet.exe` and +does not require a new FFI configuration function. + +The MXC SDK and NVX runtime package versions will be compatible. Before +launch, MXC will verify the required files, Windows architecture, signatures, +checksums, and runtime manifest compatibility. A missing or incompatible +runtime will return `backend_unavailable` with remediation identifying the +required runtime package or version. + +The integration will add a new typed MicroVM configuration to each SDK. This +configuration describes which backend and OCI image to use. Developers will +pass that configuration to the existing run, spawn, and lifecycle operations; +the integration will not introduce separate NVX-specific execution methods. + +| SDK | New typed configuration | Existing operations that will use it | +| --- | --- | --- | +| Rust | `MicrovmConfig { image: String }` and `Containment::Microvm(MicrovmConfig)`; add `microvm` to lifecycle containment choices | `v1::run`, `v1::spawn`, and `v1::container::*` | +| Node | `{ type: 'microvm', config: { image: string } }`; add `'microvm'` to `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | +| .NET | `Microvm : Containment` with required `Image`; add `Microvm` to `ContainmentBackend` and lifecycle choices | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | + +These will be new versioned SDK types. The SDKs will map the typed requests to +the `1.1.0-alpha` wire contract shown in section 4.1. The backend will continue +to require experimental authorisation until it is promoted. + ## 3. Architecture ### 3.1 NVX, OpenVMM, and Alpine Linux @@ -142,6 +177,21 @@ flowchart LR Alpine --> Workload["Non-root workload"] ``` +The NVX Rust interface exposes the following operations: + +| Operation | NVX Rust API | +| --- | --- | +| Validate provision | `validate_provision` | +| Validate execution | `validate_exec` | +| Check runtime availability | `probe` | +| Provision VM | `provision` | +| Start VM | `start` | +| Execute workload | `exec` | +| Stop VM | `stop` | +| Remove provisioned state | `deprovision` | +| Wait for execution outcome | `Execution::wait` | +| Cancel execution | `Execution::canceller` | + ### 3.2 MXC integration The integration will keep `containment: "microvm"` and will route it to NVX @@ -167,17 +217,17 @@ The integration will target the MXC `1.1.0-alpha` development schema. "version": "1.1.0-alpha", "containment": "microvm", "microvm": { - "image": "alpine:latest" + "image": "python:3.12-alpine" }, "process": { - "commandLine": "cat /mnt/c/input/message.txt > /mnt/c/output/result.txt", + "commandLine": "cat /mnt/c/nvx-work/input/message.txt > /mnt/c/nvx-work/output/result.txt", "cwd": "/", "timeout": 30000 }, "filesystem": { - "readonlyPaths": ["C:\\input"], - "readwritePaths": ["C:\\output"], - "deniedPaths": ["C:\\input\\private"] + "readonlyPaths": ["C:\\nvx-work\\input"], + "readwritePaths": ["C:\\nvx-work\\output"], + "deniedPaths": ["C:\\nvx-work\\input\\private"] }, "network": { "egress": { @@ -214,6 +264,12 @@ Read-write mappings are live: a guest write changes the mapped host file immediately. Denied entries are hidden by OpenVMM within the exported host tree. +All mapped paths must exist, be on the same Windows volume, and share a common +directory below the volume root. MXC will translate Windows paths into guest +paths: for example, `C:\nvx-work\input` will be available as +`/mnt/c/nvx-work/input`. Exporting an entire volume such as `C:\` will be +rejected. + | MXC Schema field | Works today | | --- | --- | | `filesystem.readonlyPaths` | Yes | @@ -253,6 +309,15 @@ flowchart LR Unsupported network forms are rejected before the VM starts. +| Network rule behaviour | Current NVX limit | +| --- | --- | +| Deny precedence | A matching deny rule overrides an allow rule | +| Expanded rules | Maximum 256 final allow rules and 256 final deny rules | +| Port ranges | Expanded to individual ports; a range may contain at most 256 ports | +| `protocol: any` with a port | Expands to one TCP and one UDP rule per port | +| TCP/UDP without a port | Rejected | +| Fully denied or omitted network | No virtual network device is attached | + ### 4.4 Lifecycle State-aware MicroVM provision types are not currently registered in the MXC @@ -268,18 +333,17 @@ The following shows only the state-aware difference: "phase": "provision", "microvm": { "provision": { - "memoryMib": 256, - "image": "my-python-filesystem:latest", - "imageTarPath": "C:\\images\\my-python-filesystem.tar" + "memoryMb": 256, + "image": "python:3.12-alpine" } } } ``` The `process` section from the one-shot example will be omitted during -provision. The image fields will reuse the WSLC provision schema under the -`microvm` top-level element. A later `exec` request will supply the process -configuration. +provision. The new `microvm.provision` schema will be a separate MicroVM +provision type with required `image` and optional `memoryMb`. A later `exec` +request will supply the process configuration. | MXC phase | How NVX handles it | | --- | --- | @@ -290,10 +354,26 @@ configuration. | `deprovision` | Removes the provisioned state | | One-shot | MXC will compose provision, start, exec, stop, and deprovision | +| Action | Workload effect | VM or state effect | +| --- | --- | --- | +| One-shot completion | Returns the workload outcome | MXC stops and deprovisions the VM | +| One-shot failure during provision, start, or exec | The workload may not start or will be terminated | MXC performs bounded stop and deprovision cleanup while preserving the original error | +| `process.timeout` | NVX terminates the workload and its descendants | A state-aware VM remains running; a one-shot VM is cleaned up | +| Cancel or kill an execution handle | Cancels the current workload and its descendants | Does not deprovision a state-aware VM | +| State-aware exec caller exits or loses its control session | The guest agent terminates the active workload | The running VM remains available for a later lifecycle call | +| `stop` during an exec | Waits for the active exec up to the stop timeout, then terminates the VM if required | The VM returns to provisioned state and the active exec fails | +| `deprovision` | Requires the VM to be stopped | Removes the provisioned state | + The same running VM can serve repeated `exec` calls. Only one workload runs at a time; another `exec` waits up to the configured control timeout. Guest-memory state does not survive `stop`, while changes to mapped host files do. +- NVX runs one workload at a time. A second `exec` waits for up to 60 seconds + by default. +- `stop` waits for up to 30 seconds before forcing the VM to shut down. +- Cancelling an SDK operation must cancel the NVX workload, not only stop the + SDK from waiting. + ## 5. UI and other support | Schema field or capability | Works today | Notes | @@ -309,114 +389,140 @@ state does not survive `stop`, while changes to mapped host files do. | Guest memory override | Yes | Default is 256 MB | | `fallback` | No | Not supported by design, NVX does not select another backend | +| Process behaviour | Developer-visible result | +| --- | --- | +| Command execution | `process.commandLine` runs through BusyBox `/bin/sh -c` | +| Environment | Omitted environment uses guest defaults; an explicit list replaces or layers over those defaults; the shell may set `PWD` and `SHLVL` | +| Output limit | Combined stdout and stderr are limited to 1 MB; exceeding the limit terminates the workload rather than truncating output | +| Nonzero exit | Returned as a workload result, not an SDK or FFI failure | +| Invalid request or unavailable backend | Returned as an MXC error rather than a workload exit code | + NVX lifecycle errors align with the existing [MXC SDK error classifications](reference/rust/v1/types.md). The integration will map the additional NVX execution outcomes as described in [Appendix B](#appendix-b-nvx-execution-outcome-mapping). -## 6. Image model support +## 6. Image support -The implementation will support two image-delivery models. Both models will -ultimately provide MXC with a tar file containing the filesystem used by the -workload. +The implementation will support a standard OCI image reference. The NVX image +tool will convert that image into the internal filesystem artifact consumed by +NVX. -### 6.1 Standard image and workload +### 6.1 Standard OCI image and workload -The developer will provide the workload and select a supported OCI image -through `image`. The image will default to `alpine:latest` when it is omitted. +The developer will provide a standard OCI image reference through `image`. +The signed NVX image tool will download the image and convert it into the +filesystem format consumed by NVX. + +OCI-image-backed execution is a prerequisite for the MXC integration. The +selected NVX Rust API does not currently accept an image or attach a converted +workload filesystem. The NVX team will provide the signed conversion tool and +extend the runtime contract so the converted image can be attached and used as +the workload root while the init agent remains in the initramfs. The required +conversion and runtime contract is listed in +[Appendix C](#appendix-c-oci-image-conversion-contract). ```json { "microvm": { - "image": "alpine:latest" + "image": "python:3.12-alpine" } } ``` The complete request in section 4.1 is an example of this model: -- `microvm.image` selects the standard `alpine:latest` image. -- `imageTarPath` is omitted because the developer is not providing an image - tar. +- `microvm.image` identifies the standard OCI image. - `process.commandLine` defines the workload to execute. - The filesystem policy makes the workload's input available read-only and its output location available read-write. - The network policy limits the workload to the requested destination and port. -The integration will reuse the existing WSLC cache and registry handling: +Image reference registries will be confirmed by the NVX team later. + +### 6.2 Schema additions -1. Use the image from the local cache when it is already available. -2. Otherwise, pull the image from its registry and cache it. -3. Use the signed NVX image tool to combine the standard image with the - workload and produce the NVX-compatible tar. +The `1.1.0-alpha` development contract will add a `microvm` image +configuration for both one-shot and state-aware provision requests. -Image references without an explicit registry will resolve against Docker Hub. -Explicitly named permitted registries such as MCR or GHCR will also be -supported. +| Contract surface | Schema addition | +| --- | --- | +| One-shot | Add `microvm.image` | +| State-aware provision | Register `containment: "microvm"` and add `microvm.provision.image` and `microvm.provision.memoryMb` | +| Start, exec, stop, and deprovision | No image fields; these requests use the `sandboxId` created during provision | -### 6.2 Bring Your Own Image +`image` will be required and must contain a non-empty OCI image reference. +Omitting it will fail schema or request validation. -The developer will provide a custom local image tar through `imageTarPath`. -The integration will reuse the existing WSLC image handling code and schema. -The `image` and `imageTarPath` fields will keep the same meaning; only the -top-level element will change from `wslc` to `microvm`. +One-shot standard-image addition: ```json { "microvm": { - "image": "my-python-filesystem:latest", - "imageTarPath": "C:\\images\\my-python-filesystem.tar" + "image": "python:3.12-alpine" } } ``` -`image` will identify the custom image after import. `imageTarPath` will point -to the local tar containing that image. When the named image already exists in -the local cache, MXC will use the cached image and will not re-import the tar. +State-aware provision will use the same fields under `microvm.provision`: -| Image content source | Configuration | Supported input | -| --- | --- | --- | -| Docker image archive | `imageTarPath` points to a local tar created by `docker save` | Archive containing a root-level `manifest.json` | -| Root filesystem tar | `imageTarPath` points to a local tar created by `docker export` | Root filesystem containing directories such as `bin`, `etc`, `usr`, `lib`, `sbin`, or `var` | -| NVX conversion tool output | `imageTarPath` points to the generated local tar | A supported Docker archive or root filesystem tar | +```json +{ + "phase": "provision", + "containment": "microvm", + "microvm": { + "provision": { + "image": "python:3.12-alpine", + "memoryMb": 256 + } + } +} +``` -`imageTarPath` will be a path to a local tar file accessible to MXC. It will -not be a registry reference, URL, directory, named pipe, or input stream. The -tar format will be detected automatically. Unreadable files and unrecognised -tar formats will be rejected. +The contract changes will also require regenerated development schema and +wire types, plus matching versioned Rust, Node, and .NET SDK types. -The integration will reuse the existing WSLC image download, conversion, -validation, cache, and tar-handling implementation rather than duplicate it -for NVX. +## 7. Windows requirement -MXC will validate the declared checksums for both models. It will also validate -the signatures of the signed implementation DLL, OpenVMM executable, and image -tool before use. +The initial implementation will support Windows x64. Windows ARM is planned +but will not be initially available. -## 7. Windows requirement +| Requirement | Developer action | +| --- | --- | +| Hardware virtualisation | Enable Intel VT-x or AMD-V in firmware | +| Windows Hypervisor Platform | Enable the `HypervisorPlatform` Windows optional feature from an elevated shell and reboot | +| Running Windows hypervisor | Ensure hypervisor launch has not been disabled | +| NVX runtime | Install the matching x64 SDK runtime package; MXC will validate the architecture, required files, signatures, checksums, and manifest compatibility | + +Developers should check platform support before launch using Rust +`platform_support()`, Node `getPlatformSupport()`, or .NET +`MxcPlatform.GetPlatformSupport()`. -The implementation will support Windows x64 and ARM only. Windows Hypervisor Platform -(WHP) must already be installed and enabled on the system. +Missing WHP, disabled hardware virtualisation, absent runtime files, +incompatible guest/runtime versions, and unsupported architectures will return +`backend_unavailable` with remediation. OpenVMM startup failures should +identify the associated log path. ## 8. Requirements and end-to-end tests The NVX backend will be complete when the following areas pass through the -packaged MXC executor and all three SDKs on Windows x64 and ARM with WHP. +packaged MXC executor and all three SDKs on Windows x64 with WHP. | Area | Required coverage | | --- | --- | | Integration | `microvm` routes to NVX for one-shot and state-aware execution | -| Lifecycle | Provision, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, and stale IDs | +| Lifecycle | Provision, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, stale IDs, and one-shot failure cleanup | +| State-aware process lifetime | After the process performing `start` exits, OpenVMM remains running and a later `exec` process reconnects successfully; test from a Windows job that would normally terminate child processes | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct | | Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; tar conversion; static NVX tar staging; automatic runtime discovery; and missing/corrupt artifacts | +| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; OCI image conversion; automatic runtime discovery; and missing/corrupt artifacts | | Signing | Validate signatures and checksums and reject unapproved or tampered artifacts | -| Host | Real execution on Windows x64 and ARM with WHP installed and enabled | -| Image models | Verify standard-image cache and registry resolution, workload conversion, BYOI `docker save` archives, `docker export` rootfs tars, conversion-tool output, invalid tar rejection, and WSLC code/schema reuse | +| Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | +| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | Negative filesystem and network tests must include a working positive control so infrastructure failures are not mistaken for policy enforcement. @@ -427,17 +533,13 @@ so infrastructure failures are not mistaken for policy enforcement. - Reuse and align session, image, SDK, and runtime concepts with WSLC. - Allow MXC to replace its direct NVX integration without changing the developer-facing contract. -- Preserve NVX policy capabilities while the common API evolves across the - WSL runtime options. -- Resolve long-term branding and component ownership as part of the WSL - integration. ## Appendix A: Planned MXC to OpenVMM communication | Connection | Mechanism | Purpose | | --- | --- | --- | | MXC to NVX Rust interface | In-process Rust API calls | Will invoke provision, start, execute, stop, and deprovision | -| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation supplied in the production ZIP | +| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation shipped in the NVX Rust crate | | NVX implementation DLL to `openvmm.exe` | Process launch with CLI arguments | Will supply the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | | NVX implementation DLL to `openvmm.exe`, during startup only | OpenVMM stdin | Will pass a one-time 32-byte authentication capability; stdin will not be the ongoing command channel | | NVX implementation DLL to `openvmm.exe` | Windows named pipe | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | @@ -456,11 +558,31 @@ so infrastructure failures are not mistaken for policy enforcement. | Other `Failed(...)` outcomes | Will return `backend_error` with the NVX failure reason | | No outcome available | Will return `backend_error`; it will not invent a workload exit code | +## Appendix C: OCI image conversion contract + +| Contract area | Required definition | +| --- | --- | +| Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | +| Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | +| Converted artifact | Produce a versioned NVX artifact with a manifest identifying the source digest, converter version, runtime compatibility, and checksums | +| Guest integration | Attach the converted artifact to OpenVMM and make it the workload root while `/init` and the managed agent remain in the initramfs outside the workload root | +| OCI metadata | Define how `ENTRYPOINT`, `CMD`, `ENV`, `WORKDIR`, and `USER` interact with MXC `process` settings | +| Writable state | Define the writable layer or scratch lifetime and whether it is discarded on stop or deprovision | +| Cache identity | Key conversions by image digest plus converter and runtime format version rather than by mutable image tag alone | +| Failures | Surface registry, conversion, compatibility, and attachment failures through actionable MXC errors | + +The signed image tool, image schema, converted-artifact format, and runtime +attachment are required deliverables before `microvm.image` is usable. + ## References and open decisions -### Open decisions +### Awaited support -- Future PTY support. +- NVX supported image registries +- NVX signed binaries support +- NVX crate binary packaging and extraction contract +- Windows ARM runtime and SDK package availability +- Future PTY support ### References From 25130c3c1411df18656ca0384e08980d84594fec Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 13:31:51 -0700 Subject: [PATCH 06/19] Address NVX document review feedback Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 59 ++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 10 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index ae82a1f67..462195edb 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -34,6 +34,13 @@ NVX will ship a Rust crate containing: - the Linux kernel and initramfs with the NVX init agent; and - manifests, checksums, provenance, licences, and package inventories. +Because the crate distributes `vmlinux`, the NVX team will publish the exact +corresponding Linux kernel source for every runtime version. The crate will +include the Linux licence, third-party notices, and a source manifest that +identifies the kernel version, configuration, patches, source artifact, and +hashes. The source may be a separate published artifact rather than part of +every SDK package. + ### 2.2 How MXC will consume NVX MXC will use a pinned NVX crate version. The crate will expose the Rust @@ -48,6 +55,11 @@ MXC will be implementing: - offline builds using a pre-fetched crate and dependencies; and - packaging for the executor, Node SDK, and .NET SDK. +Before packaging the NVX runtime, MXC will verify that the NVX source manifest +references an available corresponding-source artifact for the shipped +`vmlinux`. Packaging will fail when the reference is absent or does not match +the runtime version. + ### 2.3 Key NVX files that MXC will use | File | Approximate size | Contents | @@ -134,20 +146,47 @@ checksums, and runtime manifest compatibility. A missing or incompatible runtime will return `backend_unavailable` with remediation identifying the required runtime package or version. +The co-located checksum manifest will not be trusted by itself. The pinned NVX +crate will provide the expected runtime-manifest identity, and the compiled +native integration will authenticate that manifest before using its file +hashes. MXC will then: + +1. validate the Authenticode certificate chain and expected Microsoft signer + for the NVX implementation DLL, `openvmm.exe`, and image tool; +2. verify `vmlinux`, the initramfs, and all other runtime files against the + authenticated manifest; and +3. reject a missing signature, invalid certificate chain, unexpected signer, + manifest mismatch, checksum mismatch, or runtime directory that is writable + by an untrusted user. + The integration will add a new typed MicroVM configuration to each SDK. This configuration describes which backend and OCI image to use. Developers will pass that configuration to the existing run, spawn, and lifecycle operations; the integration will not introduce separate NVX-specific execution methods. -| SDK | New typed configuration | Existing operations that will use it | -| --- | --- | --- | -| Rust | `MicrovmConfig { image: String }` and `Containment::Microvm(MicrovmConfig)`; add `microvm` to lifecycle containment choices | `v1::run`, `v1::spawn`, and `v1::container::*` | -| Node | `{ type: 'microvm', config: { image: string } }`; add `'microvm'` to `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | -| .NET | `Microvm : Containment` with required `Image`; add `Microvm` to `ContainmentBackend` and lifecycle choices | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | +| SDK | One-shot configuration | State-aware provision addition | Existing operations that will use it | +| --- | --- | --- | --- | +| Rust | `MicrovmConfig { image: String }` and `Containment::Microvm(MicrovmConfig)` | Add `ProvisionRequest::microvm(image: String, memory_mb: Option)` | `v1::run`, `v1::spawn`, and `v1::container::*` | +| Node | `{ type: 'microvm', config: { image: string } }` | Add `MicrovmProvisionConfig { image: string; memoryMb?: number; filesystem?; network? }` to `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | +| .NET | `Microvm : Containment` with required `Image` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register its JSON discriminator as `microvm` | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | + +The wire and SDK changes will be delivered in this order: + +1. Add `microvm.image`, state-aware MicroVM provision, engine binding, and + runtime tests to the development `1.1.0-alpha` contract. +2. Keep the wire work development-only while that contract remains mutable. +3. Publish the fields in stable `1.1.0` when the contract is ready. +4. Advance `schemas/schema-version.json` `sdkMajorTargets["1"]` from `1.0.0` + to `1.1.0`. +5. Publish the new Rust, Node, and .NET V1 MicroVM types and update their + versioned references. + +Schema publication and runtime authorisation are independent. Even after +stable `1.1.0` becomes the V1 SDK target, `microvm` may continue to require +the experimental option until the backend meets its promotion bar. -These will be new versioned SDK types. The SDKs will map the typed requests to -the `1.1.0-alpha` wire contract shown in section 4.1. The backend will continue -to require experimental authorisation until it is promoted. +The existing lifecycle operation methods will remain unchanged; only the +backend-specific request types they accept will expand. ## 3. Architecture @@ -519,8 +558,8 @@ packaged MXC executor and all three SDKs on Windows x64 with WHP. | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; OCI image conversion; automatic runtime discovery; and missing/corrupt artifacts | -| Signing | Validate signatures and checksums and reject unapproved or tampered artifacts | +| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that the exact corresponding kernel source is published and referenced | +| Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | From 275db1458b87676346528d96333acc24493cc32d Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 14:24:57 -0700 Subject: [PATCH 07/19] Address NVX integration review gaps Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 178 ++++++++++++++++++++++++++++++---------- 1 file changed, 133 insertions(+), 45 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 462195edb..794628ba4 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -34,12 +34,14 @@ NVX will ship a Rust crate containing: - the Linux kernel and initramfs with the NVX init agent; and - manifests, checksums, provenance, licences, and package inventories. -Because the crate distributes `vmlinux`, the NVX team will publish the exact -corresponding Linux kernel source for every runtime version. The crate will -include the Linux licence, third-party notices, and a source manifest that -identifies the kernel version, configuration, patches, source artifact, and -hashes. The source may be a separate published artifact rather than part of -every SDK package. +Because the crate distributes `vmlinux` and an Alpine initramfs, the NVX team +will publish the exact corresponding Linux and Alpine source artifacts for +every runtime version. The crate will include `SOURCE-MANIFEST.json`, the +Alpine package inventory, licences, and third-party notices. The source +manifest will identify the NVX project source, kernel version and +configuration, patches, Alpine package sources, published source artifacts, +and hashes. The source archives may be published separately rather than +embedded in every SDK package. ### 2.2 How MXC will consume NVX @@ -55,10 +57,13 @@ MXC will be implementing: - offline builds using a pre-fetched crate and dependencies; and - packaging for the executor, Node SDK, and .NET SDK. -Before packaging the NVX runtime, MXC will verify that the NVX source manifest -references an available corresponding-source artifact for the shipped -`vmlinux`. Packaging will fail when the reference is absent or does not match -the runtime version. +Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json` and +verify that it references the matching Linux and Alpine source artifacts +published by the NVX team. MXC will copy the source manifest, Alpine package +inventory, licences, and notices into the npm and NuGet runtime packages. MXC +will not generate the source archives. Packaging will fail when required +metadata, source references, or hashes are missing or do not match the runtime +version. ### 2.3 Key NVX files that MXC will use @@ -88,7 +93,7 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | | Rust | `mxc-sdk` with the `microvm` feature | The feature will include the NVX crate and stage its signed runtime assets | -| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The MXC build will package the NVX crate assets beside `mxc_ffi.dll` | +| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The runtime package will contain a matched `mxc_ffi.dll` and co-located NVX crate assets | | .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The MXC build will package the RID-specific NVX crate assets beside `mxc_ffi.dll` | #### Rust @@ -121,30 +126,36 @@ The developer experience will be the same in all three ecosystems: Developers will not provide paths to `openvmm.exe`, `vmlinux`, or `initramfs.cpio.gz`, and will not communicate with OpenVMM directly. -For Node and .NET, the published application output will use a co-located -native layout: +The Node runtime package will use this layout: ```text -\ - mxc_ffi.dll - nvx\ - .dll - openvmm.exe - vmlinux - initramfs.cpio.gz - - +@microsoft/mxc-nvx-runtime\ + bin\win-x64\ + mxc_ffi.dll + nvx\ + .dll + openvmm.exe + vmlinux + initramfs.cpio.gz + + ``` -The native MXC layer will resolve the `nvx` directory relative to the loaded -`mxc_ffi.dll`. This avoids searching beside `node.exe` or `dotnet.exe` and -does not require a new FFI configuration function. +The Node SDK native-library resolver will check whether +`@microsoft/mxc-nvx-runtime` is installed. When present, it will load the +matched `mxc_ffi.dll` from that package, where the NVX runtime is already +co-located. Without the runtime package, Node will continue to load the normal +MXC native library and will report `microvm` as unavailable. -The MXC SDK and NVX runtime package versions will be compatible. Before -launch, MXC will verify the required files, Windows architecture, signatures, -checksums, and runtime manifest compatibility. A missing or incompatible -runtime will return `backend_unavailable` with remediation identifying the -required runtime package or version. +For .NET, NuGet will copy the RID-specific `mxc_ffi.dll` and `nvx` directory +into the application output. The native MXC layer will resolve the `nvx` +directory relative to the loaded `mxc_ffi.dll`. + +The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. +Before launch, MXC will verify the required files, Windows architecture, +signatures, checksums, and runtime manifest compatibility. A missing or +incompatible runtime will return an actionable load or `backend_unavailable` +error identifying the required runtime package or version. The co-located checksum manifest will not be trusted by itself. The pinned NVX crate will provide the expected runtime-manifest identity, and the compiled @@ -166,14 +177,15 @@ the integration will not introduce separate NVX-specific execution methods. | SDK | One-shot configuration | State-aware provision addition | Existing operations that will use it | | --- | --- | --- | --- | -| Rust | `MicrovmConfig { image: String }` and `Containment::Microvm(MicrovmConfig)` | Add `ProvisionRequest::microvm(image: String, memory_mb: Option)` | `v1::run`, `v1::spawn`, and `v1::container::*` | -| Node | `{ type: 'microvm', config: { image: string } }` | Add `MicrovmProvisionConfig { image: string; memoryMb?: number; filesystem?; network? }` to `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | -| .NET | `Microvm : Containment` with required `Image` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register its JSON discriminator as `microvm` | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | +| Rust | `MicrovmConfig { image: String, memory_mb: Option }` and `Containment::Microvm(MicrovmConfig)` | Add `ProvisionRequest::microvm(image: String, memory_mb: Option)` | `v1::run`, `v1::spawn`, and `v1::container::*` | +| Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Add `MicrovmProvisionConfig { image: string; memoryMb?: number; filesystem?; network? }` to `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | +| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register its JSON discriminator as `microvm` | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | The wire and SDK changes will be delivered in this order: -1. Add `microvm.image`, state-aware MicroVM provision, engine binding, and - runtime tests to the development `1.1.0-alpha` contract. +1. Add `microvm.image`, `microvm.memoryMb`, state-aware MicroVM provision, + engine binding, and runtime tests to the development `1.1.0-alpha` + contract. 2. Keep the wire work development-only while that contract remains mutable. 3. Publish the fields in stable `1.1.0` when the contract is ready. 4. Advance `schemas/schema-version.json` `sdkMajorTargets["1"]` from `1.0.0` @@ -236,6 +248,14 @@ The NVX Rust interface exposes the following operations: The integration will keep `containment: "microvm"` and will route it to NVX through `mxc_engine`. +`mxc_engine` will also integrate the existing NVX `probe()` operation into +platform discovery. The NVX probe checks that the host platform, OpenVMM, +kernel, initramfs, and configured hypervisor are available. MXC will wrap that +probe with its production-package checks for the matched runtime manifest, +required files, architecture, signatures, checksums, image tool, and +guest/runtime compatibility. This discovery path will be read-only and will +not start a VM. + | Surface | New NVX integration work | | --- | --- | | Rust SDK | Will enable the NVX backend in the SDK and engine build | @@ -256,7 +276,8 @@ The integration will target the MXC `1.1.0-alpha` development schema. "version": "1.1.0-alpha", "containment": "microvm", "microvm": { - "image": "python:3.12-alpine" + "image": "python:3.12-alpine", + "memoryMb": 256 }, "process": { "commandLine": "cat /mnt/c/nvx-work/input/message.txt > /mnt/c/nvx-work/output/result.txt", @@ -393,6 +414,24 @@ request will supply the process configuration. | `deprovision` | Removes the provisioned state | | One-shot | MXC will compose provision, start, exec, stop, and deprovision | +MXC will preserve the NVX sandbox ID unchanged. NVX provision returns IDs in +the following form: + +```text +aci-edge-sandboxes:<32 lowercase hexadecimal characters> +``` + +MXC will register the `aci-edge-sandboxes:` prefix in native lifecycle +dispatch so `start`, `exec`, `stop`, and `deprovision` route back to the +MicroVM/NVX backend. + +| Surface | Required prefix work | +| --- | --- | +| Native engine | Map `aci-edge-sandboxes:` to the `microvm` backend in `backend_from_prefix` and state-aware dispatch | +| Rust SDK | Accept and preserve the opaque ID through `ContainerId` | +| Node SDK | Brand returned IDs as `ContainerId<'microvm'>` and include the prefix in lifecycle routing and validation maps | +| .NET SDK | Map the prefix to `ContainmentBackend.Microvm` and preserve it in `ContainerId` | + | Action | Workload effect | VM or state effect | | --- | --- | --- | | One-shot completion | Returns the workload outcome | MXC stops and deprovisions the VM | @@ -425,7 +464,7 @@ state does not survive `stop`, while changes to mapped host files do. | Separate stdout and stderr | Yes | Combined output is limited to 1 MB | | Live stdin | No | Future work if needed. Workload receives EOF | | PTY | No | Future work if needed | -| Guest memory override | Yes | Default is 256 MB | +| Guest memory override | Yes | One-shot uses `microvm.memoryMb`; state-aware provision uses `microvm.provision.memoryMb`; default is 256 MB | | `fallback` | No | Not supported by design, NVX does not select another backend | | Process behaviour | Developer-visible result | @@ -464,7 +503,8 @@ conversion and runtime contract is listed in ```json { "microvm": { - "image": "python:3.12-alpine" + "image": "python:3.12-alpine", + "memoryMb": 256 } } ``` @@ -478,7 +518,40 @@ The complete request in section 4.1 is an example of this model: - The network policy limits the workload to the requested destination and port. -Image reference registries will be confirmed by the NVX team later. +The integration will reuse the existing WSLC cache and registry pattern: + +1. Use the image from the local cache when it is already available. +2. Otherwise, resolve the registry host and check a shared backend-neutral MXC + administrative registry policy. +3. When the registry is permitted, invoke the signed NVX image tool to pull + the image and resolve its immutable digest. +4. Convert and cache the NVX-compatible artifact. + +Image references without an explicit registry will resolve against Docker Hub. +Explicit registry references will be permitted only when the registry is +allowed by the machine policy. + +The policy will be stored under +`HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mxc` and will use the existing WSLC +policy behaviour: + +| Policy state | Behaviour | +| --- | --- | +| Value absent | Unmanaged; any registry host may be contacted | +| One or more hosts | Only those registry hosts may be contacted | +| Present but empty | No registry may be contacted | +| Present but unreadable | No registry may be contacted | + +MXC will own registry authorisation and will validate the registry host before +invoking the image tool. The signed NVX image tool will own DNS resolution, +TLS, registry protocol, redirects, download, digest resolution, and +conversion while enforcing the policy supplied by MXC. + +`microvm.image` will accept an OCI image reference rather than an arbitrary +URL. Redirects to another registry host will require that host to be permitted +by the same policy. Private-registry credentials will not be part of the +initial contract and will require a separate approved host +credential-provider design. ### 6.2 Schema additions @@ -487,7 +560,7 @@ configuration for both one-shot and state-aware provision requests. | Contract surface | Schema addition | | --- | --- | -| One-shot | Add `microvm.image` | +| One-shot | Add `microvm.image` and `microvm.memoryMb` | | State-aware provision | Register `containment: "microvm"` and add `microvm.provision.image` and `microvm.provision.memoryMb` | | Start, exec, stop, and deprovision | No image fields; these requests use the `sandboxId` created during provision | @@ -499,7 +572,8 @@ One-shot standard-image addition: ```json { "microvm": { - "image": "python:3.12-alpine" + "image": "python:3.12-alpine", + "memoryMb": 256 } } ``` @@ -536,7 +610,17 @@ but will not be initially available. Developers should check platform support before launch using Rust `platform_support()`, Node `getPlatformSupport()`, or .NET -`MxcPlatform.GetPlatformSupport()`. +`MxcPlatform.GetPlatformSupport()`. These existing APIs will call the native +MXC discovery path; no separate public NVX probe API will be added. + +The discovery result will include `microvm` in `availableMethods` only when the +NVX probe and MXC package-integrity checks succeed. An unavailable MicroVM +backend will include a backend-specific reason in the SDK's platform-support +result, with remediation such as installing the matching runtime package, +enabling WHP, or repairing corrupt assets. + +Execution will repeat the authoritative preflight before launch. A cached or +stale discovery result will not bypass runtime validation. Missing WHP, disabled hardware virtualisation, absent runtime files, incompatible guest/runtime versions, and unsupported architectures will return @@ -551,14 +635,15 @@ packaged MXC executor and all three SDKs on Windows x64 with WHP. | Area | Required coverage | | --- | --- | | Integration | `microvm` routes to NVX for one-shot and state-aware execution | -| Lifecycle | Provision, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, stale IDs, and one-shot failure cleanup | +| Discovery | Rust, Node, and .NET platform-support APIs include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; each failure mode returns actionable backend-specific remediation without starting a VM | +| Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | | State-aware process lifetime | After the process performing `start` exits, OpenVMM remains running and a later `exec` process reconnects successfully; test from a Windows job that would normally terminate child processes | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct | | Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, and initramfs; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that the exact corresponding kernel source is published and referenced | +| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | @@ -602,6 +687,9 @@ so infrastructure failures are not mistaken for policy enforcement. | Contract area | Required definition | | --- | --- | | Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | +| Registry authorisation | MXC validates the registry against a shared backend-neutral administrative allowlist before invoking the NVX image tool | +| Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | +| Credentials | No private-registry credentials in the initial contract; future credentials must come from an approved host provider and remain out of requests, command lines, logs, and telemetry | | Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | | Converted artifact | Produce a versioned NVX artifact with a manifest identifying the source digest, converter version, runtime compatibility, and checksums | | Guest integration | Attach the converted artifact to OpenVMM and make it the workload root while `/init` and the managed agent remain in the initramfs outside the workload root | @@ -617,7 +705,7 @@ attachment are required deliverables before `microvm.image` is usable. ### Awaited support -- NVX supported image registries +- Initial permitted registry set and migration from the existing WSLC-specific policy name - NVX signed binaries support - NVX crate binary packaging and extraction contract - Windows ARM runtime and SDK package availability From db3d0ca143bbb3b27d3e35eea4d103ec2b798208 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 14:45:10 -0700 Subject: [PATCH 08/19] Clarify NVX SDK deployment contracts Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 87 ++++++++++++++++++++++++++++++++++------- 1 file changed, 72 insertions(+), 15 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 794628ba4..043309067 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -46,13 +46,16 @@ embedded in every SDK package. ### 2.2 How MXC will consume NVX MXC will use a pinned NVX crate version. The crate will expose the Rust -interface and stage its matching signed runtime assets during the Cargo build. -MXC will use the interface and signed DLL implementation from the same crate, -so the API and runtime files remain versioned together. +interface and matching signed runtime assets. The build script for each +consuming executable will invoke a shared MXC build helper to stage those +assets beside that executable. MXC will use the interface and signed DLL +implementation from the same crate, so the API and runtime files remain +versioned together. MXC will be implementing: -- build-time staging of the runtime assets from the NVX crate; +- a public Rust build helper that stages the runtime assets from the NVX crate + beside the consuming executable; - checksum and signature verification; - offline builds using a pre-fetched crate and dependencies; and - packaging for the executor, Node SDK, and .NET SDK. @@ -92,14 +95,26 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | -| Rust | `mxc-sdk` with the `microvm` feature | The feature will include the NVX crate and stage its signed runtime assets | +| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will stage the signed NVX crate assets beside that executable | | Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The runtime package will contain a matched `mxc_ffi.dll` and co-located NVX crate assets | -| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The MXC build will package the RID-specific NVX crate assets beside `mxc_ffi.dll` | +| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will add the RID-specific `nvx` assets beside it | #### Rust ```toml +[dependencies] mxc-sdk = { version = "...", features = ["microvm"] } + +[build-dependencies] +mxc-sdk-build = { version = "...", features = ["nvx-runtime"] } +``` + +```rust +// build.rs +fn main() { + mxc_sdk_build::stage_nvx_runtime() + .expect("failed to stage the NVX runtime"); +} ``` #### Node @@ -117,7 +132,8 @@ npm install @microsoft/mxc-sdk @microsoft/mxc-nvx-runtime The developer experience will be the same in all three ecosystems: -1. Developers will add the MXC SDK and its NVX runtime dependency. +1. Developers will add the MXC SDK and its NVX runtime dependency. Rust + applications will also invoke the staging helper from their build script. 2. They will select `microvm` in the MXC request. 3. The SDK will locate the packaged NVX runtime for the current platform. 4. MXC will validate the signatures and checksums, then launch OpenVMM through @@ -147,9 +163,28 @@ matched `mxc_ffi.dll` from that package, where the NVX runtime is already co-located. Without the runtime package, Node will continue to load the normal MXC native library and will report `microvm` as unavailable. -For .NET, NuGet will copy the RID-specific `mxc_ffi.dll` and `nvx` directory -into the application output. The native MXC layer will resolve the `nvx` -directory relative to the loaded `mxc_ffi.dll`. +For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns +`mxc_ffi.dll`. Its Windows native library will include the MicroVM integration +but will report `microvm` as unavailable when the NVX assets are absent. +`Microsoft.Mxc.Sdk.Nvx.Runtime` will contain only the RID-specific `nvx` +directory and will declare an exact-version dependency on +`Microsoft.Mxc.Sdk`. NuGet will copy both into the application output without +a native-library collision: + +```text +\ + mxc_ffi.dll + nvx\ + .dll + openvmm.exe + vmlinux + initramfs.cpio.gz + + +``` + +The native MXC layer will resolve the `nvx` directory relative to the loaded +`mxc_ffi.dll`. The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. Before launch, MXC will verify the required files, Windows architecture, @@ -179,7 +214,14 @@ the integration will not introduce separate NVX-specific execution methods. | --- | --- | --- | --- | | Rust | `MicrovmConfig { image: String, memory_mb: Option }` and `Containment::Microvm(MicrovmConfig)` | Add `ProvisionRequest::microvm(image: String, memory_mb: Option)` | `v1::run`, `v1::spawn`, and `v1::container::*` | | Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Add `MicrovmProvisionConfig { image: string; memoryMb?: number; filesystem?; network? }` to `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | -| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register its JSON discriminator as `microvm` | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | +| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register both MicroVM request shapes and their `microvm` discriminators with the source-generated JSON context | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | + +The .NET one-shot `Microvm` containment type and +`MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` +roots in `MxcJsonContext`. Their serialization and deserialization will use +the existing `MxcJson` helpers without reflection fallback. +`Microsoft.Mxc.Sdk.AotSmokeTest` will serialize and deserialize representative +one-shot and state-aware MicroVM requests through that production JSON path. The wire and SDK changes will be delivered in this order: @@ -259,7 +301,7 @@ not start a VM. | Surface | New NVX integration work | | --- | --- | | Rust SDK | Will enable the NVX backend in the SDK and engine build | -| .NET SDK | Will add the `microvm` choice through the existing `mxc_ffi` boundary and package the NVX-enabled native runtime | +| .NET SDK | Will add the `microvm` choice through the existing SDK-owned `mxc_ffi` boundary; the separate exact-version runtime package will contribute only the `nvx` assets | | Node SDK | Will add the `microvm` choice to typed configuration and package the NVX-enabled native runtime | Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will @@ -619,6 +661,21 @@ backend will include a backend-specific reason in the SDK's platform-support result, with remediation such as installing the matching runtime package, enabling WHP, or repairing corrupt assets. +The native `PlatformSupport` payload will add an `unavailableReasons` map from +backend wire name to actionable reason. The existing platform-wide `reason` +will remain reserved for a host on which MXC itself is unsupported. + +| SDK | Backend-specific discovery field | +| --- | --- | +| Rust | `PlatformSupport::unavailable_reasons` | +| Node | `PlatformSupport.unavailableReasons` | +| .NET | `PlatformSupport.UnavailableReasons`, keyed by `ContainmentBackend` | + +For example, a Windows host on which ProcessContainer works but the NVX +runtime package is missing will remain platform-supported, omit `microvm` from +`availableMethods`, and report the MicroVM remediation in +`unavailableReasons`. + Execution will repeat the authoritative preflight before launch. A cached or stale discovery result will not bypass runtime validation. @@ -635,15 +692,15 @@ packaged MXC executor and all three SDKs on Windows x64 with WHP. | Area | Required coverage | | --- | --- | | Integration | `microvm` routes to NVX for one-shot and state-aware execution | -| Discovery | Rust, Node, and .NET platform-support APIs include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; each failure mode returns actionable backend-specific remediation without starting a VM | +| Discovery | Add the native `unavailableReasons` payload and its Rust, Node, and .NET projections; include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; verify each failure mode returns actionable backend-specific remediation without starting a VM | | Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | | State-aware process lifetime | After the process performing `start` exits, OpenVMM remains running and a later `exec` process reconnects successfully; test from a Windows job that would normally terminate child processes | -| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct | +| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | | Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; inclusion of the DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | +| Packaging | Rust crate, npm, and NuGet installation; single ownership of the .NET `mxc_ffi.dll`; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | From 7049fc6e9a1d342faf5b6bac79531f900582a166 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:06:38 -0700 Subject: [PATCH 09/19] Update NVX integration documentation Enhance NVX integration documentation with detailed updates on state-aware process lifetime, discovery payloads, lifecycle management, SDK behavior, filesystem access, and network rules. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 043309067..16702b07d 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -694,7 +694,7 @@ packaged MXC executor and all three SDKs on Windows x64 with WHP. | Integration | `microvm` routes to NVX for one-shot and state-aware execution | | Discovery | Add the native `unavailableReasons` payload and its Rust, Node, and .NET projections; include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; verify each failure mode returns actionable backend-specific remediation without starting a VM | | Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | -| State-aware process lifetime | After the process performing `start` exits, OpenVMM remains running and a later `exec` process reconnects successfully; test from a Windows job that would normally terminate child processes | +| State-aware process lifetime | Configure NVX with `OpenVmmConfig::breakaway_from_job = true` and return actionable `backend_unavailable` when the caller's Windows job disallows breakaway; from a kill-on-close Windows job, verify that OpenVMM remains running after the `start` process exits and that a later `exec` process reconnects successfully | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | | Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | From 7748258e5dea3b7e3321d4cdabefd87a16bf11ce Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:10:05 -0700 Subject: [PATCH 10/19] Unify NVX runtime package ownership Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 54 ++++++++++++++++++++++++++++++----------- 1 file changed, 40 insertions(+), 14 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 16702b07d..63dd444c9 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -58,7 +58,7 @@ MXC will be implementing: beside the consuming executable; - checksum and signature verification; - offline builds using a pre-fetched crate and dependencies; and -- packaging for the executor, Node SDK, and .NET SDK. +- packaging for the three SDKs and the internal MXC end-to-end executor bundle. Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json` and verify that it references the matching Linux and Alpine source artifacts @@ -96,7 +96,7 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | | Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will stage the signed NVX crate assets beside that executable | -| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The runtime package will contain a matched `mxc_ffi.dll` and co-located NVX crate assets | +| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will contain only the NVX assets and its directory will be registered with the native layer | | .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will add the RID-specific `nvx` assets beside it | #### Rust @@ -136,18 +136,23 @@ The developer experience will be the same in all three ecosystems: applications will also invoke the staging helper from their build script. 2. They will select `microvm` in the MXC request. 3. The SDK will locate the packaged NVX runtime for the current platform. + Node will register the separately installed runtime package directory with + the native layer. 4. MXC will validate the signatures and checksums, then launch OpenVMM through the NVX Rust integration. Developers will not provide paths to `openvmm.exe`, `vmlinux`, or `initramfs.cpio.gz`, and will not communicate with OpenVMM directly. -The Node runtime package will use this layout: +The Node packages will use this layout: ```text -@microsoft/mxc-nvx-runtime\ +@microsoft/mxc-sdk\ bin\win-x64\ mxc_ffi.dll + +@microsoft/mxc-nvx-runtime\ + bin\win-x64\ nvx\ .dll openvmm.exe @@ -158,10 +163,23 @@ The Node runtime package will use this layout: ``` The Node SDK native-library resolver will check whether -`@microsoft/mxc-nvx-runtime` is installed. When present, it will load the -matched `mxc_ffi.dll` from that package, where the NVX runtime is already -co-located. Without the runtime package, Node will continue to load the normal -MXC native library and will report `microvm` as unavailable. +`@microsoft/mxc-nvx-runtime` is installed. `@microsoft/mxc-sdk` will remain the +only npm package that owns and loads `mxc_ffi.dll`. When the runtime package is +present, the Node loader will resolve its absolute platform directory and call +the backend-neutral native registration API before platform discovery or +execution: + +```text +mxc_register_backend_runtime_directory("microvm", absolutePath) +``` + +Registration will be process-scoped, accept only registered backend names and +absolute canonical paths, and be idempotent for the same backend and path. A +conflicting second registration will fail. Registration locates the package; +it does not trust it. The native probe will still validate the expected +package version, architecture, authenticated manifest, signatures, and +checksums. Without the runtime package, Node will not register a MicroVM +runtime directory and will report `microvm` as unavailable. For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns `mxc_ffi.dll`. Its Windows native library will include the MicroVM integration @@ -302,10 +320,16 @@ not start a VM. | --- | --- | | Rust SDK | Will enable the NVX backend in the SDK and engine build | | .NET SDK | Will add the `microvm` choice through the existing SDK-owned `mxc_ffi` boundary; the separate exact-version runtime package will contribute only the `nvx` assets | -| Node SDK | Will add the `microvm` choice to typed configuration and package the NVX-enabled native runtime | +| Node SDK | Will add the `microvm` choice to typed configuration, keep `mxc_ffi.dll` in the base SDK, resolve the exact-version NVX runtime package, and register its directory through the backend-neutral native API | Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will be linked on the Rust side; no separate NVX FFI library will be required. +The NVX developer contract covers the Rust, Node V1, and .NET SDKs. The Node +V1 run, spawn, PTY, and state-aware paths call `mxc_ffi` in-process. NVX will +not add support commitments for legacy Node executor paths or introduce a new +NVX CLI or executor. The existing generic `wxc-exec` may be built with NVX +support only as an internal end-to-end harness over the same `mxc_engine` +implementation. ## 4. Filesystem, lifecycle, and network support @@ -686,21 +710,23 @@ identify the associated log path. ## 8. Requirements and end-to-end tests -The NVX backend will be complete when the following areas pass through the -packaged MXC executor and all three SDKs on Windows x64 with WHP. +The NVX backend will be complete when the following areas pass through all +three SDKs on Windows x64 with WHP. MXC may additionally run the same cases +through the generic packaged executor as an internal end-to-end harness; that +executor is not an NVX developer-facing surface. | Area | Required coverage | | --- | --- | -| Integration | `microvm` routes to NVX for one-shot and state-aware execution | +| Integration | `microvm` routes to NVX for one-shot and state-aware execution through Rust, Node V1, and .NET; the generic executor is used only as an internal harness | | Discovery | Add the native `unavailableReasons` payload and its Rust, Node, and .NET projections; include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; verify each failure mode returns actionable backend-specific remediation without starting a VM | | Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | | State-aware process lifetime | Configure NVX with `OpenVmmConfig::breakaway_from_job = true` and return actionable `backend_unavailable` when the caller's Windows job disallows breakaway; from a kill-on-close Windows job, verify that OpenVMM remains running after the `start` process exits and that a later `exec` process reconnects successfully | -| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | +| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; Node registers the separately installed MicroVM runtime directory before discovery or execution and rejects conflicting registration; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | | Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; single ownership of the .NET `mxc_ffi.dll`; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | +| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | From 712e16895291f79a13a48dc98fb7e16186e81608 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:31:33 -0700 Subject: [PATCH 11/19] Preserve NVX NuGet runtime layout Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 63dd444c9..5def4cde7 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -186,8 +186,14 @@ For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns but will report `microvm` as unavailable when the NVX assets are absent. `Microsoft.Mxc.Sdk.Nvx.Runtime` will contain only the RID-specific `nvx` directory and will declare an exact-version dependency on -`Microsoft.Mxc.Sdk`. NuGet will copy both into the application output without -a native-library collision: +`Microsoft.Mxc.Sdk`. + +NuGet's default RID-native asset handling flattens files beneath +`runtimes/{rid}/native`, so the runtime package will not place the NVX tree +there. It will package the assets under `runtimes/{rid}/nvx/**` and include a +`buildTransitive/Microsoft.Mxc.Sdk.Nvx.Runtime.targets` file. That target will +recursively copy the selected RID's complete `nvx` tree, preserving +`%(RecursiveDir)`, into both normal build and publish outputs: ```text \ @@ -202,7 +208,8 @@ a native-library collision: ``` The native MXC layer will resolve the `nvx` directory relative to the loaded -`mxc_ffi.dll`. +`mxc_ffi.dll`. The copy target will reject unsupported RID and architecture +combinations rather than producing a partial runtime. The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. Before launch, MXC will verify the required files, Windows architecture, @@ -726,7 +733,7 @@ executor is not an NVX developer-facing surface. | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | +| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | From d93be93691b5bd59a58eb0a153dd4b96227c2f3d Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:37:47 -0700 Subject: [PATCH 12/19] Update NVX workload cancellation semantics Clarify behavior of cancelling SDK operations related to NVX workloads. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 5def4cde7..4c246b803 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -522,8 +522,7 @@ state does not survive `stop`, while changes to mapped host files do. - NVX runs one workload at a time. A second `exec` waits for up to 60 seconds by default. - `stop` waits for up to 30 seconds before forcing the VM to shut down. -- Cancelling an SDK operation must cancel the NVX workload, not only stop the - SDK from waiting. +- Cancelling an SDK wait preserves existing SDK semantics and does not kill the workload; explicit kill or cancel operations on an execution handle must cancel the NVX workload and its descendants. ## 5. UI and other support From 22266e1be45d92a446f554f685733cd9f800c9fb Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:50:30 -0700 Subject: [PATCH 13/19] Enhance 'provision' phase description in NVX integration Expanded the description of the 'provision' phase to include details about resolving OCI references and persisting identity. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 4c246b803..a5f607175 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -480,7 +480,7 @@ request will supply the process configuration. | MXC phase | How NVX handles it | | --- | --- | -| `provision` | Stores the configuration and returns an NVX sandbox ID | +| `provision` | Resolves the OCI reference to an immutable digest, selects or creates the converted artifact, persists that identity with the configuration, and returns an NVX sandbox ID | | `start` | Launches OpenVMM and waits for the guest agent | | `exec` | Runs a workload in the running VM | | `stop` | Stops the VM while retaining provisioned state | From a1f2c7a46d50e1e0a05ab8666d77e33f81a0fcb4 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 15:52:20 -0700 Subject: [PATCH 14/19] Respect egress policy during NVX image pulls Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index a5f607175..3047c9372 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -593,11 +593,22 @@ The complete request in section 4.1 is an example of this model: The integration will reuse the existing WSLC cache and registry pattern: 1. Use the image from the local cache when it is already available. -2. Otherwise, resolve the registry host and check a shared backend-neutral MXC +2. On a cache miss, reject normal execution when the request uses + deny-by-default egress. MXC will not perform host registry traffic on behalf + of that request before the VM exists. +3. Otherwise, resolve the registry host and check a shared backend-neutral MXC administrative registry policy. -3. When the registry is permitted, invoke the signed NVX image tool to pull +4. When the registry is permitted, invoke the signed NVX image tool to pull the image and resolve its immutable digest. -4. Convert and cache the NVX-compatible artifact. +5. Convert and cache the NVX-compatible artifact. + +An administrator or deployment pipeline can warm the same cache through a +separate explicit MXC host-setup operation. That operation will invoke the +signed NVX image tool, enforce the machine registry policy, and report the +resolved digest and converted-artifact identity. It will not create a sandbox +or inherit a sandbox request's workload network policy. After prefetch, +deny-by-default requests can use the cached artifact without host network +traffic. Image references without an explicit registry will resolve against Docker Hub. Explicit registry references will be permitted only when the registry is @@ -735,7 +746,7 @@ executor is not an NVX developer-facing surface. | Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | -| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, and generated SDK types | +| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by offline cache use, and permitted execution-time pull | Negative filesystem and network tests must include a working positive control so infrastructure failures are not mistaken for policy enforcement. @@ -777,6 +788,8 @@ so infrastructure failures are not mistaken for policy enforcement. | --- | --- | | Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | | Registry authorisation | MXC validates the registry against a shared backend-neutral administrative allowlist before invoking the NVX image tool | +| Execution-time host fetch | A cache miss under deny-by-default request egress is rejected without registry traffic; automatic pull is available only when the request permits egress | +| Explicit prefetch | A separate MXC host-setup operation pulls and converts the image under the machine registry policy without creating a sandbox, then stores it in the same runtime cache | | Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | | Credentials | No private-registry credentials in the initial contract; future credentials must come from an approved host provider and remain out of requests, command lines, logs, and telemetry | | Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | From e09ba2793c31274d939a50df4f32555ae56a384c Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Wed, 7 Oct 2026 16:09:18 -0700 Subject: [PATCH 15/19] Define NVX converter security ownership Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index 3047c9372..d10a49911 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -630,6 +630,15 @@ invoking the image tool. The signed NVX image tool will own DNS resolution, TLS, registry protocol, redirects, download, digest resolution, and conversion while enforcing the policy supplied by MXC. +The NVX team will also own the converter's security model for processing +untrusted OCI manifests and layers. This includes its threat model, +least-privilege containment or no-host-extraction design, archive and path +handling, resource limits, staging and cache boundaries, failure cleanup, and +adversarial security tests. Signing authenticates the converter but does not +by itself contain parser or archive-processing vulnerabilities. MXC will not +enable arbitrary OCI image input until the NVX conversion-security contract +has completed security review. + `microvm.image` will accept an OCI image reference rather than an arbitrary URL. Redirects to another registry host will require that host to be permitted by the same policy. Private-registry credentials will not be part of the @@ -746,7 +755,7 @@ executor is not an NVX developer-facing surface. | Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | -| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by offline cache use, and permitted execution-time pull | +| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by offline cache use, permitted execution-time pull, the NVX-owned adversarial conversion-security suite, and MXC rejection of malformed or incompatible converter output | Negative filesystem and network tests must include a working positive control so infrastructure failures are not mistaken for policy enforcement. @@ -793,6 +802,7 @@ so infrastructure failures are not mistaken for policy enforcement. | Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | | Credentials | No private-registry credentials in the initial contract; future credentials must come from an approved host provider and remain out of requests, command lines, logs, and telemetry | | Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | +| Converter security ownership | NVX defines and security-reviews the least-privilege containment or no-host-extraction model, archive/path rules, resource bounds, output confinement, cleanup, and adversarial tests for untrusted OCI input | | Converted artifact | Produce a versioned NVX artifact with a manifest identifying the source digest, converter version, runtime compatibility, and checksums | | Guest integration | Attach the converted artifact to OpenVMM and make it the workload root while `/init` and the managed agent remain in the initramfs outside the workload root | | OCI metadata | Define how `ENTRYPOINT`, `CMD`, `ENV`, `WORKDIR`, and `USER` interact with MXC `process` settings | @@ -800,14 +810,16 @@ so infrastructure failures are not mistaken for policy enforcement. | Cache identity | Key conversions by image digest plus converter and runtime format version rather than by mutable image tag alone | | Failures | Surface registry, conversion, compatibility, and attachment failures through actionable MXC errors | -The signed image tool, image schema, converted-artifact format, and runtime -attachment are required deliverables before `microvm.image` is usable. +The signed image tool, reviewed conversion-security contract, image schema, +converted-artifact format, and runtime attachment are required deliverables +before `microvm.image` is usable. ## References and open decisions ### Awaited support - Initial permitted registry set and migration from the existing WSLC-specific policy name +- NVX-owned OCI converter threat model, containment and bounded-extraction contract, security review, and adversarial test evidence - NVX signed binaries support - NVX crate binary packaging and extraction contract - Windows ARM runtime and SDK package availability From 0c265dacce3e98033bcdca5ad8c7469ad7c4069f Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Thu, 8 Oct 2026 12:11:59 -0700 Subject: [PATCH 16/19] Refine NVX integration proposal Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 273 +++++++++++++++++++++++----------------- 1 file changed, 158 insertions(+), 115 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index d10a49911..a3abd498c 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -7,9 +7,9 @@ tense describes work proposed for the MXC integration. ## 1. Introduction -[NVX][nvx-readme] is an ultra-light microVM sandbox for running untrusted -workloads with hardware-enforced isolation. It is built on OpenVMM and runs -Linux as its guest. It is a cross-platform microVM sandbox for agentic workloads. +[NVX][nvx-readme] provides ultra-light microVMs for running +untrusted workloads with hardware-enforced isolation. It is built on OpenVMM, +runs Linux as its guest, and targets cross-platform agentic workloads. Customers will use NVX through the MXC APIs and will not need to understand OpenVMM, the guest agent, or image conversion. @@ -28,45 +28,57 @@ Nanvix as the backend for the `microvm` option. NVX will ship a Rust crate containing: - the Rust host API; -- the signed NVX implementation DLL; -- the signed OpenVMM executable; -- the signed image download and conversion tool; -- the Linux kernel and initramfs with the NVX init agent; and -- manifests, checksums, provenance, licences, and package inventories. - -Because the crate distributes `vmlinux` and an Alpine initramfs, the NVX team -will publish the exact corresponding Linux and Alpine source artifacts for -every runtime version. The crate will include `SOURCE-MANIFEST.json`, the -Alpine package inventory, licences, and third-party notices. The source -manifest will identify the NVX project source, kernel version and -configuration, patches, Alpine package sources, published source artifacts, -and hashes. The source archives may be published separately rather than -embedded in every SDK package. +- pinned runtime artifact identities, versions, locations, and digests; and +- build logic that fetches and verifies the matching signed artifacts from + NVX-controlled registries or repositories. + +The fetched runtime artifacts will include the signed NVX implementation DLL, +signed OpenVMM executable, signed image download and conversion tool, Linux +kernel, minimal control initramfs with the NVX init agent, manifests, +checksums, provenance, licences, and package inventories. + +NVX already publishes the corresponding source for its Linux kernel and +Alpine-based control initramfs. The NVX repository contains the owned guest +and build inputs under `kernel/`, `guest/common/`, and `guest/alpine/`. Its +release tooling collects the exact patched kernel source and the recipes and +upstream sources for the minimal initramfs packages, then publishes them as separate +`source/nvx-linux-source-.tar.gz` and +`source/nvx-alpine-source-.tar.gz` artifacts. + +The existing `SOURCE-MANIFEST.json`, control-initramfs package inventory, +licences, and third-party notices identify the NVX project source, kernel +version and configuration, patches, Alpine package sources used by the +control initramfs, published source artifacts, and hashes. MXC will consume +and validate this existing source-delivery contract rather than requiring the +NVX team to create a new one. ### 2.2 How MXC will consume NVX MXC will use a pinned NVX crate version. The crate will expose the Rust -interface and matching signed runtime assets. The build script for each -consuming executable will invoke a shared MXC build helper to stage those -assets beside that executable. MXC will use the interface and signed DLL -implementation from the same crate, so the API and runtime files remain -versioned together. +interface and resolve its matching signed runtime artifacts. The build script +for each consuming executable will invoke a shared MXC build helper, which +uses the crate's fetch and verification contract before staging those +artifacts beside the executable. The crate's pinned artifact manifest will +keep the API and runtime files versioned together. MXC will be implementing: -- a public Rust build helper that stages the runtime assets from the NVX crate - beside the consuming executable; +- a public Rust build helper that stages the runtime artifacts resolved and + verified through the NVX crate beside the consuming executable; - checksum and signature verification; -- offline builds using a pre-fetched crate and dependencies; and +- offline builds using a pre-fetched runtime artifact directory plus the + crate and its dependencies; and - packaging for the three SDKs and the internal MXC end-to-end executor bundle. Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json` and -verify that it references the matching Linux and Alpine source artifacts -published by the NVX team. MXC will copy the source manifest, Alpine package -inventory, licences, and notices into the npm and NuGet runtime packages. MXC -will not generate the source archives. Packaging will fail when required -metadata, source references, or hashes are missing or do not match the runtime -version. +verify that it references the matching Linux kernel and minimal control +initramfs corresponding-source artifacts published by the NVX team. MXC will +copy the source manifest, control-initramfs package inventory, licences, and +notices into the npm and NuGet runtime packages. The published +`nvx-alpine-source-*` archive covers only the Alpine packages included in that +minimal trusted control environment. MXC will not generate the source +archives. Packaging will fail when required metadata, source references, or +hashes are missing or do not match the runtime version. ### 2.3 Key NVX files that MXC will use @@ -76,17 +88,19 @@ version. | `openvmm.exe` | 22 MB | Windows OpenVMM executable | | Image download and conversion tool | Not yet published | Will download and convert standard OCI images | | `vmlinux` | 24 MB | NVX Linux kernel | -| `initramfs.cpio.gz` | 7.4 MB | Alpine userspace and NVX guest agent | +| `initramfs.cpio.gz` | Not yet published | Minimal trusted control userspace and NVX guest agent; customer workload tooling is supplied by the OCI image | The release also includes the supporting checksum, manifest, provenance, -licence, and package-inventory files. -These are the current development-bundle sizes and can change. The final Rust -crate size will also include the implementation DLL and image tool. +licence, and package-inventory files. The numeric OpenVMM and kernel sizes are +from the selected development baseline and can change. The new implementation +DLL, image tool, and minimal control initramfs sizes have not yet been +published. These artifacts contribute to the staged runtime and SDK package +footprint rather than the published Rust crate archive size. The current NVX binaries are not signed yet. The production Rust crate will -contain the signed implementation DLL, OpenVMM executable, and image tool. -MXC will validate their signatures and the published checksums before staging -or using the files. +fetch the required signed implementation DLL, OpenVMM executable, image tool, +and guest artifacts from pinned NVX-controlled locations. MXC will validate +their signatures and published checksums before staging or using the files. ### 2.4 Developer packaging and usage @@ -95,7 +109,7 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | -| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will stage the signed NVX crate assets beside that executable | +| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will fetch or use pre-fetched artifacts through the pinned NVX crate contract, verify them, and stage them beside that executable | | Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will contain only the NVX assets and its directory will be registered with the native layer | | .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will add the RID-specific `nvx` assets beside it | @@ -218,9 +232,9 @@ incompatible runtime will return an actionable load or `backend_unavailable` error identifying the required runtime package or version. The co-located checksum manifest will not be trusted by itself. The pinned NVX -crate will provide the expected runtime-manifest identity, and the compiled -native integration will authenticate that manifest before using its file -hashes. MXC will then: +crate will provide the expected artifact locations, digests, and +runtime-manifest identity, and the compiled native integration will +authenticate that manifest before using its file hashes. MXC will then: 1. validate the Authenticode certificate chain and expected Microsoft signer for the NVX implementation DLL, `openvmm.exe`, and image tool; @@ -237,9 +251,9 @@ the integration will not introduce separate NVX-specific execution methods. | SDK | One-shot configuration | State-aware provision addition | Existing operations that will use it | | --- | --- | --- | --- | -| Rust | `MicrovmConfig { image: String, memory_mb: Option }` and `Containment::Microvm(MicrovmConfig)` | Add `ProvisionRequest::microvm(image: String, memory_mb: Option)` | `v1::run`, `v1::spawn`, and `v1::container::*` | -| Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Add `MicrovmProvisionConfig { image: string; memoryMb?: number; filesystem?; network? }` to `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | -| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` with required `Image`, optional `MemoryMb`, `Filesystem`, and `Network`; register both MicroVM request shapes and their `microvm` discriminators with the source-generated JSON context | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | +| Rust | `MicrovmConfig { image: String, memory_mb: Option }` and `Containment::Microvm(MicrovmConfig)` | Reuse `MicrovmConfig` in the state-aware `ProvisionRequest` | `v1::run`, `v1::spawn`, and `v1::container::*` | +| Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Reuse the same `{ image, memoryMb? }` MicroVM configuration in `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | +| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` that reuses the same required `Image` and optional `MemoryMb` fields, plus `Filesystem` and `Network`; register both MicroVM request shapes and their `microvm` discriminators with the source-generated JSON context | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | The .NET one-shot `Microvm` containment type and `MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` @@ -254,61 +268,77 @@ The wire and SDK changes will be delivered in this order: engine binding, and runtime tests to the development `1.1.0-alpha` contract. 2. Keep the wire work development-only while that contract remains mutable. -3. Publish the fields in stable `1.1.0` when the contract is ready. -4. Advance `schemas/schema-version.json` `sdkMajorTargets["1"]` from `1.0.0` - to `1.1.0`. -5. Publish the new Rust, Node, and .NET V1 MicroVM types and update their - versioned references. - -Schema publication and runtime authorisation are independent. Even after -stable `1.1.0` becomes the V1 SDK target, `microvm` may continue to require -the experimental option until the backend meets its promotion bar. +3. If NVX is ready when `1.1.0` is frozen, publish the fields in stable + `1.1.0`, advance `schemas/schema-version.json` `sdkMajorTargets["1"]` from + `1.0.0` to `1.1.0`, and publish the new Rust, Node, and .NET V1 MicroVM + types and versioned references. The stable API will not require an + experimental option. +4. If NVX is not ready, stable `1.1.0` will omit the MicroVM fields. Move them + into the next mutable contract, such as `1.2.0-alpha`, and do not publish + the V1 MicroVM SDK types until a stable contract contains those fields. + +Contract publication and host availability remain separate. A published +MicroVM backend can still return `backend_unavailable` when the host lacks the +NVX runtime, WHP, or another required capability. The existing lifecycle operation methods will remain unchanged; only the backend-specific request types they accept will expand. ## 3. Architecture -### 3.1 NVX, OpenVMM, and Alpine Linux +### 3.1 NVX, OpenVMM, and the guest control environment MXC will use the NVX Rust interface and its signed DLL implementation. The DLL -will launch OpenVMM, which will boot the NVX kernel and unpack the Alpine -initramfs into the VM's in-memory root filesystem. +will launch OpenVMM, which will boot the NVX kernel and unpack the minimal +control initramfs into the VM's in-memory control root filesystem. -The initramfs contains two relevant agent components: +The initramfs contains only the binaries and libraries required to initialise +and control the microVM, including these two components: | Guest path | Role | | --- | --- | | `/init` | Runs first, mounts the guest pseudo-filesystems, reads the kernel command line, prepares networking and host filesystem mappings, and resolves the workload identity | | `/sbin/nvx-managed-agent` | Replaces `/init` for the managed lifecycle and remains running as PID 1 while workloads execute | -The managed agent stays in the initramfs root filesystem and is not exposed -inside the workload's execution context. +The managed agent remains in the control initramfs and outside the OCI +workload root filesystem. Workload processes see the converted OCI filesystem +as `/`; they do not see or execute `/sbin/nvx-managed-agent`, which remains +PID 1 and handles lifecycle and process-control messages. ```mermaid flowchart LR MXC["MXC"] --> API["NVX Rust interface"] API --> DLL["Signed NVX implementation DLL"] DLL --> OpenVMM["Signed OpenVMM executable"] - OpenVMM --> Init["Alpine initramfs: /init"] - Init --> Alpine["/sbin/nvx-managed-agent (PID 1)"] - Alpine --> Workload["Non-root workload"] + OpenVMM --> Init["Control initramfs: /init"] + Init --> Agent["/sbin/nvx-managed-agent (PID 1)"] + Agent --> Workload["OCI workload environment"] ``` The NVX Rust interface exposes the following operations: -| Operation | NVX Rust API | -| --- | --- | -| Validate provision | `validate_provision` | -| Validate execution | `validate_exec` | -| Check runtime availability | `probe` | -| Provision VM | `provision` | -| Start VM | `start` | -| Execute workload | `exec` | -| Stop VM | `stop` | -| Remove provisioned state | `deprovision` | -| Wait for execution outcome | `Execution::wait` | -| Cancel execution | `Execution::canceller` | +| Operation | NVX Rust API | Input structure | Output structure | +| --- | --- | --- | --- | +| Validate provision | `validate_provision` | `&ProvisionRequest { filesystem?, network?, microvm.provision.memoryMib? }` | `Result<()>` | +| Validate execution | `validate_exec` | `&ExecRequest { process { commandLine or argv, cwd?, env?, inheritDefaultEnv?, timeout? }, stdin }` | `Result<()>` | +| Check runtime availability | `probe` | None | `Result<()>` | +| Provision VM | `provision` | `&ProvisionRequest` | `Result` | +| Start VM | `start` | `&SandboxId` | `Result` | +| Execute workload | `exec` | `&SandboxId`, `&ExecRequest` | `Result` with live stdout and stderr pipes, optional stdin pipe, cancellation handle, and wait methods | +| Stop VM | `stop` | `&SandboxId` | `Result` | +| Remove provisioned state | `deprovision` | `&SandboxId` | `Result`; the ID becomes stale | +| Wait for execution outcome | `Execution::wait` | Owned `Execution` handle | `Result`: `Exited(code)`, `Signaled(signal)`, `TimedOut`, `Cancelled`, or `Failed(reason)` | +| Wait and collect output | `Execution::wait_with_output` | Owned `Execution` handle | `Result, stderr: Vec }>` | +| Get cancellation handle | `Execution::canceller` | `&Execution` | `Canceller` | +| Cancel execution | `Canceller::cancel` | `&Canceller` | `Result<()>` | + +`ProvisionRequest` in the selected NVX API does not yet contain an OCI image. +It also currently nests memory under `microvm.provision`. The MXC wire +contract will use the shared flat `microvm { image, memoryMb }` structure for +one-shot and state-aware provision requests; the integration adapter will map +that structure into the NVX Rust input. The image integration will extend the +NVX provision input with the resolved OCI image or converted-artifact identity +described in section 6 and Appendix C. ### 3.2 MXC integration @@ -381,7 +411,7 @@ The integration will target the MXC `1.1.0-alpha` development schema. ### 4.2 Filesystem MXC will pass the configured host paths to NVX. NVX currently asks OpenVMM for -one virtio-fs export rooted at the common host directory. The Alpine guest +one virtio-fs export rooted at the common host directory. The guest control agent then bind-mounts each requested path separately as read-only or read-write: @@ -389,38 +419,52 @@ read-write: flowchart LR Policy["MXC filesystem policy"] --> NVX["NVX mapping plan"] NVX --> OpenVMM["OpenVMM: one virtio-fs export"] - OpenVMM --> Alpine["Alpine: per-path RO/RW bind mounts"] - Alpine --> Workload["Workload paths"] + OpenVMM --> Agent["Control agent: per-path RO/RW bind mounts"] + Agent --> Workload["OCI workload paths"] ``` Read-write mappings are live: a guest write changes the mapped host file immediately. Denied entries are hidden by OpenVMM within the exported host tree. +On Windows, host access remains governed by the Windows identity running +OpenVMM and the mapped directory's NTFS ACLs. NVX will define and guarantee +the Windows owner and inherited ACL applied when a guest creates a file; MXC +will verify that behaviour end to end before release. + All mapped paths must exist, be on the same Windows volume, and share a common directory below the volume root. MXC will translate Windows paths into guest paths: for example, `C:\nvx-work\input` will be available as `/mnt/c/nvx-work/input`. Exporting an entire volume such as `C:\` will be rejected. +The initial integration will also reject mappings at or below protected +Windows host roots, including the Windows directory, Program Files, Program +Files (x86), and ProgramData. Validation will use canonical paths and Windows +Known Folder resolution so case differences, short names, junctions, and +reparse points cannot bypass the rule. Both read-only and read-write mappings +will fail before OpenVMM starts and identify the protected root. + | MXC Schema field | Works today | | --- | --- | | `filesystem.readonlyPaths` | Yes | | `filesystem.readwritePaths` | Yes | | `filesystem.deniedPaths` | Yes | + ### 4.3 Network MXC will pass the directional network policy to NVX. NVX currently converts -the supported rules into OpenVMM network options. OpenVMM attaches a virtual -network device to the guest and uses its portable profile as the host-side, -in-process NAT and filtering data plane. +the supported rules into OpenVMM network options. OpenVMM presents a virtual +network adapter to the guest. Its portable networking mode processes the +adapter's traffic inside the OpenVMM host process, where it performs NAT and +enforces the configured allow and deny rules. ```mermaid flowchart LR Policy["MXC network policy"] --> NVX["NVX rule conversion"] NVX --> OpenVMM["OpenVMM virtual NIC + portable profile"] - OpenVMM --> Alpine["Alpine virtual network device"] + OpenVMM --> Workload["OCI workload virtual network device"] ``` @@ -465,29 +509,27 @@ The following shows only the state-aware difference: { "phase": "provision", "microvm": { - "provision": { - "memoryMb": 256, - "image": "python:3.12-alpine" - } + "memoryMb": 256, + "image": "python:3.12-alpine" } } ``` The `process` section from the one-shot example will be omitted during -provision. The new `microvm.provision` schema will be a separate MicroVM -provision type with required `image` and optional `memoryMb`. A later `exec` +provision. One-shot and state-aware provision will use the same `microvm` +structure with required `image` and optional `memoryMb`. A later `exec` request will supply the process configuration. | MXC phase | How NVX handles it | | --- | --- | -| `provision` | Resolves the OCI reference to an immutable digest, selects or creates the converted artifact, persists that identity with the configuration, and returns an NVX sandbox ID | +| `provision` | Resolves the OCI reference to an immutable digest, selects or creates the converted artifact, persists that identity with the configuration, and returns an NVX instance ID | | `start` | Launches OpenVMM and waits for the guest agent | | `exec` | Runs a workload in the running VM | | `stop` | Stops the VM while retaining provisioned state | | `deprovision` | Removes the provisioned state | | One-shot | MXC will compose provision, start, exec, stop, and deprovision | -MXC will preserve the NVX sandbox ID unchanged. NVX provision returns IDs in +MXC will preserve the NVX instance ID unchanged. NVX provision returns IDs in the following form: ```text @@ -529,24 +571,26 @@ state does not survive `stop`, while changes to mapped host files do. | Schema field or capability | Works today | Notes | | --- | --- | --- | | `ui` | No | Not supported by design, reject when supplied | -| `process.commandLine` | Yes | Runs as a Linux shell command | +| `process.commandLine` | Yes | Runs through the workload image's execution environment; the control initramfs does not supply workload commands or a fallback shell | | `process.cwd` | Yes | Must be an absolute guest path | | `process.env` / `inheritDefaultEnv` | Yes | | | `process.timeout` | Yes | Maximum one hour; omitted or `0` means no execution deadline | | Separate stdout and stderr | Yes | Combined output is limited to 1 MB | | Live stdin | No | Future work if needed. Workload receives EOF | | PTY | No | Future work if needed | -| Guest memory override | Yes | One-shot uses `microvm.memoryMb`; state-aware provision uses `microvm.provision.memoryMb`; default is 256 MB | +| Guest memory override | Yes | One-shot and state-aware provision use `microvm.memoryMb`; default is 256 MB | | `fallback` | No | Not supported by design, NVX does not select another backend | | Process behaviour | Developer-visible result | | --- | --- | -| Command execution | `process.commandLine` runs through BusyBox `/bin/sh -c` | -| Environment | Omitted environment uses guest defaults; an explicit list replaces or layers over those defaults; the shell may set `PWD` and `SHLVL` | +| Command execution | The OCI workload image must provide the shell or executable required by the final execution contract; the control initramfs is not used as the workload command environment | +| Environment | Omitted environment uses the OCI image defaults; an explicit list replaces or layers over those defaults; the selected shell may set `PWD` and `SHLVL` | | Output limit | Combined stdout and stderr are limited to 1 MB; exceeding the limit terminates the workload rather than truncating output | | Nonzero exit | Returned as a workload result, not an SDK or FFI failure | | Invalid request or unavailable backend | Returned as an MXC error rather than a workload exit code | +**Open question: Should NVX change the output-limit behaviour, for example by truncating output instead of terminating the workload?** + NVX lifecycle errors align with the existing [MXC SDK error classifications](reference/rust/v1/types.md). The integration will map the additional NVX execution outcomes as described in @@ -590,7 +634,8 @@ The complete request in section 4.1 is an example of this model: - The network policy limits the workload to the requested destination and port. -The integration will reuse the existing WSLC cache and registry pattern: +The integration will use an NVX converted-image cache and a shared +backend-neutral MXC registry policy: 1. Use the image from the local cache when it is already available. 2. On a cache miss, reject normal execution when the request uses @@ -605,18 +650,18 @@ The integration will reuse the existing WSLC cache and registry pattern: An administrator or deployment pipeline can warm the same cache through a separate explicit MXC host-setup operation. That operation will invoke the signed NVX image tool, enforce the machine registry policy, and report the -resolved digest and converted-artifact identity. It will not create a sandbox -or inherit a sandbox request's workload network policy. After prefetch, +resolved digest and converted-artifact identity. It will not create a +NVX instance or inherit a workload request's network policy. After prefetch, deny-by-default requests can use the cached artifact without host network traffic. -Image references without an explicit registry will resolve against Docker Hub. -Explicit registry references will be permitted only when the registry is -allowed by the machine policy. +`microvm.image` may reference OCI registries such as Docker Hub. The NVX image +tool will define how short image references are resolved. Before any network +access, MXC will identify the effective registry host and apply the same +machine policy to both implicit and explicitly named registries. The policy will be stored under -`HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mxc` and will use the existing WSLC -policy behaviour: +`HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mxc` with the following behaviour: | Policy state | Behaviour | | --- | --- | @@ -653,7 +698,7 @@ configuration for both one-shot and state-aware provision requests. | Contract surface | Schema addition | | --- | --- | | One-shot | Add `microvm.image` and `microvm.memoryMb` | -| State-aware provision | Register `containment: "microvm"` and add `microvm.provision.image` and `microvm.provision.memoryMb` | +| State-aware provision | Register `containment: "microvm"` and reuse `microvm.image` and `microvm.memoryMb` | | Start, exec, stop, and deprovision | No image fields; these requests use the `sandboxId` created during provision | `image` will be required and must contain a non-empty OCI image reference. @@ -670,17 +715,15 @@ One-shot standard-image addition: } ``` -State-aware provision will use the same fields under `microvm.provision`: +State-aware provision will use the same `microvm` structure: ```json { "phase": "provision", "containment": "microvm", "microvm": { - "provision": { - "image": "python:3.12-alpine", - "memoryMb": 256 - } + "image": "python:3.12-alpine", + "memoryMb": 256 } } ``` @@ -748,11 +791,11 @@ executor is not an NVX developer-facing surface. | Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | | State-aware process lifetime | Configure NVX with `OpenVmmConfig::breakaway_from_job = true` and return actionable `backend_unavailable` when the caller's Windows job disallows breakaway; from a kill-on-close Windows job, verify that OpenVMM remains running after the `start` process exits and that a later `exec` process reconnects successfully | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; Node registers the separately installed MicroVM runtime directory before discovery or execution and rejects conflicting registration; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | -| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, and invalid combinations | +| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, access to pre-existing host files, the Windows owner and inherited ACL of guest-created files, protected Windows root rejection, and canonical-path alias and reparse-point bypass attempts | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | | Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, initramfs, source manifest, Alpine package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and Alpine source artifacts are published and referenced | +| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, control initramfs, source manifest, control-initramfs package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and control-initramfs source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | | Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by offline cache use, permitted execution-time pull, the NVX-owned adversarial conversion-security suite, and MXC rejection of malformed or incompatible converter output | @@ -772,11 +815,11 @@ so infrastructure failures are not mistaken for policy enforcement. | Connection | Mechanism | Purpose | | --- | --- | --- | | MXC to NVX Rust interface | In-process Rust API calls | Will invoke provision, start, execute, stop, and deprovision | -| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation shipped in the NVX Rust crate | +| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation fetched and verified through the pinned NVX crate artifact contract | | NVX implementation DLL to `openvmm.exe` | Process launch with CLI arguments | Will supply the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | | NVX implementation DLL to `openvmm.exe`, during startup only | OpenVMM stdin | Will pass a one-time 32-byte authentication capability; stdin will not be the ongoing command channel | | NVX implementation DLL to `openvmm.exe` | Windows named pipe | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | -| OpenVMM to Alpine guest agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | +| OpenVMM to guest control agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | ## Appendix B: NVX execution outcome mapping @@ -798,7 +841,7 @@ so infrastructure failures are not mistaken for policy enforcement. | Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | | Registry authorisation | MXC validates the registry against a shared backend-neutral administrative allowlist before invoking the NVX image tool | | Execution-time host fetch | A cache miss under deny-by-default request egress is rejected without registry traffic; automatic pull is available only when the request permits egress | -| Explicit prefetch | A separate MXC host-setup operation pulls and converts the image under the machine registry policy without creating a sandbox, then stores it in the same runtime cache | +| Explicit prefetch | A separate MXC host-setup operation pulls and converts the image under the machine registry policy without creating an NVX instance, then stores it in the same runtime cache | | Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | | Credentials | No private-registry credentials in the initial contract; future credentials must come from an approved host provider and remain out of requests, command lines, logs, and telemetry | | Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | @@ -818,10 +861,10 @@ before `microvm.image` is usable. ### Awaited support -- Initial permitted registry set and migration from the existing WSLC-specific policy name +- Initial permitted registry set and final backend-neutral registry policy name - NVX-owned OCI converter threat model, containment and bounded-extraction contract, security review, and adversarial test evidence - NVX signed binaries support -- NVX crate binary packaging and extraction contract +- NVX crate artifact registry, fetching, verification, offline-prefetch, and staging contract - Windows ARM runtime and SDK package availability - Future PTY support From 30bca398d0d9066b5f656cf534a0cc38a816c8a1 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Thu, 8 Oct 2026 12:52:46 -0700 Subject: [PATCH 17/19] Clarify NVX packaging and runtime contracts Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 165 +++++++++++++++++++++++----------------- 1 file changed, 95 insertions(+), 70 deletions(-) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index a3abd498c..f23b53d5d 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -32,25 +32,25 @@ NVX will ship a Rust crate containing: - build logic that fetches and verifies the matching signed artifacts from NVX-controlled registries or repositories. -The fetched runtime artifacts will include the signed NVX implementation DLL, -signed OpenVMM executable, signed image download and conversion tool, Linux -kernel, minimal control initramfs with the NVX init agent, manifests, +The fetched host artifacts will include the signed NVX implementation DLL, +OpenVMM executable, and image download and conversion tool. The artifact set +will also include the Linux kernel, minimal control initramfs, manifests, checksums, provenance, licences, and package inventories. NVX already publishes the corresponding source for its Linux kernel and Alpine-based control initramfs. The NVX repository contains the owned guest and build inputs under `kernel/`, `guest/common/`, and `guest/alpine/`. Its -release tooling collects the exact patched kernel source and the recipes and -upstream sources for the minimal initramfs packages, then publishes them as separate -`source/nvx-linux-source-.tar.gz` and -`source/nvx-alpine-source-.tar.gz` artifacts. - -The existing `SOURCE-MANIFEST.json`, control-initramfs package inventory, -licences, and third-party notices identify the NVX project source, kernel -version and configuration, patches, Alpine package sources used by the -control initramfs, published source artifacts, and hashes. MXC will consume -and validate this existing source-delivery contract rather than requiring the -NVX team to create a new one. +release tooling collects the exact patched kernel source. It also collects the +recipes and upstream sources for the minimal initramfs packages. The release +publishes them as separate `source/nvx-linux-source-.tar.gz` +and `source/nvx-alpine-source-.tar.gz` artifacts. + +NVX records its source details in `SOURCE-MANIFEST.json` and the +control-initramfs package inventory. These files identify the project source, +kernel build, patches, initramfs package sources, published source artifacts, +and hashes. The release also includes the required licences and third-party +notices. MXC will validate and consume this existing contract rather than +require a new one. ### 2.2 How MXC will consume NVX @@ -66,8 +66,6 @@ MXC will be implementing: - a public Rust build helper that stages the runtime artifacts resolved and verified through the NVX crate beside the consuming executable; - checksum and signature verification; -- offline builds using a pre-fetched runtime artifact directory plus the - crate and its dependencies; and - packaging for the three SDKs and the internal MXC end-to-end executor bundle. Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json` and @@ -109,11 +107,11 @@ Developers will not separately install or invoke OpenVMM. | Ecosystem | Developer dependency | Packaging behaviour | | --- | --- | --- | -| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will fetch or use pre-fetched artifacts through the pinned NVX crate contract, verify them, and stage them beside that executable | +| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will fetch artifacts through the pinned NVX crate contract, verify them, and stage them beside that executable | | Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will contain only the NVX assets and its directory will be registered with the native layer | | .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will add the RID-specific `nvx` assets beside it | -#### Rust +#### 2.4.1 Rust ```toml [dependencies] @@ -131,33 +129,12 @@ fn main() { } ``` -#### Node +#### 2.4.2 npm ```bash npm install @microsoft/mxc-sdk @microsoft/mxc-nvx-runtime ``` -#### .NET - -```xml - - -``` - -The developer experience will be the same in all three ecosystems: - -1. Developers will add the MXC SDK and its NVX runtime dependency. Rust - applications will also invoke the staging helper from their build script. -2. They will select `microvm` in the MXC request. -3. The SDK will locate the packaged NVX runtime for the current platform. - Node will register the separately installed runtime package directory with - the native layer. -4. MXC will validate the signatures and checksums, then launch OpenVMM through - the NVX Rust integration. - -Developers will not provide paths to `openvmm.exe`, `vmlinux`, or -`initramfs.cpio.gz`, and will not communicate with OpenVMM directly. - The Node packages will use this layout: ```text @@ -195,6 +172,13 @@ package version, architecture, authenticated manifest, signatures, and checksums. Without the runtime package, Node will not register a MicroVM runtime directory and will report `microvm` as unavailable. +#### 2.4.3 NuGet + +```xml + + +``` + For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns `mxc_ffi.dll`. Its Windows native library will include the MicroVM integration but will report `microvm` as unavailable when the NVX assets are absent. @@ -204,10 +188,12 @@ directory and will declare an exact-version dependency on NuGet's default RID-native asset handling flattens files beneath `runtimes/{rid}/native`, so the runtime package will not place the NVX tree -there. It will package the assets under `runtimes/{rid}/nvx/**` and include a -`buildTransitive/Microsoft.Mxc.Sdk.Nvx.Runtime.targets` file. That target will -recursively copy the selected RID's complete `nvx` tree, preserving -`%(RecursiveDir)`, into both normal build and publish outputs: +there. The `Microsoft.Mxc.Sdk.Nvx.Runtime` NuGet package will store the assets +under `runtimes/{rid}/nvx/**`. It will also include +`buildTransitive/Microsoft.Mxc.Sdk.Nvx.Runtime.targets`, which NuGet +automatically imports into projects that reference the runtime package. That +MSBuild target will recursively copy the selected RID's complete `nvx` tree, +preserving `%(RecursiveDir)`, into both normal build and publish outputs: ```text \ @@ -225,6 +211,29 @@ The native MXC layer will resolve the `nvx` directory relative to the loaded `mxc_ffi.dll`. The copy target will reject unsupported RID and architecture combinations rather than producing a partial runtime. +The .NET one-shot `Microvm` containment type and +`MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` +roots in `MxcJsonContext`. Their serialization and deserialization will use +the existing `MxcJson` helpers without reflection fallback. +`Microsoft.Mxc.Sdk.AotSmokeTest` will serialize and deserialize representative +one-shot and state-aware MicroVM requests through that production JSON path. + +#### 2.4.4 Common packaging and validation + +The developer experience will be the same in all three ecosystems: + +1. Developers will add the MXC SDK and its NVX runtime dependency. Rust + applications will also invoke the staging helper from their build script. +2. They will select `microvm` in the MXC request. +3. The SDK will locate the packaged NVX runtime for the current platform. + Node will register the separately installed runtime package directory with + the native layer. +4. MXC will validate the signatures and checksums, then launch OpenVMM through + the NVX Rust integration. + +Developers will not provide paths to `openvmm.exe`, `vmlinux`, or +`initramfs.cpio.gz`, and will not communicate with OpenVMM directly. + The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. Before launch, MXC will verify the required files, Windows architecture, signatures, checksums, and runtime manifest compatibility. A missing or @@ -244,6 +253,8 @@ authenticate that manifest before using its file hashes. MXC will then: manifest mismatch, checksum mismatch, or runtime directory that is writable by an untrusted user. +#### 2.4.5 SDK API exposure + The integration will add a new typed MicroVM configuration to each SDK. This configuration describes which backend and OCI image to use. Developers will pass that configuration to the existing run, spawn, and lifecycle operations; @@ -255,13 +266,6 @@ the integration will not introduce separate NVX-specific execution methods. | Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Reuse the same `{ image, memoryMb? }` MicroVM configuration in `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | | .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` that reuses the same required `Image` and optional `MemoryMb` fields, plus `Filesystem` and `Network`; register both MicroVM request shapes and their `microvm` discriminators with the source-generated JSON context | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | -The .NET one-shot `Microvm` containment type and -`MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` -roots in `MxcJsonContext`. Their serialization and deserialization will use -the existing `MxcJson` helpers without reflection fallback. -`Microsoft.Mxc.Sdk.AotSmokeTest` will serialize and deserialize representative -one-shot and state-aware MicroVM requests through that production JSON path. - The wire and SDK changes will be delivered in this order: 1. Add `microvm.image`, `microvm.memoryMb`, state-aware MicroVM provision, @@ -361,8 +365,8 @@ not start a VM. Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will be linked on the Rust side; no separate NVX FFI library will be required. -The NVX developer contract covers the Rust, Node V1, and .NET SDKs. The Node -V1 run, spawn, PTY, and state-aware paths call `mxc_ffi` in-process. NVX will +The NVX developer contract covers the Rust, Node, and .NET SDKs. The Node run, +spawn, PTY, and state-aware paths call `mxc_ffi` in-process. NVX will not add support commitments for legacy Node executor paths or introduce a new NVX CLI or executor. The existing generic `wxc-exec` may be built with NVX support only as an internal end-to-end harness over the same `mxc_engine` @@ -427,10 +431,20 @@ Read-write mappings are live: a guest write changes the mapped host file immediately. Denied entries are hidden by OpenVMM within the exported host tree. -On Windows, host access remains governed by the Windows identity running -OpenVMM and the mapped directory's NTFS ACLs. NVX will define and guarantee -the Windows owner and inherited ACL applied when a guest creates a file; MXC -will verify that behaviour end to end before release. +Filesystem access is enforced at two layers: + +| Access control | Enforcer | +| --- | --- | +| Which host paths are exposed or hidden | NVX and OpenVMM | +| Whether an exposed path is read-only or read-write | The NVX guest control agent and OpenVMM | +| Whether the OpenVMM process may access a host file | Windows and NTFS, using OpenVMM's Windows process identity | + +If a path is not mapped, or is explicitly denied, NVX and OpenVMM prevent the +workload from reaching it even when the OpenVMM process could access it. If a +mapped path is denied to the OpenVMM process by its NTFS ACL, Windows still +rejects the operation. MXC validates the requested mappings and launches +OpenVMM under the intended Windows identity, but it does not perform each file +access check itself. All mapped paths must exist, be on the same Windows volume, and share a common directory below the volume root. MXC will translate Windows paths into guest @@ -451,6 +465,14 @@ will fail before OpenVMM starts and identify the protected root. | `filesystem.readwritePaths` | Yes | | `filesystem.deniedPaths` | Yes | +| Current NVX filesystem limit | Behaviour | +| --- | --- | +| Denied paths | At most 128 hidden paths | +| Hidden relative path syntax | Whitespace, `:`, `\`, and links are rejected | +| Nested Windows mappings | A read-only file inside a read-write directory is rejected | +| Mapping count and length | All bind-mount arguments must fit in the 1024-byte kernel-command-line budget, allowing roughly a dozen typical mappings | +| If any limit above is exceeded | NVX rejects the request before OpenVMM starts; no workload runs | + ### 4.3 Network @@ -610,10 +632,10 @@ filesystem format consumed by NVX. OCI-image-backed execution is a prerequisite for the MXC integration. The selected NVX Rust API does not currently accept an image or attach a converted -workload filesystem. The NVX team will provide the signed conversion tool and -extend the runtime contract so the converted image can be attached and used as -the workload root while the init agent remains in the initramfs. The required -conversion and runtime contract is listed in +workload filesystem. The NVX team will provide the signed conversion tool. It +will also extend the runtime contract so the converted image can be attached +as the workload root. The init agent will remain in the control initramfs. The +required conversion and runtime contract is listed in [Appendix C](#appendix-c-oci-image-conversion-contract). ```json @@ -780,25 +802,27 @@ identify the associated log path. ## 8. Requirements and end-to-end tests The NVX backend will be complete when the following areas pass through all -three SDKs on Windows x64 with WHP. MXC may additionally run the same cases -through the generic packaged executor as an internal end-to-end harness; that -executor is not an NVX developer-facing surface. +three SDKs on Windows x64 with WHP. MXC may additionally run applicable cases +through the generic packaged executor as an internal end-to-end harness. The +relayed executor path is excluded from state-aware timeout expectations +because it cannot represent a distinct timed-out result. The executor is not +an NVX developer-facing surface. | Area | Required coverage | | --- | --- | -| Integration | `microvm` routes to NVX for one-shot and state-aware execution through Rust, Node V1, and .NET; the generic executor is used only as an internal harness | +| Integration | `microvm` routes to NVX for one-shot and state-aware execution through Rust, Node, and .NET; the generic executor is used only as an internal harness | | Discovery | Add the native `unavailableReasons` payload and its Rust, Node, and .NET projections; include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; verify each failure mode returns actionable backend-specific remediation without starting a VM | | Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | | State-aware process lifetime | Configure NVX with `OpenVmmConfig::breakaway_from_job = true` and return actionable `backend_unavailable` when the caller's Windows job disallows breakaway; from a kill-on-close Windows job, verify that OpenVMM remains running after the `start` process exits and that a later `exec` process reconnects successfully | | SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; Node registers the separately installed MicroVM runtime directory before discovery or execution and rejects conflicting registration; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | -| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, access to pre-existing host files, the Windows owner and inherited ACL of guest-created files, protected Windows root rejection, and canonical-path alias and reparse-point bypass attempts | +| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, access to pre-existing host files, the Windows owner and inherited ACL of guest-created files, protected Windows root rejection, canonical-path alias and reparse-point bypass attempts, 128/129 denied-path boundaries, valid and invalid hidden-path syntax, read-only files nested under read-write directories, and mapping sets immediately below and above the kernel-command-line budget | | Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | -| Process | Command, CWD, environment, timeout, cancellation, output limits, nonzero exits, and descendant cleanup | +| Process | Command, CWD, environment, cancellation, output limits, nonzero exits, and descendant cleanup through all supported paths; timeout results through the three in-process SDK paths, excluding the generic executor's relayed state-aware path | | PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | | Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, control initramfs, source manifest, control-initramfs package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and control-initramfs source artifacts are published and referenced | | Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | | Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | -| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by offline cache use, permitted execution-time pull, the NVX-owned adversarial conversion-security suite, and MXC rejection of malformed or incompatible converter output | +| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by cache-only use, permitted execution-time pull, the NVX-owned adversarial conversion-security suite, and MXC rejection of malformed or incompatible converter output | Negative filesystem and network tests must include a working positive control so infrastructure failures are not mistaken for policy enforcement. @@ -828,7 +852,8 @@ so infrastructure failures are not mistaken for policy enforcement. | --- | --- | | `Exited(code)` | Will return the workload exit code | | `Signaled(signal)` | Will return `128 + signal` through the existing integer exit result | -| `TimedOut` | Will return the existing MXC timed-out result | +| `TimedOut` through an in-process Piped SDK path | Will return the existing MXC timed-out result | +| `TimedOut` through the internal executor's Relayed path | A distinct timeout is not supported by the relay and is excluded from timeout harness expectations; returning `TimedOut` currently produces `backend_error` as a relay contract violation | | `Cancelled` | Will return exit code `137` | | `Failed(WorkingDirectory)` | Will return `backend_error` with the working-directory failure | | Other `Failed(...)` outcomes | Will return `backend_error` with the NVX failure reason | @@ -864,7 +889,7 @@ before `microvm.image` is usable. - Initial permitted registry set and final backend-neutral registry policy name - NVX-owned OCI converter threat model, containment and bounded-extraction contract, security review, and adversarial test evidence - NVX signed binaries support -- NVX crate artifact registry, fetching, verification, offline-prefetch, and staging contract +- NVX crate artifact registry, fetching, verification, and staging contract - Windows ARM runtime and SDK package availability - Future PTY support From 196183c35b5d00f01d9460955ba5407046a937c1 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Thu, 8 Oct 2026 12:56:12 -0700 Subject: [PATCH 18/19] Document MXC to NVX call flow Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-integration.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index f23b53d5d..bcd3f649d 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -372,6 +372,42 @@ NVX CLI or executor. The existing generic `wxc-exec` may be built with NVX support only as an internal end-to-end harness over the same `mxc_engine` implementation. +#### 3.2.1 MXC request and NVX call flow + +The SDKs expose typed MXC requests. Node and .NET serialize those requests and +cross the existing `mxc_ffi` boundary. Rust passes typed requests through +`mxc-sdk` directly. Both paths reach the MicroVM adapter in `mxc_engine`. + +```mermaid +flowchart LR + Rust["Rust SDK"] --> RustSdk["mxc-sdk"] + Node["Node SDK"] --> FFI["mxc_ffi"] + DotNet[".NET SDK"] --> FFI + FFI --> Parser["Exact contract parsing and binding"] + Parser --> Engine["mxc_engine MicroVM adapter"] + RustSdk --> Engine + Engine --> NVX["NVX Rust interface"] +``` + +The adapter converts the MXC MicroVM, filesystem, network, and process fields +into the phase-specific NVX request types. It then calls the NVX Rust surface +as follows: + +| MXC operation | Input reaching the adapter | NVX Rust calls | +| --- | --- | --- | +| Platform discovery | Registered runtime directory and packaged runtime metadata | `probe` | +| One-shot `run` or `spawn` | MicroVM image and memory, filesystem policy, network policy, and process request | `validate_provision` → `provision` → `start` → `validate_exec` → `exec`; the one-shot owner later waits or cancels, then calls `stop` and `deprovision` | +| State-aware provision | MicroVM image and memory, filesystem policy, and network policy | `validate_provision` → `provision` | +| State-aware start | NVX instance ID | `start` | +| State-aware execution | NVX instance ID and process request | `validate_exec` → `exec`; the SDK waits through `Execution::wait` or `Execution::wait_with_output` | +| Execution cancellation | `Execution` handle | `Execution::canceller` → `Canceller::cancel` | +| State-aware stop | NVX instance ID | `stop` | +| State-aware deprovision | NVX instance ID | `deprovision` | + +The adapter maps NVX lifecycle results, execution outcomes, and errors back +to the existing MXC SDK result and error types. Appendix B defines the +execution-outcome mapping. + ## 4. Filesystem, lifecycle, and network support The integration will target the MXC `1.1.0-alpha` development schema. From 0e73264129e8ec169ef167f5ff93f6669f148632 Mon Sep 17 00:00:00 2001 From: Huzaifa Danish Date: Fri, 9 Oct 2026 11:44:25 -0700 Subject: [PATCH 19/19] Refine NVX integration documentation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/nvx-dotnet-examples.md | 240 ++++++ docs/nvx-integration.md | 1492 ++++++++++++++++++++--------------- docs/nvx-npm-examples.md | 222 ++++++ 3 files changed, 1333 insertions(+), 621 deletions(-) create mode 100644 docs/nvx-dotnet-examples.md create mode 100644 docs/nvx-npm-examples.md diff --git a/docs/nvx-dotnet-examples.md b/docs/nvx-dotnet-examples.md new file mode 100644 index 000000000..b59e1b9a6 --- /dev/null +++ b/docs/nvx-dotnet-examples.md @@ -0,0 +1,240 @@ +# NVX .NET examples + +> [!NOTE] +> These examples describe the proposed completed NVX-backed MicroVM SDK +> surface. The MicroVM types are not available in the current NuGet package. + +For architecture, packaging, policy limits, and implementation details, see +[NVX integration in MXC](./nvx-integration.md). + +For npm examples, see +[NVX npm examples](./nvx-npm-examples.md). + +## Install the packages + +Reference matching SDK and runtime package versions: + +```xml + + +``` + +## Check MicroVM availability + +```csharp +using Microsoft.Mxc.Sdk.V1; + +PlatformSupport support = MxcPlatform.GetPlatformSupport(); +if (!support.AvailableMethods.Contains(ContainmentBackend.Microvm)) +{ + string reason = support.UnavailableReasons.TryGetValue( + ContainmentBackend.Microvm, + out string? unavailable) + ? unavailable + : "MicroVM is unavailable"; + + throw new InvalidOperationException(reason); +} +``` + +## Run and capture output + +```csharp +using Microsoft.Mxc.Sdk.V1; + +var request = new ContainerRequest( + "python -c \"print('hello from NVX')\"") +{ + TimeoutMs = 30_000, + Containment = new Containment.Microvm + { + Image = "python:3.12-alpine", + MemoryMb = 256, + }, +}; + +ExecutionResult result = await MxcContainer.RunAsync(request); +Console.Write(result.Stdout); +Console.Error.Write(result.Stderr); +Console.WriteLine($"exit={result.ExitCode} timedOut={result.TimedOut}"); +``` + +Cancelling the `RunAsync` await does not necessarily stop native execution. +Use a request timeout or a live process handle when the application must stop +the workload. + +## Spawn with live output + +```csharp +using Microsoft.Mxc.Sdk.V1; + +var request = new ContainerRequest( + "python -u -c \"import sys,time; print('out'); " + + "print('err', file=sys.stderr); time.sleep(1)\"") +{ + TimeoutMs = 30_000, + Containment = new Containment.Microvm + { + Image = "python:3.12-alpine", + MemoryMb = 256, + }, +}; + +using var process = await MxcContainer.SpawnAsync(request); +Task stdout = process.StandardOutput is { } output + ? output.CopyToAsync(Console.OpenStandardOutput()) + : Task.CompletedTask; +Task stderr = process.StandardError is { } error + ? error.CopyToAsync(Console.OpenStandardError()) + : Task.CompletedTask; + +try +{ + WaitResult result = await process.WaitAsync(); + await Task.WhenAll(stdout, stderr); + Console.WriteLine($"exit={result.ExitCode} timedOut={result.TimedOut}"); +} +catch +{ + process.Kill(); + throw; +} +``` + +Consume stdout and stderr concurrently. Call `Kill()` when the application +must stop the workload. + +## Reuse an NVX instance + +The state-aware lifecycle is: + +```text +provision → start → repeated execution → stop → deprovision +``` + +```csharp +using Microsoft.Mxc.Sdk.V1; + +var provisioned = MxcLifecycle.ProvisionContainer( + new MicrovmProvisionRequest + { + Image = "python:3.12-alpine", + MemoryMb = 256, + }); +ContainerId id = provisioned.ContainerId; + +Exception? primaryError = null; +var cleanupErrors = new List(); +bool startAttempted = false; + +try +{ + startAttempted = true; + MxcLifecycle.StartContainer(id); + + foreach (string value in new[] { "first", "second" }) + { + ExecutionResult result = await MxcLifecycle.RunInContainerAsync( + id, + new ExecutionRequest($"echo {value}")); + Console.WriteLine(result.Stdout); + } +} +catch (Exception error) +{ + primaryError = error; +} +finally +{ + if (startAttempted) + { + try + { + MxcLifecycle.StopContainer(id); + } + catch (Exception error) + { + cleanupErrors.Add(error); + } + } + + try + { + MxcLifecycle.DeprovisionContainer(id); + } + catch (Exception error) + { + cleanupErrors.Add(error); + } +} + +if (primaryError is not null || cleanupErrors.Count != 0) +{ + var errors = new List(cleanupErrors); + if (primaryError is not null) + { + errors.Insert(0, primaryError); + } + throw new AggregateException($"NVX lifecycle failed for {id}", errors); +} +``` + +Keep `ContainerId` unchanged. If cleanup fails, keep the ID in application +logs so an operator can retry cleanup. + +## Filesystem and network policy + +```json +{ + "filesystem": { + "readonlyPaths": ["C:\\nvx-work\\input"], + "readwritePaths": ["C:\\nvx-work\\output"], + "deniedPaths": ["C:\\nvx-work\\input\\private"] + }, + "network": { + "egress": { + "default": "deny", + "allow": [{ + "to": [{ "cidr": "203.0.113.0/24" }], + "ports": [{ "protocol": "tcp", "port": 443 }] + }] + }, + "ingress": { + "default": "deny", + "hostLoopback": "deny" + } + } +} +``` + +Set `Filesystem` and `Network` on the one-shot request or the provision +request. Host paths must exist before launch. + +Inside the Linux workload: + +```text +C:\nvx-work\input → /mnt/c/nvx-work/input +C:\nvx-work\output → /mnt/c/nvx-work/output +``` + +Use Linux guest paths in commands and `WorkingDirectory`. Read-write mappings +modify host files immediately. + +## PTY and live stdin + +PTY support is not available initially. `MxcContainer.SpawnWithPty` and +`MxcLifecycle.SpawnInContainerWithPty` reject MicroVM requests. + +Use `MxcContainer.Spawn` or `MxcLifecycle.SpawnInContainer` for +non-interactive workloads. These operations provide stdout and stderr pipes. +They do not provide a terminal. Live stdin is not available initially, and +the workload receives EOF. + +## Errors + +| Result | Meaning | +| --- | --- | +| `BackendUnavailable` | The NVX runtime, WHP, runtime files, architecture, or runtime version is unavailable | +| Policy validation error | The request uses a policy form or value that NVX cannot enforce | +| Nonzero workload exit | The workload ran and returned a nonzero status | + diff --git a/docs/nvx-integration.md b/docs/nvx-integration.md index bcd3f649d..c5df1038c 100644 --- a/docs/nvx-integration.md +++ b/docs/nvx-integration.md @@ -2,8 +2,7 @@ **Status:** Integration proposal. -Future -tense describes work proposed for the MXC integration. +Future tense describes work proposed for the MXC integration. ## 1. Introduction @@ -23,110 +22,84 @@ Nanvix as the backend for the `microvm` option. ## 2. NVX distribution and MXC consumption -### 2.1 NVX shipping +### 2.1 NVX crate and runtime files -NVX will ship a Rust crate containing: +NVX currently provides the `aci_edge_sandboxes` Rust crate. It contains: -- the Rust host API; -- pinned runtime artifact identities, versions, locations, and digests; and -- build logic that fetches and verifies the matching signed artifacts from - NVX-controlled registries or repositories. +- the public `AciEdgeSandbox` and `AsyncAciEdgeSandbox` lifecycle APIs +- the `OpenVmmBackend` Rust implementation +- `artifacts.json`, which pins the platform release +- the optional `bundled` Cargo feature +- the download logic in `build.rs` -The fetched host artifacts will include the signed NVX implementation DLL, -OpenVMM executable, and image download and conversion tool. The artifact set -will also include the Linux kernel, minimal control initramfs, manifests, -checksums, provenance, licences, and package inventories. +With `bundled` enabled, `build.rs` downloads the pinned NVX platform archive +and stages: -NVX already publishes the corresponding source for its Linux kernel and -Alpine-based control initramfs. The NVX repository contains the owned guest -and build inputs under `kernel/`, `guest/common/`, and `guest/alpine/`. Its -release tooling collects the exact patched kernel source. It also collects the -recipes and upstream sources for the minimal initramfs packages. The release -publishes them as separate `source/nvx-linux-source-.tar.gz` -and `source/nvx-alpine-source-.tar.gz` artifacts. +- `openvmm.exe` +- `vmlinux` +- `initramfs.cpio.gz` +- `SOURCE-MANIFEST.json` +- `SHA256SUMS` -NVX records its source details in `SOURCE-MANIFEST.json` and the -control-initramfs package inventory. These files identify the project source, -kernel build, patches, initramfs package sources, published source artifacts, -and hashes. The release also includes the required licences and third-party -notices. MXC will validate and consume this existing contract rather than -require a new one. +[Appendix D.1](#d1-archive-and-source-validation) describes archive validation +and corresponding-source delivery. ### 2.2 How MXC will consume NVX -MXC will use a pinned NVX crate version. The crate will expose the Rust -interface and resolve its matching signed runtime artifacts. The build script -for each consuming executable will invoke a shared MXC build helper, which -uses the crate's fetch and verification contract before staging those -artifacts beside the executable. The crate's pinned artifact manifest will -keep the API and runtime files versioned together. - -MXC will be implementing: - -- a public Rust build helper that stages the runtime artifacts resolved and - verified through the NVX crate beside the consuming executable; -- checksum and signature verification; -- packaging for the three SDKs and the internal MXC end-to-end executor bundle. - -Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json` and -verify that it references the matching Linux kernel and minimal control -initramfs corresponding-source artifacts published by the NVX team. MXC will -copy the source manifest, control-initramfs package inventory, licences, and -notices into the npm and NuGet runtime packages. The published -`nvx-alpine-source-*` archive covers only the Alpine packages included in that -minimal trusted control environment. MXC will not generate the source -archives. Packaging will fail when required metadata, source references, or -hashes are missing or do not match the runtime version. +MXC will pin a specific `aci_edge_sandboxes` revision. The NVX runner in +`mxc_engine` will call `AciEdgeSandbox` directly. Enabling the crate's +`bundled` feature will run the NVX `build.rs` download and checksum checks +during the Cargo build. + +The final executable or SDK package cannot depend on a Cargo dependency's +`OUT_DIR`. Its build or packaging step must copy the staged `nvx` directory +from `DEP_ACI_EDGE_SANDBOXES_ARTIFACTS_DIR` into the final output. + +MXC will add: + +- `mxc_build_common::stage_nvx_runtime`, following the existing build-helper + pattern, to copy the crate's staged files beside a Rust application +- npm and NuGet packaging for the same files +- Authenticode verification when NVX begins publishing signed Windows + executables +- packaging for the three SDKs and the internal MXC end-to-end executor bundle + +[Appendix D](#appendix-d-runtime-packaging-and-validation-details) describes +the package validation and source-file requirements. ### 2.3 Key NVX files that MXC will use | File | Approximate size | Contents | | --- | ---: | --- | -| NVX implementation DLL | Not yet published | Will provide the signed implementation of the Rust crate interface | | `openvmm.exe` | 22 MB | Windows OpenVMM executable | | Image download and conversion tool | Not yet published | Will download and convert standard OCI images | | `vmlinux` | 24 MB | NVX Linux kernel | -| `initramfs.cpio.gz` | Not yet published | Minimal trusted control userspace and NVX guest agent; customer workload tooling is supplied by the OCI image | +| `initramfs.cpio.gz` | Not yet published | Contains the trusted control userspace and NVX guest agent. The OCI image supplies the workload tools. | -The release also includes the supporting checksum, manifest, provenance, -licence, and package-inventory files. The numeric OpenVMM and kernel sizes are -from the selected development baseline and can change. The new implementation -DLL, image tool, and minimal control initramfs sizes have not yet been -published. These artifacts contribute to the staged runtime and SDK package -footprint rather than the published Rust crate archive size. +The current NVX release is checksum-verified but not Authenticode-signed. MXC +will continue to verify the `artifacts.json` archive digest and `SHA256SUMS`. +When NVX publishes signed Windows executables, MXC will also verify their +Authenticode signer before use. -The current NVX binaries are not signed yet. The production Rust crate will -fetch the required signed implementation DLL, OpenVMM executable, image tool, -and guest artifacts from pinned NVX-controlled locations. MXC will validate -their signatures and published checksums before staging or using the files. +[Appendix D.3](#d3-release-files-and-future-dll) contains more release-file +details and the open DLL design question. ### 2.4 Developer packaging and usage The NVX runtime will be distributed with the SDK for each ecosystem. Developers will not separately install or invoke OpenVMM. -| Ecosystem | Developer dependency | Packaging behaviour | -| --- | --- | --- | -| Rust | `mxc-sdk` with the `microvm` feature and the NVX staging build helper | The consuming executable's build script will fetch artifacts through the pinned NVX crate contract, verify them, and stage them beside that executable | -| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will contain only the NVX assets and its directory will be registered with the native layer | -| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | The base SDK will be the sole owner of `mxc_ffi.dll`; the exact-version runtime package will add the RID-specific `nvx` assets beside it | +| Ecosystem | Developer dependency | +| --- | --- | +| Rust | `mxc-sdk` with the `microvm` feature | +| Node | `@microsoft/mxc-sdk` and `@microsoft/mxc-nvx-runtime` | +| .NET | `Microsoft.Mxc.Sdk` and `Microsoft.Mxc.Sdk.Nvx.Runtime` | #### 2.4.1 Rust ```toml [dependencies] mxc-sdk = { version = "...", features = ["microvm"] } - -[build-dependencies] -mxc-sdk-build = { version = "...", features = ["nvx-runtime"] } -``` - -```rust -// build.rs -fn main() { - mxc_sdk_build::stage_nvx_runtime() - .expect("failed to stage the NVX runtime"); -} ``` #### 2.4.2 npm @@ -135,42 +108,9 @@ fn main() { npm install @microsoft/mxc-sdk @microsoft/mxc-nvx-runtime ``` -The Node packages will use this layout: - -```text -@microsoft/mxc-sdk\ - bin\win-x64\ - mxc_ffi.dll - -@microsoft/mxc-nvx-runtime\ - bin\win-x64\ - nvx\ - .dll - openvmm.exe - vmlinux - initramfs.cpio.gz - - -``` - -The Node SDK native-library resolver will check whether -`@microsoft/mxc-nvx-runtime` is installed. `@microsoft/mxc-sdk` will remain the -only npm package that owns and loads `mxc_ffi.dll`. When the runtime package is -present, the Node loader will resolve its absolute platform directory and call -the backend-neutral native registration API before platform discovery or -execution: - -```text -mxc_register_backend_runtime_directory("microvm", absolutePath) -``` - -Registration will be process-scoped, accept only registered backend names and -absolute canonical paths, and be idempotent for the same backend and path. A -conflicting second registration will fail. Registration locates the package; -it does not trust it. The native probe will still validate the expected -package version, architecture, authenticated manifest, signatures, and -checksums. Without the runtime package, Node will not register a MicroVM -runtime directory and will report `microvm` as unavailable. +The runtime package supplies the NVX files. The base SDK owns +`mxc_ffi.dll`. [Appendix D.4](#d4-sdk-runtime-packaging) describes the package +layout and registration flow. #### 2.4.3 NuGet @@ -180,43 +120,9 @@ runtime directory and will report `microvm` as unavailable. ``` For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns -`mxc_ffi.dll`. Its Windows native library will include the MicroVM integration -but will report `microvm` as unavailable when the NVX assets are absent. -`Microsoft.Mxc.Sdk.Nvx.Runtime` will contain only the RID-specific `nvx` -directory and will declare an exact-version dependency on -`Microsoft.Mxc.Sdk`. - -NuGet's default RID-native asset handling flattens files beneath -`runtimes/{rid}/native`, so the runtime package will not place the NVX tree -there. The `Microsoft.Mxc.Sdk.Nvx.Runtime` NuGet package will store the assets -under `runtimes/{rid}/nvx/**`. It will also include -`buildTransitive/Microsoft.Mxc.Sdk.Nvx.Runtime.targets`, which NuGet -automatically imports into projects that reference the runtime package. That -MSBuild target will recursively copy the selected RID's complete `nvx` tree, -preserving `%(RecursiveDir)`, into both normal build and publish outputs: - -```text -\ - mxc_ffi.dll - nvx\ - .dll - openvmm.exe - vmlinux - initramfs.cpio.gz - - -``` - -The native MXC layer will resolve the `nvx` directory relative to the loaded -`mxc_ffi.dll`. The copy target will reject unsupported RID and architecture -combinations rather than producing a partial runtime. - -The .NET one-shot `Microvm` containment type and -`MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` -roots in `MxcJsonContext`. Their serialization and deserialization will use -the existing `MxcJson` helpers without reflection fallback. -`Microsoft.Mxc.Sdk.AotSmokeTest` will serialize and deserialize representative -one-shot and state-aware MicroVM requests through that production JSON path. +`mxc_ffi.dll`. The runtime package supplies the matching NVX files. +[Appendix D.4](#d4-sdk-runtime-packaging) describes the NuGet copy and +validation requirements. #### 2.4.4 Common packaging and validation @@ -227,186 +133,206 @@ The developer experience will be the same in all three ecosystems: 2. They will select `microvm` in the MXC request. 3. The SDK will locate the packaged NVX runtime for the current platform. Node will register the separately installed runtime package directory with - the native layer. + `mxc_ffi`. 4. MXC will validate the signatures and checksums, then launch OpenVMM through the NVX Rust integration. Developers will not provide paths to `openvmm.exe`, `vmlinux`, or `initramfs.cpio.gz`, and will not communicate with OpenVMM directly. -The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. -Before launch, MXC will verify the required files, Windows architecture, -signatures, checksums, and runtime manifest compatibility. A missing or -incompatible runtime will return an actionable load or `backend_unavailable` -error identifying the required runtime package or version. - -The co-located checksum manifest will not be trusted by itself. The pinned NVX -crate will provide the expected artifact locations, digests, and -runtime-manifest identity, and the compiled native integration will -authenticate that manifest before using its file hashes. MXC will then: - -1. validate the Authenticode certificate chain and expected Microsoft signer - for the NVX implementation DLL, `openvmm.exe`, and image tool; -2. verify `vmlinux`, the initramfs, and all other runtime files against the - authenticated manifest; and -3. reject a missing signature, invalid certificate chain, unexpected signer, - manifest mismatch, checksum mismatch, or runtime directory that is writable - by an untrusted user. +[Appendix D.6](#d6-package-compatibility-and-runtime-file-validation) defines +version matching, signature checks, and checksum checks. #### 2.4.5 SDK API exposure -The integration will add a new typed MicroVM configuration to each SDK. This +MXC will add a typed MicroVM configuration to each SDK. This configuration describes which backend and OCI image to use. Developers will -pass that configuration to the existing run, spawn, and lifecycle operations; -the integration will not introduce separate NVX-specific execution methods. +pass that configuration to the existing run, spawn, and lifecycle operations. +MXC will not add separate NVX-specific execution methods. -| SDK | One-shot configuration | State-aware provision addition | Existing operations that will use it | -| --- | --- | --- | --- | -| Rust | `MicrovmConfig { image: String, memory_mb: Option }` and `Containment::Microvm(MicrovmConfig)` | Reuse `MicrovmConfig` in the state-aware `ProvisionRequest` | `v1::run`, `v1::spawn`, and `v1::container::*` | -| Node | `{ type: 'microvm', config: { image: string; memoryMb?: number } }` | Reuse the same `{ image, memoryMb? }` MicroVM configuration in `LifecycleConfigRegistry`, and include `'microvm'` in `LifecycleContainmentKind` | `run`, `spawn`, `provisionContainer`, and the existing lifecycle functions | -| .NET | `Microvm : Containment` with required `Image` and optional `MemoryMb` | Add `MicrovmProvisionRequest : ProvisionRequest` that reuses the same required `Image` and optional `MemoryMb` fields, plus `Filesystem` and `Network`; register both MicroVM request shapes and their `microvm` discriminators with the source-generated JSON context | `MxcContainer.Run`, `MxcContainer.Spawn`, and the existing `MxcLifecycle` methods | - -The wire and SDK changes will be delivered in this order: - -1. Add `microvm.image`, `microvm.memoryMb`, state-aware MicroVM provision, - engine binding, and runtime tests to the development `1.1.0-alpha` - contract. -2. Keep the wire work development-only while that contract remains mutable. -3. If NVX is ready when `1.1.0` is frozen, publish the fields in stable - `1.1.0`, advance `schemas/schema-version.json` `sdkMajorTargets["1"]` from - `1.0.0` to `1.1.0`, and publish the new Rust, Node, and .NET V1 MicroVM - types and versioned references. The stable API will not require an - experimental option. -4. If NVX is not ready, stable `1.1.0` will omit the MicroVM fields. Move them - into the next mutable contract, such as `1.2.0-alpha`, and do not publish - the V1 MicroVM SDK types until a stable contract contains those fields. - -Contract publication and host availability remain separate. A published -MicroVM backend can still return `backend_unavailable` when the host lacks the -NVX runtime, WHP, or another required capability. - -The existing lifecycle operation methods will remain unchanged; only the -backend-specific request types they accept will expand. - -## 3. Architecture - -### 3.1 NVX, OpenVMM, and the guest control environment - -MXC will use the NVX Rust interface and its signed DLL implementation. The DLL -will launch OpenVMM, which will boot the NVX kernel and unpack the minimal -control initramfs into the VM's in-memory control root filesystem. +The language-specific examples are in these locations: -The initramfs contains only the binaries and libraries required to initialise -and control the microVM, including these two components: - -| Guest path | Role | -| --- | --- | -| `/init` | Runs first, mounts the guest pseudo-filesystems, reads the kernel command line, prepares networking and host filesystem mappings, and resolves the workload identity | -| `/sbin/nvx-managed-agent` | Replaces `/init` for the managed lifecycle and remains running as PID 1 while workloads execute | +- [npm examples](./nvx-npm-examples.md) +- [.NET examples](./nvx-dotnet-examples.md) -The managed agent remains in the control initramfs and outside the OCI -workload root filesystem. Workload processes see the converted OCI filesystem -as `/`; they do not see or execute `/sbin/nvx-managed-agent`, which remains -PID 1 and handles lifecycle and process-control messages. +##### Rust availability check -```mermaid -flowchart LR - MXC["MXC"] --> API["NVX Rust interface"] - API --> DLL["Signed NVX implementation DLL"] - DLL --> OpenVMM["Signed OpenVMM executable"] - OpenVMM --> Init["Control initramfs: /init"] - Init --> Agent["/sbin/nvx-managed-agent (PID 1)"] - Agent --> Workload["OCI workload environment"] +```rust +let support = mxc_sdk::v1::platform_support(); +if !support + .available_methods + .iter() + .any(|method| method == "microvm") +{ + let reason = support + .unavailable_reasons + .get("microvm") + .map(String::as_str) + .unwrap_or("MicroVM is unavailable"); + return Err(reason.into()); +} ``` -The NVX Rust interface exposes the following operations: +##### Rust `run` -| Operation | NVX Rust API | Input structure | Output structure | -| --- | --- | --- | --- | -| Validate provision | `validate_provision` | `&ProvisionRequest { filesystem?, network?, microvm.provision.memoryMib? }` | `Result<()>` | -| Validate execution | `validate_exec` | `&ExecRequest { process { commandLine or argv, cwd?, env?, inheritDefaultEnv?, timeout? }, stdin }` | `Result<()>` | -| Check runtime availability | `probe` | None | `Result<()>` | -| Provision VM | `provision` | `&ProvisionRequest` | `Result` | -| Start VM | `start` | `&SandboxId` | `Result` | -| Execute workload | `exec` | `&SandboxId`, `&ExecRequest` | `Result` with live stdout and stderr pipes, optional stdin pipe, cancellation handle, and wait methods | -| Stop VM | `stop` | `&SandboxId` | `Result` | -| Remove provisioned state | `deprovision` | `&SandboxId` | `Result`; the ID becomes stale | -| Wait for execution outcome | `Execution::wait` | Owned `Execution` handle | `Result`: `Exited(code)`, `Signaled(signal)`, `TimedOut`, `Cancelled`, or `Failed(reason)` | -| Wait and collect output | `Execution::wait_with_output` | Owned `Execution` handle | `Result, stderr: Vec }>` | -| Get cancellation handle | `Execution::canceller` | `&Execution` | `Canceller` | -| Cancel execution | `Canceller::cancel` | `&Canceller` | `Result<()>` | +```rust +use mxc_sdk::v1::{ + self, + configs::MicrovmConfig, + ContainerRequest, + Containment, + WaitResult, +}; + +fn main() -> Result<(), Box> { + let request = ContainerRequest { + containment: Containment::Microvm(MicrovmConfig { + image: "python:3.12-alpine".into(), + memory_mb: Some(256), + }), + timeout_ms: Some(30_000), + ..ContainerRequest::new("python -c \"print('hello from NVX')\"") + }; + + let result = v1::run(request, Default::default())?; + print!("{}", String::from_utf8_lossy(&result.stdout)); + eprint!("{}", String::from_utf8_lossy(&result.stderr)); + + match result.outcome { + WaitResult::Exited(code) => println!("exit={code}"), + WaitResult::TimedOut => println!("timed out"), + } -`ProvisionRequest` in the selected NVX API does not yet contain an OCI image. -It also currently nests memory under `microvm.provision`. The MXC wire -contract will use the shared flat `microvm { image, memoryMb }` structure for -one-shot and state-aware provision requests; the integration adapter will map -that structure into the NVX Rust input. The image integration will extend the -NVX provision input with the resolved OCI image or converted-artifact identity -described in section 6 and Appendix C. + Ok(()) +} +``` -### 3.2 MXC integration +##### Rust `spawn` -The integration will keep `containment: "microvm"` and will route it to NVX -through `mxc_engine`. +```rust +use std::io::Read; +use std::thread; + +use mxc_sdk::v1::{ + self, + configs::MicrovmConfig, + ContainerRequest, + Containment, +}; + +fn main() -> Result<(), Box> { + let request = ContainerRequest { + containment: Containment::Microvm(MicrovmConfig { + image: "python:3.12-alpine".into(), + memory_mb: Some(256), + }), + timeout_ms: Some(30_000), + ..ContainerRequest::new(concat!( + "python -u -c \"import sys,time; print('out'); ", + "print('err', file=sys.stderr); time.sleep(1)\"", + )) + }; + + let mut process = v1::spawn(request, Default::default())?; + let mut stdout = process.take_stdout().expect("stdout is piped"); + let mut stderr = process.take_stderr().expect("stderr is piped"); + + let stdout_reader = thread::spawn(move || { + let mut bytes = Vec::new(); + stdout.read_to_end(&mut bytes).map(|_| bytes) + }); + let stderr_reader = thread::spawn(move || { + let mut bytes = Vec::new(); + stderr.read_to_end(&mut bytes).map(|_| bytes) + }); + + let outcome = process.wait()?; + let stdout = stdout_reader.join().expect("stdout reader panicked")?; + let stderr = stderr_reader.join().expect("stderr reader panicked")?; + + print!("{}", String::from_utf8_lossy(&stdout)); + eprint!("{}", String::from_utf8_lossy(&stderr)); + println!("{outcome:?}"); + Ok(()) +} +``` -`mxc_engine` will also integrate the existing NVX `probe()` operation into -platform discovery. The NVX probe checks that the host platform, OpenVMM, -kernel, initramfs, and configured hypervisor are available. MXC will wrap that -probe with its production-package checks for the matched runtime manifest, -required files, architecture, signatures, checksums, image tool, and -guest/runtime compatibility. This discovery path will be read-only and will -not start a VM. +Call `process.kill()` when the application must stop the workload. -| Surface | New NVX integration work | -| --- | --- | -| Rust SDK | Will enable the NVX backend in the SDK and engine build | -| .NET SDK | Will add the `microvm` choice through the existing SDK-owned `mxc_ffi` boundary; the separate exact-version runtime package will contribute only the `nvx` assets | -| Node SDK | Will add the `microvm` choice to typed configuration, keep `mxc_ffi.dll` in the base SDK, resolve the exact-version NVX runtime package, and register its directory through the backend-neutral native API | +##### Rust state-aware lifecycle -Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will -be linked on the Rust side; no separate NVX FFI library will be required. -The NVX developer contract covers the Rust, Node, and .NET SDKs. The Node run, -spawn, PTY, and state-aware paths call `mxc_ffi` in-process. NVX will -not add support commitments for legacy Node executor paths or introduce a new -NVX CLI or executor. The existing generic `wxc-exec` may be built with NVX -support only as an internal end-to-end harness over the same `mxc_engine` -implementation. +```rust +use mxc_sdk::v1::{ + self, + configs::MicrovmConfig, + ExecutionRequest, + ProvisionRequest, +}; + +fn main() -> Result<(), Box> { + let provisioned = v1::container::provision_container( + ProvisionRequest::microvm(MicrovmConfig { + image: "python:3.12-alpine".into(), + memory_mb: Some(256), + }), + Default::default(), + )?; + let id = provisioned.container_id; + let mut start_attempted = false; + let mut primary_error: Option> = None; + + if let Err(error) = (|| -> Result<(), Box> { + start_attempted = true; + v1::container::start_container(&id, Default::default())?; + + for value in ["first", "second"] { + let result = v1::container::run_in_container( + &id, + ExecutionRequest::new(format!("echo {value}")), + Default::default(), + )?; + println!("{}", String::from_utf8_lossy(&result.stdout)); + } + Ok(()) + })() { + primary_error = Some(error); + } -#### 3.2.1 MXC request and NVX call flow + if start_attempted { + if let Err(error) = + v1::container::stop_container(&id, Default::default()) + { + eprintln!("stop failed for {id}: {error}"); + primary_error.get_or_insert_with(|| Box::new(error)); + } + } -The SDKs expose typed MXC requests. Node and .NET serialize those requests and -cross the existing `mxc_ffi` boundary. Rust passes typed requests through -`mxc-sdk` directly. Both paths reach the MicroVM adapter in `mxc_engine`. + if let Err(error) = + v1::container::deprovision_container(&id, Default::default()) + { + eprintln!("deprovision failed for {id}: {error}"); + primary_error.get_or_insert_with(|| Box::new(error)); + } -```mermaid -flowchart LR - Rust["Rust SDK"] --> RustSdk["mxc-sdk"] - Node["Node SDK"] --> FFI["mxc_ffi"] - DotNet[".NET SDK"] --> FFI - FFI --> Parser["Exact contract parsing and binding"] - Parser --> Engine["mxc_engine MicroVM adapter"] - RustSdk --> Engine - Engine --> NVX["NVX Rust interface"] + if let Some(error) = primary_error { + return Err(error); + } + Ok(()) +} ``` -The adapter converts the MXC MicroVM, filesystem, network, and process fields -into the phase-specific NVX request types. It then calls the NVX Rust surface -as follows: +[Appendix F.3](#f3-contract-and-sdk-publication) describes contract and SDK +publication. -| MXC operation | Input reaching the adapter | NVX Rust calls | -| --- | --- | --- | -| Platform discovery | Registered runtime directory and packaged runtime metadata | `probe` | -| One-shot `run` or `spawn` | MicroVM image and memory, filesystem policy, network policy, and process request | `validate_provision` → `provision` → `start` → `validate_exec` → `exec`; the one-shot owner later waits or cancels, then calls `stop` and `deprovision` | -| State-aware provision | MicroVM image and memory, filesystem policy, and network policy | `validate_provision` → `provision` | -| State-aware start | NVX instance ID | `start` | -| State-aware execution | NVX instance ID and process request | `validate_exec` → `exec`; the SDK waits through `Execution::wait` or `Execution::wait_with_output` | -| Execution cancellation | `Execution` handle | `Execution::canceller` → `Canceller::cancel` | -| State-aware stop | NVX instance ID | `stop` | -| State-aware deprovision | NVX instance ID | `deprovision` | +## 3. Architecture -The adapter maps NVX lifecycle results, execution outcomes, and errors back -to the existing MXC SDK result and error types. Appendix B defines the -execution-outcome mapping. +### 3.1 MXC integration + +`mxc_engine` will resolve `containment: "microvm"` to the NVX runner. +[Appendix E.2](#e2-mxc-integration) describes the runner interfaces, SDK +paths, and NVX calls. +[Appendix A](#appendix-a-nvx-openvmm-and-guest-architecture) describes the +NVX, OpenVMM, and guest architecture. ## 4. Filesystem, lifecycle, and network support @@ -414,86 +340,13 @@ The integration will target the MXC `1.1.0-alpha` development schema. ### 4.1 Schema example -```json -{ - "version": "1.1.0-alpha", - "containment": "microvm", - "microvm": { - "image": "python:3.12-alpine", - "memoryMb": 256 - }, - "process": { - "commandLine": "cat /mnt/c/nvx-work/input/message.txt > /mnt/c/nvx-work/output/result.txt", - "cwd": "/", - "timeout": 30000 - }, - "filesystem": { - "readonlyPaths": ["C:\\nvx-work\\input"], - "readwritePaths": ["C:\\nvx-work\\output"], - "deniedPaths": ["C:\\nvx-work\\input\\private"] - }, - "network": { - "egress": { - "default": "deny", - "allow": [{ - "to": [{ "cidr": "203.0.113.0/24" }], - "ports": [{ "protocol": "tcp", "port": 443 }] - }] - }, - "ingress": { - "default": "deny", - "hostLoopback": "deny" - } - } -} -``` +[Appendix F.1](#f1-complete-request-example) contains the complete request +example. ### 4.2 Filesystem -MXC will pass the configured host paths to NVX. NVX currently asks OpenVMM for -one virtio-fs export rooted at the common host directory. The guest control -agent then bind-mounts each requested path separately as read-only or -read-write: - -```mermaid -flowchart LR - Policy["MXC filesystem policy"] --> NVX["NVX mapping plan"] - NVX --> OpenVMM["OpenVMM: one virtio-fs export"] - OpenVMM --> Agent["Control agent: per-path RO/RW bind mounts"] - Agent --> Workload["OCI workload paths"] -``` - -Read-write mappings are -live: a guest write changes the mapped host file immediately. Denied entries -are hidden by OpenVMM within the exported host tree. - -Filesystem access is enforced at two layers: - -| Access control | Enforcer | -| --- | --- | -| Which host paths are exposed or hidden | NVX and OpenVMM | -| Whether an exposed path is read-only or read-write | The NVX guest control agent and OpenVMM | -| Whether the OpenVMM process may access a host file | Windows and NTFS, using OpenVMM's Windows process identity | - -If a path is not mapped, or is explicitly denied, NVX and OpenVMM prevent the -workload from reaching it even when the OpenVMM process could access it. If a -mapped path is denied to the OpenVMM process by its NTFS ACL, Windows still -rejects the operation. MXC validates the requested mappings and launches -OpenVMM under the intended Windows identity, but it does not perform each file -access check itself. - -All mapped paths must exist, be on the same Windows volume, and share a common -directory below the volume root. MXC will translate Windows paths into guest -paths: for example, `C:\nvx-work\input` will be available as -`/mnt/c/nvx-work/input`. Exporting an entire volume such as `C:\` will be -rejected. - -The initial integration will also reject mappings at or below protected -Windows host roots, including the Windows directory, Program Files, Program -Files (x86), and ProgramData. Validation will use canonical paths and Windows -Known Folder resolution so case differences, short names, junctions, and -reparse points cannot bypass the rule. Both read-only and read-write mappings -will fail before OpenVMM starts and identify the protected root. +[Appendix F](#appendix-f-schema-and-policy-implementation-details) describes +path mapping, access checks, and Windows path restrictions. | MXC Schema field | Works today | | --- | --- | @@ -507,23 +360,13 @@ will fail before OpenVMM starts and identify the protected root. | Hidden relative path syntax | Whitespace, `:`, `\`, and links are rejected | | Nested Windows mappings | A read-only file inside a read-write directory is rejected | | Mapping count and length | All bind-mount arguments must fit in the 1024-byte kernel-command-line budget, allowing roughly a dozen typical mappings | -| If any limit above is exceeded | NVX rejects the request before OpenVMM starts; no workload runs | +| If any limit above is exceeded | NVX rejects the request before OpenVMM starts. No workload runs. | ### 4.3 Network -MXC will pass the directional network policy to NVX. NVX currently converts -the supported rules into OpenVMM network options. OpenVMM presents a virtual -network adapter to the guest. Its portable networking mode processes the -adapter's traffic inside the OpenVMM host process, where it performs NAT and -enforces the configured allow and deny rules. - -```mermaid -flowchart LR - Policy["MXC network policy"] --> NVX["NVX rule conversion"] - NVX --> OpenVMM["OpenVMM virtual NIC + portable profile"] - OpenVMM --> Workload["OCI workload virtual network device"] -``` +[Appendix F](#appendix-f-schema-and-policy-implementation-details) describes +how NVX maps the network policy to OpenVMM. | MXC Schema field or rule | Works today | @@ -542,45 +385,21 @@ flowchart LR | `network.ingress.hostLoopback: allow` | No | | `runtimeConfig.networkProxy` | No | -Unsupported network forms are rejected before the VM starts. - -| Network rule behaviour | Current NVX limit | -| --- | --- | -| Deny precedence | A matching deny rule overrides an allow rule | -| Expanded rules | Maximum 256 final allow rules and 256 final deny rules | -| Port ranges | Expanded to individual ports; a range may contain at most 256 ports | -| `protocol: any` with a port | Expands to one TCP and one UDP rule per port | -| TCP/UDP without a port | Rejected | -| Fully denied or omitted network | No virtual network device is attached | +[Appendix F](#appendix-f-schema-and-policy-implementation-details) lists the +network rule limits and rejection behavior. ### 4.4 Lifecycle -State-aware MicroVM provision types are not currently registered in the MXC -`1.1.0-alpha` contract. The integration will add them before publishing the -NVX lifecycle through the SDKs. - -The proposed provision request will use the same `version`, `containment`, -`filesystem`, and `network` fields as the one-shot example in section 4.1. -The following shows only the state-aware difference: - -```json -{ - "phase": "provision", - "microvm": { - "memoryMb": 256, - "image": "python:3.12-alpine" - } -} -``` +The MXC `1.1.0-alpha` contract does not contain state-aware MicroVM provision +types. MXC will add these types before the SDKs publish the NVX lifecycle. -The `process` section from the one-shot example will be omitted during -provision. One-shot and state-aware provision will use the same `microvm` -structure with required `image` and optional `memoryMb`. A later `exec` -request will supply the process configuration. +One-shot and state-aware provision use the same MicroVM configuration. +[Appendix F.4](#f4-state-aware-provision-shape) contains the state-aware +request shape. | MXC phase | How NVX handles it | | --- | --- | -| `provision` | Resolves the OCI reference to an immutable digest, selects or creates the converted artifact, persists that identity with the configuration, and returns an NVX instance ID | +| `provision` | Resolves the OCI reference to an immutable digest. Selects or creates the converted artifact. Stores that identity with the configuration. Returns an NVX instance ID. | | `start` | Launches OpenVMM and waits for the guest agent | | `exec` | Runs a workload in the running VM | | `stop` | Stops the VM while retaining provisioned state | @@ -594,85 +413,58 @@ the following form: aci-edge-sandboxes:<32 lowercase hexadecimal characters> ``` -MXC will register the `aci-edge-sandboxes:` prefix in native lifecycle -dispatch so `start`, `exec`, `stop`, and `deprovision` route back to the -MicroVM/NVX backend. +MXC will register the `aci-edge-sandboxes:` prefix in +`mxc_engine::backend_from_prefix` and state-aware dispatch so `start`, `exec`, +`stop`, and `deprovision` route back to the NVX runner. -| Surface | Required prefix work | -| --- | --- | -| Native engine | Map `aci-edge-sandboxes:` to the `microvm` backend in `backend_from_prefix` and state-aware dispatch | -| Rust SDK | Accept and preserve the opaque ID through `ContainerId` | -| Node SDK | Brand returned IDs as `ContainerId<'microvm'>` and include the prefix in lifecycle routing and validation maps | -| .NET SDK | Map the prefix to `ContainmentBackend.Microvm` and preserve it in `ContainerId` | - -| Action | Workload effect | VM or state effect | -| --- | --- | --- | -| One-shot completion | Returns the workload outcome | MXC stops and deprovisions the VM | -| One-shot failure during provision, start, or exec | The workload may not start or will be terminated | MXC performs bounded stop and deprovision cleanup while preserving the original error | -| `process.timeout` | NVX terminates the workload and its descendants | A state-aware VM remains running; a one-shot VM is cleaned up | -| Cancel or kill an execution handle | Cancels the current workload and its descendants | Does not deprovision a state-aware VM | -| State-aware exec caller exits or loses its control session | The guest agent terminates the active workload | The running VM remains available for a later lifecycle call | -| `stop` during an exec | Waits for the active exec up to the stop timeout, then terminates the VM if required | The VM returns to provisioned state and the active exec fails | -| `deprovision` | Requires the VM to be stopped | Removes the provisioned state | - -The same running VM can serve repeated `exec` calls. Only one workload runs at -a time; another `exec` waits up to the configured control timeout. Guest-memory -state does not survive `stop`, while changes to mapped host files do. +[Appendix F](#appendix-f-schema-and-policy-implementation-details) describes +ID routing, cleanup behavior, and repeated `exec` calls. - NVX runs one workload at a time. A second `exec` waits for up to 60 seconds by default. - `stop` waits for up to 30 seconds before forcing the VM to shut down. -- Cancelling an SDK wait preserves existing SDK semantics and does not kill the workload; explicit kill or cancel operations on an execution handle must cancel the NVX workload and its descendants. +- Cancelling an SDK wait does not kill the workload. Use the execution handle + to cancel the workload and its descendants. ## 5. UI and other support | Schema field or capability | Works today | Notes | | --- | --- | --- | | `ui` | No | Not supported by design, reject when supplied | -| `process.commandLine` | Yes | Runs through the workload image's execution environment; the control initramfs does not supply workload commands or a fallback shell | +| `process.commandLine` | Yes | Runs in the workload image. The control initramfs does not supply workload commands or a fallback shell. | | `process.cwd` | Yes | Must be an absolute guest path | | `process.env` / `inheritDefaultEnv` | Yes | | -| `process.timeout` | Yes | Maximum one hour; omitted or `0` means no execution deadline | +| `process.timeout` | Yes | The maximum is one hour. An omitted value or `0` means no execution deadline. | | Separate stdout and stderr | Yes | Combined output is limited to 1 MB | | Live stdin | No | Future work if needed. Workload receives EOF | | PTY | No | Future work if needed | -| Guest memory override | Yes | One-shot and state-aware provision use `microvm.memoryMb`; default is 256 MB | +| Guest memory override | Yes | One-shot and state-aware provision use `microvm.memoryMb`. The default is 256 MB. | | `fallback` | No | Not supported by design, NVX does not select another backend | -| Process behaviour | Developer-visible result | -| --- | --- | -| Command execution | The OCI workload image must provide the shell or executable required by the final execution contract; the control initramfs is not used as the workload command environment | -| Environment | Omitted environment uses the OCI image defaults; an explicit list replaces or layers over those defaults; the selected shell may set `PWD` and `SHLVL` | -| Output limit | Combined stdout and stderr are limited to 1 MB; exceeding the limit terminates the workload rather than truncating output | -| Nonzero exit | Returned as a workload result, not an SDK or FFI failure | -| Invalid request or unavailable backend | Returned as an MXC error rather than a workload exit code | +[Appendix F](#appendix-f-schema-and-policy-implementation-details) describes +command execution, environment, output, and result behavior. **Open question: Should NVX change the output-limit behaviour, for example by truncating output instead of terminating the workload?** NVX lifecycle errors align with the existing -[MXC SDK error classifications](reference/rust/v1/types.md). The integration +[MXC SDK error classifications](reference/rust/v1/types.md). The NVX runner will map the additional NVX execution outcomes as described in [Appendix B](#appendix-b-nvx-execution-outcome-mapping). ## 6. Image support -The implementation will support a standard OCI image reference. The NVX image -tool will convert that image into the internal filesystem artifact consumed by -NVX. +The NVX runner will accept a standard OCI image reference. The NVX image tool +will convert that image into the filesystem image attached by OpenVMM. ### 6.1 Standard OCI image and workload -The developer will provide a standard OCI image reference through `image`. -The signed NVX image tool will download the image and convert it into the -filesystem format consumed by NVX. +The developer will provide an OCI image reference through `microvm.image`. +The NVX image tool will convert and cache the image for OpenVMM. -OCI-image-backed execution is a prerequisite for the MXC integration. The -selected NVX Rust API does not currently accept an image or attach a converted -workload filesystem. The NVX team will provide the signed conversion tool. It -will also extend the runtime contract so the converted image can be attached -as the workload root. The init agent will remain in the control initramfs. The -required conversion and runtime contract is listed in -[Appendix C](#appendix-c-oci-image-conversion-contract). +The current NVX API does not accept an image or attach a workload filesystem. +NVX must add this support before MXC can run OCI-image-backed workloads. +[Appendix C](#appendix-c-oci-image-conversion-requirements) defines the +conversion, cache, registry, and security requirements. ```json { @@ -683,35 +475,206 @@ required conversion and runtime contract is listed in } ``` -The complete request in section 4.1 is an example of this model: +MXC will use the cached artifact when available. A cache miss with +deny-by-default egress will fail without host registry traffic. The planned +prefetch API can populate the same cache before execution. + +[Appendix F.2](#f2-schema-additions) describes the schema additions. + +## 7. Windows requirement + +The initial implementation will support Windows x64. Windows ARM is planned +but will not be initially available. + +| Requirement | Developer action | +| --- | --- | +| Hardware virtualisation | Enable Intel VT-x or AMD-V in firmware | +| Windows Hypervisor Platform | Enable the `HypervisorPlatform` Windows optional feature from an elevated shell and reboot | +| Running Windows hypervisor | Ensure hypervisor launch has not been disabled | +| NVX runtime | Install the matching x64 SDK runtime package. MXC will validate the architecture, files, signatures, checksums, and manifest. | + +Enable Windows Hypervisor Platform from an elevated PowerShell session: + +```powershell +Enable-WindowsOptionalFeature -Online -FeatureName HypervisorPlatform -All +``` + +[Appendix E.3](#e3-platform-support) describes `platform_support()`, +`unavailableReasons`, and runtime checks. + +## 8. Requirements and end-to-end tests + +The NVX backend will be complete when the following areas pass through all +three SDKs on Windows x64 with WHP. MXC may also run applicable cases through +`wxc-exec` as an internal end-to-end test path. The relayed `wxc-exec` path is +excluded from state-aware timeout expectations because it cannot represent a +distinct timed-out result. `wxc-exec` is not part of the public NVX SDK API. + +| Area | Required coverage | +| --- | --- | +| Integration | `microvm` routes to the NVX runner for one-shot and state-aware execution. Rust, Node, and .NET use this runner. `wxc-exec` is used only for internal E2E tests. | +| Platform support | Add `unavailableReasons` to `mxc_engine::PlatformSupport`. Return it through Rust, Node, and .NET. Include `microvm` only after the NVX runner and `AciEdgeSandbox::probe()` succeed. | +| Lifecycle | Test each lifecycle phase and the exact ID prefix. Test repeated and overlapping exec calls. Test reconnect, invalid transitions, stale IDs, prefix isolation, and one-shot cleanup. | +| State-aware process lifetime | Set `OpenVmmConfig::breakaway_from_job = true`. Return `backend_unavailable` when the Windows job blocks breakaway. Verify that OpenVMM remains active after the `start` process exits. Verify that a later `exec` process reconnects. | +| SDKs and FFI | Rust, Node, and .NET produce the same policy and results. Verify `mxc_ffi` handle ownership and cleanup. Node registers the NVX runtime directory before `getPlatformSupport()` or execution. .NET registers both request shapes in `MxcJsonContext` and tests them in `Microsoft.Mxc.Sdk.AotSmokeTest`. | +| Filesystem | Test read-only, read-write, and denied paths. Test files, directories, multiple mappings, and invalid combinations. Test file ownership, ACL inheritance, protected roots, aliases, reparse points, path limits, and command-line limits. | +| Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | +| Process | Test command, CWD, environment, cancellation, output limits, nonzero exits, and descendant cleanup. Test timeouts through the three in-process SDK paths. Exclude the relayed `wxc-exec` path from timeout tests. | +| PTY | Verify that the initial implementation rejects PTY requests. Add terminal tests when PTY support is available. | +| Packaging | Test Rust, npm, and NuGet installation. Verify that one package owns `mxc_ffi.dll` in each SDK. Test npm directory registration. Test the NuGet `buildTransitive` copy into `nvx/**`. Test invalid RID combinations and missing runtime files. Verify source references. | +| Signing | Validate the Authenticode chain and Microsoft signer. Check all file hashes. Reject runtime directories that an untrusted user can modify. | +| Host | Run tests on Windows x64 with WHP enabled. ARM support remains planned. | +| Image support | Test registry conversion and required-image validation. Test one-shot and state-aware requests. Test generated SDK types, cache misses, prefetch, allowed pulls, converter security tests, and invalid converter output. | + +Negative filesystem and network tests must include a working positive control +so infrastructure failures are not mistaken for policy enforcement. + +## 9. Long-term plan + +- Converge the NVX MicroVM integration under the broader WSL platform. +- Reuse and align session, image, SDK, and runtime concepts with WSLC. +- Allow MXC to replace its direct NVX integration without changing the + public SDK API. + +## Appendices + +- [Appendix A: NVX, OpenVMM, and guest architecture](#appendix-a-nvx-openvmm-and-guest-architecture) +- [Appendix B: NVX execution outcome mapping](#appendix-b-nvx-execution-outcome-mapping) +- [Appendix C: OCI image conversion requirements](#appendix-c-oci-image-conversion-requirements) +- [Appendix D: Runtime packaging and validation details](#appendix-d-runtime-packaging-and-validation-details) +- [Appendix E: NVX runner, probe, and platform-support details](#appendix-e-nvx-runner-probe-and-platform-support-details) +- [Appendix F: Schema and policy implementation details](#appendix-f-schema-and-policy-implementation-details) -- `microvm.image` identifies the standard OCI image. -- `process.commandLine` defines the workload to execute. -- The filesystem policy makes the workload's input available read-only and - its output location available read-write. -- The network policy limits the workload to the requested destination and - port. +## Appendix A: NVX, OpenVMM, and guest architecture -The integration will use an NVX converted-image cache and a shared -backend-neutral MXC registry policy: +### A.1 Guest architecture + +The NVX runner will construct `OpenVmmConfig` and call +`AciEdgeSandbox::openvmm()`. That creates the crate's `OpenVmmBackend`, which +launches `openvmm.exe`. OpenVMM boots the NVX kernel and unpacks the minimal +control initramfs into the VM's in-memory control root filesystem. + +The initramfs contains only the binaries and libraries required to initialise +and control the microVM, including these two components: + +| Guest path | Role | +| --- | --- | +| `/init` | Runs first, mounts the guest pseudo-filesystems, reads the kernel command line, prepares networking and host filesystem mappings, and resolves the workload identity | +| `/sbin/nvx-managed-agent` | Replaces `/init` for the managed lifecycle and remains running as PID 1 while workloads execute | + +The managed agent remains in the control initramfs and outside the OCI +workload root filesystem. Workload processes see the converted OCI filesystem +as `/`. They do not see or execute `/sbin/nvx-managed-agent`. The agent +remains PID 1 and handles lifecycle and process-control messages. + +```mermaid +flowchart LR + Engine["mxc_engine NVX runner"] --> API["AciEdgeSandbox"] + API --> Backend["OpenVmmBackend"] + Backend --> OpenVMM["openvmm.exe"] + OpenVMM --> Init["Control initramfs: /init"] + Init --> Agent["/sbin/nvx-managed-agent (PID 1)"] + Agent --> Workload["OCI workload environment"] +``` + +### A.2 NVX Rust interface + +The NVX Rust interface exposes the following operations: + +| Operation | NVX Rust API | Input structure | Output structure | +| --- | --- | --- | --- | +| Validate provision | `AciEdgeSandbox::validate_provision` | `&ProvisionRequest { filesystem?, network?, microvm.provision.memoryMib? }` | `Result<()>` | +| Validate execution | `AciEdgeSandbox::validate_exec` | `&ExecRequest { process { commandLine or argv, cwd?, env?, inheritDefaultEnv?, timeout? }, stdin }` | `Result<()>` | +| Check runtime files and hypervisor | `AciEdgeSandbox::probe` | None | `Result<()>` | +| Provision VM | `AciEdgeSandbox::provision` | `&ProvisionRequest` | `Result` | +| Start VM | `AciEdgeSandbox::start` | `&SandboxId` | `Result` | +| Execute workload | `AciEdgeSandbox::exec` | `&SandboxId`, `&ExecRequest` | `Result` with live stdout and stderr pipes, optional stdin pipe, cancellation handle, and wait methods | +| Stop VM | `AciEdgeSandbox::stop` | `&SandboxId` | `Result` | +| Remove provisioned state | `AciEdgeSandbox::deprovision` | `&SandboxId` | Returns `DeprovisionResult`. The ID becomes stale. | +| Wait for execution outcome | `Execution::wait` | Owned `Execution` handle | `Result`: `Exited(code)`, `Signaled(signal)`, `TimedOut`, `Cancelled`, or `Failed(reason)` | +| Wait and collect output | `Execution::wait_with_output` | Owned `Execution` handle | `Result, stderr: Vec }>` | +| Get cancellation handle | `Execution::canceller` | `&Execution` | `Canceller` | +| Cancel execution | `Canceller::cancel` | `&Canceller` | `Result<()>` | + +[Appendix E.1](#e1-current-probe-behavior) describes the current `probe()` +result and checks. + +`ProvisionRequest` in the selected NVX API does not yet contain an OCI image. +It also currently nests memory under `microvm.provision`. The MXC wire +contract will use the shared flat `microvm { image, memoryMb }` structure for +one-shot and state-aware provision requests. The NVX runner will map that +structure into `aci_edge_sandboxes::ProvisionRequest`. The image work will +extend the NVX provision input. It will add the resolved OCI image or the +converted filesystem-image path and manifest. [Section 6](#6-image-support) +and [Appendix C](#appendix-c-oci-image-conversion-requirements) describe these +fields. + +### A.3 Communication boundaries + +| Connection | Mechanism | Purpose | +| --- | --- | --- | +| `mxc_engine` NVX runner to `AciEdgeSandbox` | In-process Rust API calls | Will invoke provision, start, exec, stop, and deprovision | +| `OpenVmmBackend` to `openvmm.exe` | Process launch with CLI arguments | Supplies the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | +| `OpenVmmBackend` to `openvmm.exe`, during startup only | OpenVMM stdin | Passes a one-time 32-byte authentication capability. This stdin pipe is not the ongoing command channel. | +| `OpenVmmBackend` to `openvmm.exe` | Windows named pipe | Carries ongoing lifecycle and workload control through the NVX framed binary protocol | +| OpenVMM to guest control agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | + + +## Appendix B: NVX execution outcome mapping + +| NVX outcome | MXC result | +| --- | --- | +| `Exited(code)` | Will return the workload exit code | +| `Signaled(signal)` | Will return `128 + signal` through the existing integer exit result | +| `TimedOut` through an in-process Piped SDK path | Will return the existing MXC timed-out result | +| `TimedOut` through the `wxc-exec` Relayed path | The relay cannot return a distinct timeout. Tests exclude this path. A returned `TimedOut` value produces `backend_error`. | +| `Cancelled` | Will return exit code `137` | +| `Failed(WorkingDirectory)` | Will return `backend_error` with the working-directory failure | +| Other `Failed(...)` outcomes | Will return `backend_error` with the NVX failure reason | +| No outcome available | Will return `backend_error`. It will not invent a workload exit code. | + +## Appendix C: OCI image conversion requirements + +| Area | Required definition | +| --- | --- | +| Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | +| Registry authorisation | MXC validates the registry against the MXC registry allowlist before invoking the NVX image tool | +| Execution-time host fetch | A cache miss with deny-by-default egress fails without registry traffic. An automatic pull requires allowed egress. | +| Explicit prefetch | The planned MXC image-prefetch API pulls and converts the image. It uses the registry allowlist and does not create an NVX instance. It stores the result in the same cache. | +| Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | +| Credentials | The initial release will not support private-registry credentials. Future credentials must use an approved host provider. Requests, command lines, logs, and telemetry must not contain them. | +| Conversion timing | Pull and convert before VM start. Reuse a compatible cached conversion when available. | +| Converter security ownership | NVX defines and reviews the containment or no-host-extraction design. NVX also defines path rules, resource limits, output limits, cleanup, and security tests. | +| Converted artifact | Produce a versioned NVX artifact with a manifest identifying the source digest, converter version, runtime compatibility, and checksums | +| Guest integration | Attach the converted artifact to OpenVMM and make it the workload root while `/init` and the managed agent remain in the initramfs outside the workload root | +| OCI metadata | Define how `ENTRYPOINT`, `CMD`, `ENV`, `WORKDIR`, and `USER` interact with MXC `process` settings | +| Writable state | Define the writable layer or scratch lifetime and whether it is discarded on stop or deprovision | +| Cache identity | Key conversions by image digest plus converter and runtime format version rather than by mutable image tag alone | +| Failures | Return registry, conversion, compatibility, and attachment failures as `MxcError` values that name the failed operation and reason | + +The signed image tool, completed converter security review, image schema, +converted-artifact format, and runtime attachment are required deliverables +before `microvm.image` is usable. + +### C.1 Registry and cache behavior + +MXC will use an NVX converted-image cache and an MXC registry allowlist: 1. Use the image from the local cache when it is already available. 2. On a cache miss, reject normal execution when the request uses deny-by-default egress. MXC will not perform host registry traffic on behalf of that request before the VM exists. -3. Otherwise, resolve the registry host and check a shared backend-neutral MXC - administrative registry policy. +3. Otherwise, resolve the registry host and check the MXC registry allowlist. 4. When the registry is permitted, invoke the signed NVX image tool to pull the image and resolve its immutable digest. 5. Convert and cache the NVX-compatible artifact. -An administrator or deployment pipeline can warm the same cache through a -separate explicit MXC host-setup operation. That operation will invoke the -signed NVX image tool, enforce the machine registry policy, and report the -resolved digest and converted-artifact identity. It will not create a -NVX instance or inherit a workload request's network policy. After prefetch, -deny-by-default requests can use the cached artifact without host network -traffic. +An administrator or deployment pipeline can warm the same cache through the +planned MXC image-prefetch API. That API will invoke the signed NVX image tool, +enforce the registry allowlist, and report the resolved digest and converted +artifact. It will not create an NVX instance or use a workload request's +network policy. After prefetch, deny-by-default requests can use the cached +artifact without host network traffic. `microvm.image` may reference OCI registries such as Docker Hub. The NVX image tool will define how short image references are resolved. Before any network @@ -723,7 +686,7 @@ The policy will be stored under | Policy state | Behaviour | | --- | --- | -| Value absent | Unmanaged; any registry host may be contacted | +| Value absent | The policy does not restrict registry hosts. | | One or more hosts | Only those registry hosts may be contacted | | Present but empty | No registry may be contacted | | Present but unreadable | No registry may be contacted | @@ -733,199 +696,486 @@ invoking the image tool. The signed NVX image tool will own DNS resolution, TLS, registry protocol, redirects, download, digest resolution, and conversion while enforcing the policy supplied by MXC. -The NVX team will also own the converter's security model for processing -untrusted OCI manifests and layers. This includes its threat model, -least-privilege containment or no-host-extraction design, archive and path -handling, resource limits, staging and cache boundaries, failure cleanup, and -adversarial security tests. Signing authenticates the converter but does not -by itself contain parser or archive-processing vulnerabilities. MXC will not -enable arbitrary OCI image input until the NVX conversion-security contract -has completed security review. +### C.2 Converter security + +The NVX team will own the converter's security design for processing +untrusted OCI manifests and layers. The design must include a threat model and +a containment or no-host-extraction method. It must also define path checks, +resource limits, cache boundaries, cleanup, and security tests. Signing +authenticates the converter. It does not contain parser or archive-processing +vulnerabilities. MXC will not enable arbitrary OCI image input until that +design completes security review. `microvm.image` will accept an OCI image reference rather than an arbitrary URL. Redirects to another registry host will require that host to be permitted by the same policy. Private-registry credentials will not be part of the -initial contract and will require a separate approved host +initial release and will require a separate approved host credential-provider design. -### 6.2 Schema additions +## Appendix D: Runtime packaging and validation details -The `1.1.0-alpha` development contract will add a `microvm` image -configuration for both one-shot and state-aware provision requests. +### D.1 Archive and source validation + +The crate verifies the archive digest from `artifacts.json` and verifies the +staged files against `SHA256SUMS`. The OCI image download and conversion tool +is future NVX work and is not part of the current bundled artifact set. + +NVX already publishes the corresponding source for its Linux kernel and +Alpine-based control initramfs. The NVX repository contains the owned guest +and build inputs under `kernel/`, `guest/common/`, and `guest/alpine/`. Its +release tooling collects the exact patched kernel source. It also collects the +recipes and upstream sources for the minimal initramfs packages. The release +publishes them as separate `source/nvx-linux-source-.tar.gz` +and `source/nvx-alpine-source-.tar.gz` artifacts. + +NVX records its source details in `SOURCE-MANIFEST.json` and the +control-initramfs package inventory. These files identify the project source, +kernel build, patches, initramfs package sources, published source artifacts, +and hashes. The release also includes the required licences and third-party +notices. MXC will validate and consume these existing source files rather than +require new source-delivery files. + +### D.2 Package source files + +Before packaging the NVX runtime, MXC will validate `SOURCE-MANIFEST.json`. +MXC will check the references to the matching Linux kernel source. +MXC will also check the references to the control-initramfs source. +MXC will copy the source manifest into the npm and NuGet runtime packages. +It will also copy the package inventory, licences, and notices. The published +`nvx-alpine-source-*` archive covers only the Alpine packages included in that +minimal trusted control environment. MXC will not generate the source +archives. Packaging will fail when required metadata, source references, or +hashes are missing or do not match the runtime version. + +### D.3 Release files and future DLL + +The release also includes the supporting checksum, manifest, provenance, +licence, and package-inventory files. The numeric OpenVMM and kernel sizes are +from the selected development baseline and can change. The new image tool and +minimal control initramfs sizes have not yet been published. +These artifacts contribute to the staged runtime and SDK package footprint +rather than the published Rust crate archive size. + +The NVX team has discussed moving the host implementation behind a signed DLL. +That DLL and its ABI do not exist today. If NVX adopts that design, the DLL +name, exported functions, handle ownership, and versioning rules must be defined +before MXC can consume it. + +### D.4 SDK runtime packaging + +For Rust, the final executable or SDK package cannot depend on a Cargo +dependency's `OUT_DIR`. MXC will provide +`mxc_build_common::stage_nvx_runtime()` to copy the crate's staged `nvx` +directory from `DEP_ACI_EDGE_SANDBOXES_ARTIFACTS_DIR` into the final output. + +`@microsoft/mxc-sdk` will remain the only npm package that owns and loads +`mxc_ffi.dll`. The Node SDK will resolve the runtime package's absolute +platform directory before platform checks or execution. The runtime package +will preserve the `nvx` directory and its required files. + +For .NET, `Microsoft.Mxc.Sdk` will be the only package that owns +`mxc_ffi.dll`. Its Windows native library will include the MicroVM integration +but will report `microvm` as unavailable when the NVX assets are absent. +`Microsoft.Mxc.Sdk.Nvx.Runtime` will contain only the RID-specific `nvx` +directory and will declare an exact-version dependency on +`Microsoft.Mxc.Sdk`. + +NuGet's default RID-native asset handling flattens files beneath +`runtimes/{rid}/native`. The runtime package will instead store the assets +under `runtimes/{rid}/nvx/**`. It will include +`buildTransitive/Microsoft.Mxc.Sdk.Nvx.Runtime.targets`. That target will copy +the selected RID's complete `nvx` tree into build and publish outputs while +preserving `%(RecursiveDir)`. + +The Rust code linked into `mxc_ffi.dll` will resolve the `nvx` directory +relative to the loaded DLL. The copy target will reject unsupported RID and +architecture combinations rather than produce a partial runtime. -| Contract surface | Schema addition | +The .NET one-shot `Microvm` containment type and +`MicrovmProvisionRequest` will each be registered as `[JsonSerializable]` +roots in `MxcJsonContext`. Their serialization and deserialization will use +the existing `MxcJson` helpers without reflection fallback. +`Microsoft.Mxc.Sdk.AotSmokeTest` will test representative one-shot and +state-aware MicroVM requests through that production JSON path. + +### D.5 Node runtime-directory registration + +Registration will apply to the current process. It will accept only registered +backend names and absolute canonical paths. A repeated call with the same +backend and path will succeed. A call with a different path will fail. The +registration only supplies the directory. `mxc_engine` will validate the +package version, architecture, manifest, and checksums. Without the runtime +package, Node will not register an NVX runtime directory. +`getPlatformSupport()` will omit `microvm`. + +The Node SDK will call the planned registration export before +`getPlatformSupport()`, `run`, `spawn`, or a lifecycle operation: + +```text +mxc_register_backend_runtime_directory("microvm", absolutePath) +``` + +### D.6 Package compatibility and runtime-file validation + +The MXC SDK, `mxc_ffi.dll`, and NVX runtime package versions must match. +Before launch, MXC will verify the required files, Windows architecture, +signatures, checksums, and runtime manifest compatibility. A missing or +incompatible runtime will return `backend_unavailable` and identify the +required runtime package or version. + +The co-located checksum manifest will not be trusted by itself. The pinned NVX +crate's `artifacts.json` will provide the expected archive digest, and +`SOURCE-MANIFEST.json` will identify the expected control-protocol revision. +MXC will then: + +1. validate the Authenticode certificate chain and expected Microsoft signer + for `openvmm.exe` and the image tool when signed releases are available +2. verify `vmlinux`, the initramfs, and all other runtime files against the + expected checksums +3. reject a missing signature, invalid certificate chain, unexpected signer, + manifest mismatch, checksum mismatch, or runtime directory that is writable + by an untrusted user. + +## Appendix E: NVX runner, probe, and platform-support details + +### E.1 Current `probe()` behavior + +Today, `probe()` returns no structured payload. `Ok(())` means that the +current NVX runtime checks passed. A failure returns an NVX +`backend_unavailable` error with a reason. The current OpenVMM backend checks: + +- the host platform is supported +- the configured OpenVMM executable exists +- the configured NVX guest kernel exists +- the configured guest initramfs exists +- the configured hypervisor is available + +The current probe does not return versions, capabilities, or artifact +metadata. It also does not authenticate signatures, verify checksums, or check +the image tool and `SOURCE-MANIFEST.json` compatibility. The NVX runner will +perform those MXC checks before calling `probe()`. + +### E.2 MXC integration + +`mxc_engine` will keep `containment: "microvm"` and resolve it to the NVX +runner. + +The NVX runner will follow the existing MXC backend interfaces: + +- `ScriptRunner` for run-to-completion +- `SandboxBackend` for streaming `spawn` +- `StatefulSandboxBackend` for provision, start, exec, stop, and deprovision + +The existing `platform_support()` function will call the NVX runner's runtime +check. The runner will validate the runtime package, then call +`AciEdgeSandbox::probe()`. Neither check starts a VM. + +| SDK | Required changes | | --- | --- | -| One-shot | Add `microvm.image` and `microvm.memoryMb` | -| State-aware provision | Register `containment: "microvm"` and reuse `microvm.image` and `microvm.memoryMb` | -| Start, exec, stop, and deprovision | No image fields; these requests use the `sandboxId` created during provision | +| Rust SDK | Will enable the NVX backend in the SDK and engine build | +| .NET SDK | Will add the `microvm` choice through `mxc_ffi`. The matching runtime package will contain only the `nvx` files. | +| Node SDK | Will add the `microvm` choice, keep `mxc_ffi.dll` in the base SDK, resolve the matching NVX runtime package, and pass its directory through the planned `mxc_ffi` registration export | -`image` will be required and must contain a non-empty OCI image reference. -Omitting it will fail schema or request validation. +Node and .NET will continue to use the existing `mxc_ffi` boundary. NVX will +be linked on the Rust side. No separate NVX FFI library will be required. +The supported SDK APIs cover Rust, Node, and .NET. The Node run, spawn, PTY, +and state-aware paths call `mxc_ffi` in-process. NVX will not add support +commitments for legacy Node executor paths or introduce a new NVX CLI or +executor. `wxc-exec` may be built with NVX support only for internal +end-to-end tests over the same `mxc_engine` runner. -One-shot standard-image addition: +#### E.2.1 MXC request and NVX call flow -```json -{ - "microvm": { - "image": "python:3.12-alpine", - "memoryMb": 256 - } -} +The SDKs expose typed MXC requests. Node and .NET serialize those requests and +cross the existing `mxc_ffi` boundary. Rust passes typed requests through +`mxc-sdk` directly. Both paths reach the NVX runner in `mxc_engine`. + +```mermaid +flowchart LR + Rust["Rust SDK"] --> RustSdk["mxc-sdk"] + Node["Node SDK"] --> FFI["mxc_ffi"] + DotNet[".NET SDK"] --> FFI + FFI --> Parser["wxc_common exact parsing and binding"] + Parser --> Runner["mxc_engine NVX runner"] + RustSdk --> Runner + Runner --> NVX["AciEdgeSandbox"] ``` -State-aware provision will use the same `microvm` structure: +The NVX runner converts the MXC MicroVM, filesystem, network, and process +fields into the phase-specific NVX request types. It then calls +`AciEdgeSandbox` as follows: + +| MXC operation | Input passed to the NVX runner | `AciEdgeSandbox` calls | +| --- | --- | --- | +| `platform_support()` | Registered `nvx` directory, `SOURCE-MANIFEST.json`, `SHA256SUMS`, architecture, and signer information | `probe` | +| One-shot `run` or `spawn` | MicroVM image and memory, filesystem policy, network policy, and process request | Calls `validate_provision`, `provision`, `start`, `validate_exec`, and `exec`. The runner then waits or cancels. Finally, it calls `stop` and `deprovision`. | +| State-aware provision | MicroVM image and memory, filesystem policy, and network policy | `validate_provision` → `provision` | +| State-aware start | NVX instance ID | `start` | +| State-aware execution | NVX instance ID and process request | Calls `validate_exec` and `exec`. The SDK waits through `Execution::wait` or `Execution::wait_with_output`. | +| Execution cancellation | `Execution` handle | `Execution::canceller` → `Canceller::cancel` | +| State-aware stop | NVX instance ID | `stop` | +| State-aware deprovision | NVX instance ID | `deprovision` | + +The NVX runner maps lifecycle results, execution outcomes, and errors back to +the existing MXC SDK result and error types. +[Appendix B](#appendix-b-nvx-execution-outcome-mapping) defines the +execution-outcome mapping. + +### E.3 Platform support + +Developers should check platform support before launch using Rust +`platform_support()`, Node `getPlatformSupport()`, or .NET +`MxcPlatform.GetPlatformSupport()`. Node and .NET call +`mxc_platform_support_json()` in `mxc_ffi`, which returns +`mxc_engine::platform_support()`. No separate public NVX probe API will be +added. + +`PlatformSupport.availableMethods` will include `microvm` only when the NVX +runner's runtime checks and `AciEdgeSandbox::probe()` succeed. When they fail, +`PlatformSupport.unavailableReasons["microvm"]` will explain whether the +developer must install the runtime package, enable WHP, or repair runtime +files. + +`mxc_engine::PlatformSupport` will add an `unavailableReasons` map from backend +wire name to a specific failure reason. The existing platform-wide `reason` will +remain reserved for a host on which MXC itself is unsupported. + +| SDK | Backend-specific unavailable-reason field | +| --- | --- | +| Rust | `PlatformSupport::unavailable_reasons` | +| Node | `PlatformSupport.unavailableReasons` | +| .NET | `PlatformSupport.UnavailableReasons`, keyed by `ContainmentBackend` | + +For example, ProcessContainer can work when the NVX runtime package is +missing. The host remains supported. `availableMethods` omits `microvm`. +`unavailableReasons` explains how to install the NVX runtime. + +The NVX runner will repeat the same runtime checks before launch. A cached +`PlatformSupport` result will not bypass them. + +Missing WHP, disabled hardware virtualisation, absent runtime files, +incompatible guest/runtime versions, and unsupported architectures will return +`backend_unavailable` with remediation. OpenVMM startup failures should +identify the associated log path. + +## Appendix F: Schema and policy implementation details + +### F.1 Complete request example ```json { - "phase": "provision", + "version": "1.1.0-alpha", "containment": "microvm", "microvm": { "image": "python:3.12-alpine", "memoryMb": 256 + }, + "process": { + "commandLine": "cat /mnt/c/nvx-work/input/message.txt > /mnt/c/nvx-work/output/result.txt", + "cwd": "/", + "timeout": 30000 + }, + "filesystem": { + "readonlyPaths": ["C:\\nvx-work\\input"], + "readwritePaths": ["C:\\nvx-work\\output"], + "deniedPaths": ["C:\\nvx-work\\input\\private"] + }, + "network": { + "egress": { + "default": "deny", + "allow": [{ + "to": [{ "cidr": "203.0.113.0/24" }], + "ports": [{ "protocol": "tcp", "port": 443 }] + }] + }, + "ingress": { + "default": "deny", + "hostLoopback": "deny" + } } } ``` +### F.2 Schema additions + +The `1.1.0-alpha` development contract will add a `microvm` image +configuration for both one-shot and state-aware provision requests. + +| Request type | Schema addition | +| --- | --- | +| One-shot | Add `microvm.image` and `microvm.memoryMb` | +| State-aware provision | Register `containment: "microvm"` and reuse `microvm.image` and `microvm.memoryMb` | +| Start, exec, stop, and deprovision | These requests have no image fields. They use the `sandboxId` created during provision. | + +`image` will be required and must contain a non-empty OCI image reference. +Omitting it will fail schema or request validation. + The contract changes will also require regenerated development schema and wire types, plus matching versioned Rust, Node, and .NET SDK types. +[Appendix F.3](#f3-contract-and-sdk-publication) describes publication. +[Appendix F.4](#f4-state-aware-provision-shape) contains the state-aware +request shape. -## 7. Windows requirement +### F.3 Contract and SDK publication -The initial implementation will support Windows x64. Windows ARM is planned -but will not be initially available. +The wire and SDK changes will be delivered in this order: -| Requirement | Developer action | -| --- | --- | -| Hardware virtualisation | Enable Intel VT-x or AMD-V in firmware | -| Windows Hypervisor Platform | Enable the `HypervisorPlatform` Windows optional feature from an elevated shell and reboot | -| Running Windows hypervisor | Ensure hypervisor launch has not been disabled | -| NVX runtime | Install the matching x64 SDK runtime package; MXC will validate the architecture, required files, signatures, checksums, and manifest compatibility | +1. Add `microvm.image`, `microvm.memoryMb`, state-aware MicroVM provision, + engine binding, and runtime tests to the development `1.1.0-alpha` + contract. +2. Keep the wire work development-only while that contract remains mutable. +3. If NVX is ready when `1.1.0` is frozen, publish the fields in stable + `1.1.0`. Advance `schemas/schema-version.json` + `sdkMajorTargets["1"]` from `1.0.0` to `1.1.0`. Publish the new Rust, + Node, and .NET V1 MicroVM types and versioned references. The stable API + will not require an experimental option. +4. If NVX is not ready, stable `1.1.0` will omit the MicroVM fields. Move them + into the next mutable contract, such as `1.2.0-alpha`. Do not publish the + V1 MicroVM SDK types until a stable contract contains those fields. -Developers should check platform support before launch using Rust -`platform_support()`, Node `getPlatformSupport()`, or .NET -`MxcPlatform.GetPlatformSupport()`. These existing APIs will call the native -MXC discovery path; no separate public NVX probe API will be added. +Contract publication and host availability remain separate. A published +MicroVM backend can still return `backend_unavailable` when the host lacks the +NVX runtime, WHP, or another required capability. -The discovery result will include `microvm` in `availableMethods` only when the -NVX probe and MXC package-integrity checks succeed. An unavailable MicroVM -backend will include a backend-specific reason in the SDK's platform-support -result, with remediation such as installing the matching runtime package, -enabling WHP, or repairing corrupt assets. +The existing lifecycle operation methods will remain unchanged. Only their +backend-specific request types will expand. -The native `PlatformSupport` payload will add an `unavailableReasons` map from -backend wire name to actionable reason. The existing platform-wide `reason` -will remain reserved for a host on which MXC itself is unsupported. +### F.4 State-aware provision shape -| SDK | Backend-specific discovery field | -| --- | --- | -| Rust | `PlatformSupport::unavailable_reasons` | -| Node | `PlatformSupport.unavailableReasons` | -| .NET | `PlatformSupport.UnavailableReasons`, keyed by `ContainmentBackend` | +The proposed provision request will use the same `version`, `containment`, +`filesystem`, and `network` fields as the one-shot example in +[Appendix F.1](#f1-complete-request-example). The following example shows only +the state-aware difference: -For example, a Windows host on which ProcessContainer works but the NVX -runtime package is missing will remain platform-supported, omit `microvm` from -`availableMethods`, and report the MicroVM remediation in -`unavailableReasons`. +```json +{ + "phase": "provision", + "microvm": { + "memoryMb": 256, + "image": "python:3.12-alpine" + } +} +``` -Execution will repeat the authoritative preflight before launch. A cached or -stale discovery result will not bypass runtime validation. +The `process` section from the one-shot example will be omitted during +provision. One-shot and state-aware provision will use the same `microvm` +structure with required `image` and optional `memoryMb`. A later `exec` +request will supply the process configuration. -Missing WHP, disabled hardware virtualisation, absent runtime files, -incompatible guest/runtime versions, and unsupported architectures will return -`backend_unavailable` with remediation. OpenVMM startup failures should -identify the associated log path. +### F.5 Filesystem implementation -## 8. Requirements and end-to-end tests +MXC will pass the configured host paths to NVX. NVX currently asks OpenVMM for +one virtio-fs export rooted at the common host directory. The guest control +agent then bind-mounts each requested path separately as read-only or +read-write: -The NVX backend will be complete when the following areas pass through all -three SDKs on Windows x64 with WHP. MXC may additionally run applicable cases -through the generic packaged executor as an internal end-to-end harness. The -relayed executor path is excluded from state-aware timeout expectations -because it cannot represent a distinct timed-out result. The executor is not -an NVX developer-facing surface. +```mermaid +flowchart LR + Policy["MXC filesystem policy"] --> NVX["NVX mapping plan"] + NVX --> OpenVMM["OpenVMM: one virtio-fs export"] + OpenVMM --> Agent["Control agent: per-path RO/RW bind mounts"] + Agent --> Workload["OCI workload paths"] +``` -| Area | Required coverage | +Read-write mappings are live. A guest write changes the mapped host file +immediately. OpenVMM hides denied entries in the exported host tree. + +Filesystem access is enforced at two layers: + +| Access control | Enforcer | | --- | --- | -| Integration | `microvm` routes to NVX for one-shot and state-aware execution through Rust, Node, and .NET; the generic executor is used only as an internal harness | -| Discovery | Add the native `unavailableReasons` payload and its Rust, Node, and .NET projections; include `microvm` only after the native NVX probe and MXC package-integrity checks succeed; verify each failure mode returns actionable backend-specific remediation without starting a VM | -| Lifecycle | Provision, exact `aci-edge-sandboxes:` prefix routing, start, repeated and overlapping exec, caller reconnect, stop, deprovision, invalid transitions, malformed/stale IDs, backend prefix isolation, and one-shot failure cleanup | -| State-aware process lifetime | Configure NVX with `OpenVmmConfig::breakaway_from_job = true` and return actionable `backend_unavailable` when the caller's Windows job disallows breakaway; from a kill-on-close Windows job, verify that OpenVMM remains running after the `start` process exits and that a later `exec` process reconnects successfully | -| SDKs and FFI | Rust, Node, and .NET produce the same policy and result behaviour; native ownership and cleanup remain correct; Node registers the separately installed MicroVM runtime directory before discovery or execution and rejects conflicting registration; .NET registers both MicroVM request shapes in `MxcJsonContext` and exercises their production serialize/deserialize paths in `Microsoft.Mxc.Sdk.AotSmokeTest` | -| Filesystem | Read-only, read-write, denied paths, files, directories, multiple mappings, invalid combinations, access to pre-existing host files, the Windows owner and inherited ACL of guest-created files, protected Windows root rejection, canonical-path alias and reparse-point bypass attempts, 128/129 denied-path boundaries, valid and invalid hidden-path syntax, read-only files nested under read-write directories, and mapping sets immediately below and above the kernel-command-line budget | -| Network | Defaults, allow/deny precedence, CIDRs, exclusions, TCP/UDP ranges, and rejection of unsupported rules | -| Process | Command, CWD, environment, cancellation, output limits, nonzero exits, and descendant cleanup through all supported paths; timeout results through the three in-process SDK paths, excluding the generic executor's relayed state-aware path | -| PTY | Confirm unsupported in the initial implementation; add terminal tests when implemented | -| Packaging | Rust crate, npm, and NuGet installation; single ownership of `mxc_ffi.dll` in npm and NuGet; npm runtime-package resolution and native directory registration; NuGet `buildTransitive` recursive copy into `nvx/**` for both build and publish outputs; rejection of unsupported RID/package combinations; inclusion of the NVX implementation DLL, OpenVMM, image tool, kernel, control initramfs, source manifest, control-initramfs package inventory, licences, and notices; OCI image conversion; automatic runtime discovery; missing/corrupt artifacts; and verification that matching Linux and control-initramfs source artifacts are published and referenced | -| Signing | Authenticate the runtime manifest, validate the Authenticode chain and Microsoft signer for signed NVX binaries, verify all remaining file checksums, and reject untrusted runtime directories | -| Host | Real execution on Windows x64 with WHP installed and enabled; ARM remains planned | -| Image support | Verify standard-image registry conversion, required-image validation, one-shot and state-aware schema branches, generated SDK types, cache-miss rejection without host traffic for deny-by-default egress, explicit prefetch followed by cache-only use, permitted execution-time pull, the NVX-owned adversarial conversion-security suite, and MXC rejection of malformed or incompatible converter output | +| Which host paths are exposed or hidden | NVX and OpenVMM | +| Whether an exposed path is read-only or read-write | The NVX guest control agent and OpenVMM | +| Whether the OpenVMM process may access a host file | Windows and NTFS, using OpenVMM's Windows process identity | -Negative filesystem and network tests must include a working positive control -so infrastructure failures are not mistaken for policy enforcement. +If a path is not mapped, NVX and OpenVMM prevent access to it. They also +prevent access to an explicitly denied path. This restriction applies even if +the OpenVMM process can access the path. Windows rejects access when the NTFS +ACL denies access to the OpenVMM process. MXC validates the requested +mappings and starts OpenVMM with the intended Windows identity. MXC does not +check each file operation. -## 9. Long-term plan +All mapped paths must exist, be on the same Windows volume, and share a common +directory below the volume root. MXC will translate Windows paths into guest +paths: for example, `C:\nvx-work\input` will be available as +`/mnt/c/nvx-work/input`. Exporting an entire volume such as `C:\` will be +rejected. -- Converge the NVX MicroVM integration under the broader WSL platform. -- Reuse and align session, image, SDK, and runtime concepts with WSLC. -- Allow MXC to replace its direct NVX integration without changing the - developer-facing contract. +The initial integration will also reject mappings at or below protected +Windows host roots, including the Windows directory, Program Files, Program +Files (x86), and ProgramData. Validation will use canonical paths and Windows +Known Folder resolution so case differences, short names, junctions, and +reparse points cannot bypass the rule. Both read-only and read-write mappings +will fail before OpenVMM starts and identify the protected root. -## Appendix A: Planned MXC to OpenVMM communication +### F.6 Network implementation -| Connection | Mechanism | Purpose | -| --- | --- | --- | -| MXC to NVX Rust interface | In-process Rust API calls | Will invoke provision, start, execute, stop, and deprovision | -| NVX Rust interface to signed implementation DLL | In-process interface call | Will use the implementation fetched and verified through the pinned NVX crate artifact contract | -| NVX implementation DLL to `openvmm.exe` | Process launch with CLI arguments | Will supply the kernel, initramfs, hypervisor, filesystem and network configuration, and control-endpoint address | -| NVX implementation DLL to `openvmm.exe`, during startup only | OpenVMM stdin | Will pass a one-time 32-byte authentication capability; stdin will not be the ongoing command channel | -| NVX implementation DLL to `openvmm.exe` | Windows named pipe | Will carry ongoing lifecycle and workload control through the NVX framed binary protocol | -| OpenVMM to guest control agent | Dedicated virtio-console | Will carry readiness, workload commands, stdout and stderr, cancellation, shutdown, and execution outcomes | +MXC will pass the directional network policy to NVX. NVX currently converts +the supported rules into OpenVMM network options. OpenVMM presents a virtual +network adapter to the guest. Its portable networking mode processes the +adapter's traffic inside the OpenVMM host process, where it performs NAT and +enforces the configured allow and deny rules. +```mermaid +flowchart LR + Policy["MXC network policy"] --> NVX["NVX rule conversion"] + NVX --> OpenVMM["OpenVMM virtual NIC + portable profile"] + OpenVMM --> Workload["OCI workload virtual network device"] +``` -## Appendix B: NVX execution outcome mapping +Unsupported network forms are rejected before the VM starts. -| NVX outcome | MXC result | +| Network rule behaviour | Current NVX limit | | --- | --- | -| `Exited(code)` | Will return the workload exit code | -| `Signaled(signal)` | Will return `128 + signal` through the existing integer exit result | -| `TimedOut` through an in-process Piped SDK path | Will return the existing MXC timed-out result | -| `TimedOut` through the internal executor's Relayed path | A distinct timeout is not supported by the relay and is excluded from timeout harness expectations; returning `TimedOut` currently produces `backend_error` as a relay contract violation | -| `Cancelled` | Will return exit code `137` | -| `Failed(WorkingDirectory)` | Will return `backend_error` with the working-directory failure | -| Other `Failed(...)` outcomes | Will return `backend_error` with the NVX failure reason | -| No outcome available | Will return `backend_error`; it will not invent a workload exit code | +| Deny precedence | A matching deny rule overrides an allow rule | +| Expanded rules | Maximum 256 final allow rules and 256 final deny rules | +| Port ranges | NVX expands a range to individual ports. A range can contain at most 256 ports. | +| `protocol: any` with a port | Expands to one TCP and one UDP rule per port | +| TCP/UDP without a port | Rejected | +| Fully denied or omitted network | No virtual network device is attached | -## Appendix C: OCI image conversion contract +### F.7 Lifecycle routing and behavior -| Contract area | Required definition | +| Component | Required prefix work | | --- | --- | -| Input identity | Resolve the OCI reference to an immutable digest and record the registry or source | -| Registry authorisation | MXC validates the registry against a shared backend-neutral administrative allowlist before invoking the NVX image tool | -| Execution-time host fetch | A cache miss under deny-by-default request egress is rejected without registry traffic; automatic pull is available only when the request permits egress | -| Explicit prefetch | A separate MXC host-setup operation pulls and converts the image under the machine registry policy without creating an NVX instance, then stores it in the same runtime cache | -| Redirects | The image tool follows a redirect to another registry host only when that host is also permitted | -| Credentials | No private-registry credentials in the initial contract; future credentials must come from an approved host provider and remain out of requests, command lines, logs, and telemetry | -| Conversion timing | Pull and convert before VM start; reuse a compatible cached conversion when available | -| Converter security ownership | NVX defines and security-reviews the least-privilege containment or no-host-extraction model, archive/path rules, resource bounds, output confinement, cleanup, and adversarial tests for untrusted OCI input | -| Converted artifact | Produce a versioned NVX artifact with a manifest identifying the source digest, converter version, runtime compatibility, and checksums | -| Guest integration | Attach the converted artifact to OpenVMM and make it the workload root while `/init` and the managed agent remain in the initramfs outside the workload root | -| OCI metadata | Define how `ENTRYPOINT`, `CMD`, `ENV`, `WORKDIR`, and `USER` interact with MXC `process` settings | -| Writable state | Define the writable layer or scratch lifetime and whether it is discarded on stop or deprovision | -| Cache identity | Key conversions by image digest plus converter and runtime format version rather than by mutable image tag alone | -| Failures | Surface registry, conversion, compatibility, and attachment failures through actionable MXC errors | +| `mxc_engine` | Map `aci-edge-sandboxes:` to the `microvm` backend in `backend_from_prefix` and state-aware dispatch | +| Rust SDK | Accept and preserve the opaque ID through `ContainerId` | +| Node SDK | Brand returned IDs as `ContainerId<'microvm'>` and include the prefix in lifecycle routing and validation maps | +| .NET SDK | Map the prefix to `ContainmentBackend.Microvm` and preserve it in `ContainerId` | -The signed image tool, reviewed conversion-security contract, image schema, -converted-artifact format, and runtime attachment are required deliverables -before `microvm.image` is usable. +| Action | Workload effect | VM or state effect | +| --- | --- | --- | +| One-shot completion | Returns the workload outcome | MXC stops and deprovisions the VM | +| One-shot failure during provision, start, or exec | The workload may not start or will be terminated | MXC performs bounded stop and deprovision cleanup while preserving the original error | +| `process.timeout` | NVX terminates the workload and its descendants | A state-aware VM remains running. MXC cleans up a one-shot VM. | +| Cancel or kill an execution handle | Cancels the current workload and its descendants | Does not deprovision a state-aware VM | +| State-aware exec caller exits or loses its control session | The guest agent terminates the active workload | The running VM remains available for a later lifecycle call | +| `stop` during an exec | Waits for the active exec up to the stop timeout, then terminates the VM if required | The VM returns to provisioned state and the active exec fails | +| `deprovision` | Requires the VM to be stopped | Removes the provisioned state | + +The same running VM can serve repeated `exec` calls. Only one workload runs at +a time. Another `exec` waits up to the configured control timeout. Guest-memory +state does not survive `stop`. Changes to mapped host files do survive. + +### F.8 Process behavior + +| Process behaviour | Developer-visible result | +| --- | --- | +| Command execution | The OCI workload image must provide the required shell or executable. The control initramfs does not run workload commands. | +| Environment | An omitted environment uses the OCI image defaults. An explicit list replaces or extends those defaults. The selected shell can set `PWD` and `SHLVL`. | +| Output limit | Combined stdout and stderr are limited to 1 MB. NVX terminates the workload when it exceeds the limit. | +| Nonzero exit | Returned as a workload result, not an SDK or FFI failure | +| Invalid request or unavailable backend | Returned as an MXC error rather than a workload exit code | ## References and open decisions ### Awaited support -- Initial permitted registry set and final backend-neutral registry policy name -- NVX-owned OCI converter threat model, containment and bounded-extraction contract, security review, and adversarial test evidence +- Initial permitted registry set and final MXC registry policy name +- NVX-owned OCI converter threat model, containment and bounded-extraction design, security review, and adversarial test evidence - NVX signed binaries support -- NVX crate artifact registry, fetching, verification, and staging contract +- Documented NVX crate artifact fetching, verification, and staging behaviour +- Decision and ABI for any future NVX implementation DLL - Windows ARM runtime and SDK package availability - Future PTY support diff --git a/docs/nvx-npm-examples.md b/docs/nvx-npm-examples.md new file mode 100644 index 000000000..342d9aa2f --- /dev/null +++ b/docs/nvx-npm-examples.md @@ -0,0 +1,222 @@ +# NVX npm examples + +> [!NOTE] +> These examples describe the proposed completed NVX-backed MicroVM SDK +> surface. The MicroVM types are not available in the current npm package. + +For architecture, packaging, policy limits, and implementation details, see +[NVX integration in MXC](./nvx-integration.md). + +For .NET examples, see +[NVX .NET examples](./nvx-dotnet-examples.md). + +## Install the packages + +Install matching SDK and runtime package versions: + +```bash +npm install @microsoft/mxc-sdk @microsoft/mxc-nvx-runtime +``` + +## Check MicroVM availability + +```typescript +import { getPlatformSupport } from '@microsoft/mxc-sdk/v1'; + +const support = getPlatformSupport(); +if (!support.availableMethods.includes('microvm')) { + throw new Error( + support.unavailableReasons?.microvm ?? 'MicroVM is unavailable', + ); +} +``` + +## Run and capture output + +```typescript +import { runAsync } from '@microsoft/mxc-sdk/v1'; + +const result = await runAsync({ + command: 'python -c "print(\'hello from NVX\')"', + timeoutMs: 30_000, + containment: { + type: 'microvm', + config: { + image: 'python:3.12-alpine', + memoryMb: 256, + }, + }, +}); + +process.stdout.write(result.stdout); +process.stderr.write(result.stderr); +console.log(`exit=${result.exitCode} timedOut=${result.timedOut}`); +``` + +Prefer `runAsync`. The synchronous `run` operation blocks the Node event loop +until execution completes. + +## Spawn with live output + +```typescript +import { spawnAsync } from '@microsoft/mxc-sdk/v1'; + +const processHandle = await spawnAsync({ + command: + 'python -u -c "import sys,time; print(\'out\'); ' + + 'print(\'err\', file=sys.stderr); time.sleep(1)"', + timeoutMs: 30_000, + containment: { + type: 'microvm', + config: { + image: 'python:3.12-alpine', + memoryMb: 256, + }, + }, +}); + +try { + processHandle.standardOutput?.on( + 'data', + (chunk) => process.stdout.write(chunk), + ); + processHandle.standardError?.on( + 'data', + (chunk) => process.stderr.write(chunk), + ); + + const result = await processHandle.waitAsync(); + console.log(result); +} catch (error) { + processHandle.kill(); + throw error; +} finally { + processHandle.dispose(); +} +``` + +Consume stdout and stderr concurrently. Call `kill()` when the application +must stop the workload. + +## Reuse an NVX instance + +The state-aware lifecycle is: + +```text +provision → start → repeated execution → stop → deprovision +``` + +```typescript +import { + deprovisionContainer, + provisionContainer, + runInContainerAsync, + startContainer, + stopContainer, +} from '@microsoft/mxc-sdk/v1'; + +const { containerId } = await provisionContainer({ + containment: 'microvm', + image: 'python:3.12-alpine', + memoryMb: 256, +}); + +let startAttempted = false; +let primaryError: unknown; +const cleanupErrors: unknown[] = []; + +try { + startAttempted = true; + await startContainer(containerId); + + for (const value of ['first', 'second']) { + const result = await runInContainerAsync(containerId, { + command: `echo ${value}`, + }); + console.log(result.stdout); + } +} catch (error) { + primaryError = error; +} finally { + if (startAttempted) { + try { + await stopContainer(containerId); + } catch (error) { + cleanupErrors.push(error); + } + } + + try { + await deprovisionContainer(containerId); + } catch (error) { + cleanupErrors.push(error); + } +} + +if (primaryError !== undefined || cleanupErrors.length !== 0) { + const errors = [...cleanupErrors]; + if (primaryError !== undefined) { + errors.unshift(primaryError); + } + throw new AggregateError(errors, `NVX lifecycle failed for ${containerId}`); +} +``` + +Keep `containerId` unchanged. If cleanup fails, keep the ID in application +logs so an operator can retry cleanup. + +## Filesystem and network policy + +```json +{ + "filesystem": { + "readonlyPaths": ["C:\\nvx-work\\input"], + "readwritePaths": ["C:\\nvx-work\\output"], + "deniedPaths": ["C:\\nvx-work\\input\\private"] + }, + "network": { + "egress": { + "default": "deny", + "allow": [{ + "to": [{ "cidr": "203.0.113.0/24" }], + "ports": [{ "protocol": "tcp", "port": 443 }] + }] + }, + "ingress": { + "default": "deny", + "hostLoopback": "deny" + } + } +} +``` + +Add `filesystem` and `network` to the one-shot request or the provision +request. Host paths must exist before launch. + +Inside the Linux workload: + +```text +C:\nvx-work\input → /mnt/c/nvx-work/input +C:\nvx-work\output → /mnt/c/nvx-work/output +``` + +Use Linux guest paths in commands and `workingDirectory`. Read-write mappings +modify host files immediately. + +## PTY and live stdin + +PTY support is not available initially. `spawnWithPty` and +`spawnInContainerWithPty` reject MicroVM requests. + +Use `spawn` or `spawnInContainer` for non-interactive workloads. These +operations provide stdout and stderr pipes. They do not provide a terminal. +Live stdin is not available initially, and the workload receives EOF. + +## Errors + +| Result | Meaning | +| --- | --- | +| `backend_unavailable` | The NVX runtime, WHP, runtime files, architecture, or runtime version is unavailable | +| Policy validation error | The request uses a policy form or value that NVX cannot enforce | +| Nonzero workload exit | The workload ran and returned a nonzero status | +