[Epic] Integrate Relay and Switchyard with Classic Hermes #7937

Goal

Add an explicit, disabled-by-default Relay and Switchyard integration for NemoClaw-managed Classic Hermes sandboxes.

Hermes
└── native NeMo Relay runtime
    └── nvidia.switchyard plugin
        └── OpenShell-managed inference providers
            ├── efficient target
            ├── capable target
            └── judge role (capable reuse or dedicated)

The supervised Hermes process owns the Relay runtime and Switchyard plugin. The integration starts no Relay or Switchyard sidecar.

The public activation path is nemohermes onboard --switchyard-config <file>. The file supplies high-level efficient and capable provider/model selections without raw credentials or internal Relay/Switchyard configuration. Completion requires an adaptive llm-classifier route. Seeded weighted-random remains an internal smoke test, not the supported user outcome. The judge role must be resolved explicitly by qualified capable-provider reuse or an optional dedicated judge.

Ownership and placement

• Hermes owns the native Relay lifecycle and model-call interception.
• Relay owns plugin discovery, validation, activation, and lifecycle.
• Switchyard owns routing algorithms, target selection, provider calls, retries, and configured fallback.
• NemoClaw Classic owns plugin packaging, configuration generation, OpenShell provider binding, readiness, managed E2E evidence, and operator documentation.
• V1 continues to use generic Fabric pass-through. NemoClaw must not add a separate Switchyard-specific V1 schema.

The Classic implementation must remain a thin integration. It must not copy a Switchyard algorithm or create a second Relay lifecycle.

Current state

• Classic packages Hermes Agent 0.21.3. Hermes locks NeMo Relay 0.8.3 and includes the native Relay loader.
• The official switchyard-plugin-0.3.0 release provides checksum-bound Linux amd64 and arm64 bundles. It supports Relay >=0.8.0,<0.10.
• Classic does not package or activate Switchyard and has no --switchyard-config onboarding option.
• test(e2e): assert Hermes routing topology #8896 provides reusable evidence that one Hermes gateway runs without Relay or Switchyard sidecars.
• Relay 0.8.x has a known upstream-error propagation defect for model names that do not match a Switchyard route. The qualified Hermes flow must use the configured route ID for every model call.
• V1 packages its compatible Hermes, Relay, and Switchyard plugin set through feat: bundle Switchyard Relay plugin #13858. That work is a reference for artifact packaging, not a second Classic implementation owner.

Implementation plan

1. Package the compatible plugin

Use the released switchyard-plugin-0.3.0 bundles with the existing Classic Hermes 0.21.3 and Relay 0.8.3 pair.

• Add immutable Linux amd64 and arm64 archives with SHA-256 verification.
• Preserve the plugin manifest, schema, library, licenses, and notices.
• Install no upstream source checkout or plugin build toolchain in the final image.
• Verify the plugin ID, native API, Relay compatibility range, architecture, and installed library digest.
• Keep Switchyard inactive when no routing configuration is present.

Tracked by #13861.

2. Add the onboarding input and generate configuration

Expose one Hermes-only activation path:

nemohermes onboard --switchyard-config <file>

The versioned user file supplies high-level provider/model selections for an efficient target and a capable target. It may include a dedicated judge only when the released classifier cannot safely reuse the capable provider. It contains no raw credential, endpoint outside an approved provider profile, Relay activation record, or Switchyard deployment fragment.

NemoClaw generates two separate root-owned files:

1. A Relay dynamic-plugin activation file that references the installed nvidia.switchyard manifest.
2. A Switchyard version-1 deployment file that defines clients, targets, the resolved judge role, and one supported llm-classifier route.

The configuration path must:

• reject --switchyard-config for non-Hermes agents;
• validate target and judge roles, route, clients, algorithms, URLs, model IDs, and credential-placeholder names;
• reuse the capable provider as judge only when the released contract and security boundary support it, otherwise require an explicit dedicated judge;
• reject missing, unknown, malformed, incompatible, ambiguous, or mixed-revision values;
• write files atomically with read-only permissions;
• remove stale routing files when routing is disabled; and
• contain no raw credential value.

Seeded weighted-random may be generated internally for smoke testing, but it is not the supported user-facing routing outcome.

Tracked by #13862. Closed #9012 contains reusable validation and secret-boundary ideas, but its code must not be restored without revalidation against current Classic source.

3. Bind target and judge roles through OpenShell

Create two target bindings from the approved provider profiles selected in the onboarding file:

• an efficient target; and
• a capable target.

Resolve a separately identifiable classifier judge role. Reuse the capable physical provider only if the released Switchyard contract supports role reuse and security, accounting, and routing evidence remain unambiguous. Otherwise require and attach the optional dedicated judge provider from the user input. Never choose a hidden judge default.

For each physical provider, NemoClaw records the provider identity, resource version, endpoint identity, model, profile identity, and credential key name. The logical judge role is recorded separately even when it reuses the capable provider. Switchyard configuration is generated only from this observed state.

The API keys remain in the OpenShell credential store for the lifetime of each provider. The sandbox receives endpoint-scoped placeholders, not credential values. Provider cleanup removes the binding and causes stale routing state to fail readiness.

The first slice supports fresh provider creation. Provider reuse, mutation, and credential rotation remain out of scope.

Tracked by #13863.

No Switchyard target or judge role may use inference.local. One inference.local route represents one host-selected model and cannot provide the required adaptive roles.

4. Select the adaptive Switchyard route in Hermes

Configure Hermes to request the supported llm-classifier route ID as its model.

• Every model call in the qualified flow must use that route ID.
• The classifier must select between the efficient and capable targets through the resolved judge role.
• Seeded weighted-random is permitted only as an internal smoke test that both targets are reachable.
• Unknown route IDs or unresolved judge state must fail before a provider request.
• The integration must not silently continue through an unmatched Relay path.
• Ordinary Hermes behavior must remain unchanged when routing is disabled.

This boundary prevents the known Relay 0.8.x unmatched-route behavior from becoming a silent fallback path.

5. Add fail-closed readiness

Before Hermes accepts a routed turn, readiness must confirm:

• the expected Relay runtime is active;
• the installed Switchyard manifest and library match the packaged artifacts;
• nvidia.switchyard is loaded and active;
• the deployment file parses through the installed Switchyard schema;
• the configured route ID exists;
• all required target and judge role bindings match the generated clients and targets; and
• the required credential placeholders and network policy are active.

Missing, corrupt, stale, incompatible, or inactive state must produce a bounded, non-secret error before any provider call.

Tracked by #13864.

6. Enforce the network and credential boundary

• Keep raw credentials out of sandbox files, environment, process arguments, logs, errors, telemetry, evidence, and snapshots.
• Set forward_auth = false for every Switchyard client.
• Deny caller credential forwarding and cross-provider placeholder use.
• Deny direct provider egress outside the OpenShell provider path.
• Deny fallback to inference.local or an unconfigured target.
• Record the Relay 0.8.x upstream-error limitation in diagnostics and operator documentation.

The complete redirect, response-bound, deadline, retry, fallback, cancellation, and stream-failure contract is tracked by #13865.

7. Add deterministic and live adaptive-routing evidence

Use the shortest stable test layer for each behavior:

• Source tests own onboarding input validation, configuration generation, judge resolution, stale-file removal, and redaction.
• Integration tests own the --switchyard-config path, Hermes adaptive-route selection, provider and judge binding, readiness, and disabled-default behavior.
• Package-contract tests own archive digests, image layout, manifest compatibility, licenses, and both Linux architectures.
• Live E2E owns the real Hermes process, OpenShell provider traffic, adaptive target selection, restart behavior, and process topology.

Extend the existing #8896 topology fixture instead of creating a second process-discovery implementation.

The live E2E must show:

• onboarding through nemohermes onboard --switchyard-config <file>;
• one supervised Hermes gateway and zero Relay or Switchyard sidecars;
• seeded weighted-random smoke evidence that both targets are reachable, without treating that as adaptive acceptance;
• representative easy work selecting the efficient target and representative complex work selecting the capable target through llm-classifier;
• successful adaptive streaming and tool turns;
• canonical routing events that identify the judge role and selected target without exposing private reasoning;
• the same active adaptive route after a process restart;
• fail-closed behavior for invalid onboarding input, unresolved judge state, invalid generated configuration, an unknown route, a missing provider, a wrong placeholder, and blocked egress; and
• no credential value in captured artifacts.

Tracked by #13866.

8. Document onboarding, operation, and rollback

Document:

• nemohermes onboard --switchyard-config <file> and a minimal secret-free example;
• the required efficient and capable selections and the rules for judge reuse or a dedicated judge;
• the supported adaptive route, models, providers, and platforms;
• how to enable and disable routing;
• how to inspect active state, judge resolution, and target-selection decisions;
• that seeded weighted-random is a smoke test rather than the supported user outcome;
• the Relay 0.8.x limitation;
• how to diagnose onboarding and readiness failures; and
• how to return to ordinary unrouted Hermes.

Tracked by #13867.

Vertical-slice execution plan

Execute this as one adaptive-routing vertical slice rather than seven independent sequential projects.

#13861 Prove and package the plugin
           ↓
#13862 Onboarding config + adaptive route ─┐
#13863 Target + judge provider bindings ──┤ build in parallel
                                         ↓
#13864 Fail-closed readiness ─────────────┐
#13865 Security and failures ─────────────┤ harden in parallel
                                         ↓
#13866 Adaptive integration and live E2E
           ↓
#13867 Finalize onboarding and rollback documentation

1. Complete Package Switchyard plugin 0.3.0 for Classic Hermes #13861 first to prove that plugin 0.3.0 loads through the existing Classic Hermes and Relay pair. Do not expand the implementation until this compatibility check passes.
2. Build Add Switchyard onboarding configuration and adaptive Hermes routing #13862 and Bind adaptive Switchyard targets and judge through OpenShell #13863 in parallel. Use seeded weighted-random only for the earliest smoke request through both target paths. These issues are complete only when the onboarding input resolves the judge role and one real Hermes request uses the supported adaptive route.
3. Harden that working adaptive slice through Add fail-closed Relay and Switchyard readiness to Classic Hermes #13864 and Enforce Switchyard credential, network, and failure boundaries #13865 in parallel. Add fail-closed readiness, credential and network enforcement, and deterministic failure behavior without adding another routing loop.
4. Complete Prove adaptive Classic Hermes Switchyard routing with live E2E #13866 against the assembled path. The live E2E is the integration gate for easy-versus-complex target selection, topology, restart behavior, negative cases, and redaction.
5. Draft Document Classic adaptive Switchyard onboarding and rollback #13867 alongside implementation, then finalize its onboarding, verification, troubleshooting, disablement, and rollback instructions from behavior proven by the E2E.

Keep the supported scope to one llm-classifier route between the efficient and capable targets. StageRouter, response-judging escalation, other agent runtimes, provider reuse or rotation, benchmarks, and UI work do not block this implementation.

Acceptance criteria

• Routing is explicit, Hermes-only, and disabled by default.
• nemohermes onboard --switchyard-config <file> accepts the high-level target selections without exposing Relay/Switchyard internals or raw credentials.
• Classic installs immutable plugin bundles for Linux amd64 and arm64 without source builds.
• The supervised Hermes process loads Relay and nvidia.switchyard; no routing sidecar runs.
• Relay activation and Switchyard deployment use separate validated root-owned files.
• Judge reuse or dedicated-judge selection is explicit, validated, and observable; no hidden judge default is permitted.
• Readiness proves the plugin, deployment, adaptive route, target and judge roles, policy, and credential bindings before the first provider request.
• Every model call in the qualified Hermes flow uses the configured llm-classifier route.
• Deterministic evidence routes representative easy work to the efficient target and representative complex work to the capable target.
• Both real target models return distinguishable responses and appear in canonical routing events.
• Seeded weighted-random is used only for smoke testing and is not presented as adaptive feature completion.
• Raw credentials remain inside the OpenShell credential boundary.
• Caller authorization, direct-provider egress, cross-provider placeholder use, unknown routes, and fallback to inference.local are denied.
• Adaptive streaming, one tool turn, and process restart pass.
• Invalid, missing, corrupt, stale, ambiguous, or blocked state fails with bounded, non-secret diagnostics.
• The operator documentation defines onboarding, judge behavior, enablement, disablement, verification, limitations, and rollback.

Passing these technical criteria does not by itself establish broad product support. The accepted support boundary must also name lifecycle, compatibility, security, validation, and maintenance ownership.

Non-goals for the first slice

• Agent runtimes other than Hermes.
• Relay CLI wrappers, switchyard-server, or other routing sidecars.
• A NemoClaw implementation of any Switchyard algorithm.
• Stage, advisor, composite, or response-judging escalation routes beyond the supported llm-classifier path.
• Provider reuse, mutation, or credential rotation.
• Benchmarks, cost-saving claims, or model-recipe ownership from [Epic] Deliver the Hermes Harness Optimization Blueprint with NeMo Platform Insights #8501.
• A new Classic UI, route editor, or general public routing schema.
• Support claims for every API family, provider, lifecycle operation, or failure mode.

Replan conditions

Replan when any of these conditions occurs:

• The consumer requires a non-Hermes agent runtime or more than one qualified route.
• Plugin 0.3.0 cannot load on Relay 0.8.3 without a Hermes or Relay upgrade.
• Hermes emits required model calls that cannot use the configured route ID.
• A target requires raw credentials inside the sandbox or direct provider egress.
• The first slice must support provider reuse or credential rotation.
• The V1 Fabric contract changes the durable ownership decision.

Related work

• Package Switchyard plugin 0.3.0 for Classic Hermes #13861 owns plugin packaging and compatibility validation.
• Add Switchyard onboarding configuration and adaptive Hermes routing #13862 owns the onboarding input, generated configuration, judge resolution, and adaptive Hermes route.
• Bind adaptive Switchyard targets and judge through OpenShell #13863 owns target and judge provider-role binding.
• Add fail-closed Relay and Switchyard readiness to Classic Hermes #13864 owns fail-closed readiness.
• Enforce Switchyard credential, network, and failure boundaries #13865 owns credential, network, and failure boundaries.
• Prove adaptive Classic Hermes Switchyard routing with live E2E #13866 owns adaptive integration, live E2E, and negative-security evidence.
• Document Classic adaptive Switchyard onboarding and rollback #13867 owns onboarding, operator documentation, and rollback.
• test(e2e): assert Hermes routing topology #8896 supplies the current sidecar-free Hermes topology fixture.
• Closed feat(hermes): define native Switchyard routing contract #9012 contains historical Classic configuration work.
• Closed feat(inference): stage fail-closed Switchyard integration #13119 records the V1 Fabric ownership decision.
• Merged feat: bundle Switchyard Relay plugin #13858 supplies the V1 runtime and packaging reference.
• [Epic] Deliver the Hermes Harness Optimization Blueprint with NeMo Platform Insights #8501 owns benchmark-backed Hermes optimization recipes.