Skip to content

feat(telemetry): add server-side product telemetry - #914

Draft
dagardner-nv wants to merge 14 commits into
NVIDIA-NeMo:mainfrom
dagardner-nv:feat/product-telemetry
Draft

dagardner-nv wants to merge 14 commits into
NVIDIA-NeMo:mainfrom
dagardner-nv:feat/product-telemetry

Conversation

@dagardner-nv

@dagardner-nv dagardner-nv commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Add server-side product telemetry for routing outcomes. A tracing layer reads outcome metadata and sends events through a bounded background queue, or writes them to logs. Delivery is best effort, with configurable endpoints, timeouts, redirect limits, and a flush on shutdown.

Expand routing outcome metadata with observed candidate models, routing status, duration, and bounded error categories. Disable product telemetry in CI.

Why

Collect routing outcome data without waiting for telemetry delivery in the routing request path.

Closes SWITCH-1577

Notes for reviewers

  • This is a draft. Key decisions remain undetermined.
  • Documentation has been deferred.
  • Start with crates/switchyard-server/src/product_telemetry.rs, then review the outcome metadata and tracing changes in libsy.
  • The branch adds tests for metadata, payloads, configuration, background delivery, redirects, failures, timeouts, and flushing. Tests were not run as part of creating this PR.

Signed-off-by: David Gardner <dagardner@nvidia.com>
…uct telemetry

Signed-off-by: David Gardner <dagardner@nvidia.com>
…ENABLED' env variable

Signed-off-by: David Gardner <dagardner@nvidia.com>
Add dependency for humantime to the crates/libsy crate

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…rable endpoints

TODO:
* Determine destination for product_telemetry.rs
* Documentation

Signed-off-by: David Gardner <dagardner@nvidia.com>
…y.rs

Signed-off-by: David Gardner <dagardner@nvidia.com>
… spans with outcome metadata

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…oduct-telemetry

Signed-off-by: David Gardner <dagardner@nvidia.com>
…an that are needed

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…oduct-telemetry

Signed-off-by: David Gardner <dagardner@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant