Skip to content

feat(o11y): per-batch output send spans + HEC ACK-poll span (PIPE-1066) - #289

Open
Dylan-M wants to merge 2 commits into
mainfrom
dylanmyers/pipe-1066-output-send-spans
Open

feat(o11y): per-batch output send spans + HEC ACK-poll span (PIPE-1066)#289
Dylan-M wants to merge 2 commits into
mainfrom
dylanmyers/pipe-1066-output-send-spans

Conversation

@Dylan-M

@Dylan-M Dylan-M commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Proposed Change

Adds send-latency tracing to the outputs, gated by PerBatchSpans. tcp, udp, and file
carry the emit-span context through their internal channel and parent a per-message span
on the actual send: blitz.output.tcp.send, blitz.output.udp.send, and
blitz.output.file.write. otlp and hec batch many records, so they emit a standalone
send span carrying blitz.batch.size rather than parenting to one record. hec adds a
blitz.output.hec.ack_poll span around the indexing-confirmation poll. syslog delegates
to its tcp/udp transport, so it is not double-spanned.

How to validate

  • Start a TCP listener (nc -lk 5140), point blitz at a collector, and enable per-batch
    spans:
    go run ./cmd/blitz --generator-type apache-common --generator-apache-common-rate 1s --output-type tcp --output-tcp-host localhost --output-tcp-port 5140 --telemetry-traces-otlpendpoint localhost:4317 --telemetry-traces-insecure --telemetry-traces-perbatchspans
  • Confirm per-message blitz.output.tcp.send spans arrive as children of the emit span.
  • Swap the output flags for hec to see the batch span plus the ACK-poll span:
    --output-type hec --output-hec-host localhost --output-hec-port 8088 --output-hec-token test-token --output-hec-enableack.
    Confirm a blitz.output.hec.send span carrying blitz.batch.size, plus a
    blitz.output.hec.ack_poll span.
  • Drop --telemetry-traces-perbatchspans (the default) and confirm no send spans are
    emitted.
  • go test ./output/... covers each output's send span, the off path, and the otlp/hec
    batch spans via mock service clients.
Checklist
  • Changes are tested
  • CI has passed

eKuG
eKuG previously approved these changes Sep 4, 2026
@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1066-output-send-spans branch from dbf7635 to f12b841 Compare September 4, 2026 21:00
@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1066-fix-into-spine branch from b7ecb22 to 68e6626 Compare September 4, 2026 21:00
@Dylan-M
Dylan-M dismissed eKuG’s stale review September 4, 2026 21:01

Had to refactor to account for a change on main. Please re-review.

@Dylan-M

Dylan-M commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

@eKuG Had to refactor to account for a change on main. Please re-review

@Dylan-M
Dylan-M requested a review from eKuG September 4, 2026 21:01
@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1066-fix-into-spine branch from 68e6626 to f397cd0 Compare September 4, 2026 22:54
@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1066-output-send-spans branch from f12b841 to a6a38d6 Compare September 4, 2026 22:54
@graphite-app
graphite-app Bot changed the base branch from dylanmyers/pipe-1066-fix-into-spine to graphite-base/289 September 4, 2026 23:25
@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1066-output-send-spans branch from a6a38d6 to 006d135 Compare September 4, 2026 23:33
@graphite-app
graphite-app Bot changed the base branch from graphite-base/289 to main September 4, 2026 23:34
@graphite-app

graphite-app Bot commented Sep 4, 2026

Copy link
Copy Markdown

Merge activity

  • Sep 4, 11:34 PM UTC: Graphite rebased this pull request, because this pull request is set to merge when ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants