Skip to content

[WIP] perf(telemetry): serialize batches on shared thread pool - #1946

Open
jpnurmi wants to merge 13 commits into
jpnurmi/perf/thread-poolfrom
jpnurmi/perf/telemetry-pool
Open

[WIP] perf(telemetry): serialize batches on shared thread pool#1946
jpnurmi wants to merge 13 commits into
jpnurmi/perf/thread-poolfrom
jpnurmi/perf/telemetry-pool

Conversation

@jpnurmi

@jpnurmi jpnurmi commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Warning

WIP    🚧🔨⏳⛔

  • Let the telemetry module own a shared serialization pool for enabled telemetry batchers.
  • Use the pool for log and metric batch serialization while keeping completion ordered through the batcher.
image

before-vs-after.ftrace.zip

Close: #1862


Stack created with GitHub Stacks CLIGive Feedback 💬

Comment thread src/sentry_batcher.c
Comment thread src/sentry_batcher.c
@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.78571% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.30%. Comparing base (6ca0dce) to head (1134ee2).

Additional details and impacted files
@@                     Coverage Diff                      @@
##           jpnurmi/perf/thread-pool    #1946      +/-   ##
============================================================
+ Coverage                     76.08%   76.30%   +0.21%     
============================================================
  Files                            94       94              
  Lines                         22441    22678     +237     
  Branches                       4017     4051      +34     
============================================================
+ Hits                          17075    17305     +230     
- Misses                         4468     4471       +3     
- Partials                        898      902       +4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from ae3b36a to 53a8c3e Compare August 4, 2026 11:44
Comment thread src/sentry_batcher.c
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from 53a8c3e to e8103ca Compare August 4, 2026 12:37
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from e8103ca to af6d19c Compare August 4, 2026 13:51
Comment thread src/sentry_batcher.c
Comment thread src/sentry_batcher.c Outdated
Comment thread src/sentry_batcher.c
Comment thread src/sentry_batcher.c
Comment thread src/sentry_telemetry.c
Comment thread src/sentry_batcher.c
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from 7e20632 to ccf9950 Compare August 5, 2026 09:46
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from ccf9950 to a5e9fef Compare August 5, 2026 10:14
Comment thread src/sentry_batcher.c
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from a5e9fef to bd10425 Compare August 6, 2026 08:17
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from bd10425 to 9cb4261 Compare August 6, 2026 10:55

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9cb4261. Configure here.

Comment thread src/sentry_batcher.c
jpnurmi added 3 commits August 6, 2026 16:46
Let the telemetry lifecycle own a shared serialization pool for enabled
telemetry batchers.

Use the pool for log and metric batch serialization while keeping completion
ordered through the batcher flush lifecycle.
jpnurmi added 10 commits August 6, 2026 16:46
Convert the telemetry batcher from the removed sentry__spin_lock helpers to
the promoted sentry__spinlock_* API. The ref guard, in-flight task list, and
flush reentrancy guard are all spinlocks, so store them as sentry_spinlock_t
instead of raw long fields.

Use sentry__spinlock_try_lock for the normal flush path. It preserves the
existing one-shot behavior when another flush is active while using the same
primitive as the crash-safe wait path.

Remove the temporary sentry__spin_lock* compatibility aliases now that the
batcher is on the shared API.
Keep crash-safe flushes from making a started batcher thread look fully
stopped before normal shutdown has a chance to wake and join it. Always
run the final flush and task drain so linked serialization tasks cannot
outlive the batcher they reference.

Add regression coverage for crash-safe flush followed by shutdown while
thread-pool work is still pending.
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/telemetry-pool branch from 261cfe1 to 1134ee2 Compare August 6, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logs: optimize performance

1 participant