Skip to content

Sends billing.events log lines to S3 - #2522

Open
Hazel-Datastax wants to merge 69 commits into
mainfrom
hazel/billing-to-s3
Open

Sends billing.events log lines to S3#2522
Hazel-Datastax wants to merge 69 commits into
mainfrom
hazel/billing-to-s3

Conversation

@Hazel-Datastax

@Hazel-Datastax Hazel-Datastax commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

What this PR does:

Uses a buffer to batch up the logging messages for billing events, and sends to S3 when configured.

Start with the BillingS3HandlerInstaller and follow from there.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

📈 Unit Test Coverage Delta vs Main Branch

Metric Value
Main Branch 53.40%
This PR 54.00%
Delta 🟢 +0.60%
✅ Coverage improved!

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Unit Test Coverage Report

Overall Project 54% -0.22% 🍏
Files changed 87.68% 🍏

File Coverage
Billing.java 100% 🍏
BillingEventType.java 97.58% 🍏
BillingS3HandlerInstaller.java 97.5% -2.5% 🍏
DefaultBilling.java 95.63% 🍏
BatchedLogBufferMetrics.java 94.05% -5.95% 🍏
AsyncBatchedLogUploader.java 90% -10% 🍏
BatchedLogBuffer.java 89.91% -10.09% 🍏
MetricsBase.java 89.47% -10.53% 🍏
S3BatchedLogUploader.java 87.26% -12.74% 🍏
BillingUploadingLogHandler.java 85.78% -14.22% 🍏
BillingEvent.java 61.24% 🍏
BatchedLogUploaderMetrics.java 47.3% -52.7%
RequestContext.java 19.76% 🍏

@erichare
erichare self-requested a review September 1, 2026 17:46
Comment thread src/test/java/io/stargate/sgv2/jsonapi/service/billing/BatchedLogBufferTest.java Outdated
Comment thread src/main/java/io/stargate/sgv2/jsonapi/service/billing/BatchedLogBuffer.java Outdated
Comment thread src/main/java/io/stargate/sgv2/jsonapi/service/billing/BatchedLogBuffer.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/service/billing/BatchedLogBufferTest.java Outdated
@erichare erichare mentioned this pull request Sep 1, 2026
4 tasks
amorton and others added 2 commits September 4, 2026 15:50
still needs decisions on what to do when upload fails.

Installer setup to do basic starting of the uploader.
Comment thread src/test/java/io/stargate/sgv2/jsonapi/testresource/S3MockTestResource.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/testresource/S3MockTestResource.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/testresource/S3MockTestResource.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/testresource/S3MockTestResource.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/api/v1/BillingS3ExportIntegrationTest.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/api/v1/BillingS3ExportIntegrationTest.java Outdated
Comment thread src/test/java/io/stargate/sgv2/jsonapi/testresource/S3MockTestResource.java Outdated
* billing.events} JUL logger, the disabled path, and fail-loud startup on bad config. Delivery
* through an installed handler is covered by {@code BillingS3ExportIntegrationTest}.
*/
class BillingS3HandlerInstallerTest {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To re-enable soon?

* flush()/notify() on an object lands before the upload thread is in wait() - if we used
* Object.notify() and .wait()
*/
private final Semaphore wakeupPermit = new Semaphore(0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is smart. I like it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants