APMRP-360 stabilize code origin default test - #7553
Conversation
Co-authored-by: P403n1x87 <20231758+P403n1x87@users.noreply.github.com>
|
I can only run on private repositories. |
|
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1aa5dd0b1d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| self.send_weblog_request("/") | ||
| self.code_origin_enabled_by_default = self.wait_for_code_origin_span(timeout=self._CODE_ORIGIN_TIMEOUT) |
There was a problem hiding this comment.
Capture the trace threshold before sending the checked request
wait_for_code_origin_span computes its threshold from data already received and accepts only later trace files. Because it is invoked after the second synchronous weblog request here, a trace that reaches the agent before this call begins is discarded as pre-existing data, and no further request is made to generate another trace; the test then waits 30 seconds and fails. This regresses the previous ordering, which captured the threshold before issuing the request. Start the wait/threshold before the request, or let the helper accept a request boundary, so the checked request's trace cannot be filtered out.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I can only run on private repositories.
There was a problem hiding this comment.
The new helper records its trace threshold after each request. A fast trace can enter that threshold and cause a false failure after a 30-second wait.
🤖 Datadog Autotest · Commit 1aa5dd0 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| self.wait_for_code_origin_span(timeout=self._WARMUP_TIMEOUT) | ||
|
|
||
| self.send_weblog_request("/") | ||
| self.code_origin_enabled_by_default = self.wait_for_code_origin_span(timeout=self._CODE_ORIGIN_TIMEOUT) |
There was a problem hiding this comment.
Capture the trace threshold before the request
The test remains flaky and can wait 30 seconds before it reports a false failure.
Assertion details
- Input: The agent writes the trace for the second GET request before
send_weblog_request()returns. - Expected:
The check must record the trace threshold before it sends each request. - Actual:
The helper records the latest trace file after the request returns. It then rejects that file because its number is not above the threshold.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
Motivation
Test_Debugger_InProduct_Enablement_Code_Origin_Default_On.test_code_origin_enabled_by_default(tracked as APMRP-360) is flaky, failing intermittently withAssertionError: Expected code origin enabled by defaultacross all Python tracer versions and flask weblog variants.The setup had a race condition: the tracer's code origin product is started asynchronously and only instruments the framework view functions once enabled. The single request was sent immediately after startup and could be served before the
_dd.code_origin.type: entrymetadata was attached. Compounding this, the 5s wait was too tight for the full request -> metadata -> flush -> agent -> trace-file chain under CI load.Changes
setup_code_origin_enabled_by_default, send a warmup request and wait (10s) for a code origin span so instrumentation is fully initialized before the real check.TIMEOUT = 5.wait_for_code_origin_spanhelper (which handles threshold computation and state reset) for both the warmup and the assertion, replacing the inlinedinterfaces.agent.wait_forcall.interfacesimport.Timeouts are scoped as class constants (
_WARMUP_TIMEOUT,_CODE_ORIGIN_TIMEOUT) so other tests still using the globalTIMEOUT = 5are unaffected.Testing
ruff formatandruff checkpass on the modified file.mypy --config pyproject.toml tests/debugger/reports "Success: no issues found in 16 source files".DEBUGGER_INPRODUCT_ENABLEMENTscenario in CI, which exercises this test across the affected Python tracer versions and flask weblog variants.Workflow
🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist
tests/ormanifests/is modified ? I have the approval from R&P teambuild-XXX-imagelabel is presentPR by Bits - View session in Datadog
Comment @DataDog to request changes