Skip to content

Add APPSEC_OTEL_COLLECTOR scenario — DD tracer + OTel Collector, no DD Agent (APPSEC-68836) - #7586

Draft
RamyElkest wants to merge 6 commits into
ramy.elkest/add-appsec-otlp-export-scenarios-APPSEC-68836from
ramy.elkest/appsec-otlp-export-ddot
Draft

Add APPSEC_OTEL_COLLECTOR scenario — DD tracer + OTel Collector, no DD Agent (APPSEC-68836)#7586
RamyElkest wants to merge 6 commits into
ramy.elkest/add-appsec-otlp-export-scenarios-APPSEC-68836from
ramy.elkest/appsec-otlp-export-ddot

Conversation

@RamyElkest

Copy link
Copy Markdown
Contributor

Summary

Tests the DDOT topology: DD Tracer exporting via OTLP to an OTel Collector (with datadog exporter), no DD Agent, no Remote Config.

Related: #7578 (M2 — OTLP export via DD agent)

Architecture

Weblog (dd-tracer, AppSec, OTLP export, DD_AGENT_HOST="")
  → proxy (captures OTLP)
  → OTel Collector (datadog exporter)
  → proxy (captures collector→backend)
  → Datadog Backend (mocked)

No DD Agent. No Remote Config. Static rules only.

What's New

New Scenario

APPSEC_OTEL_COLLECTOR — DD tracer + OTel Collector (data plane), no DD Agent:

  • AppSecOtelCollectorScenario class in utils/_context/_scenarios/appsec_otel_collector.py
  • Reuses OpenTelemetryCollectorContainer with existing otelcol-config.yml
  • DD_AGENT_HOST set to empty string — tracer initializes AppSec with static rules, no agent needed
  • No rc_api_enabled — no Remote Config

Test Decorators

Added @scenarios.appsec_otel_collector to existing appsec test classes (same set as PR #7578, minus RC-dependent tests).

Key Findings

  1. No DD Agent neededDD_AGENT_HOST="" lets the tracer initialize AppSec with static rules only
  2. OTLP exporter package required — weblog image needs opentelemetry-exporter-otlp-proto-http installed (already in Dockerfile but image needed rebuild)
  3. WAF blocking works through the collector — AppSec data survives: dd-tracer → OTLP → collector → datadog exporter → backend
  4. OTLP adapter from M2 PR reused — same DataDogLibraryTraceOTLP adapter works for collector path

Test Results

Passed Failed Deselected
85 4 30

The 4 failures are missing_feature for Python (body filenames/files content) — manifest doesn't skip them due to otel-api version parsing issue with DD_TRACE_OTEL_ENABLED=true.

What's NOT Tested (no RC)

  • Remote Config rule updates
  • Runtime activation
  • Dynamic IP/user blocking
  • API Security schema discovery (needs RC)

Files Changed

  • New: utils/_context/_scenarios/appsec_otel_collector.py — scenario class
  • Modified: utils/_context/_scenarios/__init__.py — scenario registration
  • Modified: 20+ test files in tests/appsec/@scenarios decorators

… Agent for RC)

New scenario: APPSEC_OTEL_COLLECTOR and APPSEC_OTEL_COLLECTOR_DEFAULT_RULES
- DD Tracer exports via OTLP to OTel Collector (datadog exporter)
- DD Agent included for /info only (tracer needs agent to initialize)
- No Remote Config (no rc_api_enabled)

Key finding: weblog image needs opentelemetry-exporter-otlp-proto-http installed
for OTLP export to work. Currently only opentelemetry-api is installed.
The flask-poc base image does not include the OTLP exporter package.

Test decorators added to existing appsec test files for both scenarios.
The tracer initializes AppSec with DD_AGENT_HOST="" and WAF blocking
works without any DD Agent. The OTel Collector handles the data plane
(traces via OTLP → datadog exporter), and static rules don't need RC.

Test results: 85 passed, 4 failed (missing_feature for Python),
30 deselected — same as with the agent included.
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

utils/_context/_scenarios/appsec_otel_collector.py                      @DataDog/system-tests-core
tests/appsec/api_security/test_apisecurity_telemetry.py                 @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/api_security/test_endpoint_discovery.py                    @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/api_security/test_schemas.py                               @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/api_security/test_schemas_auth.py                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_hardcoded_passwords.py                      @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_hardcoded_secrets.py                        @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_hsts_missing_header.py                      @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_insecure_auth_protocol.py                   @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_trust_boundary_violation.py                 @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/iast/sink/test_xcontent_sniffing.py                        @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_api10.py                                         @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_cmdi.py                                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_lfi.py                                           @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_shi.py                                           @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_sqli.py                                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/rasp/test_ssrf.py                                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/smoke_tests/test_otlp_export.py                            @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_agentic_onboarding.py                                 @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_alpha.py                                              @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_automated_login_events.py                             @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_automated_payment_events.py                           @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_automated_user_and_session_tracking.py                @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_blocking_addresses.py                                 @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_conf.py                                               @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_event_tracking.py                                     @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_fingerprinting.py                                     @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_identify.py                                           @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_metastruct.py                                         @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_reports.py                                            @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_shell_execution.py                                    @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_span_tags_headers.py                                  @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_trace_tagging.py                                      @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_traces.py                                             @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/test_versions.py                                           @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_addresses.py                                      @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_blocking.py                                       @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_blocking_security_response_id.py                  @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_miscs.py                                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_rules.py                                          @DataDog/asm-libraries @DataDog/system-tests-core
tests/appsec/waf/test_truncation.py                                     @DataDog/asm-libraries @DataDog/system-tests-core
tests/test_the_test/test_group_rules.py                                 @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
docs/plans/aap-native-otel-collector-weblogs.md                         @DataDog/system-tests-core

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Aug 25, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 81dd36d | Docs | View more details | Give us feedback!

…pport (APPSEC-68836)

- Fix mypy dict invariance: cast base_env to dict[str, str | None]
- Add weblog_categories=[WeblogCategory.dd_trace] to scenario
- Add APPSEC_OTEL_COLLECTOR scenarios to _is_supported_legacy in test_ci_orchestrator
- Add scenarios to test_group_rules exclusion list
- Fix ruff: move imports to top level, add noqa for private access
- Format with ruff
@RamyElkest
RamyElkest changed the base branch from main to ramy.elkest/add-appsec-otlp-export-scenarios-APPSEC-68836 August 25, 2026 23:53
New scenarios:
- APPSEC_OTEL_COLLECTOR_RASP: RASP with rasp_ruleset.json, no RC needed
- APPSEC_OTEL_COLLECTOR_API_SECURITY: API Security with DD_API_SECURITY_ENABLED=true

Test decorators added:
- 60 RASP test classes across 6 files (test_cmdi, test_lfi, test_sqli, test_ssrf, test_shi, test_api10)
- 16 API Security test classes across 4 files (test_schemas, test_schemas_auth, test_endpoint_discovery, test_apisecurity_telemetry)

RASP results: 30 passed, 30 failed (StackTrace/Telemetry tests need adapter fixes, API10 needs InternalServerContainer)
API Security: tests running, some schema assertion failures

Added all new scenarios to test_group_rules exclusion list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant