Skip to content

Validate Java OpenFeature standalone and SSI modes - #7559

Draft
leoromanovsky wants to merge 4 commits into
mainfrom
leo.romanovsky/java-openfeature-modes
Draft

Validate Java OpenFeature standalone and SSI modes#7559
leoromanovsky wants to merge 4 commits into
mainfrom
leo.romanovsky/java-openfeature-modes

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Motivation

Java customers need two explicit OpenFeature deployment choices: a standalone dd-openfeature runtime that works without dd-java-agent, and SSI injection for applications that carry only the OpenFeature SDK. The system-test suite previously exercised neither artifact boundary, so a provider accidentally supplied by the application classpath or an implicit dependency on the Datadog Agent could go unnoticed.

Changes

This adds Java-only standalone and SSI feature-flagging scenarios backed by the existing mock UFC service. The spring-boot fixture now builds two artifacts from the same source: the standalone artifact contains dd-openfeature, while the SSI artifact deliberately excludes it. Runtime selection starts standalone without -javaagent, or starts SSI with the local Java agent and tracing disabled. The /ffe contract reports deployment mode and provider identity, waits for injected-provider readiness, evaluates a real flag, and verifies that the mock backend received the UFC request. Both scenarios are registered in the end-to-end workflow and test activation map. Weblog metadata restricts both scenarios to the Java spring-boot target so generic CI matrices do not schedule them against unsupported libraries or weblogs.

Decisions

Both scenarios remain agentless at the infrastructure level: no Datadog Agent container is present. This separates the Java agent used for SSI bytecode injection from the Datadog Agent service and proves that standalone does not load the Java agent at all. The application compiles against OpenFeature SDK 1.20.2 rather than the Datadog Provider type, so SSI cannot pass through a hidden compile-time provider dependency. These scenarios are intentionally excluded from tracer-release grouping until the deployment modes have established CI stability. This PR is downstream of DataDog/dd-trace-java#12252; the exact branch artifact is used for local cross-repository proof until that Java stack reaches master.

Validation

  • SYSTEM_TEST_BUILD_ATTEMPTS=3 ./build.sh java -i weblog -w spring-boot
  • Inspected both packaged artifacts: standalone contains sdk-1.20.2.jar and dd-openfeature, while SSI contains sdk-1.20.2.jar and no dd-openfeature.
  • ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION_JAVA_STANDALONE tests/ffe/test_java_openfeature_deployment_modes.py::Test_FFE_Java_OpenFeature_Standalone — 1 passed against java@1.66.0-SNAPSHOT+6b47adb9d3.
  • ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION_JAVA_SSI tests/ffe/test_java_openfeature_deployment_modes.py::Test_FFE_Java_OpenFeature_SSI — 1 passed against java@1.66.0-SNAPSHOT+6b47adb9d3.
  • ./run.sh TEST_THE_TEST tests/test_the_test/test_manifest.py tests/test_the_test/test_version.py tests/test_the_test/test_ci_orchestrator.py tests/test_the_test/test_group_rules.py — 186 passed.
  • ./format.sh
  • Final-head java/prod Spring Boot instance 2 completed both standalone and SSI steps successfully under the production-version manifest treatment.
  • Final-head java/dev Spring Boot instance 2 runs both scenarios against current Java master (1.66.0-SNAPSHOT~2d8d93dd6b); both fail until the downstream Java stack above lands. The exact 6b47adb9d3 branch artifact passes both local scenarios as recorded above.

Exercise the bundled-provider and agent-injected deployment shapes through dedicated Java feature-flagging scenarios.
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/ffe/test_java_openfeature_deployment_modes.py                     @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
.github/workflows/run-end-to-end.yml                                    @DataDog/system-tests-core
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
tests/test_the_test/scenarios.json                                      @DataDog/system-tests-core
tests/test_the_test/test_ci_orchestrator.py                             @DataDog/system-tests-core
tests/test_the_test/test_group_rules.py                                 @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/agentless_endtoend.py                         @DataDog/system-tests-core
utils/build/docker/java/app.sh                                          @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java/spring-boot.Dockerfile                          @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java/spring-boot/pom.xml                             @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/App.java  @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/featureflag/FeatureFlagEvaluatorController.java  @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java/weblog_metadata.yml                             @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 21, 2026

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (java, dev) / End-to-end #2 / spring-boot 2 — ❌ 2 tests failed · 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

2 failed tests in tests/ffe/test_java_openfeature_deployment_modes.py: 1) AssertionError: assert 'default' == 'on-value' at line 28; 2) AssertionError: assert None == 200 at line 26.

Showing tests most relevant to this failure.

❌ tests.ffe.test_java_openfeature_deployment_modes.Test_FFE_Java_OpenFeature_SSI.test_deployment_mode[spring-boot] from system_tests_suite   View in Datadog
AssertionError: Flag evaluation failed: None
assert None == 200
 +  where None = HttpResponse(status_code:None, headers:{}, text:None).status_code
 +    where HttpResponse(status_code:None, headers:{}, text:None) = <tests.ffe.test_java_openfeature_deployment_modes.Test_FFE_Java_OpenFeature_SSI object at 0x7f09fdd29190>.response

self = <tests.ffe.test_java_openfeature_deployment_modes.Test_FFE_Java_OpenFeature_SSI object at 0x7f09fdd29190>

    def test_deployment_mode(self) -> None:
>       assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
E       AssertionError: Flag evaluation failed: None
...
❌ tests.ffe.test_java_openfeature_deployment_modes.Test_FFE_Java_OpenFeature_Standalone.test_deployment_mode[spring-boot] from system_tests_suite   View in Datadog
AssertionError: assert 'default' == 'on-value'
  - on-value
  + default

self = <tests.ffe.test_java_openfeature_deployment_modes.Test_FFE_Java_OpenFeature_Standalone object at 0x7f793ec32f90>

    def test_deployment_mode(self) -> None:
        assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
        result = json.loads(self.response.text)
>       assert result["value"] == "on-value"
...
Testing the test | System Tests (cpp_kong, prod) / End-to-end #1 / kong 1 — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

JSONDecodeError: Expecting value at line 1 column 1 in JSON parsing

Testing the test | all-jobs-are-green

View in Datadog · View in GitHub Actions

Some CI checks or statuses failed during the pipeline execution.

📋 Copy prompt for your agent
CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Branch: leo.romanovsky/java-openfeature-modes

Testing the test | System Tests (java, dev) / End-to-end #2 / spring-boot 2
Commit: d42dfc71e2d81f37490eddc8a6bf5ad7d6b2150a
Error (code / test):
2 failed tests in tests/ffe/test_java_openfeature_deployment_modes.py: 1) AssertionError: assert 'default' == 'on-value' at line 28; 2) AssertionError: assert None == 200 at line 26.
CI job: https://github.com/DataDog/system-tests/actions/runs/32445343679/job/96666626875

Testing the test | System Tests (cpp_kong, prod) / End-to-end #1 / kong 1
Commit: d42dfc71e2d81f37490eddc8a6bf5ad7d6b2150a
Error (code / test):
JSONDecodeError: Expecting value at line 1 column 1 in JSON parsing
CI job: https://github.com/DataDog/system-tests/actions/runs/32445343679/job/96665383307

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c01f7f5 | Docs | View more details | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant