ci(workflow): Discuss selective execution of acceptance tests based on changed paths

[SHUBHANSHU602]

Description

While reviewing the pr-testing-with-test-project.yml workflow, I noticed that the acceptance tests currently run for all three language templates (js, py, and java) whenever should_test is true, regardless of which part of the repository was modified.

After analyzing the workflow, test infrastructure, and template dependencies, I think there may be an opportunity to reduce unnecessary acceptance test executions. Before working on an implementation, I'd like to discuss whether this approach aligns with the project's expectations.

Current behavior

The acceptance workflow uses a matrix:

strategy:
  matrix:
    language: [js, py, java]

Once the changes job sets should_test=true, all three acceptance suites are executed.

---

Findings from the repository analysis

1. Language-specific acceptance tests are already isolated

The acceptance workflow executes one Docker Compose profile per matrix job:

• test-js
• test-py
• test-java

Each profile starts only its corresponding tester service.

2. Shared infrastructure is independent

The workflow starts the common Microcks infrastructure using the ci profile before executing the language-specific profile.

The language-specific tester services do not depend on one another, only on the shared infrastructure.

3. Repository has both shared and language-specific areas

From tracing the acceptance test dependency chain, there appear to be two categories of paths:

Shared (should trigger all acceptance tests):

• apps/generator/**
• packages/helpers/**
• packages/components/**
• apps/react-sdk/**
• Shared acceptance infrastructure (integration-test, __fixtures__, microcks-setup, etc.)

Language-specific (could potentially trigger only one acceptance suite):

• packages/templates/clients/websocket/javascript/**
• packages/templates/clients/websocket/python/**
• packages/templates/clients/websocket/java/quarkus/**

---

Discussion

Would it make sense to extend the existing paths-filter configuration so that:

• changes to shared packages continue running all acceptance tests,
• while changes limited to a single client template execute only that client's acceptance suite?

The main goal would be to reduce unnecessary CI execution time without skipping any required validation.

---

[SHUBHANSHU602]

@derberg I have analyse the whole generator repo I belive this is practical an possible to implement , if u find any point unclear we can can discuss as i have done sufficient analysis . So ,please have a look once.

[Adi-204]

@SHUBHANSHU602 thanks for digging into this — the underlying idea (avoiding redundant acceptance runs) is valid and something we've wanted for a while (there's actually a TODO for exactly this in microcks-podman.yml).

That said, I don't agree with the approach in the Discussion section — simply extending the existing paths-filter step isn't the right fix, for a couple of reasons:

1. pr-testing-with-test-project.yml is not only about acceptance tests. The same changes job also gates the test job (the Node/test-project run). Bolting acceptance-specific, per-language logic onto that shared filter/output mixes two unrelated concerns into one config and makes the workflow harder to reason about and maintain.

2. Our templates aren't self-contained, so naive per-client path filtering breaks. Each client template depends on @asyncapi/generator-helpers, @asyncapi/generator-components, and @asyncapi/keeper. If we filter purely on packages/templates/clients/websocket/<client>/**, we get it wrong the moment someone changes only a shared package and touches no client-specific files — that shared change can still affect the generated output/behavior of every client, so we can't skip that client's acceptance suite in that case.

Before starting implementation, please do a bit more research and share your proposed approach/design first.

[SHUBHANSHU602]

ohk @Adi-204 I will explain my aproach soon.