Skip to content

.NET: Disable failing DurableTask and AzureFunctions integration tests#6774

Merged
SergeyMenshykh merged 1 commit into
microsoft:mainfrom
SergeyMenshykh:sergeymenshykh-disable-failing-integration-tests
Jun 26, 2026
Merged

.NET: Disable failing DurableTask and AzureFunctions integration tests#6774
SergeyMenshykh merged 1 commit into
microsoft:mainfrom
SergeyMenshykh:sergeymenshykh-disable-failing-integration-tests

Conversation

@SergeyMenshykh

@SergeyMenshykh SergeyMenshykh commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Description

Disables 19 integration tests that are persistently failing in CI (timing out). Each test now has a Skip attribute with a reason referencing the tracking issue #6732.

DurableTask - AgentEntityTests:

  • EntityNamePrefixAsync
  • RunAgentMethodNamesAllWorkAsync (Theory, 3 cases)
  • OrchestrationIdSetDuringOrchestrationAsync

DurableTask - ExternalClientTests:

  • SimplePromptAsync
  • CallFunctionToolsAsync
  • CallLongRunningFunctionToolsAsync

DurableTask - WorkflowConsoleAppSamplesValidation:

  • ConcurrentWorkflowSampleValidationAsync
  • WorkflowAndAgentsSampleValidationAsync

DurableTask - ConsoleAppSamplesValidation:

  • SingleAgentSampleValidationAsync
  • SingleAgentOrchestrationChainingSampleValidationAsync
  • MultiAgentConcurrencySampleValidationAsync
  • MultiAgentConditionalSampleValidationAsync

AzureFunctions - SamplesValidation:

  • SingleAgentSampleValidationAsync
  • MultiAgentOrchestrationConcurrentSampleValidationAsync
  • MultiAgentOrchestrationConditionalsSampleValidationAsync
  • LongRunningToolsSampleValidationAsync
  • AgentAsMcpToolAsync

AzureFunctions - WorkflowSamplesValidation:

  • WorkflowAndAgentsSampleValidationAsync
  • ConcurrentWorkflowSampleValidationAsync

Related to #6732

Copilot AI review requested due to automatic review settings June 26, 2026 17:39
@moonbox3 moonbox3 added the .NET Usage: [Issues, PRs], Target: .Net label Jun 26, 2026
@SergeyMenshykh SergeyMenshykh self-assigned this Jun 26, 2026
@SergeyMenshykh SergeyMenshykh moved this to In Review in Agent Framework Jun 26, 2026
@github-actions github-actions Bot changed the title Disable failing DurableTask and AzureFunctions integration tests .NET: Disable failing DurableTask and AzureFunctions integration tests Jun 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR temporarily disables a set of persistently failing .NET integration tests (DurableTask + Azure Functions sample validations) by adding Skip reasons to the affected xUnit tests, in order to unblock CI while the underlying failures are investigated.

Changes:

  • Added Skip to 5 Azure Functions sample validation integration tests.
  • Added Skip to DurableTask integration tests covering entities and external client tool calls.
  • Disabled the DurableTask console app sample validation that is failing in CI.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs Adds a shared skip-reason constant and skips multiple Azure Functions sample validation tests.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/WorkflowConsoleAppSamplesValidation.cs Skips the Workflow+Agents console app sample validation test due to persistent CI failures.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ExternalClientTests.cs Adds a shared skip-reason constant and skips function-tool related external client tests.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/AgentEntityTests.cs Adds a shared skip-reason constant and skips multiple agent entity tests.

Comment thread dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/AgentEntityTests.cs Outdated
@SergeyMenshykh SergeyMenshykh force-pushed the sergeymenshykh-disable-failing-integration-tests branch from 5cecdd3 to ffc7977 Compare June 26, 2026 17:45
@SergeyMenshykh SergeyMenshykh force-pushed the sergeymenshykh-disable-failing-integration-tests branch from ffc7977 to 19f06a2 Compare June 26, 2026 17:48

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 90%

✓ Correctness

This PR correctly disables 11 persistently failing integration tests using xUnit's Skip mechanism. The RetryFact and Fact/Theory attributes all support the Skip property via inheritance from FactAttribute. The only consistency issue is that WorkflowConsoleAppSamplesValidation.cs uses an inline string literal for the skip reason instead of a shared constant like the other test files, but this is not a correctness bug. The existing review comments about including the tracking issue (#6732) in the skip messages are the most actionable feedback.

✓ Security Reliability

This PR exclusively adds Skip attributes to disable persistently failing integration tests. From a security and reliability perspective, there are no issues — no production code is modified, no secrets are introduced, and no unsafe patterns are added. The only concern (skip reasons linking only to an ephemeral Actions job URL rather than the tracking issue #6732) has already been raised in existing unresolved review comments.

✓ Test Coverage

This PR correctly adds Skip attributes to disable persistently failing integration tests. The mechanical changes are sound. The existing review thread already flags the stale CI-job-link concern. One additional consistency issue: WorkflowConsoleAppSamplesValidation.cs inlines the skip string rather than extracting a constant like the other three files, making future bulk-updates (e.g., adding the tracking issue link) easy to miss.

✓ Failure Modes

This PR only adds Skip attributes to disable persistently failing integration tests. No production code is changed. The RetryFact attribute correctly supports the Skip parameter via xUnit inheritance. There are no silent failure paths, swallowed exceptions, or operational failure modes introduced by this change. The existing review thread already covers the main discoverability concern about stale CI job links vs. tracking issue references.

✓ Design Approach

The main design concern is that the PR rationale says these tests are failing in CI, but the implementation hard-skips them in every environment. This removes local and ad-hoc validation for the affected DurableTask/AzureFunctions scenarios instead of quarantining the CI-specific failure mode the way other tests in this repo already do.

Suggestions

  • Use a CI-gated runtime skip pattern for these tests so they still run locally when the environment is healthy, similar to the existing Assert.SkipWhen(...) approach in dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosCheckpointStoreTests.cs:123-129 and dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosChatHistoryProviderTests.cs:142-148.

Automated review by SergeyMenshykh's agents

@SergeyMenshykh SergeyMenshykh added this pull request to the merge queue Jun 26, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 26, 2026
@SergeyMenshykh SergeyMenshykh added this pull request to the merge queue Jun 26, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 26, 2026
@SergeyMenshykh SergeyMenshykh added this pull request to the merge queue Jun 26, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 26, 2026
Skip the following tests that are persistently failing in CI:

DurableTask - AgentEntityTests:
- EntityNamePrefixAsync
- RunAgentMethodNamesAllWorkAsync
- OrchestrationIdSetDuringOrchestrationAsync

DurableTask - ExternalClientTests:
- SimplePromptAsync
- CallFunctionToolsAsync
- CallLongRunningFunctionToolsAsync

DurableTask - WorkflowConsoleAppSamplesValidation:
- ConcurrentWorkflowSampleValidationAsync
- WorkflowAndAgentsSampleValidationAsync

DurableTask - ConsoleAppSamplesValidation:
- SingleAgentSampleValidationAsync
- SingleAgentOrchestrationChainingSampleValidationAsync
- MultiAgentConcurrencySampleValidationAsync
- MultiAgentConditionalSampleValidationAsync

AzureFunctions - SamplesValidation:
- SingleAgentSampleValidationAsync
- MultiAgentOrchestrationConcurrentSampleValidationAsync
- MultiAgentOrchestrationConditionalsSampleValidationAsync
- LongRunningToolsSampleValidationAsync
- AgentAsMcpToolAsync

AzureFunctions - WorkflowSamplesValidation:
- WorkflowAndAgentsSampleValidationAsync
- ConcurrentWorkflowSampleValidationAsync

Related to: microsoft#6732

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SergeyMenshykh SergeyMenshykh force-pushed the sergeymenshykh-disable-failing-integration-tests branch from 19f06a2 to 6c6b6c2 Compare June 26, 2026 21:08
@SergeyMenshykh SergeyMenshykh added this pull request to the merge queue Jun 26, 2026
Merged via the queue into microsoft:main with commit d094514 Jun 26, 2026
28 checks passed
@github-project-automation github-project-automation Bot moved this from In Review to Done in Agent Framework Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants