.NET: Disable failing DurableTask and AzureFunctions integration tests#6774
Conversation
There was a problem hiding this comment.
Pull request overview
This PR temporarily disables a set of persistently failing .NET integration tests (DurableTask + Azure Functions sample validations) by adding Skip reasons to the affected xUnit tests, in order to unblock CI while the underlying failures are investigated.
Changes:
- Added
Skipto 5 Azure Functions sample validation integration tests. - Added
Skipto DurableTask integration tests covering entities and external client tool calls. - Disabled the DurableTask console app sample validation that is failing in CI.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs | Adds a shared skip-reason constant and skips multiple Azure Functions sample validation tests. |
| dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/WorkflowConsoleAppSamplesValidation.cs | Skips the Workflow+Agents console app sample validation test due to persistent CI failures. |
| dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ExternalClientTests.cs | Adds a shared skip-reason constant and skips function-tool related external client tests. |
| dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/AgentEntityTests.cs | Adds a shared skip-reason constant and skips multiple agent entity tests. |
5cecdd3 to
ffc7977
Compare
ffc7977 to
19f06a2
Compare
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 90%
✓ Correctness
This PR correctly disables 11 persistently failing integration tests using xUnit's Skip mechanism. The RetryFact and Fact/Theory attributes all support the Skip property via inheritance from FactAttribute. The only consistency issue is that WorkflowConsoleAppSamplesValidation.cs uses an inline string literal for the skip reason instead of a shared constant like the other test files, but this is not a correctness bug. The existing review comments about including the tracking issue (#6732) in the skip messages are the most actionable feedback.
✓ Security Reliability
This PR exclusively adds Skip attributes to disable persistently failing integration tests. From a security and reliability perspective, there are no issues — no production code is modified, no secrets are introduced, and no unsafe patterns are added. The only concern (skip reasons linking only to an ephemeral Actions job URL rather than the tracking issue #6732) has already been raised in existing unresolved review comments.
✓ Test Coverage
This PR correctly adds Skip attributes to disable persistently failing integration tests. The mechanical changes are sound. The existing review thread already flags the stale CI-job-link concern. One additional consistency issue: WorkflowConsoleAppSamplesValidation.cs inlines the skip string rather than extracting a constant like the other three files, making future bulk-updates (e.g., adding the tracking issue link) easy to miss.
✓ Failure Modes
This PR only adds Skip attributes to disable persistently failing integration tests. No production code is changed. The RetryFact attribute correctly supports the Skip parameter via xUnit inheritance. There are no silent failure paths, swallowed exceptions, or operational failure modes introduced by this change. The existing review thread already covers the main discoverability concern about stale CI job links vs. tracking issue references.
✓ Design Approach
The main design concern is that the PR rationale says these tests are failing in CI, but the implementation hard-skips them in every environment. This removes local and ad-hoc validation for the affected DurableTask/AzureFunctions scenarios instead of quarantining the CI-specific failure mode the way other tests in this repo already do.
Suggestions
- Use a CI-gated runtime skip pattern for these tests so they still run locally when the environment is healthy, similar to the existing
Assert.SkipWhen(...)approach indotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosCheckpointStoreTests.cs:123-129anddotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosChatHistoryProviderTests.cs:142-148.
Automated review by SergeyMenshykh's agents
Skip the following tests that are persistently failing in CI: DurableTask - AgentEntityTests: - EntityNamePrefixAsync - RunAgentMethodNamesAllWorkAsync - OrchestrationIdSetDuringOrchestrationAsync DurableTask - ExternalClientTests: - SimplePromptAsync - CallFunctionToolsAsync - CallLongRunningFunctionToolsAsync DurableTask - WorkflowConsoleAppSamplesValidation: - ConcurrentWorkflowSampleValidationAsync - WorkflowAndAgentsSampleValidationAsync DurableTask - ConsoleAppSamplesValidation: - SingleAgentSampleValidationAsync - SingleAgentOrchestrationChainingSampleValidationAsync - MultiAgentConcurrencySampleValidationAsync - MultiAgentConditionalSampleValidationAsync AzureFunctions - SamplesValidation: - SingleAgentSampleValidationAsync - MultiAgentOrchestrationConcurrentSampleValidationAsync - MultiAgentOrchestrationConditionalsSampleValidationAsync - LongRunningToolsSampleValidationAsync - AgentAsMcpToolAsync AzureFunctions - WorkflowSamplesValidation: - WorkflowAndAgentsSampleValidationAsync - ConcurrentWorkflowSampleValidationAsync Related to: microsoft#6732 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
19f06a2 to
6c6b6c2
Compare
Description
Disables 19 integration tests that are persistently failing in CI (timing out). Each test now has a
Skipattribute with a reason referencing the tracking issue #6732.DurableTask - AgentEntityTests:
EntityNamePrefixAsyncRunAgentMethodNamesAllWorkAsync(Theory, 3 cases)OrchestrationIdSetDuringOrchestrationAsyncDurableTask - ExternalClientTests:
SimplePromptAsyncCallFunctionToolsAsyncCallLongRunningFunctionToolsAsyncDurableTask - WorkflowConsoleAppSamplesValidation:
ConcurrentWorkflowSampleValidationAsyncWorkflowAndAgentsSampleValidationAsyncDurableTask - ConsoleAppSamplesValidation:
SingleAgentSampleValidationAsyncSingleAgentOrchestrationChainingSampleValidationAsyncMultiAgentConcurrencySampleValidationAsyncMultiAgentConditionalSampleValidationAsyncAzureFunctions - SamplesValidation:
SingleAgentSampleValidationAsyncMultiAgentOrchestrationConcurrentSampleValidationAsyncMultiAgentOrchestrationConditionalsSampleValidationAsyncLongRunningToolsSampleValidationAsyncAgentAsMcpToolAsyncAzureFunctions - WorkflowSamplesValidation:
WorkflowAndAgentsSampleValidationAsyncConcurrentWorkflowSampleValidationAsyncRelated to #6732