Skip to content

fix(agui): share resume coordination state across replicas - #2890

Open
ordinary-s wants to merge 3 commits into
agentscope-ai:mainfrom
ordinary-s:fix/agui-shared-resume-state
Open

fix(agui): share resume coordination state across replicas#2890
ordinary-s wants to merge 3 commits into
agentscope-ai:mainfrom
ordinary-s:fix/agui-shared-resume-state

Conversation

@ordinary-s

Copy link
Copy Markdown
Contributor

Summary

Make AG-UI resume coordination shareable across replicas so a resume request can be validated and claimed even when it reaches a different application instance.

Root cause

AguiResumeCoordinator kept pending interrupt metadata and active-run ownership in process-local maps. In a multi-replica deployment, the replica receiving a later resume request could not see the state recorded by the original replica. Concurrent requests routed to different replicas could also both acquire ownership because each process made the decision independently.

Implementation

  • Add the AguiResumeStateStore SPI for reading pending interrupts, atomically claiming/releasing a thread run, and owner-checked pending-state replacement.
  • Preserve existing single-process behavior with a default InMemoryAguiResumeStateStore.
  • Add RedisAguiResumeStateStore, using one Redis hash per thread and Lua scripts for atomic ownership and pending-state transitions.
  • Allow explicit store injection through AguiRequestProcessor.Builder and automatically consume an optional store bean in the MVC and WebFlux Spring Boot adapters.
  • Add coverage for cross-coordinator resume, competing claims, owner-safe updates, stale RUN_FINISHED events, Redis command/script behavior, and Spring bean wiring.

Testing

  • mvn -pl :agentscope-extensions-agui -am -Dtest=AguiResumeCoordinatorTest,AguiRequestProcessorTest -Dsurefire.failIfNoSpecifiedTests=false test — 38 tests passed, 0 failures, 0 errors, 0 skipped.
  • mvn -pl :agentscope-extensions-redis -am -Dtest=RedisAguiResumeStateStoreTest -Dsurefire.failIfNoSpecifiedTests=false test — 5 tests passed, 0 failures, 0 errors, 0 skipped.
  • mvn -pl :agentscope-agui-spring-boot-starter -am -Dtest=AguiAdapterConfigAutoConfigurationTest -Dsurefire.failIfNoSpecifiedTests=false test — 19 tests passed, 0 failures, 0 errors, 0 skipped.
  • mvn -pl :agentscope-extensions-agui -am -Dtest=Agui*Test,AgentResolverTest,SubagentEventConverterTest,AgentLifecycleEventConverterTest,ExternalAgentEventConverterTest -Dsurefire.failIfNoSpecifiedTests=false test — 500 tests passed, 0 failures, 0 errors, 0 skipped.
  • mvn spotless:check — passed across all 89 reactor modules.
  • git diff --check — passed.

Known limitation

A hard process crash while Redis ownership is held may leave a stale claim. This implementation intentionally avoids a fixed TTL because a valid long-running request could outlive it and permit dual ownership. Lease renewal with fencing tokens should be addressed in a follow-up.

Fixes #2773

@ordinary-s

Copy link
Copy Markdown
Contributor Author

The CI failure appears to be caused by the known flaky @tempdir cleanup race in agentscope-harness (DirectoryNotEmptyException, tracked in #2251). This PR does not modify the agentscope-harness module, and the failure occurred before the AG-UI/Redis modules were reached in the reactor build. Could someone please re-run the failed job when convenient?

@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.13043% with 15 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...tensions/redis/agui/RedisAguiResumeStateStore.java 76.47% 6 Missing and 6 partials ⚠️
...ope/core/agui/processor/AguiResumeCoordinator.java 92.85% 1 Missing and 1 partial ⚠️
...cope/core/agui/processor/AguiRequestProcessor.java 88.88% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: AguiResumeCoordinator loses HITL interrupt state across instances

1 participant