fix(leader-election): publish leadership loss before retry

[GatewayJ]

Type of Change

• New Feature
• Bug Fix
• Documentation
• Performance Improvement
• Test/CI
• Refactor
• Other:

Related Issues

• Addresses rustfs/backlog#1082

Summary of Changes

• Move externally observable leader state publication into the election control loop instead of deriving it from user callbacks.
• Publish Pending immediately when renewal stops, before asynchronous cleanup or re-acquisition, and publish Leading only after acquisition succeeds.
• Report an observed remote lease holder as Following(identity) while acquiring.
• Invoke on_stopped_leading once for every completed leadership epoch without duplicating it when cancellation happens during re-acquisition.
• Drop the watch borrow before yielding state-stream items so consumers cannot block later state publication.
• Add deterministic integration coverage for renewal loss, re-acquisition, callback counts, and remote leader observation.

The root cause was a lifecycle mismatch: the elector retried acquisition internally after renewal loss, while the watch state was updated only by on_stopped_leading, which previously ran only when the entire election loop exited. As a result, LeaderElectorHandle::is_leader() could remain true throughout the non-leader re-acquisition window.

Checklist

• I have read and followed the CONTRIBUTING.md guidelines
• Passed make pre-commit (fmt-check + clippy + test + console-lint + console-fmt-check)
• Added/updated necessary tests
• Documentation updated (if needed)
• CHANGELOG.md updated under [Unreleased] (if user-visible change)
• CI/CD passed (if applicable)

Impact

• Breaking change (CRD/API compatibility)
• Requires doc/config/deployment update
• Other impact: LeaderElectorHandle now reports non-leader state immediately after renewal loss and emits accurate state transitions during re-acquisition.

Verification

cargo test -p kube-leader-election
make pre-commit

Additional Notes

No CRD, configuration, deployment, or public type changes are required. The operator currently uses LeaderElector::run directly, while this fix also makes the exported spawn/LeaderElectorHandle observation API safe for external consumers.

---

Thank you for your contribution! Please ensure your PR follows the community standards (CODE_OF_CONDUCT.md) and sign the CLA if this is your first contribution.

[GatewayJ]

@codex

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 2c5876290d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".