fix(payment-service): recover payments stuck in PROCESSING (#18)

[Yousefa7medmaher]

Summary

Payments can no longer get stuck in PROCESSING (#18).

Closes #18.

Changes

• Claim stamp and owner. Claiming a payment (PENDING → PROCESSING) stamps processingClaimedAt (database clock) and a fresh processingToken. Migration 004 adds both columns as nullable.
• Active claims stay protected. A retry within PAYMENT_PROCESSING_TIMEOUT_MS (default 60 s) gets 409, as before.
• Expired claims are taken over. After the timeout, a retry takes the claim over with one atomic UPDATE … WHERE "processingClaimedAt" < now() - timeout and settles the payment. Of many concurrent takeovers exactly one wins.
• Settlement is fenced. It needs the current token (WHERE "processingToken" = <own>). A request whose claim was taken over writes nothing and returns what the new holder settled, so a payment is settled exactly once.
• Abandoned payments are recovered. If the customer never retries, the existing payment sweep settles payments still PROCESSING after PAYMENT_PROCESSING_ABANDON_MS (default 15 min) as FAILED: "Payment processing was interrupted, so you were not charged." Then payment.failed is published, the order moves to FAILED and the customer is notified.
• Deploy-safe. Rows already stuck have no claim time and count as abandoned, so they become recoverable right after deploy.
• Simulator only. Re-deciding the outcome on a takeover, and "not charged", are right because the simulator charges nothing until it settles. The docs state that a real gateway must look the charge up by its idempotency key at those two points.

Docs: docs/services/payment-service.md ("Stuck PROCESSING recovery"), .project-context/04-database-design.md and .project-context/05-event-driven-design.md.

Lease ownership for side effects is the next issue, #19.

Test plan

• Unit tests: 66 service tests, including:
  • a fresh claim gets 409;
  • an abandoned claim is taken over and settled once;
  • a legacy row with no claim time is recoverable;
  • of two concurrent takeovers, one settles;
  • a request that was taken over can't settle, and returns the new holder's result;
  • the sweep fails a long-abandoned payment and leaves recent claims alone.
• PostgreSQL 16 integration (PAYMENT_TEST_DATABASE_URL), 21 tests, including:
  • 10 concurrent takeovers of an expired claim → exactly 1 winner;
  • the old holder's token can't settle; the new holder settles and the token is cleared;
  • a legacy PROCESSING row is listed as abandoned and reclaimable;
  • the legacy-upgrade test now also applies migration 004 and still matches the entity.
• Migrations on a fresh database add processingClaimedAt timestamptz and processingToken uuid.
• tsc; lint shows 0 errors.
• CI

🤖 Generated with Claude Code