fix: stop stranding Stripe customers on failed signups - #404
Merged
Conversation
`User#create_stripe_customer` hung off `before_validation, on: :create`, which fires even when the record is about to be REJECTED. Every failed signup therefore called `Stripe::Customer.create`, kept the customer, and threw the user away — leaving a real, billable Stripe customer with no `users` row pointing at it. This was not theoretical. `terms_accepted` is validated `acceptance: true, on: :create`, and until #403 the LevelCode Google flow called `User.from_google` without the flag, so it rejected EVERY new Google signup and minted one orphan customer per attempt. #403 stopped that particular caller; this stops the whole class of it, for duplicate-email attempts and every other validation failure. `before_create` is the correct hook for an external side effect: it runs only once validation has passed, and still before the INSERT, so `stripe_id` is part of the initial row and `create_default_subscription` (before_commit) still sees it. Checked before moving it: nothing validates `stripe_id` — the only `validates` on User is the terms acceptance — the column is nullable, and no callback between validation and insert reads it. This narrows the orphan window rather than closing it absolutely: the call sits inside the create transaction, so a Stripe success followed by a failed INSERT (a unique-email race) can still strand one. That is a far smaller target than "every rejected signup", and creating the customer after commit instead would need a second write and leave `stripe_id` nil for the subscription callback. Tests: two examples pin the timing from both sides — a rejected create must not reach Stripe, an accepted one still must. They build `User.new` directly rather than using the factory, which no-ops `create_stripe_customer` and would have tested nothing. Verified non-vacuous: with the callback back on `before_validation` the negative fails with "received: 1 time". 1018 examples green; rubocop clean.
Cleanup for the customers already stranded in Stripe by the `before_validation`
callback moved in the previous commit — chiefly the burst from the LevelCode
Google outage, where every new signup was rejected and every attempt left a
customer behind.
`stripe:orphans:report` is READ-ONLY and is the entry point. It compares Stripe
against `users.stripe_id` AND `users.levelcode_stripe_id`, restricted to a window,
and classifies what it finds:
ORPHAN_RETRY a user exists with the same email and a different stripe_id —
rejected, retried, eventually got in. Safest to delete.
ORPHAN_DUPLICATE several unreferenced customers share one email: a retry burst.
ORPHAN_UNMATCHED no user, no activity — PROBABLY an orphan, but this is also
exactly what a legitimately deleted user looks like, so it is
quarantined rather than proposed for deletion.
REVIEW_ACTIVITY has a subscription/invoice/charge/payment method. Never a
signup that failed, so never deleted here.
`stripe:orphans:delete` refuses without CONFIRM=DELETE, accepts only ORPHAN_RETRY
by default, and re-checks the live database and Stripe activity per customer
before each delete — a person rejected during the outage may since have signed up
and been assigned that very customer, which would make the manifest stale.
Both tasks refuse to run a LIVE key outside production: the report is a diff
against this database's users table, so a live key with a dev/staging database
would call every real customer an orphan.
Two limits worth knowing, both documented in the task output. thin.ly and
LevelCode share ONE Stripe account and `Stripe::Customer.create(email:)` writes no
metadata, so nothing on a customer says which product created it — the time window
is the only separator. And the `auth_events` failure rows carry no email (the
OAuth callback has no user by then), so they corroborate the orphan COUNT and time
span, not the identities.
Exercised end to end against test-mode Stripe: 321 scanned, 305 unreferenced, all
four classifications hit, and the delete task correctly refused without CONFIRM.
Rubocop clean.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR prevents orphaned, billable Stripe customers from being created when user signups fail validation by moving User#create_stripe_customer from a validation callback to a create callback, and adds tooling to identify and (optionally) delete already-orphaned Stripe customers.
Changes:
- Move Stripe customer creation from
before_validationtobefore_createto avoid side effects on rejected creates. - Add model specs that assert Stripe is not called for invalid creates, and is called for valid creates (storing
stripe_id). - Add rake tasks to report and (gated) delete Stripe customers not referenced by
users.stripe_id/users.levelcode_stripe_id.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| app/models/user.rb | Moves Stripe customer creation to before_create and documents the transactional trade-off. |
| spec/models/user_spec.rb | Adds regression specs to pin when the Stripe API call may occur during user creation. |
| lib/tasks/stripe_orphans.rake | Introduces reconciliation tasks to report/delete unreferenced Stripe customers within a time window. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+120
to
+122
| emails = candidates.filter_map { |c| c.email.presence&.downcase }.uniq | ||
| users_by_email = User.where(email: emails).pluck(:email, :stripe_id).to_h | ||
| email_counts = candidates.filter_map { |c| c.email.presence&.downcase }.tally |
Comment on lines
+256
to
+272
| row = { | ||
| id: customer.id, | ||
| email: customer.email, | ||
| created: Time.zone.at(customer.created), | ||
| user_with_same_email: users_by_email[email].present? | ||
| } | ||
|
|
||
| classification = | ||
| if commercial_activity?(customer.id) | ||
| "REVIEW_ACTIVITY" | ||
| elsif email && users_by_email.key?(email) && users_by_email[email] != customer.id | ||
| "ORPHAN_RETRY" | ||
| elsif email && email_counts[email].to_i > 1 | ||
| "ORPHAN_DUPLICATE" | ||
| else | ||
| "ORPHAN_UNMATCHED" | ||
| end |
Addresses review on #404. ORPHAN_RETRY is the one class `stripe:orphans:delete` acts on by default, and its predicate was too loose: users_by_email[email] != customer.id When an account existed with the candidate's email but its `stripe_id` was NULL, this read `nil != "cus_..."` → true → ORPHAN_RETRY → queued for deletion. The claim that class makes ("the person retried and their live customer is a different one") was not supported: an account with no stripe id has no other customer for this one to be the discard of. That is an anomaly worth a human, not a deletion. The map now collects, per email, every customer id the account actually references across BOTH columns, dropping blanks — so the three states are distinguishable: nil -> no account with this email [] -> account exists, references no Stripe customer (anomaly -> UNMATCHED) [..] -> account exists, references these customers (retry -> ORPHAN_RETRY) It also folds in `levelcode_stripe_id`, which the `known` set already treats as a live reference; leaving it out of this map was inconsistent. Verified directly on the flagged case — account present, stripe_id NULL: the old predicate returns true (deletable), the new one false. A genuine retry (account referencing a different customer) still classifies as ORPHAN_RETRY. The end-to-end test-mode run is unchanged at 266/15/12/12 because no account in that dataset has a NULL stripe_id, which is precisely why the defect was invisible there. Also requires "set" explicitly. Not a live defect — the task runs under `:environment`, where ActiveSupport has already loaded it, and Ruby has autoloaded Set since 3.1 — but it costs nothing and makes the file self-contained.
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The leak
User#create_stripe_customerhung offbefore_validation, on: :create, which fires even when the record is about to be rejected. So every failed signup calledStripe::Customer.create, kept the customer, and threw the user away — leaving a real, billable Stripe customer with nousersrow pointing at it.#403 is the reason this surfaced. Until it shipped, the LevelCode Google flow called
User.from_googlewithoutterms_accepted:, trippingvalidates :terms_accepted, acceptance: true, on: :createon every new Google signup — one orphaned Stripe customer per attempt, for the whole outage.#403 fixed that one caller. This fixes the class: duplicate-email attempts and every other validation failure leak the same way, at a lower background rate, and still do in production today.
The fix
before_validation→before_create. That is the correct hook for an external side effect: it runs only once validation has passed, and still before the INSERT, sostripe_idis part of the initial row andcreate_default_subscription(before_commit) still sees it.Checked before moving it — nothing validates
stripe_id, the onlyvalidatesonUseris the terms acceptance, the column is nullable, and no callback between validation and insert reads it.This narrows the window rather than closing it absolutely. The call sits inside the create transaction, so a Stripe success followed by a failed INSERT (a unique-email race) can still strand one. That is a far smaller target than every rejected signup, and creating the customer after commit instead would need a second write and leave
stripe_idnil for the subscription callback. The trade-off is written down at the callback.Cleanup for what is already stranded
Second commit adds
stripe:orphans:report(read-only) andstripe:orphans:delete(gated). The report diffs Stripe againstusers.stripe_idandusers.levelcode_stripe_idover a window and classifies:ORPHAN_RETRYstripe_id— rejected, retried, got in. Safest to delete.ORPHAN_DUPLICATEORPHAN_UNMATCHEDREVIEW_ACTIVITYdeleterefuses withoutCONFIRM=DELETE, takes onlyORPHAN_RETRYby default, and re-checks the live database and Stripe activity per customer — someone rejected during the outage may since have signed up and been assigned that very customer, which would make the manifest stale. Both tasks refuse a live key outside production, since the report is a diff against this database's users table and a live key with a dev database would call every real customer an orphan.Two limits, both printed in the report output:
Stripe::Customer.create(email:)writes no metadata, so nothing on a customer says which product made it. The time window is the only separator — thin.ly signups that failed validation in the same window land in the report too.auth_eventsfailure rows carry no email (the OAuth callback has no user by then), so they corroborate the orphan count and time span, not the identities.Testing
Two examples pin the timing from both sides: a rejected create must not reach Stripe, an accepted one still must. They build
User.newdirectly rather than using the factory, which no-opscreate_stripe_customerand would have tested nothing.Verified non-vacuous — with the callback put back on
before_validation, the negative fails with exactly the orphan symptom:1018 examples, 0 failures; rubocop clean. The rake task was exercised end to end against test-mode Stripe: 321 scanned, 305 unreferenced, all four classifications hit, and
deletecorrectly refused withoutCONFIRM.After merge
Deploy stops the ongoing leak. Then, to reconcile what is already there:
bundle exec rake stripe:orphans:report SINCE=2026-07-22SINCEis an inference, not an observation — the LevelCode Google flow could not have completed beforeecace42(2026-07-22) because the authorize URL was built from an unsetGOOGLE_OAUTH_ID. Check it against the observed first/last failure the report prints fromauth_events, and sanity-check the failure count against the candidate count before deleting anything.