Skip to content

fix: stop stranding Stripe customers on failed signups - #404

Merged
ndemianc merged 3 commits into
developfrom
fix/stripe-orphan-customers-on-failed-signup
Jul 29, 2026
Merged

fix: stop stranding Stripe customers on failed signups#404
ndemianc merged 3 commits into
developfrom
fix/stripe-orphan-customers-on-failed-signup

Conversation

@ndemianc

Copy link
Copy Markdown
Member

The leak

User#create_stripe_customer hung off before_validation, on: :create, which fires even when the record is about to be rejected. So every failed signup called Stripe::Customer.create, kept the customer, and threw the user away — leaving a real, billable Stripe customer with no users row pointing at it.

#403 is the reason this surfaced. Until it shipped, the LevelCode Google flow called User.from_google without terms_accepted:, tripping validates :terms_accepted, acceptance: true, on: :create on every new Google signup — one orphaned Stripe customer per attempt, for the whole outage.

#403 fixed that one caller. This fixes the class: duplicate-email attempts and every other validation failure leak the same way, at a lower background rate, and still do in production today.

The fix

before_validationbefore_create. That is the correct hook for an external side effect: it runs only once validation has passed, and still before the INSERT, so stripe_id is part of the initial row and create_default_subscription (before_commit) still sees it.

Checked before moving it — nothing validates stripe_id, the only validates on User is the terms acceptance, the column is nullable, and no callback between validation and insert reads it.

This narrows the window rather than closing it absolutely. The call sits inside the create transaction, so a Stripe success followed by a failed INSERT (a unique-email race) can still strand one. That is a far smaller target than every rejected signup, and creating the customer after commit instead would need a second write and leave stripe_id nil for the subscription callback. The trade-off is written down at the callback.

Cleanup for what is already stranded

Second commit adds stripe:orphans:report (read-only) and stripe:orphans:delete (gated). The report diffs Stripe against users.stripe_id and users.levelcode_stripe_id over a window and classifies:

Class Meaning
ORPHAN_RETRY a user exists with the same email and a different stripe_id — rejected, retried, got in. Safest to delete.
ORPHAN_DUPLICATE several unreferenced customers share one email: a retry burst.
ORPHAN_UNMATCHED no user, no activity. Probably an orphan — but this is also exactly what a legitimately deleted user looks like, so it is quarantined, not proposed.
REVIEW_ACTIVITY has a subscription/invoice/charge/payment method. Never a signup that failed, so never deleted here.

delete refuses without CONFIRM=DELETE, takes only ORPHAN_RETRY by default, and re-checks the live database and Stripe activity per customer — someone rejected during the outage may since have signed up and been assigned that very customer, which would make the manifest stale. Both tasks refuse a live key outside production, since the report is a diff against this database's users table and a live key with a dev database would call every real customer an orphan.

Two limits, both printed in the report output:

  • thin.ly and LevelCode share one Stripe account and Stripe::Customer.create(email:) writes no metadata, so nothing on a customer says which product made it. The time window is the only separator — thin.ly signups that failed validation in the same window land in the report too.
  • The auth_events failure rows carry no email (the OAuth callback has no user by then), so they corroborate the orphan count and time span, not the identities.

Testing

Two examples pin the timing from both sides: a rejected create must not reach Stripe, an accepted one still must. They build User.new directly rather than using the factory, which no-ops create_stripe_customer and would have tested nothing.

Verified non-vacuous — with the callback put back on before_validation, the negative fails with exactly the orphan symptom:

expected: 0 times with any arguments
received: 1 time

1018 examples, 0 failures; rubocop clean. The rake task was exercised end to end against test-mode Stripe: 321 scanned, 305 unreferenced, all four classifications hit, and delete correctly refused without CONFIRM.

After merge

Deploy stops the ongoing leak. Then, to reconcile what is already there:

bundle exec rake stripe:orphans:report SINCE=2026-07-22

SINCE is an inference, not an observation — the LevelCode Google flow could not have completed before ecace42 (2026-07-22) because the authorize URL was built from an unset GOOGLE_OAUTH_ID. Check it against the observed first/last failure the report prints from auth_events, and sanity-check the failure count against the candidate count before deleting anything.

ndemianc added 2 commits July 28, 2026 15:20
`User#create_stripe_customer` hung off `before_validation, on: :create`, which
fires even when the record is about to be REJECTED. Every failed signup therefore
called `Stripe::Customer.create`, kept the customer, and threw the user away —
leaving a real, billable Stripe customer with no `users` row pointing at it.

This was not theoretical. `terms_accepted` is validated `acceptance: true,
on: :create`, and until #403 the LevelCode Google flow called `User.from_google`
without the flag, so it rejected EVERY new Google signup and minted one orphan
customer per attempt. #403 stopped that particular caller; this stops the whole
class of it, for duplicate-email attempts and every other validation failure.

`before_create` is the correct hook for an external side effect: it runs only once
validation has passed, and still before the INSERT, so `stripe_id` is part of the
initial row and `create_default_subscription` (before_commit) still sees it.

Checked before moving it: nothing validates `stripe_id` — the only `validates` on
User is the terms acceptance — the column is nullable, and no callback between
validation and insert reads it.

This narrows the orphan window rather than closing it absolutely: the call sits
inside the create transaction, so a Stripe success followed by a failed INSERT (a
unique-email race) can still strand one. That is a far smaller target than "every
rejected signup", and creating the customer after commit instead would need a
second write and leave `stripe_id` nil for the subscription callback.

Tests: two examples pin the timing from both sides — a rejected create must not
reach Stripe, an accepted one still must. They build `User.new` directly rather
than using the factory, which no-ops `create_stripe_customer` and would have
tested nothing. Verified non-vacuous: with the callback back on
`before_validation` the negative fails with "received: 1 time". 1018 examples
green; rubocop clean.
Cleanup for the customers already stranded in Stripe by the `before_validation`
callback moved in the previous commit — chiefly the burst from the LevelCode
Google outage, where every new signup was rejected and every attempt left a
customer behind.

`stripe:orphans:report` is READ-ONLY and is the entry point. It compares Stripe
against `users.stripe_id` AND `users.levelcode_stripe_id`, restricted to a window,
and classifies what it finds:

  ORPHAN_RETRY      a user exists with the same email and a different stripe_id —
                    rejected, retried, eventually got in. Safest to delete.
  ORPHAN_DUPLICATE  several unreferenced customers share one email: a retry burst.
  ORPHAN_UNMATCHED  no user, no activity — PROBABLY an orphan, but this is also
                    exactly what a legitimately deleted user looks like, so it is
                    quarantined rather than proposed for deletion.
  REVIEW_ACTIVITY   has a subscription/invoice/charge/payment method. Never a
                    signup that failed, so never deleted here.

`stripe:orphans:delete` refuses without CONFIRM=DELETE, accepts only ORPHAN_RETRY
by default, and re-checks the live database and Stripe activity per customer
before each delete — a person rejected during the outage may since have signed up
and been assigned that very customer, which would make the manifest stale.

Both tasks refuse to run a LIVE key outside production: the report is a diff
against this database's users table, so a live key with a dev/staging database
would call every real customer an orphan.

Two limits worth knowing, both documented in the task output. thin.ly and
LevelCode share ONE Stripe account and `Stripe::Customer.create(email:)` writes no
metadata, so nothing on a customer says which product created it — the time window
is the only separator. And the `auth_events` failure rows carry no email (the
OAuth callback has no user by then), so they corroborate the orphan COUNT and time
span, not the identities.

Exercised end to end against test-mode Stripe: 321 scanned, 305 unreferenced, all
four classifications hit, and the delete task correctly refused without CONFIRM.
Rubocop clean.
Copilot AI review requested due to automatic review settings July 28, 2026 19:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prevents orphaned, billable Stripe customers from being created when user signups fail validation by moving User#create_stripe_customer from a validation callback to a create callback, and adds tooling to identify and (optionally) delete already-orphaned Stripe customers.

Changes:

  • Move Stripe customer creation from before_validation to before_create to avoid side effects on rejected creates.
  • Add model specs that assert Stripe is not called for invalid creates, and is called for valid creates (storing stripe_id).
  • Add rake tasks to report and (gated) delete Stripe customers not referenced by users.stripe_id / users.levelcode_stripe_id.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
app/models/user.rb Moves Stripe customer creation to before_create and documents the transactional trade-off.
spec/models/user_spec.rb Adds regression specs to pin when the Stripe API call may occur during user creation.
lib/tasks/stripe_orphans.rake Introduces reconciliation tasks to report/delete unreferenced Stripe customers within a time window.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/tasks/stripe_orphans.rake
Comment on lines +120 to +122
emails = candidates.filter_map { |c| c.email.presence&.downcase }.uniq
users_by_email = User.where(email: emails).pluck(:email, :stripe_id).to_h
email_counts = candidates.filter_map { |c| c.email.presence&.downcase }.tally
Comment on lines +256 to +272
row = {
id: customer.id,
email: customer.email,
created: Time.zone.at(customer.created),
user_with_same_email: users_by_email[email].present?
}

classification =
if commercial_activity?(customer.id)
"REVIEW_ACTIVITY"
elsif email && users_by_email.key?(email) && users_by_email[email] != customer.id
"ORPHAN_RETRY"
elsif email && email_counts[email].to_i > 1
"ORPHAN_DUPLICATE"
else
"ORPHAN_UNMATCHED"
end
Addresses review on #404.

ORPHAN_RETRY is the one class `stripe:orphans:delete` acts on by default, and its
predicate was too loose:

    users_by_email[email] != customer.id

When an account existed with the candidate's email but its `stripe_id` was NULL,
this read `nil != "cus_..."` → true → ORPHAN_RETRY → queued for deletion. The
claim that class makes ("the person retried and their live customer is a different
one") was not supported: an account with no stripe id has no other customer for
this one to be the discard of. That is an anomaly worth a human, not a deletion.

The map now collects, per email, every customer id the account actually references
across BOTH columns, dropping blanks — so the three states are distinguishable:

    nil  -> no account with this email
    []   -> account exists, references no Stripe customer  (anomaly -> UNMATCHED)
    [..] -> account exists, references these customers     (retry -> ORPHAN_RETRY)

It also folds in `levelcode_stripe_id`, which the `known` set already treats as a
live reference; leaving it out of this map was inconsistent.

Verified directly on the flagged case — account present, stripe_id NULL: the old
predicate returns true (deletable), the new one false. A genuine retry (account
referencing a different customer) still classifies as ORPHAN_RETRY. The end-to-end
test-mode run is unchanged at 266/15/12/12 because no account in that dataset has
a NULL stripe_id, which is precisely why the defect was invisible there.

Also requires "set" explicitly. Not a live defect — the task runs under
`:environment`, where ActiveSupport has already loaded it, and Ruby has autoloaded
Set since 3.1 — but it costs nothing and makes the file self-contained.
@ndemianc
ndemianc merged commit 19761f6 into develop Jul 29, 2026
3 checks passed
@ndemianc
ndemianc deleted the fix/stripe-orphan-customers-on-failed-signup branch July 29, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants