Skip to content

fix(docs): runbook must not assume levelcode_stripe_id exists - #407

Merged
ndemianc merged 1 commit into
developfrom
chore/stripe-orphan-cleanup-runbook
Jul 29, 2026
Merged

fix(docs): runbook must not assume levelcode_stripe_id exists#407
ndemianc merged 1 commit into
developfrom
chore/stripe-orphan-cleanup-runbook

Conversation

@ndemianc

Copy link
Copy Markdown
Member

Follow-up to #406, which merged before this landed. Running step 1 on production died immediately:

PG::UndefinedColumn: ERROR:  column "levelcode_stripe_id" does not exist

The actual finding

levelcode_stripe_id is in db/schema.rb and in the model annotations, but it is not in the production database — and no migration adds it:

$ grep -rl levelcode_stripe_id db/migrate/
(nothing)

It entered the tree in b7b2cfc ("Add annotations", 2026-07-22), a commit that touched only db/schema.rb and annotation comments. An annotate run regenerated the schema from a local database where the column had been added by hand, and that snapshot was committed.

The consequence is a clean split by how each database was built:

Built by Has the column?
db:schema:load (development, test) yes — User.first.levelcode_stripe_idnil
migrations (production) no — the attribute does not exist

db:migrate will never create it, because nothing defines it. This is why it looked like a harmless dead column from the application code and only surfaced when the runbook actually queried it on production.

The fix

Step 1 now asks the database rather than assuming:

id_cols = User.column_names & %w[stripe_id levelcode_stripe_id]

That drives the known-ids scan, the per-email account map, and the pre-delete re-check, and it prints what it matched on so the operator can see which shape they are running against.

Verification

Blocks extracted from the markdown and run twice against test-mode Stripe:

  • as-ismatching on: stripe_id, levelcode_stripe_id
  • with User.column_names patched to hide the column, reproducing production exactly → matching on: stripe_id

Both complete without error, with identical classification (15 RETRY / 12 REVIEW_ACTIVITY / 278 other of 305) and the customer count unchanged at 321.

Not fixed here

The schema drift itself needs a decision, not a quiet edit in a docs PR: either add a real migration if LevelCode billing separation is genuinely wanted, or drop the column from db/schema.rb and the annotations if it is not. Recorded in the runbook's limits section. Worth noting the annotations in app/models/user.rb, spec/models/user_spec.rb and spec/factories/users.rb currently describe a column production does not have, including a UNIQUE index that does not exist there.

Running step 1 on production died on:

    PG::UndefinedColumn: column "levelcode_stripe_id" does not exist

The column is in db/schema.rb and in the model annotations, but it is NOT in the
production database, and no migration adds it — `grep -rl levelcode_stripe_id
db/migrate/` is empty. It entered the tree in b7b2cfc ("Add annotations",
2026-07-22), which touched only schema.rb and annotation comments: an annotate run
regenerated the schema from a local database where the column had been added by
hand.

So it exists wherever the database was built by `db:schema:load` — development and
test — and nowhere that was built by migrations. `db:migrate` will never create it,
because nothing defines it. Locally `User.first.levelcode_stripe_id` returns nil;
on production the attribute does not exist at all.

Step 1 now asks the database which columns are actually there:

    id_cols = User.column_names & %w[stripe_id levelcode_stripe_id]

and uses that for the known-ids scan, the per-email account map, and the
pre-delete re-check, printing what it matched on. Correct on both shapes rather
than correct on the one that happened to be in front of me.

Verified by extracting the blocks and running them twice: as-is (prints "matching
on: stripe_id, levelcode_stripe_id") and with User.column_names patched to hide the
column, reproducing production exactly (prints "matching on: stripe_id"). Both
complete with identical classification and no error; customer count unchanged at
321.

The drift itself is worth a separate decision — add a real migration if LevelCode
billing separation is actually wanted, or drop the column from schema.rb if it is
not. Noted in the runbook's limits rather than fixed here, since this PR should not
be quietly changing the schema.
Copilot AI review requested due to automatic review settings July 29, 2026 02:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Stripe orphan customer cleanup runbook to be resilient to schema drift between environments by determining which Stripe ID columns actually exist on users at runtime (instead of assuming levelcode_stripe_id exists).

Changes:

  • Detects available Stripe customer-id columns via User.column_names and uses them throughout Step 1 (scan + account map).
  • Prints which columns were matched so operators can confirm the database “shape” they’re running against.
  • Documents the underlying schema drift and calls out that it requires a separate decision/migration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +57
id_cols = User.column_names & %w[stripe_id levelcode_stripe_id]
puts "matching on: #{id_cols.join(', ')}"
@ndemianc
ndemianc merged commit 5f11953 into develop Jul 29, 2026
4 checks passed
@ndemianc
ndemianc deleted the chore/stripe-orphan-cleanup-runbook branch July 29, 2026 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants