fix(docs): runbook must not assume levelcode_stripe_id exists - #407
Merged
Conversation
Running step 1 on production died on:
PG::UndefinedColumn: column "levelcode_stripe_id" does not exist
The column is in db/schema.rb and in the model annotations, but it is NOT in the
production database, and no migration adds it — `grep -rl levelcode_stripe_id
db/migrate/` is empty. It entered the tree in b7b2cfc ("Add annotations",
2026-07-22), which touched only schema.rb and annotation comments: an annotate run
regenerated the schema from a local database where the column had been added by
hand.
So it exists wherever the database was built by `db:schema:load` — development and
test — and nowhere that was built by migrations. `db:migrate` will never create it,
because nothing defines it. Locally `User.first.levelcode_stripe_id` returns nil;
on production the attribute does not exist at all.
Step 1 now asks the database which columns are actually there:
id_cols = User.column_names & %w[stripe_id levelcode_stripe_id]
and uses that for the known-ids scan, the per-email account map, and the
pre-delete re-check, printing what it matched on. Correct on both shapes rather
than correct on the one that happened to be in front of me.
Verified by extracting the blocks and running them twice: as-is (prints "matching
on: stripe_id, levelcode_stripe_id") and with User.column_names patched to hide the
column, reproducing production exactly (prints "matching on: stripe_id"). Both
complete with identical classification and no error; customer count unchanged at
321.
The drift itself is worth a separate decision — add a real migration if LevelCode
billing separation is actually wanted, or drop the column from schema.rb if it is
not. Noted in the runbook's limits rather than fixed here, since this PR should not
be quietly changing the schema.
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Stripe orphan customer cleanup runbook to be resilient to schema drift between environments by determining which Stripe ID columns actually exist on users at runtime (instead of assuming levelcode_stripe_id exists).
Changes:
- Detects available Stripe customer-id columns via
User.column_namesand uses them throughout Step 1 (scan + account map). - Prints which columns were matched so operators can confirm the database “shape” they’re running against.
- Documents the underlying schema drift and calls out that it requires a separate decision/migration.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+56
to
+57
| id_cols = User.column_names & %w[stripe_id levelcode_stripe_id] | ||
| puts "matching on: #{id_cols.join(', ')}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #406, which merged before this landed. Running step 1 on production died immediately:
The actual finding
levelcode_stripe_idis indb/schema.rband in the model annotations, but it is not in the production database — and no migration adds it:It entered the tree in
b7b2cfc("Add annotations", 2026-07-22), a commit that touched onlydb/schema.rband annotation comments. An annotate run regenerated the schema from a local database where the column had been added by hand, and that snapshot was committed.The consequence is a clean split by how each database was built:
db:schema:load(development, test)User.first.levelcode_stripe_id→nildb:migratewill never create it, because nothing defines it. This is why it looked like a harmless dead column from the application code and only surfaced when the runbook actually queried it on production.The fix
Step 1 now asks the database rather than assuming:
That drives the known-ids scan, the per-email account map, and the pre-delete re-check, and it prints what it matched on so the operator can see which shape they are running against.
Verification
Blocks extracted from the markdown and run twice against test-mode Stripe:
matching on: stripe_id, levelcode_stripe_idUser.column_namespatched to hide the column, reproducing production exactly →matching on: stripe_idBoth complete without error, with identical classification (15 RETRY / 12 REVIEW_ACTIVITY / 278 other of 305) and the customer count unchanged at 321.
Not fixed here
The schema drift itself needs a decision, not a quiet edit in a docs PR: either add a real migration if LevelCode billing separation is genuinely wanted, or drop the column from
db/schema.rband the annotations if it is not. Recorded in the runbook's limits section. Worth noting the annotations inapp/models/user.rb,spec/models/user_spec.rbandspec/factories/users.rbcurrently describe a column production does not have, including aUNIQUEindex that does not exist there.