Repository navigation
Rewrite large object migration as a direct relation copy. - #56
ibrarahmad wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Up to standards ✅🟢 Issues
|
| Category | Results |
|---|---|
| Compatibility | 16 high (4 false positives) |
| Complexity | 5 medium |
🟢 Metrics 67 complexity · 2 duplication
Metric Results Complexity 67 Duplication 2
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
3ce3474 to
80691ac
Compare
6088141 to
ad9862d
Compare
80691ac to
acb70e2
Compare
ad9862d to
c3d60b9
Compare
acb70e2 to
078a19b
Compare
c3d60b9 to
1f7b1cf
Compare
078a19b to
87386b9
Compare
1f7b1cf to
b5f7d0f
Compare
87386b9 to
8928846
Compare
b5f7d0f to
b553171
Compare
8928846 to
beb25e7
Compare
b553171 to
da6520b
Compare
beb25e7 to
85f6c84
Compare
da6520b to
a0a9408
Compare
85f6c84 to
a38bd97
Compare
a0a9408 to
9b1bf14
Compare
a38bd97 to
c72e094
Compare
54f085b to
04a66f0
Compare
c72e094 to
757dfd6
Compare
04a66f0 to
b49849f
Compare
757dfd6 to
e535164
Compare
b49849f to
02a1772
Compare
e535164 to
3c3780c
Compare
02a1772 to
48d51e1
Compare
Migration rewrote every object through the large object API, which lost information the API cannot carry. Copy tuples between the two relations instead: their layouts match by construction, and that layout is now verified at run time rather than assumed. Comments were silently discarded, having nowhere to live while an object sits in lolor storage; they are parked in lolor.pg_largeobject_description and reinstated on the way back. Ownership was applied with a raw catalog UPDATE, leaving no pg_shdepend row, so DROP ROLE succeeded on a role that still owned migrated objects and DROP OWNED could delete another user's; the dependencies are now recorded properly. Pages are copied verbatim, so a sparse 10 MB object no longer materialises as 10 MB of pages. Native objects go through performMultipleDeletions(), the path DROP uses, so core removes their dependencies, comments and labels. Hold ShareRowExclusiveLock on both stores until commit. Large object access takes RowExclusiveLock, which does not conflict with itself, so a concurrent lo_write() could commit between the page copy and the emptying of the source store and be lost without trace. The locks are taken in the same order in either direction, so opposing migrations cannot deadlock. migrate_to_native() no longer calls the renamed _orig functions, so it works whether or not lolor is enabled, and DROP EXTENSION in the disabled state now rescues the objects instead of failing. Security labels cannot be represented in lolor storage and cannot be replayed without their provider, so migration refuses rather than discarding them. Add lolor.digest() and lolor.native_lo_oids() with a peer_oids pre-flight, which turns the documented cross-node OID collision hazard into a refusal.
48d51e1 to
2ea271d
Compare
3c3780c to
a727e14
Compare
Based on #55.
Migration used to rewrite every large object through the LO API, which drops things the API cannot carry. It is now a direct copy of tuples between the native and lolor relations, in C (
lolor.migrate_storage(),src/lolor_migrate.c). The layouts match by construction and are checked at run time.What that fixes: comments were lost (now parked in
lolor.pg_largeobject_descriptionand put back); ownership was set with a raw UPDATE, soDROP ROLE,REASSIGN OWNEDandDROP OWNEDdid not see migrated objects (now recorded inpg_shdepend); sparse objects were zero-filled (pages are copied as is); native objects are removed in batches throughperformMultipleDeletions().Migration holds
ShareRowExclusiveLockon both stores until commit, so a concurrentlo_write()cannot land between the copy and the delete and be lost. Locks are taken in a fixed order.Both migrations also refuse while any other session is connected to the database, for the same reason
enable()anddisable()do in #55. The concurrent-write test int/010is now a refusal test, and the lock is checked from the migrating session itself.Also:
migrate_to_native()no longer needs lolor enabled, soDROP EXTENSIONworks in the disabled state; migration refuses on security labels and on objects that refer to a dropped role, with a clear message instead of "role N was concurrently dropped"; OID conflicts list the OIDs;lolor.digest()andlolor.native_lo_oids()with apeer_oidscheck for cross-node collisions;lo_unlink()removes a parked comment so a reused OID cannot inherit it. The README no longer suggests running the migration throughspock.replicate_ddl(), since that runs it inside apply workers.1.3.0 is unreleased, so the SQL goes into
lolor--1.2.2--1.3.0.sql.Tests:
t/009_migration.plandt/010_migration_locking.pl, plus regression coverage for sparse objects, comments,pg_shdepend, orphans and the helpers.