Rewrite large object migration as a direct relation copy.

Makefile

@@ -8,7 +8,8 @@ DATA = lolor--1.0.sql \
 	   lolor--1.2.2--1.3.0.sql
 PGFILEDESC = "lolor - drop in large objects replacement for logical replication"
 
-OBJS = src/lolor.o src/lolor_fsstubs.o src/lolor_inv_api.o src/lolor_largeobject.o
+OBJS = src/lolor.o src/lolor_fsstubs.o src/lolor_inv_api.o src/lolor_largeobject.o \
+       src/lolor_migrate.o
 
 REGRESS = lolor
 TAP_TESTS = 1

README.md

@@ -88,7 +88,25 @@ SELECT lolor.migrate_to_native();  -- manual
 DROP EXTENSION lolor;              -- automatic
 ```
 
-Both directions preserve original OIDs, owners, ACLs, and data.
+Both directions preserve original OIDs, owners, ACLs, comments and data, and
+copy pages verbatim so sparse large objects stay sparse. Moving an object to
+native storage also reinstates its `pg_shdepend` entries, so `DROP ROLE`,
+`REASSIGN OWNED` and `DROP OWNED` continue to see it.
+
+`migrate_to_native()` does not require lolor to be enabled; it operates on the
+catalogs directly.
+
+Security labels on large objects cannot be represented in lolor storage and
+cannot be reinstated without their label provider, so `migrate_from_native()`
+refuses rather than discarding them. Remove them first if you hit this.
+
+Two helpers support verifying a migration:
+
+```sql
+SELECT * FROM lolor.digest();         -- per-object checksum, compare across nodes
+SELECT * FROM lolor.check_orphans();  -- objects referring to a dropped role
+SELECT lolor.fix_orphans('some_role');  -- reassign them and drop dead ACL entries
+```
 
 When the spock extension is installed, both migration functions run under
 `spock.repair_mode()`, so the row-shuffling migration DML is **not**
@@ -101,6 +119,7 @@ Even with spock, migration is refused if a non-spock logical replication slot
 own output plugin and those consumers would still decode the migration DML:
 `migrate_to_native()` raises an `ERROR`, while `migrate_from_native()` warns and
 returns -1 without doing anything. Drop the offending slots before migrating.
+The check identifies spock's slots by their `spock_output` plugin.
 
 Without spock, the migration DML cannot be excluded from logical decoding, so
 both functions refuse to migrate while logical replication slots exist in the
@@ -123,8 +142,8 @@ database user. Upgrading to 1.3.0 revokes the privilege; see the release notes.
 ### Limitations
 
 - Native large object functionality cannot be used while you are using the lolor extension.
-- lolor does not support the following statements: `ALTER LARGE OBJECT`, `GRANT ON LARGE OBJECT`, `COMMENT ON LARGE OBJECT`, and `REVOKE ON LARGE OBJECT`.
-- `lolor.enable()` and `lolor.disable()` change which function OID owns each `pg_catalog.lo_*` name. A rename keeps the OID, so a session that has already resolved those functions keeps calling the previous implementation. Both therefore refuse while any other session is connected to the database, the same rule as `ALTER DATABASE ... RENAME`, and the calling session must reconnect afterwards.
-- Objects in lolor storage are rows in ordinary tables and so cannot participate in `pg_shdepend`. `DROP ROLE`, `DROP OWNED BY` and `REASSIGN OWNED BY` do not see them: a role that owns them or appears in their ACL can be dropped without a warning, and `REASSIGN OWNED` / `DROP OWNED` leave them untouched. Before dropping a role, run `REASSIGN OWNED BY` or `DROP OWNED BY` in each database that has lolor, then `DROP ROLE`. Afterwards run `lolor.check_orphans()` in each such database and repair anything it reports with `lolor.fix_orphans(new_owner)`.
+- lolor does not support the following statements against objects held in lolor storage: `ALTER LARGE OBJECT`, `GRANT ON LARGE OBJECT`, `COMMENT ON LARGE OBJECT`, and `REVOKE ON LARGE OBJECT`. Owners, ACLs and comments set while an object was in native storage are preserved across migration in both directions.
+- `lolor.enable()`, `lolor.disable()` and the migration functions refuse while any other session is connected to the database, the same rule as `ALTER DATABASE ... RENAME`: a renamed function keeps its OID, so a session that already resolved the `lo_*` functions keeps calling the previous implementation, and the row movement cannot be made atomic for other backends either. The calling session must reconnect after `enable()` or `disable()`.
+- Objects in lolor storage are rows in ordinary tables and so cannot participate in `pg_shdepend`. `DROP ROLE`, `DROP OWNED BY` and `REASSIGN OWNED BY` do not see them: a role that owns them or appears in their ACL can be dropped without a warning, and `REASSIGN OWNED` / `DROP OWNED` leave them untouched. Before dropping a role, run `REASSIGN OWNED BY` or `DROP OWNED BY` in each database that has lolor, then `DROP ROLE`. Afterwards run `lolor.check_orphans()` in each such database and repair anything it reports with `lolor.fix_orphans(new_owner)`; `lolor.migrate_to_native()` refuses while orphans exist, and since `DROP EXTENSION` runs that migration, so does the drop.
 - Role OIDs come from a cluster-wide counter that wraps around, so a new role can receive a dropped role's OID and silently become the owner or grantee of that role's orphaned objects. This cannot be detected after the fact, which is another reason to run `lolor.check_orphans()` promptly after dropping roles.
-- Large object migration is node-local. Native large objects live in `pg_catalog.pg_largeobject`, which is never replicated, so each node holds an independent set and `migrate_from_native()` migrates only the local node's objects; with spock installed, the migration DML runs in repair mode and is not replicated. Run the migration on every node that holds native large objects — for example with `spock.replicate_ddl('SELECT lolor.migrate_from_native()')`, which queues the command so that each node executes it locally. Migrated objects keep their original native OIDs, which are not node-encoded: if different nodes hold different objects under the same OID, the nodes' lolor contents will diverge and later replicated changes to those objects can conflict. Newly created large objects are collision-free, since new OIDs are node-encoded via `lolor.node` and checked against existing rows.
+- Large object migration is node-local. Native large objects live in `pg_catalog.pg_largeobject`, which is never replicated, so each node holds an independent set and `migrate_from_native()` migrates only the local node's objects; with spock installed, the migration DML runs in repair mode and is not replicated. Run the migration on every node that holds native large objects, connecting to each node directly. Do not send it through DDL replication: that runs the migration inside every subscriber's apply worker, where a refusal (an OID conflict, a security label) leaves that node's replication retrying forever, and where a node can receive another node's post-migration changes before it has migrated itself. Migrated objects keep their original native OIDs, which are not node-encoded: if different nodes hold different objects under the same OID, the nodes' lolor contents will diverge and later replicated changes to those objects can conflict. Newly created large objects are collision-free, since new OIDs are node-encoded via `lolor.node` and checked against existing rows. To make that hazard an error rather than silent divergence, collect the other nodes' OIDs with `lolor.native_lo_oids()` and pass them in: `SELECT lolor.migrate_from_native(peer_oids => ARRAY[...])` refuses when any of them collide. After migrating every node, compare `lolor.digest()` across nodes to confirm they converged.

docs/lolor_release_notes.md

@@ -2,22 +2,25 @@
 
 ## lolor 1.3.0
 
-* Add bidirectional large object migration between native PostgreSQL and lolor storage:
-  * `lolor.migrate_from_native()` migrates existing native large objects into lolor storage. This is a manual step (run after `CREATE EXTENSION lolor`) and requires superuser privileges.
-  * `lolor.migrate_to_native()` migrates lolor large objects back to native storage.
-  * Reverse migration runs automatically on `DROP EXTENSION lolor`, so large objects are never lost when the extension is removed.
-  * With the spock extension installed, both migration functions run under `spock.repair_mode()` so migration is replication-safe. If a logical replication slot that spock cannot suppress is present — a non-spock slot, or any logical slot when spock is absent — the functions refuse to migrate rather than risk losing objects.
-  * Migration is node-local: native large objects are never replicated, so each node holds an independent set and `migrate_from_native()` migrates only the local node's objects. Run it on every node that holds native large objects, for example via `spock.replicate_ddl('SELECT lolor.migrate_from_native()')`. Migrated objects keep their original native OIDs, which are not node-encoded and can collide across nodes if different nodes hold different objects under the same OID; newly created large objects are collision-free, since new OIDs are node-encoded via `lolor.node` and checked against existing rows.
-* **Security fix: `lo_import()` and `lo_export()` were executable by any database user.** These read and write files on the server as the account PostgreSQL runs under, so core revokes `EXECUTE` on them from `PUBLIC`. lolor replaces them by renaming the originals to `*_orig`; an ACL belongs to a function rather than a name, so the restriction stayed on the parked original while each replacement got the default `EXECUTE TO PUBLIC`. Any user could read an arbitrary server file with `lo_import()` or overwrite one with `lo_export()`. The replacements are now locked down at install time, and upgrading revokes the privilege on existing installations in either state. Versions 1.0 through 1.2.2 are affected.
-* The extension is no longer marked `trusted`. Installing lolor renames functions in `pg_catalog` for the whole database, which a non-superuser should not be able to do.
-* Fixed the `lolor.node` upper bound. The GUC accepted 0..16 while a generated OID reserves four bits for the node id, so node 16 did not fit: the node field of every OID it generated read back as 0. The bound is now derived from the encoding (`LOLOR_MAX_NODE_ID`), giving 0..15. **If you have `lolor.node = 16` configured**, the server still starts but logs `16 is outside the valid range for parameter "lolor.node" (0 .. 15)` and falls back to 0, which is the node id it was effectively using already. Set it to a value in 0..15, and check for OID collisions against whichever node is genuinely 0.
+* Add bidirectional large object migration between native PostgreSQL and lolor storage, implemented as a direct relation-to-relation copy in C (`lolor.migrate_storage()`) rather than a loop through the large object API:
+  * `lolor.migrate_from_native(peer_oids oid[] DEFAULT NULL)` moves native large objects into lolor storage. It is a manual step after `CREATE EXTENSION lolor`, requires superuser, and returns the number of objects moved or -1 if the migration was refused.
+  * `lolor.migrate_to_native()` moves them back. It runs automatically when the extension is dropped, whether by `DROP EXTENSION`, `DROP SCHEMA lolor CASCADE` or `DROP OWNED BY` the extension owner, and works whether or not lolor is enabled.
+  * Both directions preserve OIDs, owners, ACLs, comments and the exact page layout, so sparse objects stay sparse. Ownership and ACLs are recorded in `pg_shdepend` on the way back, so `DROP ROLE`, `REASSIGN OWNED` and `DROP OWNED` see the objects again. Comments are parked in `lolor.pg_largeobject_description` while an object is in lolor storage. Native objects are removed through `performMultipleDeletions()`, the path `DROP` uses, so their dependencies, comments and labels are cleaned up.
+  * Storage layouts are verified against the running server at migration time. Security labels, which lolor storage cannot represent, cause `migrate_from_native()` to refuse. OID conflicts are reported with the conflicting OIDs.
+  * Migration holds `ShareRowExclusiveLock` on both stores until commit. Ordinary large object access takes `RowExclusiveLock`, which does not conflict with itself, so a concurrent `lo_write()` could otherwise be lost silently. Opposing migrations take their locks in a fixed order and cannot deadlock.
+  * Both migration functions refuse while any other session is connected to the database, for the same reason `enable()` and `disable()` do.
+  * With spock installed, both functions run under `spock.repair_mode()`. If a logical slot spock cannot suppress exists, a non-spock slot or any slot when spock is absent, they refuse rather than let subscribers decode the migration as row changes.
+  * Migration is node-local: native large objects are never replicated, so run the migration on each node by connecting to it directly. Do not send it through DDL replication, which would run it inside every subscriber's apply worker. Native OIDs are not node-encoded, so pass the other nodes' OIDs (from `lolor.native_lo_oids()`) as `peer_oids` to turn a cross-node collision into a refusal, and compare `lolor.digest()` across nodes afterwards.
+  * `migrate_to_native()` refuses with a clear error while any object refers to a dropped role, instead of failing part-way with "role N was concurrently dropped".
+* New helpers: `lolor.digest()` and `lolor.native_lo_oids()` for cross-node checks, and `lolor.check_orphans()` and `lolor.fix_orphans(new_owner)` for objects whose owner, grantee or grantor has been dropped. Objects in lolor storage cannot participate in `pg_shdepend`, so `DROP ROLE` does not notice them; `check_orphans()` lists them and which reference is dangling, and `fix_orphans()` reassigns dead owners, replacing them in the ACL as `REASSIGN OWNED` would so that grants they made survive, and drops entries that still name a missing role.
+* **Security fix: `lo_import()` and `lo_export()` were executable by any database user.** These read and write files on the server as the account PostgreSQL runs under, so core revokes `EXECUTE` on them from `PUBLIC`. lolor replaces them by renaming the originals to `*_orig`; an ACL belongs to a function rather than a name, so the restriction stayed on the parked original while each replacement got the default `EXECUTE TO PUBLIC`. Any user could read an arbitrary server file with `lo_import()` or overwrite one with `lo_export()`. The replacements are now locked down at install time, and upgrading revokes the privilege on existing installations in either state. Versions 1.0 through 1.2.2 are affected. A pre-release build installed from the development branch under the 1.3.0 number before this change does not pick the fix up through `ALTER EXTENSION lolor UPDATE`, which does nothing when the version is unchanged; reinstall the extension, or run the three `REVOKE` statements from `lolor--1.2.2--1.3.0.sql` by hand.
 * **Fixed `DROP ROLE` failing with "unrecognized object class".** Creating a large object recorded a `pg_shdepend` row whose `classId` was the OID of `lolor.pg_largeobject`, an ordinary table rather than a catalog. Any role that had created a large object became undroppable, and the rows were never cleaned up because `inv_drop()` deleted with `PERFORM_DELETION_SKIP_ORIGINAL`. lolor no longer records these rows, and the upgrade removes the ones already present.
   * The cleanup is per database, so `ALTER EXTENSION lolor UPDATE` must be run in every database that has lolor. Until it is, `DROP ROLE` anywhere in the cluster keeps reporting "owner of objects in database X".
   * A database that ran `DROP EXTENSION lolor` on an earlier version still has the rows, now pointing at a table that no longer exists, and no upgrade script will run there. Clean it by hand as superuser in that database: find the stale class OIDs with `SELECT DISTINCT classid FROM pg_shdepend WHERE dbid = (SELECT oid FROM pg_database WHERE datname = current_database()) AND classid NOT IN (SELECT oid FROM pg_class)`, then `DELETE FROM pg_shdepend WHERE dbid = <that dbid> AND classid IN (<those OIDs>)`.
-* New helpers `lolor.check_orphans()` and `lolor.fix_orphans(new_owner)` for objects whose owner, grantee or grantor has been dropped. Objects in lolor storage cannot participate in `pg_shdepend`, so `DROP ROLE` does not notice them; `check_orphans()` lists them and which reference is dangling, and `fix_orphans()` reassigns dead owners, replacing them in the ACL as `REASSIGN OWNED` would so that grants they made survive, and drops entries that still name a missing role.
-* Cleanup now runs for every spelling of the drop. `DROP SCHEMA lolor CASCADE` and `DROP OWNED BY` reach the extension by dependency cascade rather than as `DROP EXTENSION`; the event trigger did not fire for them, so the large objects were destroyed along with the lolor tables and `pg_catalog` was left without a working `lo_open()`.
 * `lolor.enable()`, `lolor.disable()` and `lolor.is_enabled()` now probe exact function signatures in `pg_catalog`. They previously matched on `proname` across every schema, so any user with `CREATE` on any schema could define a function named `lolor_lo_open` and wedge lolor into a permanent "inconsistent state" that also blocked `DROP EXTENSION`. Both now emit a notice that client sessions must reconnect, since libpq caches the large object function OIDs per connection, and refuse while any other session is connected to the database, since a renamed function keeps its OID and other sessions cannot be made to see the switch.
-* Expanded test coverage: TAP tests for dump/restore, streaming and logical replication, and standby promotion; regression tests for `lo_lseek`, `lo_tell`, and `lo_truncate`.
+* Fixed the `lolor.node` upper bound. The GUC accepted 0..16 while a generated OID reserves four bits for the node id, so node 16 did not fit: the node field of every OID it generated read back as 0. The bound is now derived from the encoding (`LOLOR_MAX_NODE_ID`), giving 0..15. **If you have `lolor.node = 16` configured**, the server still starts but logs `16 is outside the valid range for parameter "lolor.node" (0 .. 15)` and falls back to 0, which is the node id it was effectively using already. Set it to a value in 0..15, and check for OID collisions against whichever node is genuinely 0.
+* The extension is no longer marked `trusted`. Installing lolor renames functions in `pg_catalog` for the whole database, which a non-superuser should not be able to do.
+* Expanded test coverage: TAP tests for dump/restore, streaming and logical replication, standby promotion, migration fidelity and locking, drop paths, server-side file privileges and cross-node OID collisions; regression tests for `lo_lseek`, `lo_tell`, `lo_truncate`, the 64-bit interface, sparse objects, comment round trips, `pg_shdepend` restoration, orphaned roles and name squatting.
 * Security hardening: addressed Codacy/Flawfinder warnings.
 
 ## lolor 1.2.2