Fix the pg_shdepend rows, guard the drop, and harden enable and disable.

Makefile

@@ -5,12 +5,18 @@ MODULE_big = lolor
 EXTENSION = lolor
 DATA = lolor--1.0.sql \
 	   lolor--1.0--1.2.1.sql lolor--1.2.1--1.2.2.sql \
-	   lolor--1.2.2--1.3.0.sql
+	   lolor--1.2.2--1.2.3.sql lolor--1.2.3--1.3.0.sql
 PGFILEDESC = "lolor - drop in large objects replacement for logical replication"
 
 OBJS = src/lolor.o src/lolor_fsstubs.o src/lolor_inv_api.o src/lolor_largeobject.o
 
 REGRESS = lolor
+# The drop guard is an object_access_hook, so lolor has to be preloaded and the
+# regression suite can only run against a server that preloads it.  Run it in a
+# temporary instance configured that way instead of against whatever server
+# pg_config points at; this also keeps the suite's cluster-global test roles out
+# of any shared server.
+REGRESS_OPTS = --temp-instance=./tmp_check --temp-config=regress.conf
 TAP_TESTS = 1
 
 ifdef USE_PGXS

README.md

@@ -38,15 +38,24 @@ make USE_PGXS=1
 make USE_PGXS=1 install
 ```
 
-After installing the lolor extension, connect to your Postgres database and create the extension with the command:
+After installing the lolor extension, add it to `shared_preload_libraries` and restart the server (see [Configuring lolor](#configuring-lolor)), then connect to your Postgres database and create the extension with the command:
 
 ```
 CREATE EXTENSION lolor;
 ```
 
 ### Configuring lolor
 
-You must set the `lolor.node` parameter before using the extension. The value can be from 1 to 2^28; the value is used to help in generation of new large object OID.
+lolor must be loaded at server start. Add it to `shared_preload_libraries` in
+`postgresql.conf` and restart; `CREATE EXTENSION lolor`, `LOAD 'lolor'` and any call that reaches one of lolor's functions fail with `lolor must be loaded via "shared_preload_libraries"` when the library was loaded on demand; the native `pg_catalog.lo_*` functions are not affected while lolor is not installed or is disabled. The guard that protects large
+objects when the extension is dropped is an object access hook, which is only
+in place in every backend when the library is preloaded.
+
+```
+shared_preload_libraries = 'lolor'
+```
+
+You must set the `lolor.node` parameter before using the extension. The value can be from 1 to 15 (0 means unset); it is encoded in the four low bits of every large object OID lolor generates, so each node must use a different value.
 
 ```
 lolor.node = 1
@@ -70,8 +79,8 @@ SELECT spock.repset_add_table('spock_replication_set', 'lolor.pg_largeobject_met
 
 ### Migrating large objects
 
-Migration from native to lolor is **manual**; migration back is **automatic**
-on `DROP EXTENSION` so no objects are ever lost.
+Migration is a manual step in both directions. Dropping the extension never
+moves data: it is refused while any object remains in lolor storage.
 
 Migrate existing native large objects into lolor storage (requires superuser):
 
@@ -80,16 +89,31 @@ CREATE EXTENSION lolor;
 SELECT lolor.migrate_from_native();
 ```
 
-Reverse migration happens automatically when the extension is dropped, or can
-be triggered manually:
+To remove the extension, migrate the objects back first:
 
 ```sql
-SELECT lolor.migrate_to_native();  -- manual
-DROP EXTENSION lolor;              -- automatic
+SELECT lolor.migrate_to_native();
+DROP EXTENSION lolor;
 ```
 
+An object access hook refuses to remove the extension while it is enabled or
+while any object remains in lolor storage, on every path that reaches it
+(`DROP EXTENSION`, `DROP SCHEMA lolor CASCADE`, `DROP OWNED BY`). The drop
+itself only puts the native `pg_catalog.lo_*` names back. To discard whatever
+is in lolor storage instead, a superuser can set `lolor.allow_unsafe_drop = on`
+for the session that runs the drop; it skips both checks. The drop renames the
+native functions back, so it has the same requirements as `lolor.disable()`:
+no other session connected to the database, and a client session.
+
 Both directions preserve original OIDs, owners, ACLs, and data.
 
+Two helpers find and repair objects whose roles have been dropped:
+
+```sql
+SELECT * FROM lolor.check_orphans();  -- objects referring to a dropped role
+SELECT lolor.fix_orphans('some_role');  -- reassign them and drop dead ACL entries
+```
+
 When the spock extension is installed, both migration functions run under
 `spock.repair_mode()`, so the row-shuffling migration DML is **not**
 replicated to other nodes. This is essential for `migrate_to_native()`: its
@@ -124,4 +148,7 @@ database user. Upgrading to 1.3.0 revokes the privilege; see the release notes.
 
 - Native large object functionality cannot be used while you are using the lolor extension.
 - lolor does not support the following statements: `ALTER LARGE OBJECT`, `GRANT ON LARGE OBJECT`, `COMMENT ON LARGE OBJECT`, and `REVOKE ON LARGE OBJECT`.
+- `lolor.enable()`, `lolor.disable()`, and dropping the extension while lolor is enabled, refuse while any other session is connected to the database, the same rule as `ALTER DATABASE ... RENAME`: a renamed function keeps its OID, so a session that already resolved the `lo_*` functions keeps calling the previous implementation, and the row movement cannot be made atomic for other backends either. The calling session must reconnect after `enable()` or `disable()`. They also refuse to run from anything but a client session, so replicated DDL cannot drive them from an apply worker.
+- Objects in lolor storage are rows in ordinary tables and so cannot participate in `pg_shdepend`. `DROP ROLE`, `DROP OWNED BY` and `REASSIGN OWNED BY` do not see them: a role that owns them or appears in their ACL can be dropped without a warning, and `REASSIGN OWNED` / `DROP OWNED` leave them untouched. Before dropping a role, run `REASSIGN OWNED BY` or `DROP OWNED BY` in each database that has lolor, then `DROP ROLE`. Afterwards run `lolor.check_orphans()` in each such database and repair anything it reports with `lolor.fix_orphans(new_owner)`; `lolor.migrate_to_native()` refuses while orphans exist.
+- Role OIDs come from a cluster-wide counter that wraps around, so a new role can receive a dropped role's OID and silently become the owner or grantee of that role's orphaned objects. This cannot be detected after the fact, which is another reason to run `lolor.check_orphans()` promptly after dropping roles.
 - Large object migration is node-local. Native large objects live in `pg_catalog.pg_largeobject`, which is never replicated, so each node holds an independent set and `migrate_from_native()` migrates only the local node's objects; with spock installed, the migration DML runs in repair mode and is not replicated. Run the migration on every node that holds native large objects — for example with `spock.replicate_ddl('SELECT lolor.migrate_from_native()')`, which queues the command so that each node executes it locally. Migrated objects keep their original native OIDs, which are not node-encoded: if different nodes hold different objects under the same OID, the nodes' lolor contents will diverge and later replicated changes to those objects can conflict. Newly created large objects are collision-free, since new OIDs are node-encoded via `lolor.node` and checked against existing rows.

docker/entrypoint.sh

@@ -137,6 +137,13 @@ cd /tmp/lolor-build
 make USE_PGXS=1 with_llvm=no
 make USE_PGXS=1 with_llvm=no install
 
+# lolor refuses to load on demand, and the library did not exist when the
+# server above was started.  Preload it from here on; the later setting wins.
+cat >> "$PGDATA/postgresql.conf" <<_EOF_
+shared_preload_libraries = 'spock, lolor'
+_EOF_
+pg_ctl -D "$PGDATA" -l /home/pgedge/logfile.log -o "-k /tmp" -m fast -w restart
+
 psql -U admin -d demo -h /tmp -v ON_ERROR_STOP=1 <<_EOF_
 create extension lolor;
 alter system set lolor.node to ${HOSTNAME: -1};

docs/index.md

@@ -18,9 +18,10 @@ Use of the `lolor` extension requires Postgres 16 or newer.
 
 ## Migrating large objects
 
-Migration from native to lolor storage is **manual** — you decide when to move
-existing large objects.  Migration back to native is **automatic** — dropping
-the extension moves all objects back so nothing is lost.
+Migration is a manual step in both directions: you decide when to move
+existing large objects into lolor storage, and you move them back before
+dropping the extension.  Dropping the extension never moves data: it is
+refused while any object remains in lolor storage.
 
 ### Native to lolor (manual)
 
@@ -39,25 +40,39 @@ when there are no native large objects.  This step is intentionally not
 automatic: it requires superuser privileges and should be performed during a
 maintenance window.
 
-### Lolor to native (automatic on DROP EXTENSION)
+### Lolor to native (before DROP EXTENSION)
 
-Large objects are automatically migrated back to native Postgres storage when the
-extension is dropped:
+Move the large objects back to native Postgres storage, then drop the
+extension:
 
 ```sql
+SELECT lolor.migrate_to_native();
 DROP EXTENSION lolor;
 ```
 
-This ensures that no large objects are lost if the extension is removed.  You
-can also trigger the reverse migration manually while the extension is still
-installed:
+Both paths preserve OIDs, owners, ACLs, and data.
+
+An object access hook refuses to remove the extension while it is enabled or
+while any object remains in lolor storage, on every path that reaches it
+(`DROP EXTENSION`, `DROP SCHEMA lolor CASCADE`, `DROP OWNED BY`). The drop
+itself only puts the native `pg_catalog.lo_*` names back. To discard whatever
+is in lolor storage instead, a superuser can set `lolor.allow_unsafe_drop = on`
+for the session that runs the drop; it skips both checks. The drop renames the
+native functions back, so it has the same requirements as `lolor.disable()`:
+no other session connected to the database, and a client session.
+
+`lolor.enable()` and `lolor.disable()` refuse while
+any other session is connected to the database, and refuse to run from
+anything but a client session. Reconnect client sessions after `enable()` or
+`disable()`: libpq caches the large object function OIDs per connection.
+
+Two helpers find and repair objects whose roles have been dropped:
 
 ```sql
-SELECT lolor.migrate_to_native();
+SELECT * FROM lolor.check_orphans();  -- objects referring to a dropped role
+SELECT lolor.fix_orphans('some_role');  -- reassign them and drop dead ACL entries
 ```
 
-Both paths preserve OIDs, owners, ACLs, and data.
-
 When the spock extension is installed, both migration functions run under
 `spock.repair_mode()`, so the row-shuffling migration DML is **not** replicated
 to other nodes.  This is essential for `migrate_to_native()`: its deletes from
@@ -83,4 +98,7 @@ retains the changes and delivers them when replication resumes.
 
 - Native Postgres large object functionality cannot be used while you are using the lolor extension.
 - lolor does not support the following statements: `ALTER LARGE OBJECT`, `GRANT ON LARGE OBJECT`, `COMMENT ON LARGE OBJECT`, and `REVOKE ON LARGE OBJECT`.
+- `lolor.enable()`, `lolor.disable()`, and dropping the extension while lolor is enabled, refuse while any other session is connected to the database, the same rule as `ALTER DATABASE ... RENAME`: a renamed function keeps its OID, so a session that already resolved the `lo_*` functions keeps calling the previous implementation, and the row movement cannot be made atomic for other backends either. The calling session must reconnect after `enable()` or `disable()`. They also refuse to run from anything but a client session, so replicated DDL cannot drive them from an apply worker.
+- Objects in lolor storage are rows in ordinary tables and so cannot participate in `pg_shdepend`. `DROP ROLE`, `DROP OWNED BY` and `REASSIGN OWNED BY` do not see them: a role that owns them or appears in their ACL can be dropped without a warning, and `REASSIGN OWNED` / `DROP OWNED` leave them untouched. Before dropping a role, run `REASSIGN OWNED BY` or `DROP OWNED BY` in each database that has lolor, then `DROP ROLE`. Afterwards run `lolor.check_orphans()` in each such database and repair anything it reports with `lolor.fix_orphans(new_owner)`; `lolor.migrate_to_native()` refuses while orphans exist.
+- Role OIDs come from a cluster-wide counter that wraps around, so a new role can receive a dropped role's OID and silently become the owner or grantee of that role's orphaned objects. This cannot be detected after the fact, which is another reason to run `lolor.check_orphans()` promptly after dropping roles.
 - Large object migration is node-local. Native large objects live in `pg_catalog.pg_largeobject`, which is never replicated, so each node holds an independent set and `migrate_from_native()` migrates only the local node's objects; with spock installed, the migration DML runs in repair mode and is not replicated. Run the migration on every node that holds native large objects — for example with `spock.replicate_ddl('SELECT lolor.migrate_from_native()')`, which queues the command so that each node executes it locally. Migrated objects keep their original native OIDs, which are not node-encoded: if different nodes hold different objects under the same OID, the nodes' lolor contents will diverge and later replicated changes to those objects can conflict. Newly created large objects are collision-free, since new OIDs are node-encoded via `lolor.node` and checked against existing rows.

docs/install_configure.md

@@ -14,15 +14,21 @@ make USE_PGXS=1
 make USE_PGXS=1 install
 ```
 
-After the lolor extension is installed, connect to your Postgres database and create the extension with the command:
+lolor must be loaded at server start. Add it to `shared_preload_libraries` in `postgresql.conf` and restart; `CREATE EXTENSION lolor`, `LOAD 'lolor'` and any call that reaches one of lolor's functions fail with `lolor must be loaded via "shared_preload_libraries"` when the library was loaded on demand; the native `pg_catalog.lo_*` functions are not affected while lolor is not installed or is disabled. The guard that protects large objects when the extension is dropped is an object access hook, which is only in place in every backend when the library is preloaded.
+
+```
+shared_preload_libraries = 'lolor'
+```
+
+Then connect to your Postgres database and create the extension with the command:
 
 ```
 CREATE EXTENSION lolor;
 ```
 
 ## Configuring lolor
 
-You must set the `lolor.node` parameter on each node in your replication cluster before using the extension. The value can be from 1 to 2^28; the value is used to help in generation of new large object OID.
+You must set the `lolor.node` parameter on each node in your replication cluster before using the extension. The value can be from 1 to 15 (0 means unset); it is encoded in the four low bits of every large object OID lolor generates, so each node must use a different value.
 
 ```
 lolor.node = 1