Replace explicit PIN entries in pg_depend with an OID range test.

As of v14, pg_depend contains almost 7000 "pin" entries recording
the OIDs of built-in objects.  This is a fair amount of bloat for
every database, and it adds time to pg_depend lookups as well as
initdb.  We can get rid of all of those entries in favor of an OID
range check, i.e. "OIDs below FirstUnpinnedObjectId are pinned".

(template1 and the public schema are exceptions.  Those exceptions
are now wired into IsPinnedObject() instead of initdb's code for
filling pg_depend, but it's the same amount of cruft either way.)

The contents of pg_shdepend are modified likewise.

Discussion: https://postgr.es/m/3737988.1618451008@sss.pgh.pa.us

Author: tglsfdc

doc/src/sgml/bki.sgml

@@ -418,19 +418,33 @@
    <para>
     If <filename>genbki.pl</filename> needs to assign an OID to a catalog
     entry that does not have a manually-assigned OID, it will use a value in
-    the range 10000&mdash;11999.  The server's OID counter is set to 12000
-    at the start of a bootstrap run.  Thus objects created by regular SQL
-    commands during the later phases of bootstrap, such as objects created
-    while running the <filename>information_schema.sql</filename> script,
-    receive OIDs of 12000 or above.
+    the range 10000&mdash;11999.  The server's OID counter is set to 10000
+    at the start of a bootstrap run, so that any objects created on-the-fly
+    during bootstrap processing also receive OIDs in this range.  (The
+    usual OID assignment mechanism takes care of preventing any conflicts.)
+   </para>
+
+   <para>
+    Objects with OIDs below <symbol>FirstUnpinnedObjectId</symbol> (12000)
+    are considered <quote>pinned</quote>, preventing them from being
+    deleted.  (There are a small number of exceptions, which are
+    hard-wired into <function>IsPinnedObject()</function>.)
+    <application>initdb</application> forces the OID counter up
+    to <symbol>FirstUnpinnedObjectId</symbol> as soon as it's ready to
+    create unpinned objects.  Thus objects created during the later phases
+    of <application>initdb</application>, such as objects created while
+    running the <filename>information_schema.sql</filename> script, will
+    not be pinned, while all objects known
+    to <filename>genbki.pl</filename> will be.
    </para>
 
    <para>
     OIDs assigned during normal database operation are constrained to be
     16384 or higher.  This ensures that the range 10000&mdash;16383 is free
     for OIDs assigned automatically by <filename>genbki.pl</filename> or
-    during bootstrap.  These automatically-assigned OIDs are not considered
-    stable, and may change from one installation to another.
+    during <application>initdb</application>.  These
+    automatically-assigned OIDs are not considered stable, and may change
+    from one installation to another.
    </para>
   </sect2>
 

doc/src/sgml/catalogs.sgml

@@ -3264,8 +3264,7 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
        (references <link linkend="catalog-pg-class"><structname>pg_class</structname></link>.<structfield>oid</structfield>)
       </para>
       <para>
-       The OID of the system catalog the dependent object is in,
-       or zero for a <symbol>DEPENDENCY_PIN</symbol> entry
+       The OID of the system catalog the dependent object is in
       </para></entry>
      </row>
 
@@ -3275,8 +3274,7 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
        (references any OID column)
       </para>
       <para>
-       The OID of the specific dependent object,
-       or zero for a <symbol>DEPENDENCY_PIN</symbol> entry
+       The OID of the specific dependent object
       </para></entry>
      </row>
 
@@ -3467,19 +3465,6 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
       </para>
      </listitem>
     </varlistentry>
-
-    <varlistentry>
-     <term><symbol>DEPENDENCY_PIN</symbol> (<literal>p</literal>)</term>
-     <listitem>
-      <para>
-       There is no dependent object; this type of entry is a signal
-       that the system itself depends on the referenced object, and so
-       that object must never be deleted.  Entries of this type are
-       created only by <application>initdb</application>.  The columns for the
-       dependent object contain zeroes.
-      </para>
-     </listitem>
-    </varlistentry>
    </variablelist>
 
    Other dependency flavors might be needed in future.
@@ -3498,6 +3483,19 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
    must be satisfied.
   </para>
 
+  <para>
+   Most objects created during <application>initdb</application> are
+   considered <quote>pinned</quote>, which means that the system itself
+   depends on them.  Therefore, they are never allowed to be dropped.
+   Also, knowing that pinned objects will not be dropped, the dependency
+   mechanism doesn't bother to make <structname>pg_depend</structname>
+   entries showing dependencies on them.  Thus, for example, a table
+   column of type <type>numeric</type> notionally has
+   a <literal>NORMAL</literal> dependency on the <type>numeric</type>
+   data type, but no such entry actually appears
+   in <structname>pg_depend</structname>.
+  </para>
+
  </sect1>
 
 
@@ -6779,7 +6777,6 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
       <para>
        The OID of the database the dependent object is in,
        or zero for a shared object
-       or a <symbol>SHARED_DEPENDENCY_PIN</symbol> entry
       </para></entry>
      </row>
 
@@ -6789,8 +6786,7 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
        (references <link linkend="catalog-pg-class"><structname>pg_class</structname></link>.<structfield>oid</structfield>)
       </para>
       <para>
-       The OID of the system catalog the dependent object is in,
-       or zero for a <symbol>SHARED_DEPENDENCY_PIN</symbol> entry
+       The OID of the system catalog the dependent object is in
       </para></entry>
      </row>
 
@@ -6800,8 +6796,7 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
        (references any OID column)
       </para>
       <para>
-       The OID of the specific dependent object,
-       or zero for a <symbol>SHARED_DEPENDENCY_PIN</symbol> entry
+       The OID of the specific dependent object
       </para></entry>
      </row>
 
@@ -6889,19 +6884,6 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
      </listitem>
     </varlistentry>
 
-    <varlistentry>
-     <term><symbol>SHARED_DEPENDENCY_PIN</symbol> (<literal>p</literal>)</term>
-     <listitem>
-      <para>
-       There is no dependent object; this type of entry is a signal
-       that the system itself depends on the referenced object, and so
-       that object must never be deleted.  Entries of this type are
-       created only by <application>initdb</application>.  The columns for the
-       dependent object contain zeroes.
-      </para>
-     </listitem>
-    </varlistentry>
-
     <varlistentry>
      <term><symbol>SHARED_DEPENDENCY_TABLESPACE</symbol> (<literal>t</literal>)</term>
      <listitem>
@@ -6918,6 +6900,14 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
    objects.
   </para>
 
+  <para>
+   As in the <structname>pg_depend</structname> catalog, most objects
+   created during <application>initdb</application> are
+   considered <quote>pinned</quote>.  No entries are made
+   in <structname>pg_shdepend</structname> that would have a pinned
+   object as either referenced or dependent object.
+  </para>
+
  </sect1>
 
  <sect1 id="catalog-pg-shdescription">

src/backend/access/transam/varsup.c

@@ -541,11 +541,11 @@ GetNewObjectId(void)
 	 * FirstNormalObjectId since that range is reserved for initdb (see
 	 * IsCatalogRelationOid()).  Note we are relying on unsigned comparison.
 	 *
-	 * During initdb, we start the OID generator at FirstBootstrapObjectId, so
-	 * we only wrap if before that point when in bootstrap or standalone mode.
+	 * During initdb, we start the OID generator at FirstGenbkiObjectId, so we
+	 * only wrap if before that point when in bootstrap or standalone mode.
 	 * The first time through this routine after normal postmaster start, the
 	 * counter will be forced up to FirstNormalObjectId.  This mechanism
-	 * leaves the OIDs between FirstBootstrapObjectId and FirstNormalObjectId
+	 * leaves the OIDs between FirstGenbkiObjectId and FirstNormalObjectId
 	 * available for automatic assignment during initdb, while ensuring they
 	 * will never conflict with user-assigned OIDs.
 	 */
@@ -560,7 +560,7 @@ GetNewObjectId(void)
 		else
 		{
 			/* we may be bootstrapping, so don't enforce the full range */
-			if (ShmemVariableCache->nextOid < ((Oid) FirstBootstrapObjectId))
+			if (ShmemVariableCache->nextOid < ((Oid) FirstGenbkiObjectId))
 			{
 				/* wraparound in standalone mode (unlikely but possible) */
 				ShmemVariableCache->nextOid = FirstNormalObjectId;
@@ -586,6 +586,47 @@ GetNewObjectId(void)
 	return result;
 }
 
+/*
+ * SetNextObjectId
+ *
+ * This may only be called during initdb; it advances the OID counter
+ * to the specified value.
+ */
+static void
+SetNextObjectId(Oid nextOid)
+{
+	/* Safety check, this is only allowable during initdb */
+	if (IsPostmasterEnvironment)
+		elog(ERROR, "cannot advance OID counter anymore");
+
+	/* Taking the lock is, therefore, just pro forma; but do it anyway */
+	LWLockAcquire(OidGenLock, LW_EXCLUSIVE);
+
+	if (ShmemVariableCache->nextOid > nextOid)
+		elog(ERROR, "too late to advance OID counter to %u, it is now %u",
+			 nextOid, ShmemVariableCache->nextOid);
+
+	ShmemVariableCache->nextOid = nextOid;
+	ShmemVariableCache->oidCount = 0;
+
+	LWLockRelease(OidGenLock);
+}
+
+/*
+ * StopGeneratingPinnedObjectIds
+ *
+ * This is called once during initdb to force the OID counter up to
+ * FirstUnpinnedObjectId.  This supports letting initdb's post-bootstrap
+ * processing create some pinned objects early on.  Once it's done doing
+ * so, it calls this (via pg_stop_making_pinned_objects()) so that the
+ * remaining objects it makes will be considered un-pinned.
+ */
+void
+StopGeneratingPinnedObjectIds(void)
+{
+	SetNextObjectId(FirstUnpinnedObjectId);
+}
+
 
 #ifdef USE_ASSERT_CHECKING
 

src/backend/access/transam/xlog.c

@@ -5318,7 +5318,7 @@ BootStrapXLOG(void)
 	checkPoint.fullPageWrites = fullPageWrites;
 	checkPoint.nextXid =
 		FullTransactionIdFromEpochAndXid(0, FirstNormalTransactionId);
-	checkPoint.nextOid = FirstBootstrapObjectId;
+	checkPoint.nextOid = FirstGenbkiObjectId;
 	checkPoint.nextMulti = FirstMultiXactId;
 	checkPoint.nextMultiOffset = 0;
 	checkPoint.oldestXid = FirstNormalTransactionId;

src/backend/catalog/catalog.c

@@ -31,6 +31,7 @@
 #include "catalog/pg_authid.h"
 #include "catalog/pg_database.h"
 #include "catalog/pg_db_role_setting.h"
+#include "catalog/pg_largeobject.h"
 #include "catalog/pg_namespace.h"
 #include "catalog/pg_replication_origin.h"
 #include "catalog/pg_shdepend.h"
@@ -120,9 +121,9 @@ bool
 IsCatalogRelationOid(Oid relid)
 {
 	/*
-	 * We consider a relation to be a system catalog if it has an OID that was
-	 * manually assigned or assigned by genbki.pl.  This includes all the
-	 * defined catalogs, their indexes, and their TOAST tables and indexes.
+	 * We consider a relation to be a system catalog if it has a pinned OID.
+	 * This includes all the defined catalogs, their indexes, and their TOAST
+	 * tables and indexes.
 	 *
 	 * This rule excludes the relations in information_schema, which are not
 	 * integral to the system and can be treated the same as user relations.
@@ -132,7 +133,7 @@ IsCatalogRelationOid(Oid relid)
 	 * This test is reliable since an OID wraparound will skip this range of
 	 * OIDs; see GetNewObjectId().
 	 */
-	return (relid < (Oid) FirstBootstrapObjectId);
+	return (relid < (Oid) FirstUnpinnedObjectId);
 }
 
 /*
@@ -294,6 +295,64 @@ IsSharedRelation(Oid relationId)
 	return false;
 }
 
+/*
+ * IsPinnedObject
+ *		Given the class + OID identity of a database object, report whether
+ *		it is "pinned", that is not droppable because the system requires it.
+ *
+ * We used to represent this explicitly in pg_depend, but that proved to be
+ * an undesirable amount of overhead, so now we rely on an OID range test.
+ */
+bool
+IsPinnedObject(Oid classId, Oid objectId)
+{
+	/*
+	 * Objects with OIDs above FirstUnpinnedObjectId are never pinned.  Since
+	 * the OID generator skips this range when wrapping around, this check
+	 * guarantees that user-defined objects are never considered pinned.
+	 */
+	if (objectId >= FirstUnpinnedObjectId)
+		return false;
+
+	/*
+	 * Large objects are never pinned.  We need this special case because
+	 * their OIDs can be user-assigned.
+	 */
+	if (classId == LargeObjectRelationId)
+		return false;
+
+	/*
+	 * There are a few objects defined in the catalog .dat files that, as a
+	 * matter of policy, we prefer not to treat as pinned.  We used to handle
+	 * that by excluding them from pg_depend, but it's just as easy to
+	 * hard-wire their OIDs here.  (If the user does indeed drop and recreate
+	 * them, they'll have new but certainly-unpinned OIDs, so no problem.)
+	 *
+	 * Checking both classId and objectId is overkill, since OIDs below
+	 * FirstGenbkiObjectId should be globally unique, but do it anyway for
+	 * robustness.
+	 */
+
+	/* template1 is not pinned */
+	if (classId == DatabaseRelationId &&
+		objectId == TemplateDbOid)
+		return false;
+
+	/* the public namespace is not pinned */
+	if (classId == NamespaceRelationId &&
+		objectId == PG_PUBLIC_NAMESPACE)
+		return false;
+
+	/*
+	 * All other initdb-created objects are pinned.  This is overkill (the
+	 * system doesn't really depend on having every last weird datatype, for
+	 * instance) but generating only the minimum required set of dependencies
+	 * seems hard, and enforcing an accurate list would be much more expensive
+	 * than the simple range test used here.
+	 */
+	return true;
+}
+
 
 /*
  * GetNewOidWithIndex
@@ -533,7 +592,8 @@ pg_nextoid(PG_FUNCTION_ARGS)
 	if (!superuser())
 		ereport(ERROR,
 				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
-				 errmsg("must be superuser to call pg_nextoid()")));
+				 errmsg("must be superuser to call %s()",
+						"pg_nextoid")));
 
 	rel = table_open(reloid, RowExclusiveLock);
 	idx = index_open(idxoid, RowExclusiveLock);
@@ -580,5 +640,29 @@ pg_nextoid(PG_FUNCTION_ARGS)
 	table_close(rel, RowExclusiveLock);
 	index_close(idx, RowExclusiveLock);
 
-	return newoid;
+	PG_RETURN_OID(newoid);
+}
+
+/*
+ * SQL callable interface for StopGeneratingPinnedObjectIds().
+ *
+ * This is only to be used by initdb, so it's intentionally not documented in
+ * the user facing docs.
+ */
+Datum
+pg_stop_making_pinned_objects(PG_FUNCTION_ARGS)
+{
+	/*
+	 * Belt-and-suspenders check, since StopGeneratingPinnedObjectIds will
+	 * fail anyway in non-single-user mode.
+	 */
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to call %s()",
+						"pg_stop_making_pinned_objects")));
+
+	StopGeneratingPinnedObjectIds();
+
+	PG_RETURN_VOID();
 }