Separate multixact freezing parameters from xid's

Previously we were piggybacking on transaction ID parameters to freeze
multixacts; but since there isn't necessarily any relationship between
rates of Xid and multixact consumption, this turns out not to be a good
idea.

Therefore, we now have multixact-specific freezing parameters:

vacuum_multixact_freeze_min_age: when to remove multis as we come across
them in vacuum (default to 5 million, i.e. early in comparison to Xid's
default of 50 million)

vacuum_multixact_freeze_table_age: when to force whole-table scans
instead of scanning only the pages marked as not all visible in
visibility map (default to 150 million, same as for Xids).  Whichever of
both which reaches the 150 million mark earlier will cause a whole-table
scan.

autovacuum_multixact_freeze_max_age: when for cause emergency,
uninterruptible whole-table scans (default to 400 million, double as
that for Xids).  This means there shouldn't be more frequent emergency
vacuuming than previously, unless multixacts are being used very
rapidly.

Backpatch to 9.3 where multixacts were made to persist enough to require
freezing.  To avoid an ABI break in 9.3, VacuumStmt has a couple of
fields in an unnatural place, and StdRdOptions is split in two so that
the newly added fields can go at the end.

Patch by me, reviewed by Robert Haas, with additional input from Andres
Freund and Tom Lane.

Author: alvherre

doc/src/sgml/config.sgml

@@ -4730,6 +4730,33 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-autovacuum-multixact-freeze-max-age" xreflabel="autovacuum_multixact_freeze_max_age">
+      <term><varname>autovacuum_multixact_freeze_max_age</varname> (<type>integer</type>)</term>
+      <indexterm>
+       <primary><varname>autovacuum_multixact_freeze_max_age</varname> configuration parameter</primary>
+      </indexterm>
+      <listitem>
+       <para>
+        Specifies the maximum age (in multixacts) that a table's
+        <structname>pg_class</>.<structfield>relminmxid</> field can
+        attain before a <command>VACUUM</> operation is forced to
+        prevent multixact ID wraparound within the table.
+        Note that the system will launch autovacuum processes to
+        prevent wraparound even when autovacuum is otherwise disabled.
+       </para>
+
+       <para>
+        Vacuuming multixacts also allows removal of old files from the
+        <filename>pg_multixact/members</> and <filename>pg_multixact/offsets</>
+        subdirectories, which is why the default is a relatively low
+        400 million multixacts.
+        This parameter can only be set at server start, but the setting
+        can be reduced for individual tables by changing storage parameters.
+        For more information see <xref linkend="vacuum-for-multixact-wraparound">.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-autovacuum-vacuum-cost-delay" xreflabel="autovacuum_vacuum_cost_delay">
       <term><varname>autovacuum_vacuum_cost_delay</varname> (<type>integer</type>)</term>
       <indexterm>
@@ -5138,7 +5165,7 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
         <structname>pg_class</>.<structfield>relfrozenxid</> field has reached
         the age specified by this setting.  The default is 150 million
         transactions.  Although users can set this value anywhere from zero to
-        one billion, <command>VACUUM</> will silently limit the effective value
+        two billions, <command>VACUUM</> will silently limit the effective value
         to 95% of <xref linkend="guc-autovacuum-freeze-max-age">, so that a
         periodical manual <command>VACUUM</> has a chance to run before an
         anti-wraparound autovacuum is launched for the table. For more
@@ -5169,6 +5196,47 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-vacuum-multixact-freeze-table-age" xreflabel="vacuum_multixact_freeze_table_age">
+      <term><varname>vacuum_multixact_freeze_table_age</varname> (<type>integer</type>)</term>
+      <indexterm>
+       <primary><varname>vacuum_multixact_freeze_table_age</> configuration parameter</primary>
+      </indexterm>
+      <listitem>
+       <para>
+        <command>VACUUM</> performs a whole-table scan if the table's
+        <structname>pg_class</>.<structfield>relminmxid</> field has reached
+        the age specified by this setting.  The default is 150 million multixacts.
+        Although users can set this value anywhere from zero to two billions,
+        <command>VACUUM</> will silently limit the effective value to 95% of
+        <xref linkend="guc-autovacuum-multixact-freeze-max-age">, so that a
+        periodical manual <command>VACUUM</> has a chance to run before an
+        anti-wraparound is launched for the table.
+        For more information see <xref linkend="vacuum-for-multixact-wraparound">.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry id="guc-vacuum-multixact-freeze-min-age" xreflabel="vacuum_multixact_freeze_min_age">
+      <term><varname>vacuum_multixact_freeze_min_age</varname> (<type>integer</type>)</term>
+      <indexterm>
+       <primary><varname>vacuum_multixact_freeze_min_age</> configuration parameter</primary>
+      </indexterm>
+      <listitem>
+       <para>
+        Specifies the cutoff age (in multixacts) that <command>VACUUM</>
+        should use to decide whether to replace multixact IDs with a newer
+        transaction ID or multixact ID while scanning a table.  The default
+        is 5 million multixacts.
+        Although users can set this value anywhere from zero to one billion,
+        <command>VACUUM</> will silently limit the effective value to half
+        the value of <xref linkend="guc-autovacuum-multixact-freeze-max-age">,
+        so that there is not an unreasonably short time between forced
+        autovacuums.
+        For more information see <xref linkend="vacuum-for-multixact-wraparound">.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-bytea-output" xreflabel="bytea_output">
       <term><varname>bytea_output</varname> (<type>enum</type>)</term>
       <indexterm>

doc/src/sgml/maintenance.sgml

@@ -108,7 +108,8 @@
 
      <listitem>
       <simpara>To protect against loss of very old data due to
-      <firstterm>transaction ID wraparound</>.</simpara>
+      <firstterm>transaction ID wraparound</> or
+      <firstterm>multixact ID wraparound</>.</simpara>
      </listitem>
     </orderedlist>
 
@@ -379,6 +380,11 @@
     <secondary>wraparound</secondary>
    </indexterm>
 
+    <indexterm>
+     <primary>wraparound</primary>
+     <secondary>of transaction IDs</secondary>
+    </indexterm>
+
    <para>
     <productname>PostgreSQL</productname>'s MVCC transaction semantics
     depend on being able to compare transaction ID (<acronym>XID</>)
@@ -599,6 +605,54 @@ HINT:  Stop the postmaster and use a standalone backend to VACUUM in "mydb".
     page for details about using a single-user backend.
    </para>
 
+   <sect3 id="vacuum-for-multixact-wraparound">
+    <title>Multixacts and Wraparound</title>
+
+    <indexterm>
+     <primary>MultiXactId</primary>
+    </indexterm>
+
+    <indexterm>
+     <primary>wraparound</primary>
+     <secondary>of multixact IDs</secondary>
+    </indexterm>
+
+    <para>
+     <firstterm>Multixacts</> are used to implement row locking by
+     multiple transactions: since there is limited space in the tuple
+     header to store lock information, that information is stored as a
+     multixact separately in the <filename>pg_multixact</> subdirectory,
+     and only its ID is in the <structfield>xmax</> field
+     in the tuple header.
+     Similar to transaction IDs, multixact IDs are implemented as a
+     32-bit counter and corresponding storage, all of which requires
+     careful aging management, storage cleanup, and wraparound handling.
+    </para>
+
+    <para>
+     During a <command>VACUUM</> table scan, either partial or of the whole
+     table, any multixact ID older than
+     <xref linkend="guc-vacuum-multixact-freeze-min-age">
+     is replaced by a different value, which can be the zero value, a single
+     transaction ID, or a newer multixact ID.  For each table,
+     <structname>pg_class</>.<structfield>relminmxid</> stores the oldest
+     possible value still stored in any tuple of that table.  Every time this
+     value is older than
+     <xref linkend="guc-vacuum-multixact-freeze-table-age">, a whole-table
+     scan is forced.  Whole-table <command>VACUUM</> scans, regardless of
+     what causes them, enable advancing the value for that table.
+     Eventually, as all tables in all databases are scanned and their
+     oldest multixact values are advanced, on-disk storage for older
+     multixacts can be removed.
+    </para>
+
+    <para>
+     As a safety device, a whole-table vacuum scan will occur for any table
+     whose multixact-age is greater than
+     <xref linkend="guc-autovacuum-multixact-freeze-max-age">.
+     This will occur even if autovacuum is nominally disabled.
+    </para>
+   </sect3>
   </sect2>
 
   <sect2 id="autovacuum">

doc/src/sgml/ref/create_table.sgml

@@ -981,7 +981,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
      <para>
      Custom <xref linkend="guc-vacuum-freeze-min-age"> parameter. Note that
      autovacuum will ignore attempts to set a per-table
-     <literal>autovacuum_freeze_min_age</> larger than the half system-wide
+     <literal>autovacuum_freeze_min_age</> larger than half the system-wide
      <xref linkend="guc-autovacuum-freeze-max-age"> setting.
      </para>
     </listitem>
@@ -1010,6 +1010,43 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>autovacuum_multixact_freeze_min_age</literal>, <literal>toast.autovacuum_multixact_freeze_min_age</literal> (<type>integer</type>)</term>
+    <listitem>
+     <para>
+      Custom <xref linkend="guc-vacuum-multixact-freeze-min-age"> parameter.
+      Note that autovacuum will ignore attempts to set a per-table
+      <literal>autovacuum_multixact_freeze_min_age</> larger than half the
+      system-wide <xref linkend="guc-autovacuum-multixact-freeze-max-age">
+      setting.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term><literal>autovacuum_multixact_freeze_max_age</literal>, <literal>toast.autovacuum_multixact_freeze_max_age</literal> (<type>integer</type>)</term>
+    <listitem>
+     <para>
+      Custom <xref linkend="guc-autovacuum-multixact-freeze-max-age"> parameter. Note
+      that autovacuum will ignore attempts to set a per-table
+      <literal>autovacuum_multixact_freeze_max_age</> larger than the
+      system-wide setting (it can only be set smaller).  Note that while you
+      can set <literal>autovacuum_multixact_freeze_max_age</> very small,
+      or even zero, this is usually unwise since it will force frequent
+      vacuuming.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term><literal>autovacuum_multixact_freeze_table_age</literal>, <literal>toast.autovacuum_multixact_freeze_table_age</literal> (<type>integer</type>)</term>
+    <listitem>
+     <para>
+      Custom <xref linkend="guc-vacuum-multixact-freeze-table-age"> parameter.
+     </para>
+    </listitem>
+   </varlistentry>
+
    </variablelist>
 
   </refsect2>

src/backend/access/common/reloptions.c

@@ -162,6 +162,14 @@ static relopt_int intRelOpts[] =
 		},
 		-1, 0, 1000000000
 	},
+	{
+		{
+			"autovacuum_multixact_freeze_min_age",
+			"Minimum multixact age at which VACUUM should freeze a row multixact's, for autovacuum",
+			RELOPT_KIND_HEAP | RELOPT_KIND_TOAST
+		},
+		-1, 0, 1000000000
+	},
 	{
 		{
 			"autovacuum_freeze_max_age",
@@ -170,13 +178,28 @@ static relopt_int intRelOpts[] =
 		},
 		-1, 100000000, 2000000000
 	},
+	{
+		{
+			"autovacuum_multixact_freeze_max_age",
+			"Multixact age at which to autovacuum a table to prevent multixact wraparound",
+			RELOPT_KIND_HEAP | RELOPT_KIND_TOAST
+		},
+		-1, 100000000, 2000000000
+	},
 	{
 		{
 			"autovacuum_freeze_table_age",
 			"Age at which VACUUM should perform a full table sweep to replace old Xid values with FrozenXID",
 			RELOPT_KIND_HEAP | RELOPT_KIND_TOAST
 		}, -1, 0, 2000000000
 	},
+	{
+		{
+			"autovacuum_multixact_freeze_table_age",
+			"Age of multixact at which VACUUM should perform a full table sweep to replace old multixact values with newer ones",
+			RELOPT_KIND_HEAP | RELOPT_KIND_TOAST
+		}, -1, 0, 2000000000
+	},
 	/* list terminator */
 	{{NULL}}
 };
@@ -1146,6 +1169,12 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
 		offsetof(StdRdOptions, autovacuum) +offsetof(AutoVacOpts, freeze_max_age)},
 		{"autovacuum_freeze_table_age", RELOPT_TYPE_INT,
 		offsetof(StdRdOptions, autovacuum) +offsetof(AutoVacOpts, freeze_table_age)},
+		{"autovacuum_multixact_freeze_min_age", RELOPT_TYPE_INT,
+		offsetof(StdRdOptions, autovacuum2) +offsetof(AutoVacOpts2, multixact_freeze_min_age)},
+		{"autovacuum_multixact_freeze_max_age", RELOPT_TYPE_INT,
+		offsetof(StdRdOptions, autovacuum2) +offsetof(AutoVacOpts2, multixact_freeze_max_age)},
+		{"autovacuum_multixact_freeze_table_age", RELOPT_TYPE_INT,
+		offsetof(StdRdOptions, autovacuum2) +offsetof(AutoVacOpts2, multixact_freeze_table_age)},
 		{"autovacuum_vacuum_scale_factor", RELOPT_TYPE_REAL,
 		offsetof(StdRdOptions, autovacuum) +offsetof(AutoVacOpts, vacuum_scale_factor)},
 		{"autovacuum_analyze_scale_factor", RELOPT_TYPE_REAL,

src/backend/access/transam/multixact.c

@@ -2055,11 +2055,13 @@ SetMultiXactIdLimit(MultiXactId oldest_datminmxid, Oid oldest_datoid)
 	Assert(MultiXactIdIsValid(oldest_datminmxid));
 
 	/*
-	 * The place where we actually get into deep trouble is halfway around
-	 * from the oldest potentially-existing XID/multi.	(This calculation is
-	 * probably off by one or two counts for Xids, because the special XIDs
-	 * reduce the size of the loop a little bit.  But we throw in plenty of
-	 * slop below, so it doesn't matter.)
+	 * Since multixacts wrap differently from transaction IDs, this logic is
+	 * not entirely correct: in some scenarios we could go for longer than 2
+	 * billion multixacts without seeing any data loss, and in some others we
+	 * could get in trouble before that if the new pg_multixact/members data
+	 * stomps on the previous cycle's data.  For lack of a better mechanism we
+	 * use the same logic as for transaction IDs, that is, start taking action
+	 * halfway around the oldest potentially-existing multixact.
 	 */
 	multiWrapLimit = oldest_datminmxid + (MaxMultiXactId >> 1);
 	if (multiWrapLimit < FirstMultiXactId)
@@ -2093,12 +2095,13 @@ SetMultiXactIdLimit(MultiXactId oldest_datminmxid, Oid oldest_datoid)
 
 	/*
 	 * We'll start trying to force autovacuums when oldest_datminmxid gets to
-	 * be more than autovacuum_freeze_max_age mxids old.
+	 * be more than autovacuum_multixact_freeze_max_age mxids old.
 	 *
-	 * It's a bit ugly to just reuse limits for xids that way, but it doesn't
-	 * seem worth adding separate GUCs for that purpose.
+	 * Note: autovacuum_multixact_freeze_max_age is a PGC_POSTMASTER parameter
+	 * so that we don't have to worry about dealing with on-the-fly changes in
+	 * its value.  See SetTransactionIdLimit.
 	 */
-	multiVacLimit = oldest_datminmxid + autovacuum_freeze_max_age;
+	multiVacLimit = oldest_datminmxid + autovacuum_multixact_freeze_max_age;
 	if (multiVacLimit < FirstMultiXactId)
 		multiVacLimit += FirstMultiXactId;
 

src/backend/access/transam/varsup.c

@@ -313,7 +313,8 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
 	 * value.  It doesn't look practical to update shared state from a GUC
 	 * assign hook (too many processes would try to execute the hook,
 	 * resulting in race conditions as well as crashes of those not connected
-	 * to shared memory).  Perhaps this can be improved someday.
+	 * to shared memory).  Perhaps this can be improved someday.  See also
+	 * SetMultiXactIdLimit.
 	 */
 	xidVacLimit = oldest_datfrozenxid + autovacuum_freeze_max_age;
 	if (xidVacLimit < FirstNormalTransactionId)