Add code to prevent transaction ID wraparound by enforcing a safe limit

in GetNewTransactionId().  Since the limit value has to be computed
before we run any real transactions, this requires adding code to database
startup to scan pg_database and determine the oldest datfrozenxid.
This can conveniently be combined with the first stage of an attack on
the problem that the 'flat file' copies of pg_shadow and pg_group are
not properly updated during WAL recovery.  The code I've added to
startup resides in a new file src/backend/utils/init/flatfiles.c, and
it is responsible for rewriting the flat files as well as initializing
the XID wraparound limit value.  This will eventually allow us to get
rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add
a trigger to pg_database.

Author: tglsfdc

doc/src/sgml/maintenance.sgml

@@ -1,5 +1,5 @@
 <!--
-$PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.40 2004/12/27 22:30:10 tgl Exp $
+$PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.41 2005/02/20 02:21:26 tgl Exp $
 -->
 
 <chapter id="maintenance">
@@ -290,7 +290,7 @@ $PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.40 2004/12/27 22:30:10 tgl
     transaction's XID is <quote>in the future</> and should not be visible
     to the current transaction.  But since transaction IDs have limited size
     (32 bits at this writing) a cluster that runs for a long time (more
-    than 4 billion transactions) will suffer <firstterm>transaction ID
+    than 4 billion transactions) would suffer <firstterm>transaction ID
     wraparound</>: the XID counter wraps around to zero, and all of a sudden
     transactions that were in the past appear to be in the future &mdash; which
     means their outputs become invisible.  In short, catastrophic data loss.
@@ -313,8 +313,13 @@ $PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.40 2004/12/27 22:30:10 tgl
     In practice this isn't an onerous requirement, but since the
     consequences of failing to meet it can be complete data loss (not
     just wasted disk space or slow performance), some special provisions
-    have been made to help database administrators keep track of the
-    time since the last <command>VACUUM</>. The remainder of this
+    have been made to help database administrators avoid disaster.
+    For each database in the cluster, <productname>PostgreSQL</productname>
+    keeps track of the time of the last database-wide <command>VACUUM</>.
+    When any database approaches the billion-transaction danger level,
+    the system begins to emit warning messages.  If nothing is done, it
+    will eventually shut down normal operations until appropriate
+    manual maintenance is done.  The remainder of this
     section gives the details.
    </para>
 
@@ -363,7 +368,8 @@ $PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.40 2004/12/27 22:30:10 tgl
     statistics in the system table <literal>pg_database</>.  In particular,
     the <literal>datfrozenxid</> column of a database's
     <literal>pg_database</> row is updated at the completion of any
-    database-wide <command>VACUUM</command> operation (i.e., <command>VACUUM</> that does not
+    database-wide <command>VACUUM</command> operation (i.e.,
+    <command>VACUUM</> that does not
     name a specific table).  The value stored in this field is the freeze
     cutoff XID that was used by that <command>VACUUM</> command.  All normal
     XIDs older than this cutoff XID are guaranteed to have been replaced by
@@ -391,12 +397,37 @@ SELECT datname, age(datfrozenxid) FROM pg_database;
 
 <programlisting>
 play=# VACUUM;
-WARNING:  some databases have not been vacuumed in 1613770184 transactions
-HINT:  Better vacuum them within 533713463 transactions, or you may have a wraparound failure.
+WARNING:  database "mydb" must be vacuumed within 177009986 transactions
+HINT:  To avoid a database shutdown, execute a full-database VACUUM in "mydb".
 VACUUM
 </programlisting>
    </para>
 
+   <para>
+    If the warnings emitted by <command>VACUUM</> go ignored, then
+    <productname>PostgreSQL</productname> will begin to emit a warning
+    like the above on every transaction start once there are fewer than 10
+    million transactions left until wraparound.  If those warnings also are
+    ignored, the system will shut down and refuse to execute any new
+    transactions once there are fewer than 1 million transactions left
+    until wraparound:
+
+<programlisting>
+play=# select 2+2;
+ERROR:  database is shut down to avoid wraparound data loss in database "mydb"
+HINT:  Stop the postmaster and use a standalone backend to VACUUM in "mydb".
+</programlisting>
+
+    The 1-million-transaction safety margin exists to let the
+    administrator recover without data loss, by manually executing the
+    required <command>VACUUM</> commands.  However, since the system will not
+    execute commands once it has gone into the safety shutdown mode,
+    the only way to do this is to stop the postmaster and use a standalone
+    backend to execute <command>VACUUM</>.  The shutdown mode is not enforced
+    by a standalone backend.  See the <xref linkend="app-postgres"> reference
+    page for details about using a standalone backend.
+   </para>
+
    <para>
     <command>VACUUM</> with the <command>FREEZE</> option uses a more
     aggressive freezing policy: row versions are frozen if they are old enough
@@ -410,26 +441,19 @@ VACUUM
     It should also be used to prepare any user-created databases that
     are to be marked <literal>datallowconn</> = <literal>false</> in
     <literal>pg_database</>, since there isn't any convenient way to
-    <command>VACUUM</command> a database that you can't connect to. Note that
-    <command>VACUUM</command>'s automatic warning message about
-    unvacuumed databases will ignore <literal>pg_database</> entries
-    with <literal>datallowconn</> = <literal>false</>, so as to avoid
-    giving false warnings about these databases; therefore it's up to
-    you to ensure that such databases are frozen correctly.
+    <command>VACUUM</command> a database that you can't connect to.
    </para>
 
    <warning>
     <para>
-     To be sure of safety against transaction wraparound, it is necessary
-     to vacuum <emphasis>every</> table, including system catalogs, in
-     <emphasis>every</> database at least once every billion transactions.
-     We have seen data loss situations caused by people deciding that they
-     only needed to vacuum their active user tables, rather than issuing
-     database-wide vacuum commands.  That will appear to work fine ...
-     for a while.
+     A database that is marked <literal>datallowconn</> = <literal>false</>
+     in <literal>pg_database</> is assumed to be properly frozen; the
+     automatic warnings and wraparound protection shutdown do not take
+     such databases into account.  Therefore it's up to you to ensure
+     you've correctly frozen a database before you mark it with
+     <literal>datallowconn</> = <literal>false</>.
     </para>
    </warning>
-
   </sect2>
  </sect1>
 

src/backend/access/transam/varsup.c

@@ -6,7 +6,7 @@
  * Copyright (c) 2000-2005, PostgreSQL Global Development Group
  *
  * IDENTIFICATION
- *	  $PostgreSQL: pgsql/src/backend/access/transam/varsup.c,v 1.60 2005/01/01 05:43:06 momjian Exp $
+ *	  $PostgreSQL: pgsql/src/backend/access/transam/varsup.c,v 1.61 2005/02/20 02:21:28 tgl Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -16,8 +16,10 @@
 #include "access/clog.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "miscadmin.h"
 #include "storage/ipc.h"
 #include "storage/proc.h"
+#include "utils/builtins.h"
 
 
 /* Number of OIDs to prefetch (preallocate) per XLOG write */
@@ -46,6 +48,37 @@ GetNewTransactionId(bool isSubXact)
 
 	xid = ShmemVariableCache->nextXid;
 
+	/*
+	 * Check to see if it's safe to assign another XID.  This protects
+	 * against catastrophic data loss due to XID wraparound.  The basic
+	 * rules are: warn if we're past xidWarnLimit, and refuse to execute
+	 * transactions if we're past xidStopLimit, unless we are running in
+	 * a standalone backend (which gives an escape hatch to the DBA who
+	 * ignored all those warnings).
+	 *
+	 * Test is coded to fall out as fast as possible during normal operation,
+	 * ie, when the warn limit is set and we haven't violated it.
+	 */
+	if (TransactionIdFollowsOrEquals(xid, ShmemVariableCache->xidWarnLimit) &&
+		TransactionIdIsValid(ShmemVariableCache->xidWarnLimit))
+	{
+		if (IsUnderPostmaster &&
+			TransactionIdFollowsOrEquals(xid, ShmemVariableCache->xidStopLimit))
+			ereport(ERROR,
+					(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+					 errmsg("database is shut down to avoid wraparound data loss in database \"%s\"",
+							NameStr(ShmemVariableCache->limit_datname)),
+					 errhint("Stop the postmaster and use a standalone backend to VACUUM in \"%s\".",
+							 NameStr(ShmemVariableCache->limit_datname))));
+		else
+			ereport(WARNING,
+					(errmsg("database \"%s\" must be vacuumed within %u transactions",
+							NameStr(ShmemVariableCache->limit_datname),
+							ShmemVariableCache->xidWrapLimit - xid),
+					 errhint("To avoid a database shutdown, execute a full-database VACUUM in \"%s\".",
+							 NameStr(ShmemVariableCache->limit_datname))));
+	}
+
 	/*
 	 * If we are allocating the first XID of a new page of the commit log,
 	 * zero out that commit-log page before returning. We must do this
@@ -137,6 +170,84 @@ ReadNewTransactionId(void)
 	return xid;
 }
 
+/*
+ * Determine the last safe XID to allocate given the currently oldest
+ * datfrozenxid (ie, the oldest XID that might exist in any database
+ * of our cluster).
+ */
+void
+SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
+					  Name oldest_datname)
+{
+	TransactionId xidWarnLimit;
+	TransactionId xidStopLimit;
+	TransactionId xidWrapLimit;
+	TransactionId curXid;
+
+	Assert(TransactionIdIsValid(oldest_datfrozenxid));
+
+	/*
+	 * The place where we actually get into deep trouble is halfway around
+	 * from the oldest potentially-existing XID.  (This calculation is
+	 * probably off by one or two counts, because the special XIDs reduce the
+	 * size of the loop a little bit.  But we throw in plenty of slop below,
+	 * so it doesn't matter.)
+	 */
+	xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1);
+	if (xidWrapLimit < FirstNormalTransactionId)
+		xidWrapLimit += FirstNormalTransactionId;
+
+	/*
+	 * We'll refuse to continue assigning XIDs in interactive mode once
+	 * we get within 1M transactions of data loss.  This leaves lots
+	 * of room for the DBA to fool around fixing things in a standalone
+	 * backend, while not being significant compared to total XID space.
+	 * (Note that since vacuuming requires one transaction per table
+	 * cleaned, we had better be sure there's lots of XIDs left...)
+	 */
+	xidStopLimit = xidWrapLimit - 1000000;
+	if (xidStopLimit < FirstNormalTransactionId)
+		xidStopLimit -= FirstNormalTransactionId;
+
+	/*
+	 * We'll start complaining loudly when we get within 10M transactions
+	 * of the stop point.  This is kind of arbitrary, but if you let your
+	 * gas gauge get down to 1% of full, would you be looking for the
+	 * next gas station?  We need to be fairly liberal about this number
+	 * because there are lots of scenarios where most transactions are
+	 * done by automatic clients that won't pay attention to warnings.
+	 * (No, we're not gonna make this configurable.  If you know enough to
+	 * configure it, you know enough to not get in this kind of trouble in
+	 * the first place.)
+	 */
+	xidWarnLimit = xidStopLimit - 10000000;
+	if (xidWarnLimit < FirstNormalTransactionId)
+		xidWarnLimit -= FirstNormalTransactionId;
+
+	/* Grab lock for just long enough to set the new limit values */
+	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+	ShmemVariableCache->xidWarnLimit = xidWarnLimit;
+	ShmemVariableCache->xidStopLimit = xidStopLimit;
+	ShmemVariableCache->xidWrapLimit = xidWrapLimit;
+	namecpy(&ShmemVariableCache->limit_datname, oldest_datname);
+	curXid = ShmemVariableCache->nextXid;
+	LWLockRelease(XidGenLock);
+
+	/* Log the info */
+	ereport(LOG,
+			(errmsg("transaction ID wrap limit is %u, limited by database \"%s\"",
+					xidWrapLimit, NameStr(*oldest_datname))));
+	/* Give an immediate warning if past the wrap warn point */
+	if (TransactionIdFollowsOrEquals(curXid, xidWarnLimit))
+		ereport(WARNING,
+				(errmsg("database \"%s\" must be vacuumed within %u transactions",
+						NameStr(*oldest_datname),
+						xidWrapLimit - curXid),
+				 errhint("To avoid a database shutdown, execute a full-database VACUUM in \"%s\".",
+						 NameStr(*oldest_datname))));
+}
+
+
 /* ----------------------------------------------------------------
  *					object id generation support
  * ----------------------------------------------------------------

src/backend/access/transam/xact.c

@@ -10,7 +10,7 @@
  *
  *
  * IDENTIFICATION
- *	  $PostgreSQL: pgsql/src/backend/access/transam/xact.c,v 1.195 2004/12/31 21:59:29 pgsql Exp $
+ *	  $PostgreSQL: pgsql/src/backend/access/transam/xact.c,v 1.196 2005/02/20 02:21:28 tgl Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -28,14 +28,14 @@
 #include "commands/async.h"
 #include "commands/tablecmds.h"
 #include "commands/trigger.h"
-#include "commands/user.h"
 #include "executor/spi.h"
 #include "libpq/be-fsstubs.h"
 #include "miscadmin.h"
 #include "storage/fd.h"
 #include "storage/proc.h"
 #include "storage/sinval.h"
 #include "storage/smgr.h"
+#include "utils/flatfiles.h"
 #include "utils/guc.h"
 #include "utils/inval.h"
 #include "utils/memutils.h"
@@ -1354,6 +1354,7 @@ StartTransaction(void)
 	 * start processing
 	 */
 	s->state = TRANS_START;
+	s->transactionId = InvalidTransactionId; /* until assigned */
 
 	/*
 	 * Make sure we've freed any old snapshot, and reset xact state
@@ -1464,9 +1465,9 @@ CommitTransaction(void)
 	/* NOTIFY commit must come before lower-level cleanup */
 	AtCommit_Notify();
 
-	/* Update the flat password file if we changed pg_shadow or pg_group */
+	/* Update flat files if we changed pg_database, pg_shadow or pg_group */
 	/* This should be the last step before commit */
-	AtEOXact_UpdatePasswordFile(true);
+	AtEOXact_UpdateFlatFiles(true);
 
 	/* Prevent cancel/die interrupt while cleaning up */
 	HOLD_INTERRUPTS();
@@ -1654,10 +1655,14 @@ AbortTransaction(void)
 	AtAbort_Portals();
 	AtEOXact_LargeObject(false);	/* 'false' means it's abort */
 	AtAbort_Notify();
-	AtEOXact_UpdatePasswordFile(false);
+	AtEOXact_UpdateFlatFiles(false);
 
-	/* Advertise the fact that we aborted in pg_clog. */
-	RecordTransactionAbort();
+	/*
+	 * Advertise the fact that we aborted in pg_clog (assuming that we
+	 * got as far as assigning an XID to advertise).
+	 */
+	if (TransactionIdIsValid(s->transactionId))
+		RecordTransactionAbort();
 
 	/*
 	 * Let others know about no transaction in progress by me. Note that
@@ -2034,10 +2039,25 @@ AbortCurrentTransaction(void)
 
 	switch (s->blockState)
 	{
-			/*
-			 * we aren't in a transaction, so we do nothing.
-			 */
 		case TBLOCK_DEFAULT:
+			if (s->state == TRANS_DEFAULT)
+			{
+				/* we are idle, so nothing to do */
+			}
+			else
+			{
+				/*
+				 * We can get here after an error during transaction start
+				 * (state will be TRANS_START).  Need to clean up the
+				 * incompletely started transaction.  First, adjust the
+				 * low-level state to suppress warning message from
+				 * AbortTransaction.
+				 */
+				if (s->state == TRANS_START)
+					s->state = TRANS_INPROGRESS;
+				AbortTransaction();
+				CleanupTransaction();
+			}
 			break;
 
 			/*
@@ -3277,8 +3297,8 @@ CommitSubTransaction(void)
 	AtEOSubXact_LargeObject(true, s->subTransactionId,
 							s->parent->subTransactionId);
 	AtSubCommit_Notify();
-	AtEOSubXact_UpdatePasswordFile(true, s->subTransactionId,
-								   s->parent->subTransactionId);
+	AtEOSubXact_UpdateFlatFiles(true, s->subTransactionId,
+								s->parent->subTransactionId);
 
 	CallSubXactCallbacks(SUBXACT_EVENT_COMMIT_SUB, s->subTransactionId,
 						 s->parent->subTransactionId);
@@ -3387,8 +3407,8 @@ AbortSubTransaction(void)
 		AtEOSubXact_LargeObject(false, s->subTransactionId,
 								s->parent->subTransactionId);
 		AtSubAbort_Notify();
-		AtEOSubXact_UpdatePasswordFile(false, s->subTransactionId,
-									   s->parent->subTransactionId);
+		AtEOSubXact_UpdateFlatFiles(false, s->subTransactionId,
+									s->parent->subTransactionId);
 
 		/* Advertise the fact that we aborted in pg_clog. */
 		if (TransactionIdIsValid(s->transactionId))

src/backend/bootstrap/bootstrap.c

@@ -8,7 +8,7 @@
  * Portions Copyright (c) 1994, Regents of the University of California
  *
  * IDENTIFICATION
- *	  $PostgreSQL: pgsql/src/backend/bootstrap/bootstrap.c,v 1.198 2005/01/14 21:08:44 tgl Exp $
+ *	  $PostgreSQL: pgsql/src/backend/bootstrap/bootstrap.c,v 1.199 2005/02/20 02:21:31 tgl Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -39,6 +39,7 @@
 #include "storage/proc.h"
 #include "tcop/tcopprot.h"
 #include "utils/builtins.h"
+#include "utils/flatfiles.h"
 #include "utils/fmgroids.h"
 #include "utils/guc.h"
 #include "utils/lsyscache.h"
@@ -407,6 +408,7 @@ BootstrapMain(int argc, char *argv[])
 			bootstrap_signals();
 			StartupXLOG();
 			LoadFreeSpaceMap();
+			BuildFlatFiles(false);
 			proc_exit(0);		/* startup done */
 
 		case BS_XLOG_BGWRITER: