Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 6587818

Browse files
hlinnakahlinnaka
committed
Add vacuum_freeze_table_age GUC option, to control when VACUUM should
ignore the visibility map and scan the whole table, to advance relfrozenxid.
1 parent 19afb4e commit 6587818

File tree

16 files changed

+170
-54
lines changed

16 files changed

+170
-54
lines changed

‎doc/src/sgml/catalogs.sgml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- $PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.188 2009/01/09 15:02:22 momjian Exp $ -->
1+
<!-- $PostgreSQL: pgsql/doc/src/sgml/catalogs.sgml,v 2.189 2009/01/16 13:27:23 heikki Exp $ -->
22
<!--
33
Documentation of the system catalogs, directed toward PostgreSQL developers
44
-->
@@ -1361,6 +1361,13 @@
13611361
<entry></entry>
13621362
<entry>Custom <varname>autovacuum_freeze_max_age</> parameter</entry>
13631363
</row>
1364+
1365+
<row>
1366+
<entry><structfield>freeze_table_age</structfield></entry>
1367+
<entry><type>integer</type></entry>
1368+
<entry></entry>
1369+
<entry>Custom <varname>vacuum_freeze_table_age</> parameter</entry>
1370+
</row>
13641371
</tbody>
13651372
</tgroup>
13661373
</table>

‎doc/src/sgml/config.sgml

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.205 2009/01/12 05:10:44 tgl Exp $ -->
1+
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.206 2009/01/16 13:27:23 heikki Exp $ -->
22

33
<chapter Id="runtime-config">
44
<title>Server Configuration</title>
@@ -3950,6 +3950,27 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
39503950
</listitem>
39513951
</varlistentry>
39523952

3953+
<varlistentry id="guc-vacuum-freeze-table-age" xreflabel="vacuum_freeze_table_age">
3954+
<term><varname>vacuum_freeze_table_age</varname> (<type>integer</type>)</term>
3955+
<indexterm>
3956+
<primary><varname>vacuum_freeze_table_age</> configuration parameter</primary>
3957+
</indexterm>
3958+
<listitem>
3959+
<para>
3960+
<command>VACUUM</> performs a whole-table scan if the table's
3961+
<structname>pg_class</>.<structfield>relfrozenxid</> field has reached
3962+
the age specified by this setting. The default is 150 million
3963+
transactions. Although users can set this value anywhere from zero to
3964+
one billion, <command>VACUUM</> will silently limit the effective value
3965+
to 95% of <xref linkend="guc-autovacuum-freeze-max-age">, so that a
3966+
periodical manual <command>VACUUM</> has a chance to run before an
3967+
anti-wraparound autovacuum is launched for the table. For more
3968+
information see
3969+
<xref linkend="vacuum-for-wraparound">.
3970+
</para>
3971+
</listitem>
3972+
</varlistentry>
3973+
39533974
<varlistentry id="guc-vacuum-freeze-min-age" xreflabel="vacuum_freeze_min_age">
39543975
<term><varname>vacuum_freeze_min_age</varname> (<type>integer</type>)</term>
39553976
<indexterm>
@@ -3960,7 +3981,7 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
39603981
Specifies the cutoff age (in transactions) that <command>VACUUM</>
39613982
should use to decide whether to replace transaction IDs with
39623983
<literal>FrozenXID</> while scanning a table.
3963-
The default is 100 million transactions. Although
3984+
The default is 50 million transactions. Although
39643985
users can set this value anywhere from zero to one billion,
39653986
<command>VACUUM</> will silently limit the effective value to half
39663987
the value of <xref linkend="guc-autovacuum-freeze-max-age">, so

‎doc/src/sgml/maintenance.sgml

Lines changed: 34 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- $PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.88 2008/12/11 18:16:18 tgl Exp $ -->
1+
<!-- $PostgreSQL: pgsql/doc/src/sgml/maintenance.sgml,v 1.89 2009/01/16 13:27:23 heikki Exp $ -->
22

33
<chapter id="maintenance">
44
<title>Routine Database Maintenance Tasks</title>
@@ -367,10 +367,14 @@
367367
</para>
368368

369369
<para>
370-
<command>VACUUM</>'s behavior is controlled by the configuration parameter
371-
<xref linkend="guc-vacuum-freeze-min-age">: any XID older than
372-
<varname>vacuum_freeze_min_age</> transactions is replaced by
373-
<literal>FrozenXID</>. Larger values of <varname>vacuum_freeze_min_age</>
370+
<command>VACUUM</>'s behavior is controlled by the two configuration
371+
parameters: <xref linkend="guc-vacuum-freeze-min-age"> and
372+
<xref linkend="guc-vacuum-freeze-table-age">.
373+
<varname>vacuum_freeze_table_age</> controls when <command>VACUUM</>
374+
performs a full sweep of the table, in order to replace old XID values
375+
with <literal>FrozenXID</>. <varname>vacuum_freeze_min_age</>
376+
controls how old an XID value has to be before it's replaced with
377+
<literal>FrozenXID</>. Larger values of these settings
374378
preserve transactional information longer, while smaller values increase
375379
the number of transactions that can elapse before the table must be
376380
vacuumed again.
@@ -379,7 +383,8 @@
379383
<para>
380384
The maximum time that a table can go unvacuumed is two billion
381385
transactions minus the <varname>vacuum_freeze_min_age</> that was used
382-
when it was last vacuumed. If it were to go unvacuumed for longer than
386+
when <command>VACUUM</> last scanned the whole table. If it were to go
387+
unvacuumed for longer than
383388
that, data loss could result. To ensure that this does not happen,
384389
autovacuum is invoked on any table that might contain XIDs older than the
385390
age specified by the configuration parameter <xref
@@ -403,16 +408,18 @@
403408
</para>
404409

405410
<para>
406-
The sole disadvantage of increasing <varname>autovacuum_freeze_max_age</>
411+
The sole disadvantage of increasing <varname>vacuum_freeze_table_age</>
412+
and <varname>autovacuum_freeze_max_age</>
407413
is that the <filename>pg_clog</> subdirectory of the database cluster
408414
will take more space, because it must store the commit status for all
409415
transactions back to the <varname>autovacuum_freeze_max_age</> horizon.
410416
The commit status uses two bits per transaction, so if
411417
<varname>autovacuum_freeze_max_age</> has its maximum allowed value of
412418
a little less than two billion, <filename>pg_clog</> can be expected to
413419
grow to about half a gigabyte. If this is trivial compared to your
414-
total database size, setting <varname>autovacuum_freeze_max_age</> to
415-
its maximum allowed value is recommended. Otherwise, set it depending
420+
total database size, setting <varname>autovacuum_freeze_max_age</> and
421+
<varname>vacuum_freeze_table_age</varname> to their maximum allowed values
422+
is recommended. Otherwise, set them depending
416423
on what you are willing to allow for <filename>pg_clog</> storage.
417424
(The default, 200 million transactions, translates to about 50MB of
418425
<filename>pg_clog</> storage.)
@@ -455,13 +462,24 @@ SELECT datname, age(datfrozenxid) FROM pg_database;
455462
</programlisting>
456463

457464
The <literal>age</> column measures the number of transactions from the
458-
cutoff XID to the current transaction's XID. Immediately after a
459-
<command>VACUUM</>, <literal>age(relfrozenxid)</> should be a little
460-
more than the <varname>vacuum_freeze_min_age</> setting that was used
461-
(more by the number of transactions started since the <command>VACUUM</>
462-
started). If <literal>age(relfrozenxid)</> exceeds
463-
<varname>autovacuum_freeze_max_age</>, an autovacuum will soon be forced
464-
for the table.
465+
cutoff XID to the current transaction's XID. When <command>VACUUM</>
466+
scans the whole table, after it's finished <literal>age(relfrozenxid)</>
467+
should be a little more than the <varname>vacuum_freeze_min_age</> setting
468+
that was used (more by the number of transactions started since the
469+
<command>VACUUM</> started).
470+
</para>
471+
472+
<para>
473+
<command>VACUUM</> normally only scans pages that have been modified
474+
since last vacuum, but <structfield>relfrozenxid</> can only be advanced
475+
when the whole table is scanned. The whole table is scanned when
476+
<structfield>relfrozenxid</> is more than
477+
<varname>vacuum_freeze_table_age</> transactions old, if
478+
<command>VACUUM FREEZE</> command is used, or if all pages happen to
479+
require vacuuming to remove dead row versions. If no whole-table-scanning
480+
<command>VACUUM</> is issued on the table until
481+
<varname>autovacuum_freeze_max_age</> is reached, an autovacuum will soon
482+
be forced for the table.
465483
</para>
466484

467485
<para>

‎src/backend/commands/cluster.c

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
*
1212
*
1313
* IDENTIFICATION
14-
* $PostgreSQL: pgsql/src/backend/commands/cluster.c,v 1.180 2009/01/01 17:23:37 momjian Exp $
14+
* $PostgreSQL: pgsql/src/backend/commands/cluster.c,v 1.181 2009/01/16 13:27:23 heikki Exp $
1515
*
1616
*-------------------------------------------------------------------------
1717
*/
@@ -789,8 +789,8 @@ copy_heap_data(Oid OIDNewHeap, Oid OIDOldHeap, Oid OIDOldIndex)
789789
* freeze_min_age to avoid having CLUSTER freeze tuples earlier than a
790790
* plain VACUUM would.
791791
*/
792-
vacuum_set_xid_limits(-1, OldHeap->rd_rel->relisshared,
793-
&OldestXmin, &FreezeXid);
792+
vacuum_set_xid_limits(-1, -1, OldHeap->rd_rel->relisshared,
793+
&OldestXmin, &FreezeXid, NULL);
794794

795795
/*
796796
* FreezeXid will become the table's new relfrozenxid, and that mustn't

‎src/backend/commands/vacuum.c

Lines changed: 38 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
*
1414
*
1515
* IDENTIFICATION
16-
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.384 2009/01/01 17:23:40 momjian Exp $
16+
* $PostgreSQL: pgsql/src/backend/commands/vacuum.c,v 1.385 2009/01/16 13:27:23 heikki Exp $
1717
*
1818
*-------------------------------------------------------------------------
1919
*/
@@ -62,6 +62,7 @@
6262
* GUC parameters
6363
*/
6464
int vacuum_freeze_min_age;
65+
int vacuum_freeze_table_age;
6566

6667
/*
6768
* VacPage structures keep track of each page on which we find useful
@@ -590,9 +591,12 @@ get_rel_oids(Oid relid, const RangeVar *vacrel, const char *stmttype)
590591
* vacuum_set_xid_limits() -- compute oldest-Xmin and freeze cutoff points
591592
*/
592593
void
593-
vacuum_set_xid_limits(int freeze_min_age, bool sharedRel,
594+
vacuum_set_xid_limits(int freeze_min_age,
595+
int freeze_table_age,
596+
bool sharedRel,
594597
TransactionId *oldestXmin,
595-
TransactionId *freezeLimit)
598+
TransactionId *freezeLimit,
599+
TransactionId *freezeTableLimit)
596600
{
597601
int freezemin;
598602
TransactionId limit;
@@ -648,6 +652,34 @@ vacuum_set_xid_limits(int freeze_min_age, bool sharedRel,
648652
}
649653

650654
*freezeLimit = limit;
655+
656+
if (freezeTableLimit != NULL)
657+
{
658+
int freezetable;
659+
660+
/*
661+
* Determine the table freeze age to use: as specified by the caller,
662+
* or vacuum_freeze_table_age, but in any case not more than
663+
* autovacuum_freeze_max_age * 0.95, so that if you have e.g nightly
664+
* VACUUM schedule, the nightly VACUUM gets a chance to freeze tuples
665+
* before anti-wraparound autovacuum is launched.
666+
*/
667+
freezetable = freeze_min_age;
668+
if (freezetable < 0)
669+
freezetable = vacuum_freeze_table_age;
670+
freezetable = Min(freezetable, autovacuum_freeze_max_age * 0.95);
671+
Assert(freezetable >= 0);
672+
673+
/*
674+
* Compute the cutoff XID, being careful not to generate a
675+
* "permanent" XID.
676+
*/
677+
limit = ReadNewTransactionId() - freezetable;
678+
if (!TransactionIdIsNormal(limit))
679+
limit = FirstNormalTransactionId;
680+
681+
*freezeTableLimit = limit;
682+
}
651683
}
652684

653685

@@ -1219,8 +1251,9 @@ full_vacuum_rel(Relation onerel, VacuumStmt *vacstmt)
12191251
i;
12201252
VRelStats *vacrelstats;
12211253

1222-
vacuum_set_xid_limits(vacstmt->freeze_min_age, onerel->rd_rel->relisshared,
1223-
&OldestXmin, &FreezeLimit);
1254+
vacuum_set_xid_limits(vacstmt->freeze_min_age, vacstmt->freeze_table_age,
1255+
onerel->rd_rel->relisshared,
1256+
&OldestXmin, &FreezeLimit, NULL);
12241257

12251258
/*
12261259
* Flush any previous async-commit transactions. This does not guarantee

‎src/backend/commands/vacuumlazy.c

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
*
3030
*
3131
* IDENTIFICATION
32-
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.116 2009/01/06 14:55:37 heikki Exp $
32+
* $PostgreSQL: pgsql/src/backend/commands/vacuumlazy.c,v 1.117 2009/01/16 13:27:23 heikki Exp $
3333
*
3434
*-------------------------------------------------------------------------
3535
*/
@@ -144,6 +144,8 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt,
144144
BlockNumber possibly_freeable;
145145
PGRUsage ru0;
146146
TimestampTz starttime = 0;
147+
bool scan_all;
148+
TransactionId freezeTableLimit;
147149

148150
pg_rusage_init(&ru0);
149151

@@ -158,8 +160,11 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt,
158160

159161
vac_strategy = bstrategy;
160162

161-
vacuum_set_xid_limits(vacstmt->freeze_min_age, onerel->rd_rel->relisshared,
162-
&OldestXmin, &FreezeLimit);
163+
vacuum_set_xid_limits(vacstmt->freeze_min_age, vacstmt->freeze_table_age,
164+
onerel->rd_rel->relisshared,
165+
&OldestXmin, &FreezeLimit, &freezeTableLimit);
166+
scan_all = TransactionIdPrecedesOrEquals(onerel->rd_rel->relfrozenxid,
167+
freezeTableLimit);
163168

164169
vacrelstats = (LVRelStats *) palloc0(sizeof(LVRelStats));
165170

@@ -171,7 +176,7 @@ lazy_vacuum_rel(Relation onerel, VacuumStmt *vacstmt,
171176
vacrelstats->hasindex = (nindexes > 0);
172177

173178
/* Do the vacuuming */
174-
lazy_scan_heap(onerel, vacrelstats, Irel, nindexes, vacstmt->scan_all);
179+
lazy_scan_heap(onerel, vacrelstats, Irel, nindexes, scan_all);
175180

176181
/* Done with indexes */
177182
vac_close_indexes(nindexes, Irel, NoLock);

‎src/backend/nodes/copyfuncs.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
* Portions Copyright (c) 1994, Regents of the University of California
1616
*
1717
* IDENTIFICATION
18-
* $PostgreSQL: pgsql/src/backend/nodes/copyfuncs.c,v 1.419 2009/01/01 17:23:43 momjian Exp $
18+
* $PostgreSQL: pgsql/src/backend/nodes/copyfuncs.c,v 1.420 2009/01/16 13:27:23 heikki Exp $
1919
*
2020
*-------------------------------------------------------------------------
2121
*/
@@ -2865,7 +2865,7 @@ _copyVacuumStmt(VacuumStmt *from)
28652865
COPY_SCALAR_FIELD(analyze);
28662866
COPY_SCALAR_FIELD(verbose);
28672867
COPY_SCALAR_FIELD(freeze_min_age);
2868-
COPY_SCALAR_FIELD(scan_all);
2868+
COPY_SCALAR_FIELD(freeze_table_age);
28692869
COPY_NODE_FIELD(relation);
28702870
COPY_NODE_FIELD(va_cols);
28712871

‎src/backend/nodes/equalfuncs.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
* Portions Copyright (c) 1994, Regents of the University of California
2323
*
2424
* IDENTIFICATION
25-
* $PostgreSQL: pgsql/src/backend/nodes/equalfuncs.c,v 1.344 2009/01/01 17:23:43 momjian Exp $
25+
* $PostgreSQL: pgsql/src/backend/nodes/equalfuncs.c,v 1.345 2009/01/16 13:27:23 heikki Exp $
2626
*
2727
*-------------------------------------------------------------------------
2828
*/
@@ -1454,7 +1454,7 @@ _equalVacuumStmt(VacuumStmt *a, VacuumStmt *b)
14541454
COMPARE_SCALAR_FIELD(analyze);
14551455
COMPARE_SCALAR_FIELD(verbose);
14561456
COMPARE_SCALAR_FIELD(freeze_min_age);
1457-
COMPARE_SCALAR_FIELD(scan_all);
1457+
COMPARE_SCALAR_FIELD(freeze_table_age);
14581458
COMPARE_NODE_FIELD(relation);
14591459
COMPARE_NODE_FIELD(va_cols);
14601460

‎src/backend/parser/gram.y

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
*
1212
*
1313
* IDENTIFICATION
14-
* $PostgreSQL: pgsql/src/backend/parser/gram.y,v 2.654 2009/01/12 09:38:30 petere Exp $
14+
* $PostgreSQL: pgsql/src/backend/parser/gram.y,v 2.655 2009/01/16 13:27:23 heikki Exp $
1515
*
1616
* HISTORY
1717
* AUTHOR DATE MAJOR EVENT
@@ -6263,7 +6263,7 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose
62636263
n->analyze = false;
62646264
n->full = $2;
62656265
n->freeze_min_age = $3 ? 0 : -1;
6266-
n->scan_all = $2 || $3;
6266+
n->freeze_table_age = $3 ? 0 : -1;
62676267
n->verbose = $4;
62686268
n->relation = NULL;
62696269
n->va_cols = NIL;
@@ -6276,7 +6276,7 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose
62766276
n->analyze = false;
62776277
n->full = $2;
62786278
n->freeze_min_age = $3 ? 0 : -1;
6279-
n->scan_all = $2 || $3;
6279+
n->freeze_table_age = $3 ? 0 : -1;
62806280
n->verbose = $4;
62816281
n->relation = $5;
62826282
n->va_cols = NIL;
@@ -6288,7 +6288,7 @@ VacuumStmt: VACUUM opt_full opt_freeze opt_verbose
62886288
n->vacuum = true;
62896289
n->full = $2;
62906290
n->freeze_min_age = $3 ? 0 : -1;
6291-
n->scan_all = $2 || $3;
6291+
n->freeze_table_age = $3 ? 0 : -1;
62926292
n->verbose |= $4;
62936293
$$ = (Node *)n;
62946294
}
@@ -6302,6 +6302,7 @@ AnalyzeStmt:
63026302
n->analyze = true;
63036303
n->full = false;
63046304
n->freeze_min_age = -1;
6305+
n->freeze_table_age = -1;
63056306
n->verbose = $2;
63066307
n->relation = NULL;
63076308
n->va_cols = NIL;
@@ -6314,6 +6315,7 @@ AnalyzeStmt:
63146315
n->analyze = true;
63156316
n->full = false;
63166317
n->freeze_min_age = -1;
6318+
n->freeze_table_age = -1;
63176319
n->verbose = $2;
63186320
n->relation = $3;
63196321
n->va_cols = $4;

0 commit comments

Comments
 (0)