Rename wal_keep_segments to wal_keep_size.

max_slot_wal_keep_size that was added in v13 and wal_keep_segments are
the GUC parameters to specify how much WAL files to retain for
the standby servers. While max_slot_wal_keep_size accepts the number of
bytes of WAL files, wal_keep_segments accepts the number of WAL files.
This difference of setting units between those similar parameters could
be confusing to users.

To alleviate this situation, this commit renames wal_keep_segments to
wal_keep_size, and make users specify the WAL size in it instead of
the number of WAL files.

There was also the idea to rename max_slot_wal_keep_size to
max_slot_wal_keep_segments, in the discussion. But we have been moving
away from measuring in segments, for example, checkpoint_segments was
replaced by max_wal_size. So we concluded to rename wal_keep_segments
to wal_keep_size.

Back-patch to v13 where max_slot_wal_keep_size was added.

Author: Fujii Masao
Reviewed-by: Álvaro Herrera, Kyotaro Horiguchi, David Steele
Discussion: https://postgr.es/m/574b4ea3-e0f9-b175-ead2-ebea7faea855@oss.nttdata.com

diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9f00ea90e32a34fc3605dae65d916726b8ba5909..048ff284f76d97784ca7a598e2607cc5258a6c3a 100644 (file)
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -11278,7 +11278,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
          <para><literal>extended</literal> means
           that <varname>max_wal_size</varname> is exceeded but the files are
           still retained, either by the replication slot or
-          by <varname>wal_keep_segments</varname>.
+          by <varname>wal_keep_size</varname>.
          </para>
         </listitem>
         <listitem>

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 1c491fb8ffcb03516068d2c8da8ef6761ae8977a..ca6a3a523ff689a8d89f4a0bd98983a418f123a1 100644 (file)
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3151,7 +3151,7 @@ include_dir 'conf.d'
         checkpoints. This is a soft limit; WAL size can exceed
         <varname>max_wal_size</varname> under special circumstances, such as
         heavy load, a failing <varname>archive_command</varname>, or a high
-        <varname>wal_keep_segments</varname> setting.
+        <varname>wal_keep_size</varname> setting.
         If this value is specified without units, it is taken as megabytes.
         The default is 1 GB.
         Increasing this parameter can increase the amount of time needed for
@@ -3778,21 +3778,21 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"'  # Windows
        </listitem>
       </varlistentry>
 
-      <varlistentry id="guc-wal-keep-segments" xreflabel="wal_keep_segments">
-       <term><varname>wal_keep_segments</varname> (<type>integer</type>)
+      <varlistentry id="guc-wal-keep-size" xreflabel="wal_keep_size">
+       <term><varname>wal_keep_size</varname> (<type>integer</type>)
        <indexterm>
-        <primary><varname>wal_keep_segments</varname> configuration parameter</primary>
+        <primary><varname>wal_keep_size</varname> configuration parameter</primary>
        </indexterm>
        </term>
        <listitem>
        <para>
-        Specifies the minimum number of past log file segments kept in the
+        Specifies the minimum size of past log file segments kept in the
         <filename>pg_wal</filename>
         directory, in case a standby server needs to fetch them for streaming
-        replication. Each segment is normally 16 megabytes. If a standby
+        replication. If a standby
         server connected to the sending server falls behind by more than
-        <varname>wal_keep_segments</varname> segments, the sending server might remove
-        a WAL segment still needed by the standby, in which case the
+        <varname>wal_keep_size</varname> megabytes, the sending server might
+        remove a WAL segment still needed by the standby, in which case the
         replication connection will be terminated.  Downstream connections
         will also eventually fail as a result.  (However, the standby
         server can recover by fetching the segment from archive, if WAL
@@ -3800,14 +3800,15 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"'  # Windows
        </para>
 
        <para>
-        This sets only the minimum number of segments retained in
+        This sets only the minimum size of segments retained in
         <filename>pg_wal</filename>; the system might need to retain more segments
         for WAL archival or to recover from a checkpoint. If
-        <varname>wal_keep_segments</varname> is zero (the default), the system
+        <varname>wal_keep_size</varname> is zero (the default), the system
         doesn't keep any extra segments for standby purposes, so the number
         of old WAL segments available to standby servers is a function of
         the location of the previous checkpoint and status of WAL
         archiving.
+        If this value is specified without units, it is taken as megabytes.
         This parameter can only be set in the
         <filename>postgresql.conf</filename> file or on the server command line.
        </para>

diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index 6a9184f314e6458dc29eaadd7b0e8f09bd77e881..89f6d6eda6362bb270b94e581081955274fa6724 100644 (file)
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -785,7 +785,7 @@ archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r'
     archiving, the server might recycle old WAL segments before the standby
     has received them.  If this occurs, the standby will need to be
     reinitialized from a new base backup.  You can avoid this by setting
-    <varname>wal_keep_segments</varname> to a value large enough to ensure that
+    <varname>wal_keep_size</varname> to a value large enough to ensure that
     WAL segments are not recycled too early, or by configuring a replication
     slot for the standby.  If you set up a WAL archive that's accessible from
     the standby, these solutions are not required, since the standby can
@@ -929,7 +929,7 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
    </para>
    <para>
     In lieu of using replication slots, it is possible to prevent the removal
-    of old WAL segments using <xref linkend="guc-wal-keep-segments"/>, or by
+    of old WAL segments using <xref linkend="guc-wal-keep-size"/>, or by
     storing the segments in an archive using
     <xref linkend="guc-archive-command"/>.
     However, these methods often result in retaining more WAL segments than

diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index aa41fb444fa731bbe423bf061c9800b1161c334f..e246efbdb52075a038429f9aad2a0c4dd5adbe3f 100644 (file)
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -305,7 +305,7 @@ PostgreSQL documentation
            <para>
             The write-ahead log files are collected at the end of the backup.
             Therefore, it is necessary for the
-            <xref linkend="guc-wal-keep-segments"/> parameter to be set high
+            <xref linkend="guc-wal-keep-size"/> parameter to be set high
              enough that the log is not removed before the end of the backup.
              If the log has been rotated when it's time to transfer it, the
              backup will fail and be unusable.

diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml
index 1902f36291db4657f781270e6f3a5060c6c3ec9b..7a13d8d5025ee3e2fd3c28cc2bfdddfd996b4790 100644 (file)
--- a/doc/src/sgml/wal.sgml
+++ b/doc/src/sgml/wal.sgml
@@ -578,7 +578,8 @@
 
   <para>
    Independently of <varname>max_wal_size</varname>,
-   <xref linkend="guc-wal-keep-segments"/> + 1 most recent WAL files are
+   the most recent <xref linkend="guc-wal-keep-size"/> megabytes of
+   WAL files plus one additional WAL file are
    kept at all times. Also, if WAL archiving is used, old segments can not be
    removed or recycled until they are archived. If WAL archiving cannot keep up
    with the pace that WAL is generated, or if <varname>archive_command</varname>

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0a97b1d37fbefcf1f5cbe8f2c9b04ad57dac5b42..184c6672f3be0d1a6ae66c6ea43819f5c72e6b7f 100644 (file)
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -88,7 +88,7 @@ extern uint32 bootstrap_data_checksum_version;
 /* User-settable parameters */
 int            max_wal_size_mb = 1024; /* 1 GB */
 int            min_wal_size_mb = 80;   /* 80 MB */
-int            wal_keep_segments = 0;
+int            wal_keep_size_mb = 0;
 int            XLOGbuffers = -1;
 int            XLogArchiveTimeout = 0;
 int            XLogArchiveMode = ARCHIVE_MODE_OFF;
@@ -9525,7 +9525,7 @@ GetWALAvailability(XLogRecPtr targetLSN)
 
    /*
     * Calculate the oldest segment currently reserved by all slots,
-    * considering wal_keep_segments and max_slot_wal_keep_size.  Initialize
+    * considering wal_keep_size and max_slot_wal_keep_size.  Initialize
     * oldestSlotSeg to the current segment.
     */
    currpos = GetXLogWriteRecPtr();
@@ -9576,9 +9576,9 @@ GetWALAvailability(XLogRecPtr targetLSN)
 
 /*
  * Retreat *logSegNo to the last segment that we need to retain because of
- * either wal_keep_segments or replication slots.
+ * either wal_keep_size or replication slots.
  *
- * This is calculated by subtracting wal_keep_segments from the given xlog
+ * This is calculated by subtracting wal_keep_size from the given xlog
  * location, recptr and by making sure that that result is below the
  * requirement of replication slots.  For the latter criterion we do consider
  * the effects of max_slot_wal_keep_size: reserve at most that much space back
@@ -9616,14 +9616,20 @@ KeepLogSeg(XLogRecPtr recptr, XLogSegNo *logSegNo)
        }
    }
 
-   /* but, keep at least wal_keep_segments if that's set */
-   if (wal_keep_segments > 0 && currSegNo - segno < wal_keep_segments)
+   /* but, keep at least wal_keep_size if that's set */
+   if (wal_keep_size_mb > 0)
    {
-       /* avoid underflow, don't go below 1 */
-       if (currSegNo <= wal_keep_segments)
-           segno = 1;
-       else
-           segno = currSegNo - wal_keep_segments;
+       uint64      keep_segs;
+
+       keep_segs = ConvertToXSegs(wal_keep_size_mb, wal_segment_size);
+       if (currSegNo - segno < keep_segs)
+       {
+           /* avoid underflow, don't go below 1 */
+           if (currSegNo <= keep_segs)
+               segno = 1;
+           else
+               segno = currSegNo - keep_segs;
+       }
    }
 
    /* don't delete WAL segments newer than the calculated segment */
@@ -11328,7 +11334,7 @@ do_pg_stop_backup(char *labelfile, bool waitforarchive, TimeLineID *stoptli_p)
     * If archiving is enabled, wait for all the required WAL files to be
     * archived before returning. If archiving isn't enabled, the required WAL
     * needs to be transported via streaming replication (hopefully with
-    * wal_keep_segments set high enough), or some more exotic mechanism like
+    * wal_keep_size set high enough), or some more exotic mechanism like
     * polling and copying files from pg_wal with script. We have no knowledge
     * of those mechanisms, so it's up to the user to ensure that he gets all
     * the required WAL.

diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9fe147bf44ec399cd041b75c6ab6e2ccd8bf6792..f88694672fba9a2b3849a0c0b8148fe9a0a3af36 100644 (file)
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -413,19 +413,20 @@ pg_get_replication_slots(PG_FUNCTION_ARGS)
        else
        {
            XLogSegNo   targetSeg;
-           XLogSegNo   keepSegs;
+           uint64   slotKeepSegs;
+           uint64   keepSegs;
            XLogSegNo   failSeg;
            XLogRecPtr  failLSN;
 
            XLByteToSeg(slot_contents.data.restart_lsn, targetSeg, wal_segment_size);
 
-           /* determine how many segments slots can be kept by slots ... */
-           keepSegs = XLogMBVarToSegs(max_slot_wal_keep_size_mb, wal_segment_size);
-           /* ... and override by wal_keep_segments as needed */
-           keepSegs = Max(keepSegs, wal_keep_segments);
+           /* determine how many segments slots can be kept by slots */
+           slotKeepSegs = XLogMBVarToSegs(max_slot_wal_keep_size_mb, wal_segment_size);
+           /* ditto for wal_keep_size */
+           keepSegs = XLogMBVarToSegs(wal_keep_size_mb, wal_segment_size);
 
            /* if currpos reaches failLSN, we lose our segment */
-           failSeg = targetSeg + keepSegs + 1;
+           failSeg = targetSeg + Max(slotKeepSegs, keepSegs) + 1;
            XLogSegNoOffsetToRecPtr(failSeg, 0, wal_segment_size, failLSN);
 
            values[i++] = Int64GetDatum(failLSN - currlsn);

diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 99a3e4f6f6553b9fff9be6bb6169a9ee6f2c7efd..6f603cbbe8c837e8bcc0997dcd10d24df4b0642d 100644 (file)
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2636,12 +2636,13 @@ static struct config_int ConfigureNamesInt[] =
    },
 
    {
-       {"wal_keep_segments", PGC_SIGHUP, REPLICATION_SENDING,
-           gettext_noop("Sets the number of WAL files held for standby servers."),
-           NULL
+       {"wal_keep_size", PGC_SIGHUP, REPLICATION_SENDING,
+           gettext_noop("Sets the size of WAL files held for standby servers."),
+           NULL,
+           GUC_UNIT_MB
        },
-       &wal_keep_segments,
-       0, 0, INT_MAX,
+       &wal_keep_size_mb,
+       0, 0, MAX_KILOBYTES,
        NULL, NULL, NULL
    },
 

diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 29e0152196689557cebc3c3eeee9250b2ff2d57c..5a0b8e982179eae7a742ec919a608c140b75ffef 100644 (file)
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -290,7 +290,7 @@
 
 #max_wal_senders = 10      # max number of walsender processes
                # (change requires restart)
-#wal_keep_segments = 0     # in logfile segments; 0 disables
+#wal_keep_size = 0     # in megabytes; 0 disables
 #max_slot_wal_keep_size = -1   # in megabytes; -1 disables
 #wal_sender_timeout = 60s  # in milliseconds; 0 disables
 

diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 7516af7a01a64cb8d993cc78ede6e5649c1cb04e..41ed7d4b3befd82b0abb38823f757507c044d374 100644 (file)
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -135,11 +135,11 @@ sub setup_cluster
        extra            => $extra,
        auth_extra       => [ '--create-role', 'rewind_user' ]);
 
-   # Set wal_keep_segments to prevent WAL segment recycling after enforced
+   # Set wal_keep_size to prevent WAL segment recycling after enforced
    # checkpoints in the tests.
    $node_primary->append_conf(
        'postgresql.conf', qq(
-wal_keep_segments = 20
+wal_keep_size = 320MB
 ));
    return;
 }

diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index d8391aa3783ac264b54e6e4b432b2892cbd77cb4..219a7299e1f0e5c6fbe4d5dd89b5df27a4778484 100644 (file)
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -107,7 +107,7 @@ extern bool reachedConsistency;
 extern int wal_segment_size;
 extern int min_wal_size_mb;
 extern int max_wal_size_mb;
-extern int wal_keep_segments;
+extern int wal_keep_size_mb;
 extern int max_slot_wal_keep_size_mb;
 extern int XLOGbuffers;
 extern int XLogArchiveTimeout;
@@ -273,7 +273,7 @@ typedef enum WALAvailability
    WALAVAIL_INVALID_LSN,       /* parameter error */
    WALAVAIL_RESERVED,          /* WAL segment is within max_wal_size */
    WALAVAIL_EXTENDED,          /* WAL segment is reserved by a slot or
-                                * wal_keep_segments */
+                                * wal_keep_size */
    WALAVAIL_UNRESERVED,        /* no longer reserved, but not removed yet */
    WALAVAIL_REMOVED            /* WAL segment has been removed */
 } WALAvailability;

diff --git a/src/test/recovery/t/019_replslot_limit.pl b/src/test/recovery/t/019_replslot_limit.pl
index 1fced12fca50fba673381ce892f4cb9bce487157..20f4a1bbc3dcc28e972098572d3dbaa13de86a26 100644 (file)
--- a/src/test/recovery/t/019_replslot_limit.pl
+++ b/src/test/recovery/t/019_replslot_limit.pl
@@ -116,19 +116,19 @@ $start_lsn = $node_primary->lsn('write');
 $node_primary->wait_for_catchup($node_standby, 'replay', $start_lsn);
 $node_standby->stop;
 
-# wal_keep_segments overrides max_slot_wal_keep_size
+# wal_keep_size overrides max_slot_wal_keep_size
 $result = $node_primary->safe_psql('postgres',
-   "ALTER SYSTEM SET wal_keep_segments to 8; SELECT pg_reload_conf();");
+   "ALTER SYSTEM SET wal_keep_size to '8MB'; SELECT pg_reload_conf();");
 # Advance WAL again then checkpoint, reducing remain by 6 MB.
 advance_wal($node_primary, 6);
 $result = $node_primary->safe_psql('postgres',
    "SELECT wal_status as remain FROM pg_replication_slots WHERE slot_name = 'rep1'"
 );
 is($result, "extended",
-   'check that wal_keep_segments overrides max_slot_wal_keep_size');
-# restore wal_keep_segments
+   'check that wal_keep_size overrides max_slot_wal_keep_size');
+# restore wal_keep_size
 $result = $node_primary->safe_psql('postgres',
-   "ALTER SYSTEM SET wal_keep_segments to 0; SELECT pg_reload_conf();");
+   "ALTER SYSTEM SET wal_keep_size to 0; SELECT pg_reload_conf();");
 
 # The standby can reconnect to primary
 $node_standby->start;