Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 4a33bb5

Browse files
committed
Ensure that WAL pages skipped by a forced WAL switch are zero-filled.
In the previous coding, skipped pages were mostly zeroes, but they still had valid WAL page headers. That makes them very much less compressible than an unbroken string of zeroes would be --- about 10X worse for bzip2 compression, for instance. We don't need those headers, so tweak the logic so that we zero them out. Chapman Flack, reviewed by Daniel Gustafsson Discussion: https://postgr.es/m/579297F8.7020107@anastigmatix.net
1 parent e5eb4fa commit 4a33bb5

File tree

1 file changed

+32
-12
lines changed
  • src/backend/access/transam

1 file changed

+32
-12
lines changed

src/backend/access/transam/xlog.c

+32-12
Original file line numberDiff line numberDiff line change
@@ -1533,30 +1533,50 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata,
15331533

15341534
/*
15351535
* If this was an xlog-switch, it's not enough to write the switch record,
1536-
* we also have to consume all the remaining space in the WAL segment. We
1537-
* have already reserved it for us, but we still need to make sure it's
1538-
* allocated and zeroed in the WAL buffers so that when the caller (or
1539-
* someone else) does XLogWrite(), it can really write out all the zeros.
1536+
* we also have to consume all the remaining space in the WAL segment. We
1537+
* have already reserved that space, but we need to actually fill it.
15401538
*/
15411539
if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0)
15421540
{
15431541
/* An xlog-switch record doesn't contain any data besides the header */
15441542
Assert(write_len == SizeOfXLogRecord);
15451543

1546-
/*
1547-
* We do this one page at a time, to make sure we don't deadlock
1548-
* against ourselves if wal_buffers < wal_segment_size.
1549-
*/
1544+
/* Assert that we did reserve the right amount of space */
15501545
Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0);
15511546

1552-
/* Use up all the remaining space on the first page */
1547+
/* Use up all the remaining space on the current page */
15531548
CurrPos += freespace;
15541549

1550+
/*
1551+
* Cause all remaining pages in the segment to be flushed, leaving the
1552+
* XLog position where it should be, at the start of the next segment.
1553+
* We do this one page at a time, to make sure we don't deadlock
1554+
* against ourselves if wal_buffers < wal_segment_size.
1555+
*/
15551556
while (CurrPos < EndPos)
15561557
{
1557-
/* initialize the next page (if not initialized already) */
1558-
WALInsertLockUpdateInsertingAt(CurrPos);
1559-
AdvanceXLInsertBuffer(CurrPos, false);
1558+
/*
1559+
* The minimal action to flush the page would be to call
1560+
* WALInsertLockUpdateInsertingAt(CurrPos) followed by
1561+
* AdvanceXLInsertBuffer(...). The page would be left initialized
1562+
* mostly to zeros, except for the page header (always the short
1563+
* variant, as this is never a segment's first page).
1564+
*
1565+
* The large vistas of zeros are good for compressibility, but the
1566+
* headers interrupting them every XLOG_BLCKSZ (with values that
1567+
* differ from page to page) are not. The effect varies with
1568+
* compression tool, but bzip2 for instance compresses about an
1569+
* order of magnitude worse if those headers are left in place.
1570+
*
1571+
* Rather than complicating AdvanceXLInsertBuffer itself (which is
1572+
* called in heavily-loaded circumstances as well as this lightly-
1573+
* loaded one) with variant behavior, we just use GetXLogBuffer
1574+
* (which itself calls the two methods we need) to get the pointer
1575+
* and zero most of the page. Then we just zero the page header.
1576+
*/
1577+
currpos = GetXLogBuffer(CurrPos);
1578+
MemSet(currpos, 0, SizeOfXLogShortPHD);
1579+
15601580
CurrPos += XLOG_BLCKSZ;
15611581
}
15621582
}

0 commit comments

Comments
 (0)