Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 667e65a

Browse files
Use TidStore for dead tuple TIDs storage during lazy vacuum.
Previously, we used a simple array for storing dead tuple IDs during lazy vacuum, which had a number of problems: * The array used a single allocation and so was limited to 1GB. * The allocation was pessimistically sized according to table size. * Lookup with binary search was slow because of poor CPU cache and branch prediction behavior. This commit replaces that array with the TID store from commit 30e1442. Since the backing radix tree makes small allocations as needed, the 1GB limit is now gone. Further, the total memory used is now often smaller by an order of magnitude or more, depending on the distribution of blocks and offsets. These two features should make multiple rounds of heap scanning and index cleanup an extremely rare event. TID lookup during index cleanup is also several times faster, even more so when index order is correlated with heap tuple order. Since there is no longer a predictable relationship between the number of dead tuples vacuumed and the space taken up by their TIDs, the number of tuples no longer provides any meaningful insights for users, nor is the maximum number predictable. For that reason this commit also changes to byte-based progress reporting, with the relevant columns of pg_stat_progress_vacuum renamed accordingly to max_dead_tuple_bytes and dead_tuple_bytes. For parallel vacuum, both the TID store and supplemental information specific to vacuum are shared among the parallel vacuum workers. As with the previous array, we don't take any locks on TidStore during parallel vacuum since writes are still only done by the leader process. Bump catalog version. Reviewed-by: John Naylor, (in an earlier version) Dilip Kumar Discussion: https://postgr.es/m/CAD21AoAfOZvmfR0j8VmZorZjL7RhTiQdVttNuC4W-Shdc2a-AA%40mail.gmail.com
1 parent d5d2205 commit 667e65a

File tree

14 files changed

+226
-289
lines changed

14 files changed

+226
-289
lines changed

doc/src/sgml/config.sgml

-12
Original file line numberDiff line numberDiff line change
@@ -1919,11 +1919,6 @@ include_dir 'conf.d'
19191919
too high. It may be useful to control for this by separately
19201920
setting <xref linkend="guc-autovacuum-work-mem"/>.
19211921
</para>
1922-
<para>
1923-
Note that for the collection of dead tuple identifiers,
1924-
<command>VACUUM</command> is only able to utilize up to a maximum of
1925-
<literal>1GB</literal> of memory.
1926-
</para>
19271922
</listitem>
19281923
</varlistentry>
19291924

@@ -1946,13 +1941,6 @@ include_dir 'conf.d'
19461941
<filename>postgresql.conf</filename> file or on the server command
19471942
line.
19481943
</para>
1949-
<para>
1950-
For the collection of dead tuple identifiers, autovacuum is only able
1951-
to utilize up to a maximum of <literal>1GB</literal> of memory, so
1952-
setting <varname>autovacuum_work_mem</varname> to a value higher than
1953-
that has no effect on the number of dead tuples that autovacuum can
1954-
collect while scanning a table.
1955-
</para>
19561944
</listitem>
19571945
</varlistentry>
19581946

doc/src/sgml/monitoring.sgml

+4-4
Original file line numberDiff line numberDiff line change
@@ -6237,21 +6237,21 @@ FROM pg_stat_get_backend_idset() AS backendid;
62376237

62386238
<row>
62396239
<entry role="catalog_table_entry"><para role="column_definition">
6240-
<structfield>max_dead_tuples</structfield> <type>bigint</type>
6240+
<structfield>max_dead_tuple_bytes</structfield> <type>bigint</type>
62416241
</para>
62426242
<para>
6243-
Number of dead tuples that we can store before needing to perform
6243+
Amount of dead tuple data that we can store before needing to perform
62446244
an index vacuum cycle, based on
62456245
<xref linkend="guc-maintenance-work-mem"/>.
62466246
</para></entry>
62476247
</row>
62486248

62496249
<row>
62506250
<entry role="catalog_table_entry"><para role="column_definition">
6251-
<structfield>num_dead_tuples</structfield> <type>bigint</type>
6251+
<structfield>dead_tuple_bytes</structfield> <type>bigint</type>
62526252
</para>
62536253
<para>
6254-
Number of dead tuples collected since the last index vacuum cycle.
6254+
Amount of dead tuple data collected since the last index vacuum cycle.
62556255
</para></entry>
62566256
</row>
62576257

0 commit comments

Comments
 (0)