Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 4d928db

Browse files
author
Commitfest Bot
committed
[CF 5323] v20250315 - Support enabling checksums online
This branch was automatically generated by a robot using patches from an email thread registered at: https://commitfest.postgresql.org/patch/5323 The branch will be overwritten each time a new patch version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. Patch(es): https://www.postgresql.org/message-id/f528413c-477a-4ec3-a0df-e22a80ffbe41@vondra.me Author(s): Magnus Hagander, Daniel Gustafsson
2 parents 203c1b4 + 50fea27 commit 4d928db

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+3454
-56
lines changed

‎doc/src/sgml/func.sgml

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29882,6 +29882,77 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset
2988229882

2988329883
</sect2>
2988429884

29885+
<sect2 id="functions-admin-checksum">
29886+
<title>Data Checksum Functions</title>
29887+
29888+
<para>
29889+
The functions shown in <xref linkend="functions-checksums-table" /> can
29890+
be used to enable or disable data checksums in a running cluster.
29891+
See <xref linkend="checksums" /> for details.
29892+
</para>
29893+
29894+
<table id="functions-checksums-table">
29895+
<title>Data Checksum Functions</title>
29896+
<tgroup cols="1">
29897+
<thead>
29898+
<row>
29899+
<entry role="func_table_entry"><para role="func_signature">
29900+
Function
29901+
</para>
29902+
<para>
29903+
Description
29904+
</para></entry>
29905+
</row>
29906+
</thead>
29907+
29908+
<tbody>
29909+
<row>
29910+
<entry role="func_table_entry"><para role="func_signature">
29911+
<indexterm>
29912+
<primary>pg_enable_data_checksums</primary>
29913+
</indexterm>
29914+
<function>pg_enable_data_checksums</function> ( <optional><parameter>cost_delay</parameter> <type>int</type>, <parameter>cost_limit</parameter> <type>int</type></optional> )
29915+
<returnvalue>void</returnvalue>
29916+
</para>
29917+
<para>
29918+
Initiates data checksums for the cluster. This will switch the data
29919+
checksums mode to <literal>inprogress-on</literal> as well as start a
29920+
background worker that will process all pages in the database and
29921+
enable checksums on them. When all data pages have had checksums
29922+
enabled, the cluster will automatically switch data checksums mode to
29923+
<literal>on</literal>.
29924+
</para>
29925+
<para>
29926+
If <parameter>cost_delay</parameter> and <parameter>cost_limit</parameter> are
29927+
specified, the speed of the process is throttled using the same principles as
29928+
<link linkend="runtime-config-resource-vacuum-cost">Cost-based Vacuum Delay</link>.
29929+
</para></entry>
29930+
</row>
29931+
29932+
<row>
29933+
<entry role="func_table_entry"><para role="func_signature">
29934+
<indexterm>
29935+
<primary>pg_disable_data_checksums</primary>
29936+
</indexterm>
29937+
<function>pg_disable_data_checksums</function> ()
29938+
<returnvalue>void</returnvalue>
29939+
</para>
29940+
<para>
29941+
Disables data checksum validation and calculation for the cluster. This
29942+
will switch the data checksum mode to <literal>inprogress-off</literal>
29943+
while data checksums are being disabled. When all active backends have
29944+
stopped validating data checksums, the data checksum mode will be
29945+
changed to <literal>off</literal>. At this point the data pages will
29946+
still have checksums recorded but they are not updated when pages are
29947+
modified.
29948+
</para></entry>
29949+
</row>
29950+
</tbody>
29951+
</tgroup>
29952+
</table>
29953+
29954+
</sect2>
29955+
2988529956
<sect2 id="functions-admin-dbobject">
2988629957
<title>Database Object Management Functions</title>
2988729958

‎doc/src/sgml/glossary.sgml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,8 @@
159159
(but not the autovacuum workers),
160160
the <glossterm linkend="glossary-background-writer">background writer</glossterm>,
161161
the <glossterm linkend="glossary-checkpointer">checkpointer</glossterm>,
162+
the <glossterm linkend="glossary-data-checksums-worker">data checksums worker</glossterm>,
163+
the <glossterm linkend="glossary-data-checksums-worker-launcher">data checksums worker launcher</glossterm>,
162164
the <glossterm linkend="glossary-logger">logger</glossterm>,
163165
the <glossterm linkend="glossary-startup-process">startup process</glossterm>,
164166
the <glossterm linkend="glossary-wal-archiver">WAL archiver</glossterm>,
@@ -548,6 +550,27 @@
548550
</glossdef>
549551
</glossentry>
550552

553+
<glossentry id="glossary-data-checksums-worker">
554+
<glossterm>Data Checksums Worker</glossterm>
555+
<glossdef>
556+
<para>
557+
An <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
558+
which enables or disables data checksums in a specific database.
559+
</para>
560+
</glossdef>
561+
</glossentry>
562+
563+
<glossentry id="glossary-data-checksums-worker-launcher">
564+
<glossterm>Data Checksums Worker Launcher</glossterm>
565+
<glossdef>
566+
<para>
567+
An <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
568+
which starts <glossterm linkend="glossary-data-checksums-worker"> processes</glossterm>
569+
for each database.
570+
</para>
571+
</glossdef>
572+
</glossentry>
573+
551574
<glossentry id="glossary-db-cluster">
552575
<glossterm>Database cluster</glossterm>
553576
<glossdef>

‎doc/src/sgml/monitoring.sgml

Lines changed: 204 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3497,8 +3497,9 @@ description | Waiting for a newly initialized WAL file to reach durable storage
34973497
</para>
34983498
<para>
34993499
Number of data page checksum failures detected in this
3500-
database (or on a shared object), or NULL if data checksums are
3501-
disabled.
3500+
database (or on a shared object).
3501+
Detected failures are reported regardless of the
3502+
<xref linkend="guc-data-checksums"/> setting.
35023503
</para></entry>
35033504
</row>
35043505

@@ -3508,8 +3509,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage
35083509
</para>
35093510
<para>
35103511
Time at which the last data page checksum failure was detected in
3511-
this database (or on a shared object), or NULL if data checksums are
3512-
disabled.
3512+
this database (or on a shared object). Last failure is reported
3513+
regardless of the <xref linkend="guc-data-checksums"/> setting.
35133514
</para></entry>
35143515
</row>
35153516

@@ -6828,6 +6829,205 @@ FROM pg_stat_get_backend_idset() AS backendid;
68286829

68296830
</sect2>
68306831

6832+
<sect2 id="data-checksum-progress-reporting">
6833+
<title>Data Checksum Progress Reporting</title>
6834+
6835+
<indexterm>
6836+
<primary>pg_stat_progress_data_checksums</primary>
6837+
</indexterm>
6838+
6839+
<para>
6840+
When data checksums are being enabled on a running cluster, the
6841+
<structname>pg_stat_progress_data_checksums</structname> view will contain
6842+
a row for the launcher process, and one row for each worker process which
6843+
is currently calculating checksums for the data pages in one database.
6844+
</para>
6845+
6846+
<table id="pg-stat-progress-data-checksums-view" xreflabel="pg_stat_progress_data_checksums">
6847+
<title><structname>pg_stat_progress_data_checksums</structname> View</title>
6848+
<tgroup cols="1">
6849+
<thead>
6850+
<row>
6851+
<entry role="catalog_table_entry">
6852+
<para role="column_definition">
6853+
Column Type
6854+
</para>
6855+
<para>
6856+
Description>
6857+
</para>
6858+
</entry>
6859+
</row>
6860+
</thead>
6861+
6862+
<tbody>
6863+
<row>
6864+
<entry role="catalog_table_entry">
6865+
<para role="column_definition">
6866+
<structfield>pid</structfield> <type>integer</type>
6867+
</para>
6868+
<para>
6869+
Process ID of a datachecksumworker process.
6870+
</para>
6871+
</entry>
6872+
</row>
6873+
6874+
<row>
6875+
<entry role="catalog_table_entry"><para role="column_definition">
6876+
<structfield>datid</structfield> <type>oid</type>
6877+
</para>
6878+
<para>
6879+
OID of this database, or 0 for the launcher process
6880+
relation
6881+
</para></entry>
6882+
</row>
6883+
6884+
<row>
6885+
<entry role="catalog_table_entry"><para role="column_definition">
6886+
<structfield>datname</structfield> <type>name</type>
6887+
</para>
6888+
<para>
6889+
Name of this database, or <literal>NULL</literal> for the
6890+
launcher process.
6891+
</para></entry>
6892+
</row>
6893+
6894+
<row>
6895+
<entry role="catalog_table_entry">
6896+
<para role="column_definition">
6897+
<structfield>phase</structfield> <type>text</type>
6898+
</para>
6899+
<para>
6900+
Current processing phase, see <xref linkend="datachecksum-phases"/>
6901+
for description of the phases.
6902+
</para>
6903+
</entry>
6904+
</row>
6905+
6906+
<row>
6907+
<entry role="catalog_table_entry">
6908+
<para role="column_definition">
6909+
<structfield>databases_total</structfield> <type>integer</type>
6910+
</para>
6911+
<para>
6912+
The total number of databases which will be processed. Only the
6913+
launcher worker has this value set, the other worker processes
6914+
have this set to <literal>NULL</literal>.
6915+
</para>
6916+
</entry>
6917+
</row>
6918+
6919+
<row>
6920+
<entry role="catalog_table_entry">
6921+
<para role="column_definition">
6922+
<structfield>databases_done</structfield> <type>integer</type>
6923+
</para>
6924+
<para>
6925+
The number of databases which have been processed. Only the
6926+
launcher worker has this value set, the other worker processes
6927+
have this set to <literal>NULL</literal>.
6928+
</para>
6929+
</entry>
6930+
</row>
6931+
6932+
<row>
6933+
<entry role="catalog_table_entry">
6934+
<para role="column_definition">
6935+
<structfield>relations_total</structfield> <type>integer</type>
6936+
</para>
6937+
<para>
6938+
The total number of relations which will be processed, or
6939+
<literal>NULL</literal> if the data checksums worker process hasn't
6940+
calculated the number of relations yet. The launcher process has
6941+
this <literal>NULL</literal>.
6942+
</para>
6943+
</entry>
6944+
</row>
6945+
6946+
<row>
6947+
<entry role="catalog_table_entry">
6948+
<para role="column_definition">
6949+
<structfield>relations_done</structfield> <type>integer</type>
6950+
</para>
6951+
<para>
6952+
The number of relations which have been processed. The launcher
6953+
process has this <literal>NULL</literal>.
6954+
</para>
6955+
</entry>
6956+
</row>
6957+
6958+
<row>
6959+
<entry role="catalog_table_entry">
6960+
<para role="column_definition">
6961+
<structfield>blocks_total</structfield> <type>integer</type>
6962+
</para>
6963+
<para>
6964+
The number of blocks in the current relation which will be processed,
6965+
or <literal>NULL</literal> if the data checksums worker process hasn't
6966+
calculated the number of blocks yet. The launcher process has
6967+
this <literal>NULL</literal>.
6968+
</para>
6969+
</entry>
6970+
</row>
6971+
6972+
<row>
6973+
<entry role="catalog_table_entry">
6974+
<para role="column_definition">
6975+
<structfield>blocks_done</structfield> <type>integer</type>
6976+
</para>
6977+
<para>
6978+
The number of blocks in the current relation which have been processed.
6979+
The launcher process has this <literal>NULL</literal>.
6980+
</para>
6981+
</entry>
6982+
</row>
6983+
6984+
</tbody>
6985+
</tgroup>
6986+
</table>
6987+
6988+
<table id="datachecksum-phases">
6989+
<title>Data Checksum Phases</title>
6990+
<tgroup cols="2">
6991+
<colspec colname="col1" colwidth="1*"/>
6992+
<colspec colname="col2" colwidth="2*"/>
6993+
<thead>
6994+
<row>
6995+
<entry>Phase</entry>
6996+
<entry>Description</entry>
6997+
</row>
6998+
</thead>
6999+
<tbody>
7000+
<row>
7001+
<entry><literal>enabling</literal></entry>
7002+
<entry>
7003+
The command is currently enabling data checksums on the cluster.
7004+
</entry>
7005+
</row>
7006+
<row>
7007+
<entry><literal>disabling</literal></entry>
7008+
<entry>
7009+
The command is currently disabling data checksums on the cluster.
7010+
</entry>
7011+
</row>
7012+
<row>
7013+
<entry><literal>waiting on temporary tables</literal></entry>
7014+
<entry>
7015+
The command is currently waiting for all temporary tables which existed
7016+
at the time the command was started to be removed.
7017+
</entry>
7018+
</row>
7019+
<row>
7020+
<entry><literal>waiting on checkpoint</literal></entry>
7021+
<entry>
7022+
The command is currently waiting for a checkpoint to update the checksum
7023+
state before finishing.
7024+
</entry>
7025+
</row>
7026+
</tbody>
7027+
</tgroup>
7028+
</table>
7029+
</sect2>
7030+
68317031
</sect1>
68327032

68337033
<sect1 id="dynamic-trace">

‎doc/src/sgml/ref/pg_checksums.sgml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,12 @@ PostgreSQL documentation
4545
exit status is nonzero if the operation failed.
4646
</para>
4747

48+
<para>
49+
When enabling checksums, if checksums were in the process of being enabled
50+
when the cluster was shut down, <application>pg_checksums</application>
51+
will still process all relations regardless of the online processing.
52+
</para>
53+
4854
<para>
4955
When verifying checksums, every file in the cluster is scanned. When
5056
enabling checksums, each relation file block with a changed checksum is

0 commit comments

Comments
 (0)