Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 5dc92b8

Browse files
peterepetere
committed
REINDEX CONCURRENTLY
This adds the CONCURRENTLY option to the REINDEX command. A REINDEX CONCURRENTLY on a specific index creates a new index (like CREATE INDEX CONCURRENTLY), then renames the old index away and the new index in place and adjusts the dependencies, and then drops the old index (like DROP INDEX CONCURRENTLY). The REINDEX command also has the capability to run its other variants (TABLE, DATABASE) with the CONCURRENTLY option (but not SYSTEM). The reindexdb command gets the --concurrently option. Author: Michael Paquier, Andreas Karlsson, Peter Eisentraut Reviewed-by: Andres Freund, Fujii Masao, Jim Nasby, Sergei Kornilov Discussion: https://www.postgresql.org/message-id/flat/60052986-956b-4478-45ed-8bd119e9b9cf%402ndquadrant.com#74948a1044c56c5e817a5050f554ddee
1 parent d25f519 commit 5dc92b8

26 files changed

+2048
-183
lines changed

‎doc/src/sgml/mvcc.sgml

+1
Original file line numberDiff line numberDiff line change
@@ -926,6 +926,7 @@ ERROR: could not serialize access due to read/write dependencies among transact
926926
<para>
927927
Acquired by <command>VACUUM</command> (without <option>FULL</option>),
928928
<command>ANALYZE</command>, <command>CREATE INDEX CONCURRENTLY</command>,
929+
<command>REINDEX CONCURRENTLY</command>,
929930
<command>CREATE STATISTICS</command>, and certain <command>ALTER
930931
INDEX</command> and <command>ALTER TABLE</command> variants (for full
931932
details see <xref linkend="sql-alterindex"/> and <xref

‎doc/src/sgml/ref/create_index.sgml

+1
Original file line numberDiff line numberDiff line change
@@ -844,6 +844,7 @@ CREATE INDEX CONCURRENTLY sales_quantity_index ON sales_table (quantity);
844844
<simplelist type="inline">
845845
<member><xref linkend="sql-alterindex"/></member>
846846
<member><xref linkend="sql-dropindex"/></member>
847+
<member><xref linkend="sql-reindex"/></member>
847848
</simplelist>
848849
</refsect1>
849850
</refentry>

‎doc/src/sgml/ref/reindex.sgml

+188-2
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ PostgreSQL documentation
2121

2222
<refsynopsisdiv>
2323
<synopsis>
24-
REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } <replaceable class="parameter">name</replaceable>
24+
REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } [ CONCURRENTLY ] <replaceable class="parameter">name</replaceable>
2525
</synopsis>
2626
</refsynopsisdiv>
2727

@@ -68,7 +68,7 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } <replacea
6868
An index build with the <literal>CONCURRENTLY</literal> option failed, leaving
6969
an <quote>invalid</quote> index. Such indexes are useless but it can be
7070
convenient to use <command>REINDEX</command> to rebuild them. Note that
71-
<command>REINDEX</command> will not perform a concurrent build. To build the
71+
<command>REINDEX</command> will not perform a concurrent build on an invalid index. To build the
7272
index without interfering with production you should drop the index and
7373
reissue the <command>CREATE INDEX CONCURRENTLY</command> command.
7474
</para>
@@ -151,6 +151,21 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } <replacea
151151
</listitem>
152152
</varlistentry>
153153

154+
<varlistentry>
155+
<term><literal>CONCURRENTLY</literal></term>
156+
<listitem>
157+
<para>
158+
When this option is used, <productname>PostgreSQL</productname> will rebuild the
159+
index without taking any locks that prevent concurrent inserts,
160+
updates, or deletes on the table; whereas a standard reindex build
161+
locks out writes (but not reads) on the table until it's done.
162+
There are several caveats to be aware of when using this option
163+
&mdash; see <xref linkend="sql-reindex-concurrently"
164+
endterm="sql-reindex-concurrently-title"/>.
165+
</para>
166+
</listitem>
167+
</varlistentry>
168+
154169
<varlistentry>
155170
<term><literal>VERBOSE</literal></term>
156171
<listitem>
@@ -241,6 +256,159 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } <replacea
241256
Each individual partition can be reindexed separately instead.
242257
</para>
243258

259+
<refsect2 id="sql-reindex-concurrently">
260+
<title id="sql-reindex-concurrently-title">Rebuilding Indexes Concurrently</title>
261+
262+
<indexterm zone="sql-reindex-concurrently">
263+
<primary>index</primary>
264+
<secondary>rebuilding concurrently</secondary>
265+
</indexterm>
266+
267+
<para>
268+
Rebuilding an index can interfere with regular operation of a database.
269+
Normally <productname>PostgreSQL</productname> locks the table whose index is rebuilt
270+
against writes and performs the entire index build with a single scan of the
271+
table. Other transactions can still read the table, but if they try to
272+
insert, update, or delete rows in the table they will block until the
273+
index rebuild is finished. This could have a severe effect if the system is
274+
a live production database. Very large tables can take many hours to be
275+
indexed, and even for smaller tables, an index rebuild can lock out writers
276+
for periods that are unacceptably long for a production system.
277+
</para>
278+
279+
<para>
280+
<productname>PostgreSQL</productname> supports rebuilding indexes with minimum locking
281+
of writes. This method is invoked by specifying the
282+
<literal>CONCURRENTLY</literal> option of <command>REINDEX</command>. When this option
283+
is used, <productname>PostgreSQL</productname> must perform two scans of the table
284+
for each index that needs to be rebuild and in addition it must wait for
285+
all existing transactions that could potentially use the index to
286+
terminate. This method requires more total work than a standard index
287+
rebuild and takes significantly longer to complete as it needs to wait
288+
for unfinished transactions that might modify the index. However, since
289+
it allows normal operations to continue while the index is rebuilt, this
290+
method is useful for rebuilding indexes in a production environment. Of
291+
course, the extra CPU, memory and I/O load imposed by the index rebuild
292+
may slow down other operations.
293+
</para>
294+
295+
<para>
296+
The following steps occur in a concurrent reindex. Each step is run in a
297+
separate transaction. If there are multiple indexes to be rebuilt, then
298+
each step loops through all the indexes before moving to the next step.
299+
300+
<orderedlist>
301+
<listitem>
302+
<para>
303+
A new temporary index definition is added into the catalog
304+
<literal>pg_index</literal>. This definition will be used to replace
305+
the old index. A <literal>SHARE UPDATE EXCLUSIVE</literal> lock at
306+
session level is taken on the indexes being reindexed as well as its
307+
associated table to prevent any schema modification while processing.
308+
</para>
309+
</listitem>
310+
311+
<listitem>
312+
<para>
313+
A first pass to build the index is done for each new index. Once the
314+
index is built, its flag <literal>pg_index.indisready</literal> is
315+
switched to <quote>true</quote> to make ready for inserts, making it
316+
visible to other sessions once the transaction that performed the build
317+
is finished. This step is done in a separate transaction for each
318+
index.
319+
</para>
320+
</listitem>
321+
322+
<listitem>
323+
<para>
324+
Then a second pass is performed to add tuples that were added while the
325+
first pass build was running. This step is also done in a separate
326+
transaction for each index.
327+
</para>
328+
</listitem>
329+
330+
<listitem>
331+
<para>
332+
All the constraints that refer to the index are changed to refer to the
333+
new index definition, and the names of the indexes are changed. At
334+
this point <literal>pg_index.indisvalid</literal> is switched to
335+
<quote>true</quote> for the new index and to <quote>false</quote> for
336+
the old, and a cache invalidation is done so as all the sessions that
337+
referenced the old index are invalidated.
338+
</para>
339+
</listitem>
340+
341+
<listitem>
342+
<para>
343+
The old indexes have <literal>pg_index.indisready</literal> switched to
344+
<quote>false</quote> to prevent any new tuple insertions, after waiting
345+
for running queries that might reference the old index to complete.
346+
</para>
347+
</listitem>
348+
349+
<listitem>
350+
<para>
351+
The old indexes are dropped. The <literal>SHARE UPDATE
352+
EXCLUSIVE</literal> session locks for the indexes and the table ar
353+
released.
354+
</para>
355+
</listitem>
356+
</orderedlist>
357+
</para>
358+
359+
<para>
360+
If a problem arises while rebuilding the indexes, such as a
361+
uniqueness violation in a unique index, the <command>REINDEX</command>
362+
command will fail but leave behind an <quote>invalid</quote> new index on top
363+
of the existing one. This index will be ignored for querying purposes
364+
because it might be incomplete; however it will still consume update
365+
overhead. The <application>psql</application> <command>\d</command> command will report
366+
such an index as <literal>INVALID</literal>:
367+
368+
<programlisting>
369+
postgres=# \d tab
370+
Table "public.tab"
371+
Column | Type | Modifiers
372+
--------+---------+-----------
373+
col | integer |
374+
Indexes:
375+
"idx" btree (col)
376+
"idx_ccnew" btree (col) INVALID
377+
</programlisting>
378+
379+
The recommended recovery method in such cases is to drop the invalid index
380+
and try again to perform <command>REINDEX CONCURRENTLY</command>. The
381+
concurrent index created during the processing has a name ending in the
382+
suffix <literal>ccnew</literal>, or <literal>ccold</literal> if it is an
383+
old index definition which we failed to drop. Invalid indexes can be
384+
dropped using <literal>DROP INDEX</literal>, including invalid toast
385+
indexes.
386+
</para>
387+
388+
<para>
389+
Regular index builds permit other regular index builds on the same table
390+
to occur in parallel, but only one concurrent index build can occur on a
391+
table at a time. In both cases, no other types of schema modification on
392+
the table are allowed meanwhile. Another difference is that a regular
393+
<command>REINDEX TABLE</command> or <command>REINDEX INDEX</command>
394+
command can be performed within a transaction block, but <command>REINDEX
395+
CONCURRENTLY</command> cannot.
396+
</para>
397+
398+
<para>
399+
<command>REINDEX SYSTEM</command> does not support
400+
<command>CONCURRENTLY</command> since system catalogs cannot be reindexed
401+
concurrently.
402+
</para>
403+
404+
<para>
405+
Furthermore, indexes for exclusion constraints cannot be reindexed
406+
concurrently. If such an index is named directly in this command, an
407+
error is raised. If a table or database with exclusion constraint indexes
408+
is reindexed concurrently, those indexes will be skipped. (It is possible
409+
to reindex such indexes without the concurrently option.)
410+
</para>
411+
</refsect2>
244412
</refsect1>
245413

246414
<refsect1>
@@ -272,6 +440,14 @@ $ <userinput>psql broken_db</userinput>
272440
...
273441
broken_db=&gt; REINDEX DATABASE broken_db;
274442
broken_db=&gt; \q
443+
</programlisting></para>
444+
445+
<para>
446+
Rebuild a table while authorizing read and write operations on involved
447+
relations when performed:
448+
449+
<programlisting>
450+
REINDEX TABLE CONCURRENTLY my_broken_table;
275451
</programlisting></para>
276452
</refsect1>
277453

@@ -282,4 +458,14 @@ broken_db=&gt; \q
282458
There is no <command>REINDEX</command> command in the SQL standard.
283459
</para>
284460
</refsect1>
461+
462+
<refsect1>
463+
<title>See Also</title>
464+
465+
<simplelist type="inline">
466+
<member><xref linkend="sql-createindex"/></member>
467+
<member><xref linkend="sql-dropindex"/></member>
468+
<member><xref linkend="app-reindexdb"/></member>
469+
</simplelist>
470+
</refsect1>
285471
</refentry>

‎doc/src/sgml/ref/reindexdb.sgml

+10
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,16 @@ PostgreSQL documentation
118118
</listitem>
119119
</varlistentry>
120120

121+
<varlistentry>
122+
<term><option>--concurrently</option></term>
123+
<listitem>
124+
<para>
125+
Use the <literal>CONCURRENTLY</literal> option. See <xref
126+
linkend="sql-reindex"/> for further information.
127+
</para>
128+
</listitem>
129+
</varlistentry>
130+
121131
<varlistentry>
122132
<term><option><optional>-d</optional> <replaceable class="parameter">dbname</replaceable></option></term>
123133
<term><option><optional>--dbname=</optional><replaceable class="parameter">dbname</replaceable></option></term>

‎src/backend/catalog/dependency.c

+6-1
Original file line numberDiff line numberDiff line change
@@ -306,6 +306,10 @@ deleteObjectsInList(ObjectAddresses *targetObjects, Relation *depRel,
306306
* PERFORM_DELETION_SKIP_EXTENSIONS: do not delete extensions, even when
307307
* deleting objects that are part of an extension. This should generally
308308
* be used only when dropping temporary objects.
309+
*
310+
* PERFORM_DELETION_CONCURRENT_LOCK: perform the drop normally but with a lock
311+
* as if it were concurrent. This is used by REINDEX CONCURRENTLY.
312+
*
309313
*/
310314
void
311315
performDeletion(const ObjectAddress *object,
@@ -1316,9 +1320,10 @@ doDeletion(const ObjectAddress *object, int flags)
13161320
relKind == RELKIND_PARTITIONED_INDEX)
13171321
{
13181322
bool concurrent = ((flags & PERFORM_DELETION_CONCURRENTLY) != 0);
1323+
bool concurrent_lock_mode = ((flags & PERFORM_DELETION_CONCURRENT_LOCK) != 0);
13191324

13201325
Assert(object->objectSubId == 0);
1321-
index_drop(object->objectId, concurrent);
1326+
index_drop(object->objectId, concurrent, concurrent_lock_mode);
13221327
}
13231328
else
13241329
{

0 commit comments

Comments
 (0)