Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 1ce3451

Browse files
author
Amit Kapila
committed
Doc: clarify that tablesync ignores publish operation.
This patch documents that the initial data synchronization (tablesync) for logical replication does not take into account the publication 'publish' parameter when copying the existing table data. Author: Peter Smith Reviewed-by: Shi yu, Euler Taveira, Robert Haas, Amit Kapila Discussion: https://postgr.es/m/CAHut+PtbfALjFpS2MkrvQ+wWQKByP7CNh9RtFta-r=BHEU3S3w@mail.gmail.com
1 parent 75f2171 commit 1ce3451

File tree

3 files changed

+225
-3
lines changed

3 files changed

+225
-3
lines changed

‎doc/src/sgml/logical-replication.sgml

Lines changed: 215 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -117,9 +117,10 @@
117117
Publications can choose to limit the changes they produce to
118118
any combination of <command>INSERT</command>, <command>UPDATE</command>,
119119
<command>DELETE</command>, and <command>TRUNCATE</command>, similar to how triggers are fired by
120-
particular event types. By default, all operation types are replicated.
121-
(Row filters have no effect for <command>TRUNCATE</command>. See
122-
<xref linkend="logical-replication-row-filter"/>).
120+
particular event types. By default, all operation types are replicated.
121+
These publication specifications apply only for DML operations; they do not affect the initial
122+
data synchronization copy. (Row filters have no effect for
123+
<command>TRUNCATE</command>. See <xref linkend="logical-replication-row-filter"/>).
123124
</para>
124125

125126
<para>
@@ -317,6 +318,200 @@
317318
</itemizedlist>
318319
</para>
319320
</sect2>
321+
322+
<sect2 id="logical-replication-subscription-examples">
323+
<title>Examples</title>
324+
325+
<para>
326+
Create some test tables on the publisher.
327+
<programlisting>
328+
test_pub=# CREATE TABLE t1(a int, b text, PRIMARY KEY(a));
329+
CREATE TABLE
330+
test_pub=# CREATE TABLE t2(c int, d text, PRIMARY KEY(c));
331+
CREATE TABLE
332+
test_pub=# CREATE TABLE t3(e int, f text, PRIMARY KEY(e));
333+
CREATE TABLE
334+
</programlisting></para>
335+
336+
<para>
337+
Create the same tables on the subscriber.
338+
<programlisting>
339+
test_sub=# CREATE TABLE t1(a int, b text, PRIMARY KEY(a));
340+
CREATE TABLE
341+
test_sub=# CREATE TABLE t2(c int, d text, PRIMARY KEY(c));
342+
CREATE TABLE
343+
test_sub=# CREATE TABLE t3(e int, f text, PRIMARY KEY(e));
344+
CREATE TABLE
345+
</programlisting></para>
346+
347+
<para>
348+
Insert data to the tables at the publisher side.
349+
<programlisting>
350+
test_pub=# INSERT INTO t1 VALUES (1, 'one'), (2, 'two'), (3, 'three');
351+
INSERT 0 3
352+
test_pub=# INSERT INTO t2 VALUES (1, 'A'), (2, 'B'), (3, 'C');
353+
INSERT 0 3
354+
test_pub=# INSERT INTO t3 VALUES (1, 'i'), (2, 'ii'), (3, 'iii');
355+
INSERT 0 3
356+
</programlisting></para>
357+
358+
<para>
359+
Create publications for the tables. The publications <literal>pub2</literal>
360+
and <literal>pub3a</literal> disallow some <literal>publish</literal>
361+
operations. The publication <literal>pub3b</literal> has a row filter (see
362+
<xref linkend="logical-replication-row-filter"/>).
363+
<programlisting>
364+
test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1;
365+
CREATE PUBLICATION
366+
test_pub=# CREATE PUBLICATION pub2 FOR TABLE t2 WITH (publish = 'truncate');
367+
CREATE PUBLICATION
368+
test_pub=# CREATE PUBLICATION pub3a FOR TABLE t3 WITH (publish = 'truncate');
369+
CREATE PUBLICATION
370+
test_pub=# CREATE PUBLICATION pub3b FOR TABLE t3 WHERE (e > 5);
371+
CREATE PUBLICATION
372+
</programlisting></para>
373+
374+
<para>
375+
Create subscriptions for the publications. The subscription
376+
<literal>sub3</literal> subscribes to both <literal>pub3a</literal> and
377+
<literal>pub3b</literal>. All subscriptions will copy initial data by default.
378+
<programlisting>
379+
test_sub=# CREATE SUBSCRIPTION sub1
380+
test_sub-# CONNECTION 'host=localhost dbname=test_pub application_name=sub1'
381+
test_sub-# PUBLICATION pub1;
382+
CREATE SUBSCRIPTION
383+
test_sub=# CREATE SUBSCRIPTION sub2
384+
test_sub-# CONNECTION 'host=localhost dbname=test_pub application_name=sub2'
385+
test_sub-# PUBLICATION pub2;
386+
CREATE SUBSCRIPTION
387+
test_sub=# CREATE SUBSCRIPTION sub3
388+
test_sub-# CONNECTION 'host=localhost dbname=test_pub application_name=sub3'
389+
test_sub-# PUBLICATION pub3a, pub3b;
390+
CREATE SUBSCRIPTION
391+
</programlisting></para>
392+
393+
<para>
394+
Observe that initial table data is copied, regardless of the
395+
<literal>publish</literal> operation of the publication.
396+
<programlisting>
397+
test_sub=# SELECT * FROM t1;
398+
a | b
399+
---+-------
400+
1 | one
401+
2 | two
402+
3 | three
403+
(3 rows)
404+
405+
test_sub=# SELECT * FROM t2;
406+
c | d
407+
---+---
408+
1 | A
409+
2 | B
410+
3 | C
411+
(3 rows)
412+
</programlisting></para>
413+
414+
<para>
415+
Furthermore, because the initial data copy ignores the <literal>publish</literal>
416+
operation, and because publication <literal>pub3a</literal> has no row filter,
417+
it means the copied table <literal>t3</literal> contains all rows even when
418+
they do not match the row filter of publication <literal>pub3b</literal>.
419+
<programlisting>
420+
test_sub=# SELECT * FROM t3;
421+
e | f
422+
---+-----
423+
1 | i
424+
2 | ii
425+
3 | iii
426+
(3 rows)
427+
</programlisting></para>
428+
429+
<para>
430+
Insert more data to the tables at the publisher side.
431+
<programlisting>
432+
test_pub=# INSERT INTO t1 VALUES (4, 'four'), (5, 'five'), (6, 'six');
433+
INSERT 0 3
434+
test_pub=# INSERT INTO t2 VALUES (4, 'D'), (5, 'E'), (6, 'F');
435+
INSERT 0 3
436+
test_pub=# INSERT INTO t3 VALUES (4, 'iv'), (5, 'v'), (6, 'vi');
437+
INSERT 0 3
438+
</programlisting></para>
439+
440+
<para>
441+
Now the publisher side data looks like:
442+
<programlisting>
443+
test_pub=# SELECT * FROM t1;
444+
a | b
445+
---+-------
446+
1 | one
447+
2 | two
448+
3 | three
449+
4 | four
450+
5 | five
451+
6 | six
452+
(6 rows)
453+
454+
test_pub=# SELECT * FROM t2;
455+
c | d
456+
---+---
457+
1 | A
458+
2 | B
459+
3 | C
460+
4 | D
461+
5 | E
462+
6 | F
463+
(6 rows)
464+
465+
test_pub=# SELECT * FROM t3;
466+
e | f
467+
---+-----
468+
1 | i
469+
2 | ii
470+
3 | iii
471+
4 | iv
472+
5 | v
473+
6 | vi
474+
(6 rows)
475+
</programlisting></para>
476+
477+
<para>
478+
Observe that during normal replication the appropriate
479+
<literal>publish</literal> operations are used. This means publications
480+
<literal>pub2</literal> and <literal>pub3a</literal> will not replicate the
481+
<literal>INSERT</literal>. Also, publication <literal>pub3b</literal> will
482+
only replicate data that matches the row filter of <literal>pub3b</literal>.
483+
Now the subscriber side data looks like:
484+
<programlisting>
485+
test_sub=# SELECT * FROM t1;
486+
a | b
487+
---+-------
488+
1 | one
489+
2 | two
490+
3 | three
491+
4 | four
492+
5 | five
493+
6 | six
494+
(6 rows)
495+
496+
test_sub=# SELECT * FROM t2;
497+
c | d
498+
---+---
499+
1 | A
500+
2 | B
501+
3 | C
502+
(3 rows)
503+
504+
test_sub=# SELECT * FROM t3;
505+
e | f
506+
---+-----
507+
1 | i
508+
2 | ii
509+
3 | iii
510+
6 | vi
511+
(4 rows)
512+
</programlisting></para>
513+
</sect2>
514+
320515
</sect1>
321516

322517
<sect1 id="logical-replication-row-filter">
@@ -461,6 +656,16 @@
461656
<xref linkend="logical-replication-row-filter-combining"/> for details.
462657
</para>
463658

659+
<warning>
660+
<para>
661+
Because initial data synchronization does not take into account the
662+
<literal>publish</literal> parameter when copying existing table data,
663+
some rows may be copied that would not be replicated using DML. Refer to
664+
<xref linkend="logical-replication-snapshot"/>, and see
665+
<xref linkend="logical-replication-subscription-examples"/> for examples.
666+
</para>
667+
</warning>
668+
464669
<note>
465670
<para>
466671
If the subscriber is in a release prior to 15, copy pre-existing data
@@ -1095,6 +1300,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
10951300
replication of the table is given back to the main apply process where
10961301
replication continues as normal.
10971302
</para>
1303+
<note>
1304+
<para>
1305+
The publication <literal>publish</literal> parameter only affects what
1306+
DML operations will be replicated. The initial data synchronization does
1307+
not take this parameter into account when copying the existing table data.
1308+
</para>
1309+
</note>
10981310
</sect2>
10991311
</sect1>
11001312

‎doc/src/sgml/ref/create_publication.sgml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,12 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
177177
and so the default value for this option is
178178
<literal>'insert, update, delete, truncate'</literal>.
179179
</para>
180+
<para>
181+
This parameter only affects DML operations. In particular, the initial
182+
data synchronization (see <xref linkend="logical-replication-snapshot"/>)
183+
for logical replication does not take this parameter into account when
184+
copying existing table data.
185+
</para>
180186
</listitem>
181187
</varlistentry>
182188

‎doc/src/sgml/ref/create_subscription.sgml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -353,6 +353,10 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
353353
15 then any row filtering is ignored during the initial data synchronization
354354
phase. For this case, the user might want to consider deleting any initially
355355
copied data that would be incompatible with subsequent filtering.
356+
Because initial data synchronization does not take into account the publication
357+
<literal>publish</literal> parameter when copying existing table data, some rows
358+
may be copied that would not be replicated using DML. See
359+
<xref linkend="logical-replication-subscription-examples"/> for examples.
356360
</para>
357361

358362
<para>

0 commit comments

Comments
 (0)