Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 1bf1140

Browse files
author
Amit Kapila
committed
Change the default value of the streaming option to 'parallel'.
Previously the default value of streaming option for a subscription was 'off'. The parallel option indicates that the changes in large transactions (greater than logical_decoding_work_mem) are to be applied directly via one of the parallel apply workers, if available. The parallel mode was introduced in 16, but we refrain from enabling it by default to avoid seeing any unpleasant behavior in the existing applications. However we haven't found any such report yet, so this is a good time to enable it by default. Reported-by: Vignesh C Author: Hayato Kuroda, Masahiko Sawada, Peter Smith, Amit Kapila Discussion: https://postgr.es/m/CALDaNm1=MedhW23NuoePJTmonwsMSp80ddsw+sEJs0GUMC_kqQ@mail.gmail.com
1 parent 6b652e6 commit 1bf1140

File tree

6 files changed

+39
-30
lines changed

6 files changed

+39
-30
lines changed

‎doc/src/sgml/ref/create_subscription.sgml

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -271,25 +271,32 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
271271
<listitem>
272272
<para>
273273
Specifies whether to enable streaming of in-progress transactions
274-
for this subscription. The default value is <literal>off</literal>,
275-
meaning all transactions are fully decoded on the publisher and only
276-
then sent to the subscriber as a whole.
274+
for this subscription. The default value is <literal>parallel</literal>,
275+
meaning incoming changes are directly applied via one of the parallel
276+
apply workers, if available. If no parallel apply worker is free to
277+
handle streaming transactions then the changes are written to
278+
temporary files and applied after the transaction is committed. Note
279+
that if an error happens in a parallel apply worker, the finish LSN
280+
of the remote transaction might not be reported in the server log.
277281
</para>
278282

283+
<caution>
284+
<para>
285+
There is a risk of deadlock when the schemas of the publisher and
286+
subscriber differ, although such cases are rare. The apply worker
287+
is equipped to retry these transactions automatically.
288+
</para>
289+
</caution>
290+
279291
<para>
280292
If set to <literal>on</literal>, the incoming changes are written to
281293
temporary files and then applied only after the transaction is
282294
committed on the publisher and received by the subscriber.
283295
</para>
284296

285297
<para>
286-
If set to <literal>parallel</literal>, incoming changes are directly
287-
applied via one of the parallel apply workers, if available. If no
288-
parallel apply worker is free to handle streaming transactions then
289-
the changes are written to temporary files and applied after the
290-
transaction is committed. Note that if an error happens in a
291-
parallel apply worker, the finish LSN of the remote transaction
292-
might not be reported in the server log.
298+
If set to <literal>off</literal>, all transactions are fully decoded
299+
on the publisher and only then sent to the subscriber as a whole.
293300
</para>
294301
</listitem>
295302
</varlistentry>

‎src/backend/commands/subscriptioncmds.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
151151
if (IsSet(supported_opts, SUBOPT_BINARY))
152152
opts->binary = false;
153153
if (IsSet(supported_opts, SUBOPT_STREAMING))
154-
opts->streaming = LOGICALREP_STREAM_OFF;
154+
opts->streaming = LOGICALREP_STREAM_PARALLEL;
155155
if (IsSet(supported_opts, SUBOPT_TWOPHASE_COMMIT))
156156
opts->twophase = false;
157157
if (IsSet(supported_opts, SUBOPT_DISABLE_ON_ERR))

‎src/bin/pg_dump/pg_dump.c

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5235,6 +5235,8 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
52355235
appendPQExpBufferStr(query, ", streaming = on");
52365236
else if (strcmp(subinfo->substream, "p") == 0)
52375237
appendPQExpBufferStr(query, ", streaming = parallel");
5238+
else
5239+
appendPQExpBufferStr(query, ", streaming = off");
52385240

52395241
if (strcmp(subinfo->subtwophasestate, two_phase_disabled) != 0)
52405242
appendPQExpBufferStr(query, ", two_phase = on");

‎src/bin/pg_dump/t/002_pg_dump.pl

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2992,7 +2992,7 @@
29922992
CONNECTION \'dbname=doesnotexist\' PUBLICATION pub1
29932993
WITH (connect = false);',
29942994
regexp => qr/^
2995-
\QCREATE SUBSCRIPTION sub1 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub1');\E
2995+
\QCREATE SUBSCRIPTION sub1 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub1', streaming = parallel);\E
29962996
/xm,
29972997
like => { %full_runs, section_post_data => 1, },
29982998
},
@@ -3001,9 +3001,9 @@
30013001
create_order => 50,
30023002
create_sql => 'CREATE SUBSCRIPTION sub2
30033003
CONNECTION \'dbname=doesnotexist\' PUBLICATION pub1
3004-
WITH (connect = false, origin = none);',
3004+
WITH (connect = false, origin = none, streaming = off);',
30053005
regexp => qr/^
3006-
\QCREATE SUBSCRIPTION sub2 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub2', origin = none);\E
3006+
\QCREATE SUBSCRIPTION sub2 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub2', streaming = off, origin = none);\E
30073007
/xm,
30083008
like => { %full_runs, section_post_data => 1, },
30093009
},
@@ -3012,9 +3012,9 @@
30123012
create_order => 50,
30133013
create_sql => 'CREATE SUBSCRIPTION sub3
30143014
CONNECTION \'dbname=doesnotexist\' PUBLICATION pub1
3015-
WITH (connect = false, origin = any);',
3015+
WITH (connect = false, origin = any, streaming = on);',
30163016
regexp => qr/^
3017-
\QCREATE SUBSCRIPTION sub3 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub3');\E
3017+
\QCREATE SUBSCRIPTION sub3 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub3', streaming = on);\E
30183018
/xm,
30193019
like => { %full_runs, section_post_data => 1, },
30203020
},

‎src/test/regress/expected/subscription.out

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -119,15 +119,15 @@ HINT: To initiate replication, you must manually create the replication slot, e
119119
List of subscriptions
120120
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
121121
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
122-
regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
122+
regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
123123
(1 row)
124124

125125
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
126126
\dRs+ regress_testsub4
127127
List of subscriptions
128128
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
129129
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
130-
regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
130+
regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
131131
(1 row)
132132

133133
DROP SUBSCRIPTION regress_testsub3;
@@ -148,7 +148,7 @@ ERROR: invalid connection string syntax: missing "=" after "foobar" in connecti
148148
List of subscriptions
149149
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
150150
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
151-
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
151+
regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
152152
(1 row)
153153

154154
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -160,7 +160,7 @@ ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
160160
List of subscriptions
161161
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
162162
-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
163-
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
163+
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
164164
(1 row)
165165

166166
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -179,7 +179,7 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
179179
List of subscriptions
180180
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
181181
-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
182-
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
182+
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
183183
(1 row)
184184

185185
-- ok - with lsn = NONE
@@ -191,7 +191,7 @@ ERROR: invalid WAL location (LSN): 0/0
191191
List of subscriptions
192192
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
193193
-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
194-
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
194+
regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
195195
(1 row)
196196

197197
BEGIN;
@@ -226,7 +226,7 @@ HINT: Available values: local, remote_write, remote_apply, on, off.
226226
List of subscriptions
227227
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
228228
---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
229-
regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
229+
regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
230230
(1 row)
231231

232232
-- rename back to keep the rest simple
@@ -258,7 +258,7 @@ HINT: To initiate replication, you must manually create the replication slot, e
258258
List of subscriptions
259259
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
260260
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
261-
regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
261+
regress_testsub | regress_subscription_user | f | {testpub} | t | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
262262
(1 row)
263263

264264
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
@@ -267,7 +267,7 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
267267
List of subscriptions
268268
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
269269
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
270-
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
270+
regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
271271
(1 row)
272272

273273
DROP SUBSCRIPTION regress_testsub;
@@ -374,7 +374,7 @@ HINT: To initiate replication, you must manually create the replication slot, e
374374
List of subscriptions
375375
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
376376
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
377-
regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
377+
regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
378378
(1 row)
379379

380380
-- we can alter streaming when two_phase enabled
@@ -412,15 +412,15 @@ HINT: To initiate replication, you must manually create the replication slot, e
412412
List of subscriptions
413413
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
414414
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
415-
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
415+
regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
416416
(1 row)
417417

418418
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
419419
\dRs+
420420
List of subscriptions
421421
Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
422422
-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
423-
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
423+
regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
424424
(1 row)
425425

426426
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);

0 commit comments

Comments
 (0)