Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit f98d074

Browse files
author
Amit Kapila
committed
Doc: Explain about Column List feature.
Add a new logical replication section for "Column Lists" (analogous to the Row Filters page). This explains how the feature can be used and the caveats in it. Author: Peter Smith Reviewed-by: Shi yu, Vignesh C, Erik Rijkers, Amit Kapila Backpatch-through: 15, where it was introduced Discussion: https://postgr.es/m/CAHut+PvOuc9=_4TbASc5=VUqh16UWtFO3GzcKQK_5m1hrW3vqg@mail.gmail.com
1 parent 20b6847 commit f98d074

File tree

3 files changed

+224
-11
lines changed

3 files changed

+224
-11
lines changed

‎doc/src/sgml/logical-replication.sgml

+217
Original file line numberDiff line numberDiff line change
@@ -1089,6 +1089,223 @@ test_sub=# SELECT * FROM child ORDER BY a;
10891089

10901090
</sect1>
10911091

1092+
<sect1 id="logical-replication-col-lists">
1093+
<title>Column Lists</title>
1094+
1095+
<para>
1096+
By default, all columns of a published table will be replicated to the
1097+
appropriate subscribers. The subscriber table must have at least all the
1098+
columns of the published table. However, if a
1099+
<firstterm>column list</firstterm> is specified then only the columns named
1100+
in the list will be replicated. This means the subscriber-side table only
1101+
needs to have those columns named by the column list. A user might choose to
1102+
use column lists for behavioral, security or performance reasons.
1103+
</para>
1104+
1105+
<sect2 id="logical-replication-col-list-rules">
1106+
<title>Column List Rules</title>
1107+
1108+
<para>
1109+
A column list is specified per table following the table name, and enclosed
1110+
by parentheses. See <xref linkend="sql-createpublication"/> for details.
1111+
</para>
1112+
1113+
<para>
1114+
When specifying a column list, the order of columns is not important. If no
1115+
column list is specified, all columns of the table are replicated through
1116+
this publication, including any columns added later. This means a column
1117+
list which names all columns is not quite the same as having no column list
1118+
at all. For example, if additional columns are added to the table then only
1119+
those named columns mentioned in the column list will continue to be
1120+
replicated.
1121+
</para>
1122+
1123+
<para>
1124+
Column lists have no effect for <literal>TRUNCATE</literal> command.
1125+
</para>
1126+
1127+
</sect2>
1128+
1129+
<sect2 id="logical-replication-col-list-restrictions">
1130+
<title>Column List Restrictions</title>
1131+
1132+
<para>
1133+
A column list can contain only simple column references.
1134+
</para>
1135+
1136+
<para>
1137+
If a publication publishes <command>UPDATE</command> or
1138+
<command>DELETE</command> operations, any column list must include the
1139+
table's replica identity columns (see
1140+
<xref linkend="sql-altertable-replica-identity"/>).
1141+
If a publication publishes only <command>INSERT</command> operations, then
1142+
the column list is arbitrary and may omit some replica identity columns.
1143+
</para>
1144+
1145+
</sect2>
1146+
1147+
<sect2 id="logical-replication-col-list-partitioned">
1148+
<title>Partitioned Tables</title>
1149+
1150+
<para>
1151+
For partitioned tables, the publication parameter
1152+
<literal>publish_via_partition_root</literal> determines which column list
1153+
is used. If <literal>publish_via_partition_root</literal> is
1154+
<literal>true</literal>, the root partitioned table's column list is used.
1155+
Otherwise, if <literal>publish_via_partition_root</literal> is
1156+
<literal>false</literal> (default), each partition's column list is used.
1157+
</para>
1158+
1159+
</sect2>
1160+
1161+
<sect2 id="logical-replication-col-list-initial-data-sync">
1162+
<title>Initial Data Synchronization</title>
1163+
1164+
<para>
1165+
If the subscription requires copying pre-existing table data and a
1166+
publication specifies a column list, only data from those columns will be
1167+
copied.
1168+
</para>
1169+
1170+
<note>
1171+
<para>
1172+
If the subscriber is in a release prior to 15, copy pre-existing data
1173+
doesn't use column lists even if they are defined in the publication.
1174+
This is because old releases can only copy the entire table data.
1175+
</para>
1176+
</note>
1177+
1178+
</sect2>
1179+
1180+
<sect2 id="logical-replication-col-list-combining">
1181+
<title>Combining Multiple Column Lists</title>
1182+
1183+
<warning>
1184+
<para>
1185+
It is not supported to have a subscription comprising several publications
1186+
where the same table has been published with different column lists.
1187+
This means changing the column lists of the tables being subscribed could
1188+
cause inconsistency of column lists among publications, in which case
1189+
the <xref linkend="sql-alterpublication"/> will be successful but later
1190+
the walsender on the publisher, or the subscriber may throw an error. In
1191+
this scenario, the user needs to recreate the subscription after adjusting
1192+
the column list or drop the problematic publication using
1193+
<literal>ALTER SUBSCRIPTION ... DROP PUBLICATION</literal> and then add it
1194+
back after adjusting the column list.
1195+
</para>
1196+
<para>
1197+
Background: The main purpose of the column list feature is to allow
1198+
statically different table shapes on publisher and subscriber, or hide
1199+
sensitive column data. In both cases, it doesn't seem to make sense to
1200+
combine column lists.
1201+
</para>
1202+
</warning>
1203+
1204+
</sect2>
1205+
1206+
<sect2 id="logical-replication-col-list-examples">
1207+
<title>Examples</title>
1208+
1209+
<para>
1210+
Create a table <literal>t1</literal> to be used in the following example.
1211+
<programlisting>
1212+
test_pub=# CREATE TABLE t1(id int, a text, b text, c text, d text, e text, PRIMARY KEY(id));
1213+
CREATE TABLE
1214+
</programlisting></para>
1215+
1216+
<para>
1217+
Create a publication <literal>p1</literal>. A column list is defined for
1218+
table <literal>t1</literal> to reduce the number of columns that will be
1219+
replicated. Notice that the order of column names in the column list does
1220+
not matter.
1221+
<programlisting>
1222+
test_pub=# CREATE PUBLICATION p1 FOR TABLE t1 (id, b, a, d);
1223+
CREATE PUBLICATION
1224+
</programlisting></para>
1225+
1226+
<para>
1227+
<literal>psql</literal> can be used to show the column lists (if defined)
1228+
for each publication.
1229+
<programlisting>
1230+
test_pub=# \dRp+
1231+
Publication p1
1232+
Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
1233+
----------+------------+---------+---------+---------+-----------+----------
1234+
postgres | f | t | t | t | t | f
1235+
Tables:
1236+
"public.t1" (id, a, b, d)
1237+
</programlisting></para>
1238+
1239+
<para>
1240+
<literal>psql</literal> can be used to show the column lists (if defined)
1241+
for each table.
1242+
<programlisting>
1243+
test_pub=# \d t1
1244+
Table "public.t1"
1245+
Column | Type | Collation | Nullable | Default
1246+
--------+---------+-----------+----------+---------
1247+
id | integer | | not null |
1248+
a | text | | |
1249+
b | text | | |
1250+
c | text | | |
1251+
d | text | | |
1252+
e | text | | |
1253+
Indexes:
1254+
"t1_pkey" PRIMARY KEY, btree (id)
1255+
Publications:
1256+
"p1" (id, a, b, d)
1257+
</programlisting></para>
1258+
1259+
<para>
1260+
On the subscriber node, create a table <literal>t1</literal> which now
1261+
only needs a subset of the columns that were on the publisher table
1262+
<literal>t1</literal>, and also create the subscription
1263+
<literal>s1</literal> that subscribes to the publication
1264+
<literal>p1</literal>.
1265+
<programlisting>
1266+
test_sub=# CREATE TABLE t1(id int, b text, a text, d text, PRIMARY KEY(id));
1267+
CREATE TABLE
1268+
test_sub=# CREATE SUBSCRIPTION s1
1269+
test_sub-# CONNECTION 'host=localhost dbname=test_pub application_name=s1'
1270+
test_sub-# PUBLICATION p1;
1271+
CREATE SUBSCRIPTION
1272+
</programlisting></para>
1273+
1274+
<para>
1275+
On the publisher node, insert some rows to table <literal>t1</literal>.
1276+
<programlisting>
1277+
test_pub=# INSERT INTO t1 VALUES(1, 'a-1', 'b-1', 'c-1', 'd-1', 'e-1');
1278+
INSERT 0 1
1279+
test_pub=# INSERT INTO t1 VALUES(2, 'a-2', 'b-2', 'c-2', 'd-2', 'e-2');
1280+
INSERT 0 1
1281+
test_pub=# INSERT INTO t1 VALUES(3, 'a-3', 'b-3', 'c-3', 'd-3', 'e-3');
1282+
INSERT 0 1
1283+
test_pub=# SELECT * FROM t1 ORDER BY id;
1284+
id | a | b | c | d | e
1285+
----+-----+-----+-----+-----+-----
1286+
1 | a-1 | b-1 | c-1 | d-1 | e-1
1287+
2 | a-2 | b-2 | c-2 | d-2 | e-2
1288+
3 | a-3 | b-3 | c-3 | d-3 | e-3
1289+
(3 rows)
1290+
</programlisting></para>
1291+
1292+
<para>
1293+
Only data from the column list of publication <literal>p1</literal> is
1294+
replicated.
1295+
<programlisting>
1296+
test_sub=# SELECT * FROM t1 ORDER BY id;
1297+
id | b | a | d
1298+
----+-----+-----+-----
1299+
1 | b-1 | a-1 | d-1
1300+
2 | b-2 | a-2 | d-2
1301+
3 | b-3 | a-3 | d-3
1302+
(3 rows)
1303+
</programlisting></para>
1304+
1305+
</sect2>
1306+
1307+
</sect1>
1308+
10921309
<sect1 id="logical-replication-conflicts">
10931310
<title>Conflicts</title>
10941311

‎doc/src/sgml/ref/alter_publication.sgml

+3-9
Original file line numberDiff line numberDiff line change
@@ -118,15 +118,9 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r
118118
Optionally, a column list can be specified. See <xref
119119
linkend="sql-createpublication"/> for details. Note that a subscription
120120
having several publications in which the same table has been published
121-
with different column lists is not supported. So, changing the column
122-
lists of the tables being subscribed could cause inconsistency of column
123-
lists among publications, in which case <command>ALTER PUBLICATION</command>
124-
will be successful but later the walsender on the publisher or the
125-
subscriber may throw an error. In this scenario, the user needs to
126-
recreate the subscription after adjusting the column list or drop the
127-
problematic publication using
128-
<literal>ALTER SUBSCRIPTION ... DROP PUBLICATION</literal> and then add
129-
it back after adjusting the column list.
121+
with different column lists is not supported. See
122+
<xref linkend="logical-replication-col-list-combining"/> for details of
123+
potential problems when altering column lists.
130124
</para>
131125

132126
<para>

‎doc/src/sgml/ref/create_publication.sgml

+4-2
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
9191
When a column list is specified, only the named columns are replicated.
9292
If no column list is specified, all columns of the table are replicated
9393
through this publication, including any columns added later. It has no
94-
effect on <literal>TRUNCATE</literal> commands.
95-
</para>
94+
effect on <literal>TRUNCATE</literal> commands. See
95+
<xref linkend="logical-replication-col-lists"/> for details about column
96+
lists.
97+
</para>
9698

9799
<para>
98100
Only persistent base tables and partitioned tables can be part of a

0 commit comments

Comments
 (0)