Ignore invalid indexes when enforcing index rules in ALTER TABLE ATTACH PARTITION

A portion of ALTER TABLE .. ATTACH PARTITION is to ensure that the
partition being attached to the partitioned table has a correct set of
indexes, so as there is a consistent index mapping between the
partitioned table and its new-to-be partition.  However, as introduced
in 8b08f7d, the current logic could choose an invalid index as a match,
which is something that can exist when dealing with more than two levels
of partitioning, like attaching a partitioned table (that has
partitions, with an index created by CREATE INDEX ON ONLY) to another
partitioned table.

A partitioned index with indisvalid set to false is equivalent to an
incomplete partition tree, meaning that an invalid partitioned index
does not have indexes defined in all its partitions.  Hence, choosing an
invalid partitioned index can create inconsistent partition index trees,
where the parent attaching to is valid, but its partition may be
invalid.

In the report from Alexander Lakhin, this showed up as an assertion
failure when validating an index.  Without assertions enabled, the
partition index tree would be actually broken, as indisvalid should
be switched to true for a partitioned index once all its partitions are
themselves valid.  With two levels of partitioning, the top partitioned
table used a valid index and was able to link to an invalid index stored
on its partition, itself a partitioned table.

I have studied a few options here (like the possibility to switch
indisvalid to false for the parent), but came down to the conclusion
that we'd better rely on a simple rule: invalid indexes had better never
be chosen, so as the partition attached uses and creates indexes that
the parent expects.  Some regression tests are added to provide some
coverage.  Note that the existing coverage is not impacted.

This is a problem since partitioned indexes exist, so backpatch all the
way down to v11.

Reported-by: Alexander Lakhin
Discussion: https://postgr.es/14987634-43c0-0cb3-e075-94d423607e08@gmail.com
Backpatch-through: 11

Author: michaelpq

src/backend/commands/tablecmds.c

@@ -18104,8 +18104,8 @@ AttachPartitionEnsureIndexes(Relation rel, Relation attachrel)
 
 		/*
 		 * Scan the list of existing indexes in the partition-to-be, and mark
-		 * the first matching, unattached one we find, if any, as partition of
-		 * the parent index.  If we find one, we're done.
+		 * the first matching, valid, unattached one we find, if any, as
+		 * partition of the parent index.  If we find one, we're done.
 		 */
 		for (i = 0; i < list_length(attachRelIdxs); i++)
 		{
@@ -18116,6 +18116,10 @@ AttachPartitionEnsureIndexes(Relation rel, Relation attachrel)
 			if (attachrelIdxRels[i]->rd_rel->relispartition)
 				continue;
 
+			/* If this index is invalid, can't use it */
+			if (!attachrelIdxRels[i]->rd_index->indisvalid)
+				continue;
+
 			if (CompareIndexInfo(attachInfos[i], info,
 								 attachrelIdxRels[i]->rd_indcollation,
 								 idxRel->rd_indcollation,

src/test/regress/expected/indexing.out

@@ -1421,3 +1421,33 @@ Indexes:
     "parted_index_col_drop11_b_idx" btree (b)
 
 drop table parted_index_col_drop;
+-- Check that invalid indexes are not selected when attaching a partition.
+create table parted_inval_tab (a int) partition by range (a);
+create index parted_inval_idx on parted_inval_tab (a);
+create table parted_inval_tab_1 (a int) partition by range (a);
+create table parted_inval_tab_1_1 partition of parted_inval_tab_1
+  for values from (0) to (10);
+create table parted_inval_tab_1_2 partition of parted_inval_tab_1
+  for values from (10) to (20);
+-- this creates an invalid index.
+create index parted_inval_ixd_1 on only parted_inval_tab_1 (a);
+-- this creates new indexes for all the partitions of parted_inval_tab_1,
+-- discarding the invalid index created previously as what is chosen.
+alter table parted_inval_tab attach partition parted_inval_tab_1
+  for values from (1) to (100);
+select indexrelid::regclass, indisvalid,
+       indrelid::regclass, inhparent::regclass
+  from pg_index idx left join
+       pg_inherits inh on (idx.indexrelid = inh.inhrelid)
+  where indexrelid::regclass::text like 'parted_inval%'
+  order by indexrelid::regclass::text collate "C";
+         indexrelid         | indisvalid |       indrelid       |        inhparent         
+----------------------------+------------+----------------------+--------------------------
+ parted_inval_idx           | t          | parted_inval_tab     | 
+ parted_inval_ixd_1         | f          | parted_inval_tab_1   | 
+ parted_inval_tab_1_1_a_idx | t          | parted_inval_tab_1_1 | parted_inval_tab_1_a_idx
+ parted_inval_tab_1_2_a_idx | t          | parted_inval_tab_1_2 | parted_inval_tab_1_a_idx
+ parted_inval_tab_1_a_idx   | t          | parted_inval_tab_1   | parted_inval_idx
+(5 rows)
+
+drop table parted_inval_tab;

src/test/regress/sql/indexing.sql

@@ -760,3 +760,25 @@ alter table parted_index_col_drop drop column c;
 \d parted_index_col_drop2
 \d parted_index_col_drop11
 drop table parted_index_col_drop;
+
+-- Check that invalid indexes are not selected when attaching a partition.
+create table parted_inval_tab (a int) partition by range (a);
+create index parted_inval_idx on parted_inval_tab (a);
+create table parted_inval_tab_1 (a int) partition by range (a);
+create table parted_inval_tab_1_1 partition of parted_inval_tab_1
+  for values from (0) to (10);
+create table parted_inval_tab_1_2 partition of parted_inval_tab_1
+  for values from (10) to (20);
+-- this creates an invalid index.
+create index parted_inval_ixd_1 on only parted_inval_tab_1 (a);
+-- this creates new indexes for all the partitions of parted_inval_tab_1,
+-- discarding the invalid index created previously as what is chosen.
+alter table parted_inval_tab attach partition parted_inval_tab_1
+  for values from (1) to (100);
+select indexrelid::regclass, indisvalid,
+       indrelid::regclass, inhparent::regclass
+  from pg_index idx left join
+       pg_inherits inh on (idx.indexrelid = inh.inhrelid)
+  where indexrelid::regclass::text like 'parted_inval%'
+  order by indexrelid::regclass::text collate "C";
+drop table parted_inval_tab;