Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit f0c409d

Browse files
committed
Fix incorrect step generation in HASH partition pruning
get_steps_using_prefix_recurse() incorrectly assumed that it could stop recursive processing of the 'prefix' list when cur_keyno was one before the step_lastkeyno. Since hash partition pruning can prune using IS NULL quals, and these IS NULL quals are not present in the 'prefix' list, then that logic could cause more levels of recursion than what is needed and lead to there being no more items in the 'prefix' list to process. This would manifest itself as a crash in some code that expected the 'start' ListCell not to be NULL. Here we adjust the logic so that instead of stopping recursion at 1 key before the step_lastkeyno, we just look at the llast(prefix) item and ensure we only recursively process up until just before whichever the last key is. This effectively allows keys to be missing in the 'prefix' list. This change does mean that step_lastkeyno is no longer needed, so we remove that from the static functions. I also spent quite some time reading this code and testing it to try to convince myself that there are no other issues. That resulted in the irresistible temptation of rewriting some comments, many of which were just not true or inconcise. Reported-by: Sergei Glukhov Reviewed-by: Sergei Glukhov, tender wang Discussion: https://postgr.es/m/2f09ce72-315e-2a33-589a-8519ada8df61@postgrespro.ru Backpatch-through: 11, where partition pruning was introduced.
1 parent e768919 commit f0c409d

File tree

3 files changed

+315
-62
lines changed

3 files changed

+315
-62
lines changed

src/backend/partitioning/partprune.c

Lines changed: 59 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -167,15 +167,13 @@ static List *get_steps_using_prefix(GeneratePruningStepsContext *context,
167167
bool step_op_is_ne,
168168
Expr *step_lastexpr,
169169
Oid step_lastcmpfn,
170-
int step_lastkeyno,
171170
Bitmapset *step_nullkeys,
172171
List *prefix);
173172
static List *get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
174173
StrategyNumber step_opstrategy,
175174
bool step_op_is_ne,
176175
Expr *step_lastexpr,
177176
Oid step_lastcmpfn,
178-
int step_lastkeyno,
179177
Bitmapset *step_nullkeys,
180178
List *prefix,
181179
ListCell *start,
@@ -1531,7 +1529,6 @@ gen_prune_steps_from_opexps(GeneratePruningStepsContext *context,
15311529
pc->op_is_ne,
15321530
pc->expr,
15331531
pc->cmpfn,
1534-
0,
15351532
NULL,
15361533
NIL);
15371534
opsteps = list_concat(opsteps, pc_steps);
@@ -1657,7 +1654,6 @@ gen_prune_steps_from_opexps(GeneratePruningStepsContext *context,
16571654
pc->op_is_ne,
16581655
pc->expr,
16591656
pc->cmpfn,
1660-
pc->keyno,
16611657
NULL,
16621658
prefix);
16631659
opsteps = list_concat(opsteps, pc_steps);
@@ -1731,7 +1727,6 @@ gen_prune_steps_from_opexps(GeneratePruningStepsContext *context,
17311727
false,
17321728
pc->expr,
17331729
pc->cmpfn,
1734-
pc->keyno,
17351730
nullkeys,
17361731
prefix);
17371732
opsteps = list_concat(opsteps, pc_steps);
@@ -2350,40 +2345,49 @@ match_clause_to_partition_key(GeneratePruningStepsContext *context,
23502345

23512346
/*
23522347
* get_steps_using_prefix
2353-
* Generate list of PartitionPruneStepOp steps each consisting of given
2354-
* opstrategy
2355-
*
2356-
* To generate steps, step_lastexpr and step_lastcmpfn are appended to
2357-
* expressions and cmpfns, respectively, extracted from the clauses in
2358-
* 'prefix'. Actually, since 'prefix' may contain multiple clauses for the
2359-
* same partition key column, we must generate steps for various combinations
2360-
* of the clauses of different keys.
2361-
*
2362-
* For list/range partitioning, callers must ensure that step_nullkeys is
2363-
* NULL, and that prefix contains at least one clause for each of the
2364-
* partition keys earlier than one specified in step_lastkeyno if it's
2365-
* greater than zero. For hash partitioning, step_nullkeys is allowed to be
2366-
* non-NULL, but they must ensure that prefix contains at least one clause
2367-
* for each of the partition keys other than those specified in step_nullkeys
2368-
* and step_lastkeyno.
2369-
*
2370-
* For both cases, callers must also ensure that clauses in prefix are sorted
2371-
* in ascending order of their partition key numbers.
2348+
* Generate a list of PartitionPruneStepOps based on the given input.
2349+
*
2350+
* 'step_lastexpr' and 'step_lastcmpfn' are the Expr and comparison function
2351+
* belonging to the final partition key that we have a clause for. 'prefix'
2352+
* is a list of PartClauseInfos for partition key numbers prior to the given
2353+
* 'step_lastexpr' and 'step_lastcmpfn'. 'prefix' may contain multiple
2354+
* PartClauseInfos belonging to a single partition key. We will generate a
2355+
* PartitionPruneStepOp for each combination of the given PartClauseInfos
2356+
* using, at most, one PartClauseInfo per partition key.
2357+
*
2358+
* For LIST and RANGE partitioned tables, callers must ensure that
2359+
* step_nullkeys is NULL, and that prefix contains at least one clause for
2360+
* each of the partition keys prior to the key that 'step_lastexpr' and
2361+
* 'step_lastcmpfn'belong to.
2362+
*
2363+
* For HASH partitioned tables, callers must ensure that 'prefix' contains at
2364+
* least one clause for each of the partition keys apart from the final key
2365+
* (the expr and comparison function for the final key are in 'step_lastexpr'
2366+
* and 'step_lastcmpfn'). A bit set in step_nullkeys can substitute clauses
2367+
* in the 'prefix' list for any given key. If a bit is set in 'step_nullkeys'
2368+
* for a given key, then there must be no PartClauseInfo for that key in the
2369+
* 'prefix' list.
2370+
*
2371+
* For each of the above cases, callers must ensure that PartClauseInfos in
2372+
* 'prefix' are sorted in ascending order of keyno.
23722373
*/
23732374
static List *
23742375
get_steps_using_prefix(GeneratePruningStepsContext *context,
23752376
StrategyNumber step_opstrategy,
23762377
bool step_op_is_ne,
23772378
Expr *step_lastexpr,
23782379
Oid step_lastcmpfn,
2379-
int step_lastkeyno,
23802380
Bitmapset *step_nullkeys,
23812381
List *prefix)
23822382
{
2383+
/* step_nullkeys must be empty for RANGE and LIST partitioned tables */
23832384
Assert(step_nullkeys == NULL ||
23842385
context->rel->part_scheme->strategy == PARTITION_STRATEGY_HASH);
23852386

2386-
/* Quick exit if there are no values to prefix with. */
2387+
/*
2388+
* No recursive processing is required when 'prefix' is an empty list.
2389+
* This occurs when there is only 1 partition key column.
2390+
*/
23872391
if (prefix == NIL)
23882392
{
23892393
PartitionPruneStep *step;
@@ -2397,13 +2401,12 @@ get_steps_using_prefix(GeneratePruningStepsContext *context,
23972401
return list_make1(step);
23982402
}
23992403

2400-
/* Recurse to generate steps for various combinations. */
2404+
/* Recurse to generate steps for every combination of clauses. */
24012405
return get_steps_using_prefix_recurse(context,
24022406
step_opstrategy,
24032407
step_op_is_ne,
24042408
step_lastexpr,
24052409
step_lastcmpfn,
2406-
step_lastkeyno,
24072410
step_nullkeys,
24082411
prefix,
24092412
list_head(prefix),
@@ -2412,13 +2415,17 @@ get_steps_using_prefix(GeneratePruningStepsContext *context,
24122415

24132416
/*
24142417
* get_steps_using_prefix_recurse
2415-
* Recursively generate combinations of clauses for different partition
2416-
* keys and start generating steps upon reaching clauses for the greatest
2417-
* column that is less than the one for which we're currently generating
2418-
* steps (that is, step_lastkeyno)
2418+
* Generate and return a list of PartitionPruneStepOps using the 'prefix'
2419+
* list of PartClauseInfos starting at the 'start' cell.
2420+
*
2421+
* When 'prefix' contains multiple PartClauseInfos for a single partition key
2422+
* we create a PartitionPruneStepOp for each combination of duplicated
2423+
* PartClauseInfos. The returned list will contain a PartitionPruneStepOp
2424+
* for each unique combination of input PartClauseInfos containing at most one
2425+
* PartClauseInfo per partition key.
24192426
*
2420-
* 'prefix' is the list of PartClauseInfos.
2421-
* 'start' is where we should start iterating for the current invocation.
2427+
* 'prefix' is the input list of PartClauseInfos sorted by keyno.
2428+
* 'start' marks the cell that searching the 'prefix' list should start from.
24222429
* 'step_exprs' and 'step_cmpfns' each contains the expressions and cmpfns
24232430
* we've generated so far from the clauses for the previous part keys.
24242431
*/
@@ -2428,7 +2435,6 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
24282435
bool step_op_is_ne,
24292436
Expr *step_lastexpr,
24302437
Oid step_lastcmpfn,
2431-
int step_lastkeyno,
24322438
Bitmapset *step_nullkeys,
24332439
List *prefix,
24342440
ListCell *start,
@@ -2438,23 +2444,25 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
24382444
List *result = NIL;
24392445
ListCell *lc;
24402446
int cur_keyno;
2447+
int final_keyno;
24412448

24422449
/* Actually, recursion would be limited by PARTITION_MAX_KEYS. */
24432450
check_stack_depth();
24442451

2445-
/* Check if we need to recurse. */
24462452
Assert(start != NULL);
24472453
cur_keyno = ((PartClauseInfo *) lfirst(start))->keyno;
2448-
if (cur_keyno < step_lastkeyno - 1)
2454+
final_keyno = ((PartClauseInfo *) llast(prefix))->keyno;
2455+
2456+
/* Check if we need to recurse. */
2457+
if (cur_keyno < final_keyno)
24492458
{
24502459
PartClauseInfo *pc;
24512460
ListCell *next_start;
24522461

24532462
/*
2454-
* For each clause with cur_keyno, add its expr and cmpfn to
2455-
* step_exprs and step_cmpfns, respectively, and recurse after setting
2456-
* next_start to the ListCell of the first clause for the next
2457-
* partition key.
2463+
* Find the first PartClauseInfo belonging to the next partition key,
2464+
* the next recursive call must start iteration of the prefix list
2465+
* from that point.
24582466
*/
24592467
for_each_cell(lc, prefix, start)
24602468
{
@@ -2463,8 +2471,15 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
24632471
if (pc->keyno > cur_keyno)
24642472
break;
24652473
}
2474+
2475+
/* record where to start iterating in the next recursive call */
24662476
next_start = lc;
24672477

2478+
/*
2479+
* For each PartClauseInfo with keyno set to cur_keyno, add its expr
2480+
* and cmpfn to step_exprs and step_cmpfns, respectively, and recurse
2481+
* using 'next_start' as the starting point in the 'prefix' list.
2482+
*/
24682483
for_each_cell(lc, prefix, start)
24692484
{
24702485
List *moresteps;
@@ -2484,6 +2499,7 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
24842499
}
24852500
else
24862501
{
2502+
/* check the 'prefix' list is sorted correctly */
24872503
Assert(pc->keyno > cur_keyno);
24882504
break;
24892505
}
@@ -2493,7 +2509,6 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
24932509
step_op_is_ne,
24942510
step_lastexpr,
24952511
step_lastcmpfn,
2496-
step_lastkeyno,
24972512
step_nullkeys,
24982513
prefix,
24992514
next_start,
@@ -2512,8 +2527,8 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
25122527
* each clause with cur_keyno, which is all clauses from here onward
25132528
* till the end of the list. Note that for hash partitioning,
25142529
* step_nullkeys is allowed to be non-empty, in which case step_exprs
2515-
* would only contain expressions for the earlier partition keys that
2516-
* are not specified in step_nullkeys.
2530+
* would only contain expressions for the partition keys that are not
2531+
* specified in step_nullkeys.
25172532
*/
25182533
Assert(list_length(step_exprs) == cur_keyno ||
25192534
!bms_is_empty(step_nullkeys));

0 commit comments

Comments
 (0)