Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 3ca930f

Browse files
committed
Improve performance of get_actual_variable_range with recently-dead tuples.
In commit fccebe4, we hacked get_actual_variable_range() to scan the index with SnapshotDirty, so that if there are many uncommitted tuples at the end of the index range, it wouldn't laboriously scan through all of them looking for a live value to return. However, that didn't fix it for the case of many recently-dead tuples at the end of the index; SnapshotDirty recognizes those as committed dead and so we're back to the same problem. To improve the situation, invent a "SnapshotNonVacuumable" snapshot type and use that instead. The reason this helps is that, if the snapshot rejects a given index entry, we know that the indexscan will mark that index entry as killed. This means the next get_actual_variable_range() scan will proceed past that entry without visiting the heap, making the scan a lot faster. We may end up accepting a recently-dead tuple as being the estimated extremal value, but that doesn't seem much worse than the compromise we made before to accept not-yet-committed extremal values. The cost of the scan is still proportional to the number of dead index entries at the end of the range, so in the interval after a mass delete but before VACUUM's cleaned up the mess, it's still possible for get_actual_variable_range() to take a noticeable amount of time, if you've got enough such dead entries. But the constant factor is much much better than before, since all we need to do with each index entry is test its "killed" bit. We chose to back-patch commit fccebe4 at the time, but I'm hesitant to do so here, because this form of the problem seems to affect many fewer people. Also, even when it happens, it's less bad than the case fixed by commit fccebe4 because we don't get the contention effects from expensive TransactionIdIsInProgress tests. Dmitriy Sarafannikov, reviewed by Andrey Borodin Discussion: https://postgr.es/m/05C72CF7-B5F6-4DB9-8A09-5AC897653113@yandex.ru
1 parent b976499 commit 3ca930f

File tree

5 files changed

+65
-14
lines changed

5 files changed

+65
-14
lines changed

src/backend/access/heap/heapam.c

+3
Original file line numberDiff line numberDiff line change
@@ -2118,6 +2118,9 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
21182118
* If we can't see it, maybe no one else can either. At caller
21192119
* request, check whether all chain members are dead to all
21202120
* transactions.
2121+
*
2122+
* Note: if you change the criterion here for what is "dead", fix the
2123+
* planner's get_actual_variable_range() function to match.
21212124
*/
21222125
if (all_dead && *all_dead &&
21232126
!HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin))

src/backend/utils/adt/selfuncs.c

+27-13
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@
142142
#include "utils/pg_locale.h"
143143
#include "utils/rel.h"
144144
#include "utils/selfuncs.h"
145+
#include "utils/snapmgr.h"
145146
#include "utils/spccache.h"
146147
#include "utils/syscache.h"
147148
#include "utils/timestamp.h"
@@ -5328,7 +5329,7 @@ get_actual_variable_range(PlannerInfo *root, VariableStatData *vardata,
53285329
HeapTuple tup;
53295330
Datum values[INDEX_MAX_KEYS];
53305331
bool isnull[INDEX_MAX_KEYS];
5331-
SnapshotData SnapshotDirty;
5332+
SnapshotData SnapshotNonVacuumable;
53325333

53335334
estate = CreateExecutorState();
53345335
econtext = GetPerTupleExprContext(estate);
@@ -5351,7 +5352,7 @@ get_actual_variable_range(PlannerInfo *root, VariableStatData *vardata,
53515352
slot = MakeSingleTupleTableSlot(RelationGetDescr(heapRel));
53525353
econtext->ecxt_scantuple = slot;
53535354
get_typlenbyval(vardata->atttype, &typLen, &typByVal);
5354-
InitDirtySnapshot(SnapshotDirty);
5355+
InitNonVacuumableSnapshot(SnapshotNonVacuumable, RecentGlobalXmin);
53555356

53565357
/* set up an IS NOT NULL scan key so that we ignore nulls */
53575358
ScanKeyEntryInitialize(&scankeys[0],
@@ -5373,17 +5374,29 @@ get_actual_variable_range(PlannerInfo *root, VariableStatData *vardata,
53735374
* active snapshot, which is the best approximation we've got
53745375
* to what the query will see when executed. But that won't
53755376
* be exact if a new snap is taken before running the query,
5376-
* and it can be very expensive if a lot of uncommitted rows
5377-
* exist at the end of the index (because we'll laboriously
5378-
* fetch each one and reject it). What seems like a good
5379-
* compromise is to use SnapshotDirty. That will accept
5380-
* uncommitted rows, and thus avoid fetching multiple heap
5381-
* tuples in this scenario. On the other hand, it will reject
5382-
* known-dead rows, and thus not give a bogus answer when the
5383-
* extreme value has been deleted; that case motivates not
5384-
* using SnapshotAny here.
5377+
* and it can be very expensive if a lot of recently-dead or
5378+
* uncommitted rows exist at the beginning or end of the index
5379+
* (because we'll laboriously fetch each one and reject it).
5380+
* Instead, we use SnapshotNonVacuumable. That will accept
5381+
* recently-dead and uncommitted rows as well as normal
5382+
* visible rows. On the other hand, it will reject known-dead
5383+
* rows, and thus not give a bogus answer when the extreme
5384+
* value has been deleted (unless the deletion was quite
5385+
* recent); that case motivates not using SnapshotAny here.
5386+
*
5387+
* A crucial point here is that SnapshotNonVacuumable, with
5388+
* RecentGlobalXmin as horizon, yields the inverse of the
5389+
* condition that the indexscan will use to decide that index
5390+
* entries are killable (see heap_hot_search_buffer()).
5391+
* Therefore, if the snapshot rejects a tuple and we have to
5392+
* continue scanning past it, we know that the indexscan will
5393+
* mark that index entry killed. That means that the next
5394+
* get_actual_variable_range() call will not have to visit
5395+
* that heap entry. In this way we avoid repetitive work when
5396+
* this function is used a lot during planning.
53855397
*/
5386-
index_scan = index_beginscan(heapRel, indexRel, &SnapshotDirty,
5398+
index_scan = index_beginscan(heapRel, indexRel,
5399+
&SnapshotNonVacuumable,
53875400
1, 0);
53885401
index_rescan(index_scan, scankeys, 1, NULL, 0);
53895402

@@ -5415,7 +5428,8 @@ get_actual_variable_range(PlannerInfo *root, VariableStatData *vardata,
54155428
/* If max is requested, and we didn't find the index is empty */
54165429
if (max && have_data)
54175430
{
5418-
index_scan = index_beginscan(heapRel, indexRel, &SnapshotDirty,
5431+
index_scan = index_beginscan(heapRel, indexRel,
5432+
&SnapshotNonVacuumable,
54195433
1, 0);
54205434
index_rescan(index_scan, scankeys, 1, NULL, 0);
54215435

src/backend/utils/time/tqual.c

+22
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@
4545
* like HeapTupleSatisfiesSelf(), but includes open transactions
4646
* HeapTupleSatisfiesVacuum()
4747
* visible to any running transaction, used by VACUUM
48+
* HeapTupleSatisfiesNonVacuumable()
49+
* Snapshot-style API for HeapTupleSatisfiesVacuum
4850
* HeapTupleSatisfiesToast()
4951
* visible unless part of interrupted vacuum, used for TOAST
5052
* HeapTupleSatisfiesAny()
@@ -1392,6 +1394,26 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin,
13921394
return HEAPTUPLE_DEAD;
13931395
}
13941396

1397+
1398+
/*
1399+
* HeapTupleSatisfiesNonVacuumable
1400+
*
1401+
* True if tuple might be visible to some transaction; false if it's
1402+
* surely dead to everyone, ie, vacuumable.
1403+
*
1404+
* This is an interface to HeapTupleSatisfiesVacuum that meets the
1405+
* SnapshotSatisfiesFunc API, so it can be used through a Snapshot.
1406+
* snapshot->xmin must have been set up with the xmin horizon to use.
1407+
*/
1408+
bool
1409+
HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot,
1410+
Buffer buffer)
1411+
{
1412+
return HeapTupleSatisfiesVacuum(htup, snapshot->xmin, buffer)
1413+
!= HEAPTUPLE_DEAD;
1414+
}
1415+
1416+
13951417
/*
13961418
* HeapTupleIsSurelyDead
13971419
*

src/include/utils/snapshot.h

+3-1
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ typedef bool (*SnapshotSatisfiesFunc) (HeapTuple htup,
4141
* * MVCC snapshots taken during recovery (in Hot-Standby mode)
4242
* * Historic MVCC snapshots used during logical decoding
4343
* * snapshots passed to HeapTupleSatisfiesDirty()
44+
* * snapshots passed to HeapTupleSatisfiesNonVacuumable()
4445
* * snapshots used for SatisfiesAny, Toast, Self where no members are
4546
* accessed.
4647
*
@@ -56,7 +57,8 @@ typedef struct SnapshotData
5657
/*
5758
* The remaining fields are used only for MVCC snapshots, and are normally
5859
* just zeroes in special snapshots. (But xmin and xmax are used
59-
* specially by HeapTupleSatisfiesDirty.)
60+
* specially by HeapTupleSatisfiesDirty, and xmin is used specially by
61+
* HeapTupleSatisfiesNonVacuumable.)
6062
*
6163
* An MVCC snapshot can never see the effects of XIDs >= xmax. It can see
6264
* the effects of all older XIDs except those listed in the snapshot. xmin

src/include/utils/tqual.h

+10
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ extern bool HeapTupleSatisfiesToast(HeapTuple htup,
6666
Snapshot snapshot, Buffer buffer);
6767
extern bool HeapTupleSatisfiesDirty(HeapTuple htup,
6868
Snapshot snapshot, Buffer buffer);
69+
extern bool HeapTupleSatisfiesNonVacuumable(HeapTuple htup,
70+
Snapshot snapshot, Buffer buffer);
6971
extern bool HeapTupleSatisfiesHistoricMVCC(HeapTuple htup,
7072
Snapshot snapshot, Buffer buffer);
7173

@@ -100,6 +102,14 @@ extern bool ResolveCminCmaxDuringDecoding(struct HTAB *tuplecid_data,
100102
#define InitDirtySnapshot(snapshotdata) \
101103
((snapshotdata).satisfies = HeapTupleSatisfiesDirty)
102104

105+
/*
106+
* Similarly, some initialization is required for a NonVacuumable snapshot.
107+
* The caller must supply the xmin horizon to use (e.g., RecentGlobalXmin).
108+
*/
109+
#define InitNonVacuumableSnapshot(snapshotdata, xmin_horizon) \
110+
((snapshotdata).satisfies = HeapTupleSatisfiesNonVacuumable, \
111+
(snapshotdata).xmin = (xmin_horizon))
112+
103113
/*
104114
* Similarly, some initialization is required for SnapshotToast. We need
105115
* to set lsn and whenTaken correctly to support snapshot_too_old.

0 commit comments

Comments
 (0)