Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit d04c8ed

Browse files
committed
Add support for index-only scans in GiST.
This adds a new GiST opclass method, 'fetch', which is used to reconstruct the original Datum from the value stored in the index. Also, the 'canreturn' index AM interface function gains a new 'attno' argument. That makes it possible to use index-only scans on a multi-column index where some of the opclasses support index-only scans but some do not. This patch adds support in the box and point opclasses. Other opclasses can added later as follow-on patches (btree_gist would be particularly interesting). Anastasia Lubennikova, with additional fixes and modifications by me.
1 parent 8fa393a commit d04c8ed

File tree

24 files changed

+575
-73
lines changed

24 files changed

+575
-73
lines changed

doc/src/sgml/catalogs.sgml

+2-2
Original file line numberDiff line numberDiff line change
@@ -724,8 +724,8 @@
724724
<entry><structfield>amcanreturn</structfield></entry>
725725
<entry><type>regproc</type></entry>
726726
<entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
727-
<entry>Function to check whether index supports index-only scans,
728-
or zero if none</entry>
727+
<entry>Function to check whether an index column supports index-only
728+
scans. Can be zero if index-only scans are never supported.</entry>
729729
</row>
730730

731731
<row>

doc/src/sgml/gist.sgml

+70-3
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,7 @@ CREATE INDEX ON my_table USING gist (my_inet_column inet_ops);
266266

267267
<para>
268268
There are seven methods that an index operator class for
269-
<acronym>GiST</acronym> must provide, and an eighth that is optional.
269+
<acronym>GiST</acronym> must provide, and two that are optional.
270270
Correctness of the index is ensured
271271
by proper implementation of the <function>same</>, <function>consistent</>
272272
and <function>union</> methods, while efficiency (size and speed) of the
@@ -282,7 +282,8 @@ CREATE INDEX ON my_table USING gist (my_inet_column inet_ops);
282282
of the <command>CREATE OPERATOR CLASS</> command can be used.
283283
The optional eighth method is <function>distance</>, which is needed
284284
if the operator class wishes to support ordered scans (nearest-neighbor
285-
searches).
285+
searches). The optional ninth method <function>fetch</> is needed if the
286+
operator class wishes to support index-only scans.
286287
</para>
287288

288289
<variablelist>
@@ -506,7 +507,7 @@ my_compress(PG_FUNCTION_ARGS)
506507
<para>
507508
The reverse of the <function>compress</function> method. Converts the
508509
index representation of the data item into a format that can be
509-
manipulated by the database.
510+
manipulated by the other GiST methods in the operator class.
510511
</para>
511512

512513
<para>
@@ -807,6 +808,72 @@ my_distance(PG_FUNCTION_ARGS)
807808
</listitem>
808809
</varlistentry>
809810

811+
<varlistentry>
812+
<term><function>fetch</></term>
813+
<listitem>
814+
<para>
815+
Converts the compressed index representation of the data item into the
816+
original data type, for index-only scans. The returned data must be an
817+
exact, non-lossy copy of the originally indexed value.
818+
</para>
819+
820+
<para>
821+
The <acronym>SQL</> declaration of the function must look like this:
822+
823+
<programlisting>
824+
CREATE OR REPLACE FUNCTION my_fetch(internal)
825+
RETURNS internal
826+
AS 'MODULE_PATHNAME'
827+
LANGUAGE C STRICT;
828+
</programlisting>
829+
830+
The argument is a pointer to a <structname>GISTENTRY</> struct. On
831+
entry, its 'key' field contains a non-NULL leaf datum in its
832+
compressed form. The return value is another <structname>GISTENTRY</>
833+
struct, whose 'key' field contains the same datum in the original,
834+
uncompressed form. If the opclass' compress function does nothing for
835+
leaf entries, the fetch method can return the argument as is.
836+
</para>
837+
838+
<para>
839+
The matching code in the C module could then follow this skeleton:
840+
841+
<programlisting>
842+
Datum my_fetch(PG_FUNCTION_ARGS);
843+
PG_FUNCTION_INFO_V1(my_fetch);
844+
845+
Datum
846+
my_fetch(PG_FUNCTION_ARGS)
847+
{
848+
GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
849+
input_data_type *in = DatumGetP(entry->key);
850+
fetched_data_type *fetched_data;
851+
GISTENTRY *retval;
852+
853+
retval = palloc(sizeof(GISTENTRY));
854+
fetched_data = palloc(sizeof(fetched_data_type));
855+
856+
/*
857+
* Convert 'fetched_data' into the a Datum of the original datatype.
858+
*/
859+
860+
/* fill *retval from fetch_data. */
861+
gistentryinit(*retval, PointerGetDatum(converted_datum),
862+
entry->rel, entry->page, entry->offset, FALSE);
863+
864+
PG_RETURN_POINTER(retval);
865+
}
866+
</programlisting>
867+
</para>
868+
869+
<para>
870+
If the compress method is lossy for leaf entries, the operator class
871+
cannot support index-only scans, and must not define a 'fetch'
872+
function.
873+
</para>
874+
875+
</listitem>
876+
</varlistentry>
810877
</variablelist>
811878

812879
<para>

doc/src/sgml/indexam.sgml

+8-7
Original file line numberDiff line numberDiff line change
@@ -274,14 +274,15 @@ amvacuumcleanup (IndexVacuumInfo *info,
274274
<para>
275275
<programlisting>
276276
bool
277-
amcanreturn (Relation indexRelation);
277+
amcanreturn (Relation indexRelation, int attno);
278278
</programlisting>
279-
Check whether the index can support <firstterm>index-only scans</> by
280-
returning the indexed column values for an index entry in the form of an
281-
<structname>IndexTuple</structname>. Return TRUE if so, else FALSE. If the index AM can never
282-
support index-only scans (an example is hash, which stores only
283-
the hash values not the original data), it is sufficient to set its
284-
<structfield>amcanreturn</> field to zero in <structname>pg_am</>.
279+
Check whether the index can support <firstterm>index-only scans</> on the
280+
given column, by returning the indexed column values for an index entry in
281+
the form of an <structname>IndexTuple</structname>. The attribute number
282+
is 1-based, i.e. the first columns attno is 1. Returns TRUE if supported,
283+
else FALSE. If the access method does not support index-only scans at all,
284+
the <structfield>amcanreturn</> field in its <structname>pg_am</> row can
285+
be set to zero.
285286
</para>
286287

287288
<para>

src/backend/access/gist/gist.c

+8
Original file line numberDiff line numberDiff line change
@@ -1404,6 +1404,14 @@ initGISTstate(Relation index)
14041404
else
14051405
giststate->distanceFn[i].fn_oid = InvalidOid;
14061406

1407+
/* opclasses are not required to provide a Fetch method */
1408+
if (OidIsValid(index_getprocid(index, i + 1, GIST_FETCH_PROC)))
1409+
fmgr_info_copy(&(giststate->fetchFn[i]),
1410+
index_getprocinfo(index, i + 1, GIST_FETCH_PROC),
1411+
scanCxt);
1412+
else
1413+
giststate->fetchFn[i].fn_oid = InvalidOid;
1414+
14071415
/*
14081416
* If the index column has a specified collation, we should honor that
14091417
* while doing comparisons. However, we may have a collatable storage

src/backend/access/gist/gistget.c

+64-2
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,9 @@ gistindex_keytest(IndexScanDesc scan,
228228
* tuples should be reported directly into the bitmap. If they are NULL,
229229
* we're doing a plain or ordered indexscan. For a plain indexscan, heap
230230
* tuple TIDs are returned into so->pageData[]. For an ordered indexscan,
231-
* heap tuple TIDs are pushed into individual search queue items.
231+
* heap tuple TIDs are pushed into individual search queue items. In an
232+
* index-only scan, reconstructed index tuples are returned along with the
233+
* TIDs.
232234
*
233235
* If we detect that the index page has split since we saw its downlink
234236
* in the parent, we push its new right sibling onto the queue so the
@@ -239,6 +241,8 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
239241
TIDBitmap *tbm, int64 *ntids)
240242
{
241243
GISTScanOpaque so = (GISTScanOpaque) scan->opaque;
244+
GISTSTATE *giststate = so->giststate;
245+
Relation r = scan->indexRelation;
242246
Buffer buffer;
243247
Page page;
244248
GISTPageOpaque opaque;
@@ -288,6 +292,8 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
288292
}
289293

290294
so->nPageData = so->curPageData = 0;
295+
if (so->pageDataCxt)
296+
MemoryContextReset(so->pageDataCxt);
291297

292298
/*
293299
* check all tuples on page
@@ -326,10 +332,21 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
326332
else if (scan->numberOfOrderBys == 0 && GistPageIsLeaf(page))
327333
{
328334
/*
329-
* Non-ordered scan, so report heap tuples in so->pageData[]
335+
* Non-ordered scan, so report tuples in so->pageData[]
330336
*/
331337
so->pageData[so->nPageData].heapPtr = it->t_tid;
332338
so->pageData[so->nPageData].recheck = recheck;
339+
340+
/*
341+
* In an index-only scan, also fetch the data from the tuple.
342+
*/
343+
if (scan->xs_want_itup)
344+
{
345+
oldcxt = MemoryContextSwitchTo(so->pageDataCxt);
346+
so->pageData[so->nPageData].ftup =
347+
gistFetchTuple(giststate, r, it);
348+
MemoryContextSwitchTo(oldcxt);
349+
}
333350
so->nPageData++;
334351
}
335352
else
@@ -352,6 +369,12 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
352369
item->blkno = InvalidBlockNumber;
353370
item->data.heap.heapPtr = it->t_tid;
354371
item->data.heap.recheck = recheck;
372+
373+
/*
374+
* In an index-only scan, also fetch the data from the tuple.
375+
*/
376+
if (scan->xs_want_itup)
377+
item->data.heap.ftup = gistFetchTuple(giststate, r, it);
355378
}
356379
else
357380
{
@@ -412,6 +435,13 @@ getNextNearest(IndexScanDesc scan)
412435
GISTScanOpaque so = (GISTScanOpaque) scan->opaque;
413436
bool res = false;
414437

438+
if (scan->xs_itup)
439+
{
440+
/* free previously returned tuple */
441+
pfree(scan->xs_itup);
442+
scan->xs_itup = NULL;
443+
}
444+
415445
do
416446
{
417447
GISTSearchItem *item = getNextGISTSearchItem(so);
@@ -424,6 +454,10 @@ getNextNearest(IndexScanDesc scan)
424454
/* found a heap item at currently minimal distance */
425455
scan->xs_ctup.t_self = item->data.heap.heapPtr;
426456
scan->xs_recheck = item->data.heap.recheck;
457+
458+
/* in an index-only scan, also return the reconstructed tuple. */
459+
if (scan->xs_want_itup)
460+
scan->xs_itup = item->data.heap.ftup;
427461
res = true;
428462
}
429463
else
@@ -465,6 +499,8 @@ gistgettuple(PG_FUNCTION_ARGS)
465499

466500
so->firstCall = false;
467501
so->curPageData = so->nPageData = 0;
502+
if (so->pageDataCxt)
503+
MemoryContextReset(so->pageDataCxt);
468504

469505
fakeItem.blkno = GIST_ROOT_BLKNO;
470506
memset(&fakeItem.data.parentlsn, 0, sizeof(GistNSN));
@@ -483,10 +519,17 @@ gistgettuple(PG_FUNCTION_ARGS)
483519
{
484520
if (so->curPageData < so->nPageData)
485521
{
522+
486523
/* continuing to return tuples from a leaf page */
487524
scan->xs_ctup.t_self = so->pageData[so->curPageData].heapPtr;
488525
scan->xs_recheck = so->pageData[so->curPageData].recheck;
526+
527+
/* in an index-only scan, also return the reconstructed tuple */
528+
if (scan->xs_want_itup)
529+
scan->xs_itup = so->pageData[so->curPageData].ftup;
530+
489531
so->curPageData++;
532+
490533
PG_RETURN_BOOL(true);
491534
}
492535

@@ -533,6 +576,8 @@ gistgetbitmap(PG_FUNCTION_ARGS)
533576

534577
/* Begin the scan by processing the root page */
535578
so->curPageData = so->nPageData = 0;
579+
if (so->pageDataCxt)
580+
MemoryContextReset(so->pageDataCxt);
536581

537582
fakeItem.blkno = GIST_ROOT_BLKNO;
538583
memset(&fakeItem.data.parentlsn, 0, sizeof(GistNSN));
@@ -558,3 +603,20 @@ gistgetbitmap(PG_FUNCTION_ARGS)
558603

559604
PG_RETURN_INT64(ntids);
560605
}
606+
607+
/*
608+
* Can we do index-only scans on the given index column?
609+
*
610+
* Opclasses that implement a fetch function support index-only scans.
611+
*/
612+
Datum
613+
gistcanreturn(PG_FUNCTION_ARGS)
614+
{
615+
Relation index = (Relation) PG_GETARG_POINTER(0);
616+
int attno = PG_GETARG_INT32(1);
617+
618+
if (OidIsValid(index_getprocid(index, attno, GIST_FETCH_PROC)))
619+
PG_RETURN_BOOL(true);
620+
else
621+
PG_RETURN_BOOL(false);
622+
}

src/backend/access/gist/gistproc.c

+37
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,16 @@ gist_box_decompress(PG_FUNCTION_ARGS)
151151
PG_RETURN_POINTER(PG_GETARG_POINTER(0));
152152
}
153153

154+
/*
155+
* GiST Fetch method for boxes
156+
* do not do anything --- we just return the stored box as is.
157+
*/
158+
Datum
159+
gist_box_fetch(PG_FUNCTION_ARGS)
160+
{
161+
PG_RETURN_POINTER(PG_GETARG_POINTER(0));
162+
}
163+
154164
/*
155165
* The GiST Penalty method for boxes (also used for points)
156166
*
@@ -1186,6 +1196,33 @@ gist_point_compress(PG_FUNCTION_ARGS)
11861196
PG_RETURN_POINTER(entry);
11871197
}
11881198

1199+
/*
1200+
* GiST Fetch method for point
1201+
*
1202+
* Get point coordinates from its bounding box coordinates and form new
1203+
* gistentry.
1204+
*/
1205+
Datum
1206+
gist_point_fetch(PG_FUNCTION_ARGS)
1207+
{
1208+
GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
1209+
BOX *in = DatumGetBoxP(entry->key);
1210+
Point *r;
1211+
GISTENTRY *retval;
1212+
1213+
retval = palloc(sizeof(GISTENTRY));
1214+
1215+
r = (Point *) palloc(sizeof(Point));
1216+
r->x = in->high.x;
1217+
r->y = in->high.y;
1218+
gistentryinit(*retval, PointerGetDatum(r),
1219+
entry->rel, entry->page,
1220+
entry->offset, FALSE);
1221+
1222+
PG_RETURN_POINTER(retval);
1223+
}
1224+
1225+
11891226
#define point_point_distance(p1,p2) \
11901227
DatumGetFloat8(DirectFunctionCall2(point_distance, \
11911228
PointPGetDatum(p1), PointPGetDatum(p2)))

src/backend/access/gist/gistscan.c

+18
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,13 @@ gistbeginscan(PG_FUNCTION_ARGS)
8888

8989
scan->opaque = so;
9090

91+
/*
92+
* All fields required for index-only scans are null until gistrescan.
93+
* However, we set up scan->xs_itupdesc whether we'll need it or not,
94+
* since that's cheap.
95+
*/
96+
scan->xs_itupdesc = RelationGetDescr(r);
97+
9198
MemoryContextSwitchTo(oldCxt);
9299

93100
PG_RETURN_POINTER(scan);
@@ -141,6 +148,17 @@ gistrescan(PG_FUNCTION_ARGS)
141148
first_time = false;
142149
}
143150

151+
/*
152+
* If we're doing an index-only scan, also create a memory context to hold
153+
* the returned tuples.
154+
*/
155+
if (scan->xs_want_itup && so->pageDataCxt == NULL)
156+
so->pageDataCxt = AllocSetContextCreate(so->giststate->scanCxt,
157+
"GiST page data context",
158+
ALLOCSET_DEFAULT_MINSIZE,
159+
ALLOCSET_DEFAULT_INITSIZE,
160+
ALLOCSET_DEFAULT_MAXSIZE);
161+
144162
/* create new, empty RBTree for search queue */
145163
oldCxt = MemoryContextSwitchTo(so->queueCxt);
146164
so->queue = pairingheap_allocate(pairingheap_GISTSearchItem_cmp, scan);

0 commit comments

Comments
 (0)