Skip to content

Commit c2fe139

Browse files
anarazelanarazel
committed
tableam: Add and use scan APIs.
Too allow table accesses to be not directly dependent on heap, several new abstractions are needed. Specifically: 1) Heap scans need to be generalized into table scans. Do this by introducing TableScanDesc, which will be the "base class" for individual AMs. This contains the AM independent fields from HeapScanDesc. The previous heap_{beginscan,rescan,endscan} et al. have been replaced with a table_ version. There's no direct replacement for heap_getnext(), as that returned a HeapTuple, which is undesirable for a other AMs. Instead there's table_scan_getnextslot(). But note that heap_getnext() lives on, it's still used widely to access catalog tables. This is achieved by new scan_begin, scan_end, scan_rescan, scan_getnextslot callbacks. 2) The portion of parallel scans that's shared between backends need to be able to do so without the user doing per-AM work. To achieve that new parallelscan_{estimate, initialize, reinitialize} callbacks are introduced, which operate on a new ParallelTableScanDesc, which again can be subclassed by AMs. As it is likely that several AMs are going to be block oriented, block oriented callbacks that can be shared between such AMs are provided and used by heap. table_block_parallelscan_{estimate, intiialize, reinitialize} as callbacks, and table_block_parallelscan_{nextpage, init} for use in AMs. These operate on a ParallelBlockTableScanDesc. 3) Index scans need to be able to access tables to return a tuple, and there needs to be state across individual accesses to the heap to store state like buffers. That's now handled by introducing a sort-of-scan IndexFetchTable, which again is intended to be subclassed by individual AMs (for heap IndexFetchHeap). The relevant callbacks for an AM are index_fetch_{end, begin, reset} to create the necessary state, and index_fetch_tuple to retrieve an indexed tuple. Note that index_fetch_tuple implementations need to be smarter than just blindly fetching the tuples for AMs that have optimizations similar to heap's HOT - the currently alive tuple in the update chain needs to be fetched if appropriate. Similar to table_scan_getnextslot(), it's undesirable to continue to return HeapTuples. Thus index_fetch_heap (might want to rename that later) now accepts a slot as an argument. Core code doesn't have a lot of call sites performing index scans without going through the systable_* API (in contrast to loads of heap_getnext calls and working directly with HeapTuples). Index scans now store the result of a search in IndexScanDesc->xs_heaptid, rather than xs_ctup->t_self. As the target is not generally a HeapTuple anymore that seems cleaner. To be able to sensible adapt code to use the above, two further callbacks have been introduced: a) slot_callbacks returns a TupleTableSlotOps* suitable for creating slots capable of holding a tuple of the AMs type. table_slot_callbacks() and table_slot_create() are based upon that, but have additional logic to deal with views, foreign tables, etc. While this change could have been done separately, nearly all the call sites that needed to be adapted for the rest of this commit also would have been needed to be adapted for table_slot_callbacks(), making separation not worthwhile. b) tuple_satisfies_snapshot checks whether the tuple in a slot is currently visible according to a snapshot. That's required as a few places now don't have a buffer + HeapTuple around, but a slot (which in heap's case internally has that information). Additionally a few infrastructure changes were needed: I) SysScanDesc, as used by systable_{beginscan, getnext} et al. now internally uses a slot to keep track of tuples. While systable_getnext() still returns HeapTuples, and will so for the foreseeable future, the index API (see 1) above) now only deals with slots. The remainder, and largest part, of this commit is then adjusting all scans in postgres to use the new APIs. Author: Andres Freund, Haribabu Kommi, Alvaro Herrera Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
1 parent a478415 commit c2fe139

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+2031
-1266
lines changed

‎contrib/amcheck/verify_nbtree.c

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
#include "access/heapam.h"
2727
#include "access/htup_details.h"
2828
#include "access/nbtree.h"
29+
#include "access/tableam.h"
2930
#include "access/transam.h"
3031
#include "access/xact.h"
3132
#include "catalog/index.h"
@@ -481,7 +482,7 @@ bt_check_every_level(Relation rel, Relation heaprel, bool readonly,
481482
if (state->heapallindexed)
482483
{
483484
IndexInfo *indexinfo = BuildIndexInfo(state->rel);
484-
HeapScanDesc scan;
485+
TableScanDesc scan;
485486

486487
/* Report on extra downlink checks performed in readonly case */
487488
if (state->readonly)
@@ -500,12 +501,12 @@ bt_check_every_level(Relation rel, Relation heaprel, bool readonly,
500501
*
501502
* Note that IndexBuildHeapScan() calls heap_endscan() for us.
502503
*/
503-
scan = heap_beginscan_strat(state->heaprel, /* relation */
504-
snapshot, /* snapshot */
505-
0, /* number of keys */
506-
NULL, /* scan key */
507-
true, /* buffer access strategy OK */
508-
true); /* syncscan OK? */
504+
scan = table_beginscan_strat(state->heaprel, /* relation */
505+
snapshot, /* snapshot */
506+
0, /* number of keys */
507+
NULL, /* scan key */
508+
true, /* buffer access strategy OK */
509+
true); /* syncscan OK? */
509510

510511
/*
511512
* Scan will behave as the first scan of a CREATE INDEX CONCURRENTLY

‎contrib/pgrowlocks/pgrowlocks.c

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
#include "access/heapam.h"
2828
#include "access/multixact.h"
2929
#include "access/relscan.h"
30+
#include "access/tableam.h"
3031
#include "access/xact.h"
3132
#include "catalog/namespace.h"
3233
#include "catalog/pg_authid.h"
@@ -55,7 +56,7 @@ PG_FUNCTION_INFO_V1(pgrowlocks);
5556
typedef struct
5657
{
5758
Relation rel;
58-
HeapScanDesc scan;
59+
TableScanDesc scan;
5960
int ncolumns;
6061
} MyData;
6162

@@ -70,7 +71,8 @@ Datum
7071
pgrowlocks(PG_FUNCTION_ARGS)
7172
{
7273
FuncCallContext *funcctx;
73-
HeapScanDesc scan;
74+
TableScanDesc scan;
75+
HeapScanDesc hscan;
7476
HeapTuple tuple;
7577
TupleDesc tupdesc;
7678
AttInMetadata *attinmeta;
@@ -124,7 +126,8 @@ pgrowlocks(PG_FUNCTION_ARGS)
124126
aclcheck_error(aclresult, get_relkind_objtype(rel->rd_rel->relkind),
125127
RelationGetRelationName(rel));
126128

127-
scan = heap_beginscan(rel, GetActiveSnapshot(), 0, NULL);
129+
scan = table_beginscan(rel, GetActiveSnapshot(), 0, NULL);
130+
hscan = (HeapScanDesc) scan;
128131
mydata = palloc(sizeof(*mydata));
129132
mydata->rel = rel;
130133
mydata->scan = scan;
@@ -138,6 +141,7 @@ pgrowlocks(PG_FUNCTION_ARGS)
138141
attinmeta = funcctx->attinmeta;
139142
mydata = (MyData *) funcctx->user_fctx;
140143
scan = mydata->scan;
144+
hscan = (HeapScanDesc) scan;
141145

142146
/* scan the relation */
143147
while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
@@ -147,11 +151,11 @@ pgrowlocks(PG_FUNCTION_ARGS)
147151
uint16 infomask;
148152

149153
/* must hold a buffer lock to call HeapTupleSatisfiesUpdate */
150-
LockBuffer(scan->rs_cbuf, BUFFER_LOCK_SHARE);
154+
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
151155

152156
htsu = HeapTupleSatisfiesUpdate(tuple,
153157
GetCurrentCommandId(false),
154-
scan->rs_cbuf);
158+
hscan->rs_cbuf);
155159
xmax = HeapTupleHeaderGetRawXmax(tuple->t_data);
156160
infomask = tuple->t_data->t_infomask;
157161

@@ -284,7 +288,7 @@ pgrowlocks(PG_FUNCTION_ARGS)
284288
BackendXidGetPid(xmax));
285289
}
286290

287-
LockBuffer(scan->rs_cbuf, BUFFER_LOCK_UNLOCK);
291+
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
288292

289293
/* build a tuple */
290294
tuple = BuildTupleFromCStrings(attinmeta, values);
@@ -301,11 +305,11 @@ pgrowlocks(PG_FUNCTION_ARGS)
301305
}
302306
else
303307
{
304-
LockBuffer(scan->rs_cbuf, BUFFER_LOCK_UNLOCK);
308+
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
305309
}
306310
}
307311

308-
heap_endscan(scan);
312+
table_endscan(scan);
309313
table_close(mydata->rel, AccessShareLock);
310314

311315
SRF_RETURN_DONE(funcctx);

‎contrib/pgstattuple/pgstattuple.c

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
#include "access/heapam.h"
3030
#include "access/nbtree.h"
3131
#include "access/relscan.h"
32+
#include "access/tableam.h"
3233
#include "catalog/namespace.h"
3334
#include "catalog/pg_am.h"
3435
#include "funcapi.h"
@@ -317,7 +318,8 @@ pgstat_relation(Relation rel, FunctionCallInfo fcinfo)
317318
static Datum
318319
pgstat_heap(Relation rel, FunctionCallInfo fcinfo)
319320
{
320-
HeapScanDesc scan;
321+
TableScanDesc scan;
322+
HeapScanDesc hscan;
321323
HeapTuple tuple;
322324
BlockNumber nblocks;
323325
BlockNumber block = 0; /* next block to count free space in */
@@ -327,20 +329,22 @@ pgstat_heap(Relation rel, FunctionCallInfo fcinfo)
327329
SnapshotData SnapshotDirty;
328330

329331
/* Disable syncscan because we assume we scan from block zero upwards */
330-
scan = heap_beginscan_strat(rel, SnapshotAny, 0, NULL, true, false);
332+
scan = table_beginscan_strat(rel, SnapshotAny, 0, NULL, true, false);
333+
hscan = (HeapScanDesc) scan;
334+
331335
InitDirtySnapshot(SnapshotDirty);
332336

333-
nblocks = scan->rs_nblocks; /* # blocks to be scanned */
337+
nblocks = hscan->rs_nblocks; /* # blocks to be scanned */
334338

335339
/* scan the relation */
336340
while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
337341
{
338342
CHECK_FOR_INTERRUPTS();
339343

340344
/* must hold a buffer lock to call HeapTupleSatisfiesVisibility */
341-
LockBuffer(scan->rs_cbuf, BUFFER_LOCK_SHARE);
345+
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
342346

343-
if (HeapTupleSatisfiesVisibility(tuple, &SnapshotDirty, scan->rs_cbuf))
347+
if (HeapTupleSatisfiesVisibility(tuple, &SnapshotDirty, hscan->rs_cbuf))
344348
{
345349
stat.tuple_len += tuple->t_len;
346350
stat.tuple_count++;
@@ -351,7 +355,7 @@ pgstat_heap(Relation rel, FunctionCallInfo fcinfo)
351355
stat.dead_tuple_count++;
352356
}
353357

354-
LockBuffer(scan->rs_cbuf, BUFFER_LOCK_UNLOCK);
358+
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
355359

356360
/*
357361
* To avoid physically reading the table twice, try to do the
@@ -366,7 +370,7 @@ pgstat_heap(Relation rel, FunctionCallInfo fcinfo)
366370
CHECK_FOR_INTERRUPTS();
367371

368372
buffer = ReadBufferExtended(rel, MAIN_FORKNUM, block,
369-
RBM_NORMAL, scan->rs_strategy);
373+
RBM_NORMAL, hscan->rs_strategy);
370374
LockBuffer(buffer, BUFFER_LOCK_SHARE);
371375
stat.free_space += PageGetHeapFreeSpace((Page) BufferGetPage(buffer));
372376
UnlockReleaseBuffer(buffer);
@@ -379,14 +383,14 @@ pgstat_heap(Relation rel, FunctionCallInfo fcinfo)
379383
CHECK_FOR_INTERRUPTS();
380384

381385
buffer = ReadBufferExtended(rel, MAIN_FORKNUM, block,
382-
RBM_NORMAL, scan->rs_strategy);
386+
RBM_NORMAL, hscan->rs_strategy);
383387
LockBuffer(buffer, BUFFER_LOCK_SHARE);
384388
stat.free_space += PageGetHeapFreeSpace((Page) BufferGetPage(buffer));
385389
UnlockReleaseBuffer(buffer);
386390
block++;
387391
}
388392

389-
heap_endscan(scan);
393+
table_endscan(scan);
390394
relation_close(rel, AccessShareLock);
391395

392396
stat.table_len = (uint64) nblocks * BLCKSZ;

‎contrib/tsm_system_rows/tsm_system_rows.c

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,8 @@ static BlockNumber
209209
system_rows_nextsampleblock(SampleScanState *node)
210210
{
211211
SystemRowsSamplerData *sampler = (SystemRowsSamplerData *) node->tsm_state;
212-
HeapScanDesc scan = node->ss.ss_currentScanDesc;
212+
TableScanDesc scan = node->ss.ss_currentScanDesc;
213+
HeapScanDesc hscan = (HeapScanDesc) scan;
213214

214215
/* First call within scan? */
215216
if (sampler->doneblocks == 0)
@@ -221,14 +222,14 @@ system_rows_nextsampleblock(SampleScanState *node)
221222
SamplerRandomState randstate;
222223

223224
/* If relation is empty, there's nothing to scan */
224-
if (scan->rs_nblocks == 0)
225+
if (hscan->rs_nblocks == 0)
225226
return InvalidBlockNumber;
226227

227228
/* We only need an RNG during this setup step */
228229
sampler_random_init_state(sampler->seed, randstate);
229230

230231
/* Compute nblocks/firstblock/step only once per query */
231-
sampler->nblocks = scan->rs_nblocks;
232+
sampler->nblocks = hscan->rs_nblocks;
232233

233234
/* Choose random starting block within the relation */
234235
/* (Actually this is the predecessor of the first block visited) */
@@ -258,7 +259,7 @@ system_rows_nextsampleblock(SampleScanState *node)
258259
{
259260
/* Advance lb, using uint64 arithmetic to forestall overflow */
260261
sampler->lb = ((uint64) sampler->lb + sampler->step) % sampler->nblocks;
261-
} while (sampler->lb >= scan->rs_nblocks);
262+
} while (sampler->lb >= hscan->rs_nblocks);
262263

263264
return sampler->lb;
264265
}
@@ -278,7 +279,8 @@ system_rows_nextsampletuple(SampleScanState *node,
278279
OffsetNumber maxoffset)
279280
{
280281
SystemRowsSamplerData *sampler = (SystemRowsSamplerData *) node->tsm_state;
281-
HeapScanDesc scan = node->ss.ss_currentScanDesc;
282+
TableScanDesc scan = node->ss.ss_currentScanDesc;
283+
HeapScanDesc hscan = (HeapScanDesc) scan;
282284
OffsetNumber tupoffset = sampler->lt;
283285

284286
/* Quit if we've returned all needed tuples */
@@ -308,7 +310,7 @@ system_rows_nextsampletuple(SampleScanState *node,
308310
}
309311

310312
/* Found a candidate? */
311-
if (SampleOffsetVisible(tupoffset, scan))
313+
if (SampleOffsetVisible(tupoffset, hscan))
312314
{
313315
sampler->donetuples++;
314316
break;

‎contrib/tsm_system_time/tsm_system_time.c

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,8 @@ static BlockNumber
216216
system_time_nextsampleblock(SampleScanState *node)
217217
{
218218
SystemTimeSamplerData *sampler = (SystemTimeSamplerData *) node->tsm_state;
219-
HeapScanDesc scan = node->ss.ss_currentScanDesc;
219+
TableScanDesc scan = node->ss.ss_currentScanDesc;
220+
HeapScanDesc hscan = (HeapScanDesc) scan;
220221
instr_time cur_time;
221222

222223
/* First call within scan? */
@@ -229,14 +230,14 @@ system_time_nextsampleblock(SampleScanState *node)
229230
SamplerRandomState randstate;
230231

231232
/* If relation is empty, there's nothing to scan */
232-
if (scan->rs_nblocks == 0)
233+
if (hscan->rs_nblocks == 0)
233234
return InvalidBlockNumber;
234235

235236
/* We only need an RNG during this setup step */
236237
sampler_random_init_state(sampler->seed, randstate);
237238

238239
/* Compute nblocks/firstblock/step only once per query */
239-
sampler->nblocks = scan->rs_nblocks;
240+
sampler->nblocks = hscan->rs_nblocks;
240241

241242
/* Choose random starting block within the relation */
242243
/* (Actually this is the predecessor of the first block visited) */
@@ -272,7 +273,7 @@ system_time_nextsampleblock(SampleScanState *node)
272273
{
273274
/* Advance lb, using uint64 arithmetic to forestall overflow */
274275
sampler->lb = ((uint64) sampler->lb + sampler->step) % sampler->nblocks;
275-
} while (sampler->lb >= scan->rs_nblocks);
276+
} while (sampler->lb >= hscan->rs_nblocks);
276277

277278
return sampler->lb;
278279
}

‎src/backend/access/gist/gistget.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -561,7 +561,7 @@ getNextNearest(IndexScanDesc scan)
561561
if (GISTSearchItemIsHeap(*item))
562562
{
563563
/* found a heap item at currently minimal distance */
564-
scan->xs_ctup.t_self = item->data.heap.heapPtr;
564+
scan->xs_heaptid = item->data.heap.heapPtr;
565565
scan->xs_recheck = item->data.heap.recheck;
566566

567567
index_store_float8_orderby_distances(scan, so->orderByTypes,
@@ -650,7 +650,7 @@ gistgettuple(IndexScanDesc scan, ScanDirection dir)
650650
so->pageData[so->curPageData - 1].offnum;
651651
}
652652
/* continuing to return tuples from a leaf page */
653-
scan->xs_ctup.t_self = so->pageData[so->curPageData].heapPtr;
653+
scan->xs_heaptid = so->pageData[so->curPageData].heapPtr;
654654
scan->xs_recheck = so->pageData[so->curPageData].recheck;
655655

656656
/* in an index-only scan, also return the reconstructed tuple */

‎src/backend/access/hash/hashsearch.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ _hash_next(IndexScanDesc scan, ScanDirection dir)
119119

120120
/* OK, itemIndex says what to return */
121121
currItem = &so->currPos.items[so->currPos.itemIndex];
122-
scan->xs_ctup.t_self = currItem->heapTid;
122+
scan->xs_heaptid = currItem->heapTid;
123123

124124
return true;
125125
}
@@ -432,7 +432,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
432432

433433
/* OK, itemIndex says what to return */
434434
currItem = &so->currPos.items[so->currPos.itemIndex];
435-
scan->xs_ctup.t_self = currItem->heapTid;
435+
scan->xs_heaptid = currItem->heapTid;
436436

437437
/* if we're here, _hash_readpage found a valid tuples */
438438
return true;

0 commit comments

Comments
 (0)