Skip to content

Commit 5607e99

Browse files
anarazelanarazel
committed
Flush unlogged table's buffers when copying or moving databases.
CREATE DATABASE and ALTER DATABASE .. SET TABLESPACE copy the source database directory on the filesystem level. To ensure the on disk state is consistent they block out users of the affected database and force a checkpoint to flush out all data to disk. Unfortunately, up to now, that checkpoint didn't flush out dirty buffers from unlogged relations. That bug means there could be leftover dirty buffers in either the template database, or the database in its old location. Leading to problems when accessing relations in an inconsistent state; and to possible problems during shutdown in the SET TABLESPACE case because buffers belonging files that don't exist anymore are flushed. This was reported in bug #10675 by Maxim Boguk. Fix by Pavan Deolasee, modified somewhat by me. Reviewed by MauMau and Fujii Masao. Backpatch to 9.1 where unlogged tables were introduced.
1 parent 6e0a053 commit 5607e99

File tree

4 files changed

+30
-21
lines changed

4 files changed

+30
-21
lines changed

‎src/backend/access/transam/xlog.c

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7824,9 +7824,9 @@ LogCheckpointStart(int flags, bool restartpoint)
78247824
* the main message, but what about all the flags?
78257825
*/
78267826
if (restartpoint)
7827-
msg = "restartpoint starting:%s%s%s%s%s%s%s";
7827+
msg = "restartpoint starting:%s%s%s%s%s%s%s%s";
78287828
else
7829-
msg = "checkpoint starting:%s%s%s%s%s%s%s";
7829+
msg = "checkpoint starting:%s%s%s%s%s%s%s%s";
78307830

78317831
elog(LOG, msg,
78327832
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
@@ -7835,7 +7835,8 @@ LogCheckpointStart(int flags, bool restartpoint)
78357835
(flags & CHECKPOINT_FORCE) ? " force" : "",
78367836
(flags & CHECKPOINT_WAIT) ? " wait" : "",
78377837
(flags & CHECKPOINT_CAUSE_XLOG) ? " xlog" : "",
7838-
(flags & CHECKPOINT_CAUSE_TIME) ? " time" : "");
7838+
(flags & CHECKPOINT_CAUSE_TIME) ? " time" : "",
7839+
(flags & CHECKPOINT_FLUSH_ALL) ? " flush-all" :"");
78397840
}
78407841

78417842
/*

‎src/backend/commands/dbcommands.c

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -527,15 +527,17 @@ createdb(const CreatedbStmt *stmt)
527527
InvokeObjectPostCreateHook(DatabaseRelationId, dboid, 0);
528528

529529
/*
530-
* Force a checkpoint before starting the copy. This will force dirty
531-
* buffers out to disk, to ensure source database is up-to-date on disk
532-
* for the copy. FlushDatabaseBuffers() would suffice for that, but we
533-
* also want to process any pending unlink requests. Otherwise, if a
534-
* checkpoint happened while we're copying files, a file might be deleted
535-
* just when we're about to copy it, causing the lstat() call in copydir()
536-
* to fail with ENOENT.
530+
* Force a checkpoint before starting the copy. This will force all dirty
531+
* buffers, including those of unlogged tables, out to disk, to ensure
532+
* source database is up-to-date on disk for the copy.
533+
* FlushDatabaseBuffers() would suffice for that, but we also want
534+
* to process any pending unlink requests. Otherwise, if a checkpoint
535+
* happened while we're copying files, a file might be deleted just when
536+
* we're about to copy it, causing the lstat() call in copydir() to fail
537+
* with ENOENT.
537538
*/
538-
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE | CHECKPOINT_WAIT);
539+
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE | CHECKPOINT_WAIT
540+
| CHECKPOINT_FLUSH_ALL);
539541

540542
/*
541543
* Once we start copying subdirectories, we need to be able to clean 'em
@@ -1116,16 +1118,18 @@ movedb(const char *dbname, const char *tblspcname)
11161118
dst_dbpath = GetDatabasePath(db_id, dst_tblspcoid);
11171119

11181120
/*
1119-
* Force a checkpoint before proceeding. This will force dirty buffers out
1120-
* to disk, to ensure source database is up-to-date on disk for the copy.
1121+
* Force a checkpoint before proceeding. This will force all dirty
1122+
* buffers, including those of unlogged tables, out to disk, to ensure
1123+
* source database is up-to-date on disk for the copy.
11211124
* FlushDatabaseBuffers() would suffice for that, but we also want to
11221125
* process any pending unlink requests. Otherwise, the check for existing
11231126
* files in the target directory might fail unnecessarily, not to mention
11241127
* that the copy might fail due to source files getting deleted under it.
11251128
* On Windows, this also ensures that background procs don't hold any open
11261129
* files, which would cause rmdir() to fail.
11271130
*/
1128-
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE | CHECKPOINT_WAIT);
1131+
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE | CHECKPOINT_WAIT
1132+
| CHECKPOINT_FLUSH_ALL);
11291133

11301134
/*
11311135
* Check for existence of files in the target directory, i.e., objects of

‎src/backend/storage/buffer/bufmgr.c

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1204,9 +1204,10 @@ UnpinBuffer(volatile BufferDesc *buf, bool fixOwner)
12041204
*
12051205
* This is called at checkpoint time to write out all dirty shared buffers.
12061206
* The checkpoint request flags should be passed in. If CHECKPOINT_IMMEDIATE
1207-
* is set, we disable delays between writes; if CHECKPOINT_IS_SHUTDOWN is
1208-
* set, we write even unlogged buffers, which are otherwise skipped. The
1209-
* remaining flags currently have no effect here.
1207+
* is set, we disable delays between writes; if CHECKPOINT_IS_SHUTDOWN,
1208+
* CHECKPOINT_END_OF_RECOVERY or CHECKPOINT_FLUSH_ALL is set, we write even
1209+
* unlogged buffers, which are otherwise skipped. The remaining flags
1210+
* currently have no effect here.
12101211
*/
12111212
static void
12121213
BufferSync(int flags)
@@ -1221,11 +1222,12 @@ BufferSync(int flags)
12211222
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
12221223

12231224
/*
1224-
* Unless this is a shutdown checkpoint, we write only permanent, dirty
1225-
* buffers. But at shutdown or end of recovery, we write all dirty
1226-
* buffers.
1225+
* Unless this is a shutdown checkpoint or we have been explicitly told,
1226+
* we write only permanent, dirty buffers. But at shutdown or end of
1227+
* recovery, we write all dirty buffers.
12271228
*/
1228-
if (!((flags & CHECKPOINT_IS_SHUTDOWN) || (flags & CHECKPOINT_END_OF_RECOVERY)))
1229+
if (!((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
1230+
CHECKPOINT_FLUSH_ALL))))
12291231
mask |= BM_PERMANENT;
12301232

12311233
/*

‎src/include/access/xlog.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,8 @@ extern bool XLOG_DEBUG;
251251
/* These indicate the cause of a checkpoint request */
252252
#define CHECKPOINT_CAUSE_XLOG 0x0020 /* XLOG consumption */
253253
#define CHECKPOINT_CAUSE_TIME 0x0040 /* Elapsed time */
254+
#define CHECKPOINT_FLUSH_ALL 0x0080 /* Flush all pages, including those
255+
* belonging to unlogged tables */
254256

255257
/* Checkpoint statistics */
256258
typedef struct CheckpointStatsData

0 commit comments

Comments
 (0)