Fix pg_dump handling of extension config tables

sfrost · sfrost · commit 43d81f16a393 · 2015-03-02T14:12:33.000-05:00
Since 9.1, we've provided extensions with a way to denote
"configuration" tables- tables created by an extension which the user
may modify.  By marking these as "configuration" tables, the extension
is asking for the data in these tables to be pg_dump'd (tables which
are not marked in this way are assumed to be entirely handled during
CREATE EXTENSION and are not included at all in a pg_dump).

Unfortunately, pg_dump neglected to consider foreign key relationships
between extension configuration tables and therefore could end up
trying to reload the data in an order which would cause FK violations.

This patch teaches pg_dump about these dependencies, so that the data
dumped out is done so in the best order possible.  Note that there's no
way to handle circular dependencies, but those have yet to be seen in
the wild.

The release notes for this should include a caution to users that
existing pg_dump-based backups may be invalid due to this issue.  The
data is all there, but restoring from it will require extracting the
data for the configuration tables and then loading them in the correct
order by hand.

Discussed initially back in bug #6738, more recently brought up by
Gilles Darold, who provided an initial patch which was further reworked
by Michael Paquier.  Further modifications and documentation updates
by me.

Back-patch to 9.1 where we added the concept of extension configuration
tables.
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
@@ -721,6 +721,17 @@ SELECT pg_catalog.pg_extension_config_dump('my_config', 'WHERE NOT standard_entr
      a table as no longer a configuration table is to dissociate it from the
      extension with <command>ALTER EXTENSION ... DROP TABLE</>.
     </para>
+
+    <para>
+     Note that foreign key relationships between these tables will dictate the
+     order in which the tables are dumped out by pg_dump.  Specifically, pg_dump
+     will attempt to dump the referenced-by table before the referencing table.
+     As the foreign key relationships are set up at CREATE EXTENSION time (prior
+     to data being loaded into the tables) circular dependencies are not
+     supported.  When circular dependencies exist, the data will still be dumped
+     out but the dump will not be able to be restored directly and user
+     intervention will be required.
+    </para>
    </sect2>
 
    <sect2>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
@@ -14571,6 +14571,33 @@ dumpRule(Archive *fout, RuleInfo *rinfo)
 
 /*
  * getExtensionMembership --- obtain extension membership data
+ *
+ * There are three main parts to this process:
+ *
+ * 1. Identify objects which are members of extensions
+ *
+ *    Generally speaking, this is to mark them as *not* being dumped, as most
+ *    extension objects are created by the single CREATE EXTENSION command.
+ *    The one exception is binary upgrades with pg_upgrade will still dump the
+ *    non-table objects.
+ *
+ * 2. Identify and create dump records for extension configuration tables.
+ *
+ *    Extensions can mark tables as "configuration", which means that the user
+ *    is able and expected to modify those tables after the extension has been
+ *    loaded.  For these tables, we dump out only the data- the structure is
+ *    expected to be handled at CREATE EXTENSION time, including any indexes or
+ *    foriegn keys, which brings us to-
+ *
+ * 3. Record FK dependencies between configuration tables.
+ *
+ *    Due to the FKs being created at CREATE EXTENSION time and therefore before
+ *    the data is loaded, we have to work out what the best order for reloading
+ *    the data is, to avoid FK violations when the tables are restored.  This is
+ *    not perfect- we can't handle circular dependencies and if any exist they
+ *    will cause an invalid dump to be produced (though at least all of the data
+ *    is included for a user to manually restore).  This is currently documented
+ *    but perhaps we can provide a better solution in the future.
  */
 void
 getExtensionMembership(Archive *fout, ExtensionInfo extinfo[],
@@ -14583,7 +14610,9 @@ getExtensionMembership(Archive *fout, ExtensionInfo extinfo[],
 	int			i_classid,
 				i_objid,
 				i_refclassid,
-				i_refobjid;
+				i_refobjid,
+				i_conrelid,
+				i_confrelid;
 	DumpableObject *dobj,
 			   *refdobj;
 
@@ -14764,6 +14793,53 @@ getExtensionMembership(Archive *fout, ExtensionInfo extinfo[],
 			free(extconditionarray);
 	}
 
+	/*
+	 * Now that all the TableInfoData objects have been created for all
+	 * the extensions, check their FK dependencies and register them to
+	 * try and dump the data out in an order which they can be restored
+	 * in.
+	 *
+	 * Note that this is not a problem for user tables as their FKs are
+	 * recreated after the data has been loaded.
+	 */
+	printfPQExpBuffer(query,
+			"SELECT conrelid, confrelid "
+			"FROM pg_constraint "
+				"JOIN pg_depend ON (objid = confrelid) "
+			"WHERE contype = 'f' "
+			"AND refclassid = 'pg_extension'::regclass "
+			"AND classid = 'pg_class'::regclass;");
+
+	res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+	ntups = PQntuples(res);
+
+	i_conrelid = PQfnumber(res, "conrelid");
+	i_confrelid = PQfnumber(res, "confrelid");
+
+	/* Now get the dependencies and register them */
+	for (i = 0; i < ntups; i++)
+	{
+		Oid			conrelid, confrelid;
+		TableInfo  *reftable, *contable;
+
+		conrelid = atooid(PQgetvalue(res, i, i_conrelid));
+		confrelid = atooid(PQgetvalue(res, i, i_confrelid));
+		contable = findTableByOid(conrelid);
+		reftable = findTableByOid(confrelid);
+
+		if (reftable == NULL ||
+			reftable->dataObj == NULL ||
+			contable == NULL ||
+			contable->dataObj == NULL)
+			continue;
+
+		/*
+		 * Make referencing TABLE_DATA object depend on the
+		 * referenced table's TABLE_DATA object.
+		 */
+		addObjectDependency(&contable->dataObj->dobj,
+							reftable->dataObj->dobj.dumpId);
+	}
 	destroyPQExpBuffer(query);
 }
 
