Fix recently-exposed portability issue in regex optimization.

tglsfdc · tglsfdc · commit df1a2633b11a · 2024-11-17T14:14:06.000-05:00
fixempties() counts the number of in-arcs in the regex NFA and then allocates an array of that many arc pointers. If the NFA contains no arcs, this amounts to malloc(0) for which some platforms return NULL. The code mistakenly treats that as indicating out-of-memory. Thus, we can get a bogus "out of memory" failure for some unsatisfiable regexes. This happens only in v15 and earlier, since bea3d7e switched to using palloc() rather than bare malloc(). And at least of the platforms in the buildfarm, only AIX seems to return NULL. So the impact is pretty narrow. But I don't especially want to ship code that is failing its own regression tests, so let's fix this for this week's releases. A quick code survey says that there is only the one place in src/backend/regex/ that is at risk of doing malloc(0), so we'll just band-aid that place. A more future-proof fix could be to install a malloc() wrapper similar to pg_malloc(). But this code seems unlikely to change much more in the affected branches, so that's probably overkill. The only known test case for this involves a complemented character class in a bracket expression, for example [^\s\S], and we didn't support that in v13. So it may be that the problem is unreachable in v13. But I'm not 100% sure of that, so patch v13 too. Discussion: https://postgr.es/m/661fd81b-f069-8f30-1a41-e195c57449b4@gmail.com
diff --git a/src/backend/regex/regc_nfa.c b/src/backend/regex/regc_nfa.c
@@ -2172,9 +2172,12 @@ fixempties(struct nfa *nfa,
 	 * current target state.  totalinarcs is probably a considerable
 	 * overestimate of the space needed, but the NFA is unlikely to be large
 	 * enough at this point to make it worth being smarter.
+	 *
+	 * Note: totalinarcs could be zero, and some machines return NULL for
+	 * malloc(0).  Don't throw an error if so.
 	 */
 	arcarray = (struct arc **) MALLOC(totalinarcs * sizeof(struct arc *));
-	if (arcarray == NULL)
+	if (arcarray == NULL && totalinarcs != 0)
 	{
 		NERR(REG_ESPACE);
 		FREE(inarcsorig);
