Fix memory leak in Incremental Sort rescans

tvondra · tvondra · commit c1affa38c73b · 2023-07-02T20:05:14.000+02:00
The Incremental Sort had a couple issues, resulting in leaking memory during rescans, possibly triggering OOM. The code had a couple of related flaws: 1. During rescans, the sort states were reset but then also set to NULL (despite the comment saying otherwise). ExecIncrementalSort then sees NULL and initializes a new sort state, leaking the memory used by the old one. 2. Initializing the sort state also automatically rebuilt the info about presorted keys, leaking the already initialized info. presorted_keys was also unnecessarily reset to NULL. Patch by James Coleman, based on patches by Laurenz Albe and Tom Lane. Backpatch to 13, where Incremental Sort was introduced. Author: James Coleman, Laurenz Albe, Tom Lane Reported-by: Laurenz Albe, Zu-Ming Jiang Backpatch-through: 13 Discussion: https://postgr.es/m/b2bd02dff61af15e3526293e2771f874cf2a3be7.camel%40cybertec.at Discussion: https://postgr.es/m/db03c582-086d-e7cd-d4a1-3bc722f81765%40inf.ethz.ch
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
@@ -1138,7 +1138,6 @@ ExecReScanIncrementalSort(IncrementalSortState *node)
 	node->outerNodeDone = false;
 	node->n_fullsort_remaining = 0;
 	node->bound_Done = 0;
-	node->presorted_keys = NULL;
 
 	node->execution_status = INCSORT_LOADFULLSORT;
 
@@ -1151,15 +1150,9 @@ ExecReScanIncrementalSort(IncrementalSortState *node)
 	 * cause a leak.
 	 */
 	if (node->fullsort_state != NULL)
-	{
 		tuplesort_reset(node->fullsort_state);
-		node->fullsort_state = NULL;
-	}
 	if (node->prefixsort_state != NULL)
-	{
 		tuplesort_reset(node->prefixsort_state);
-		node->prefixsort_state = NULL;
-	}
 
 	/*
 	 * If chgParam of subnode is not null, then the plan will be re-scanned by
