Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit b5810de

Browse files
committed
Reduce memory consumption for multi-statement query strings.
Previously, exec_simple_query always ran parse analysis, rewrite, and planning in MessageContext, allowing all the data generated thereby to persist until the end of processing of the whole query string. That's fine for single-command strings, but if a client sends many commands in a single simple-Query message, this strategy could result in annoying memory bloat, as complained of by Andreas Seltenreich. To fix, create a child context to do this work in, and reclaim it after each command. But we only do so for parsetrees that are not last in their query string. That avoids adding any memory management overhead for the typical case of a single-command string. Memory allocated for the last parsetree would be freed immediately after finishing the command string anyway. Similarly, adjust extension.c's execute_sql_string() to reclaim memory after each command. In that usage, multi-command strings are the norm, so it's a bit surprising that no one has yet complained of bloat --- especially since the bloat extended to whatever data ProcessUtility execution might leak. Amit Langote, reviewed by Julien Rouhaud Discussion: https://postgr.es/m/87ftp6l2qr.fsf@credativ.de
1 parent 909a7b6 commit b5810de

File tree

2 files changed

+41
-5
lines changed

2 files changed

+41
-5
lines changed

src/backend/commands/extension.c

+16
Original file line numberDiff line numberDiff line change
@@ -717,9 +717,21 @@ execute_sql_string(const char *sql)
717717
foreach(lc1, raw_parsetree_list)
718718
{
719719
RawStmt *parsetree = lfirst_node(RawStmt, lc1);
720+
MemoryContext per_parsetree_context,
721+
oldcontext;
720722
List *stmt_list;
721723
ListCell *lc2;
722724

725+
/*
726+
* We do the work for each parsetree in a short-lived context, to
727+
* limit the memory used when there are many commands in the string.
728+
*/
729+
per_parsetree_context =
730+
AllocSetContextCreate(CurrentMemoryContext,
731+
"execute_sql_string per-statement context",
732+
ALLOCSET_DEFAULT_SIZES);
733+
oldcontext = MemoryContextSwitchTo(per_parsetree_context);
734+
723735
/* Be sure parser can see any DDL done so far */
724736
CommandCounterIncrement();
725737

@@ -772,6 +784,10 @@ execute_sql_string(const char *sql)
772784

773785
PopActiveSnapshot();
774786
}
787+
788+
/* Clean up per-parsetree context. */
789+
MemoryContextSwitchTo(oldcontext);
790+
MemoryContextDelete(per_parsetree_context);
775791
}
776792

777793
/* Be sure to advance the command counter after the last script command */

src/backend/tcop/postgres.c

+25-5
Original file line numberDiff line numberDiff line change
@@ -1070,6 +1070,7 @@ exec_simple_query(const char *query_string)
10701070
bool snapshot_set = false;
10711071
const char *commandTag;
10721072
char completionTag[COMPLETION_TAG_BUFSIZE];
1073+
MemoryContext per_parsetree_context = NULL;
10731074
List *querytree_list,
10741075
*plantree_list;
10751076
Portal portal;
@@ -1132,10 +1133,25 @@ exec_simple_query(const char *query_string)
11321133
/*
11331134
* OK to analyze, rewrite, and plan this query.
11341135
*
1135-
* Switch to appropriate context for constructing querytrees (again,
1136-
* these must outlive the execution context).
1136+
* Switch to appropriate context for constructing query and plan trees
1137+
* (these can't be in the transaction context, as that will get reset
1138+
* when the command is COMMIT/ROLLBACK). If we have multiple
1139+
* parsetrees, we use a separate context for each one, so that we can
1140+
* free that memory before moving on to the next one. But for the
1141+
* last (or only) parsetree, just use MessageContext, which will be
1142+
* reset shortly after completion anyway. In event of an error, the
1143+
* per_parsetree_context will be deleted when MessageContext is reset.
11371144
*/
1138-
oldcontext = MemoryContextSwitchTo(MessageContext);
1145+
if (lnext(parsetree_item) != NULL)
1146+
{
1147+
per_parsetree_context =
1148+
AllocSetContextCreate(MessageContext,
1149+
"per-parsetree message context",
1150+
ALLOCSET_DEFAULT_SIZES);
1151+
oldcontext = MemoryContextSwitchTo(per_parsetree_context);
1152+
}
1153+
else
1154+
oldcontext = MemoryContextSwitchTo(MessageContext);
11391155

11401156
querytree_list = pg_analyze_and_rewrite(parsetree, query_string,
11411157
NULL, 0, NULL);
@@ -1160,8 +1176,8 @@ exec_simple_query(const char *query_string)
11601176

11611177
/*
11621178
* We don't have to copy anything into the portal, because everything
1163-
* we are passing here is in MessageContext, which will outlive the
1164-
* portal anyway.
1179+
* we are passing here is in MessageContext or the
1180+
* per_parsetree_context, and so will outlive the portal anyway.
11651181
*/
11661182
PortalDefineQuery(portal,
11671183
NULL,
@@ -1263,6 +1279,10 @@ exec_simple_query(const char *query_string)
12631279
* aborted by error will not send an EndCommand report at all.)
12641280
*/
12651281
EndCommand(completionTag, dest);
1282+
1283+
/* Now we may drop the per-parsetree context, if one was created. */
1284+
if (per_parsetree_context)
1285+
MemoryContextDelete(per_parsetree_context);
12661286
} /* end loop over parsetrees */
12671287

12681288
/*

0 commit comments

Comments
 (0)