Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit d1001a7

Browse files
committed
Reduce memory consumption inside inheritance_planner().
Avoid eating quite so much memory for large inheritance trees, by reclaiming the space used by temporary copies of the original parsetree and range table, as well as the workspace needed during planning. The cost is needing to copy the finished plan trees out of the child memory context. Although this looks like it ought to slow things down, my testing shows it actually is faster, apparently because fewer interactions with malloc() are needed and/or we can do the work within a more readily cacheable amount of memory. That result might be platform-dependent, but I'll take it. Per a gripe from John Papandriopoulos, in which it was pointed out that the memory consumption actually grew as O(N^2) for sufficiently many child tables, since we were creating N copies of the N-element range table.
1 parent d1f5a92 commit d1001a7

File tree

1 file changed

+53
-15
lines changed

1 file changed

+53
-15
lines changed

src/backend/optimizer/plan/planner.c

Lines changed: 53 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
#include "parser/parse_oper.h"
4242
#include "parser/parsetree.h"
4343
#include "utils/lsyscache.h"
44+
#include "utils/memutils.h"
4445
#include "utils/syscache.h"
4546

4647

@@ -741,19 +742,52 @@ inheritance_planner(PlannerInfo *root)
741742
List *rowMarks;
742743
List *tlist;
743744
PlannerInfo subroot;
745+
MemoryContext childcxt;
744746
ListCell *l;
745747

748+
/*
749+
* Memory management here is a bit messy, because the planning process can
750+
* scribble on both the query parsetree and the rangetable. We need to
751+
* ensure that each call of grouping_planner gets an un-scribbled-on copy
752+
* to start with, so we start by copying the given query tree each time
753+
* (as a byproduct of adjust_appendrel_attrs). However, we also want to
754+
* make sure that the modified rangetable ultimately gets propagated back
755+
* to the master copy, to pick up any changes of the Query structures
756+
* inside subquery RTEs. We do that by copying back the rangetable from
757+
* the first successful child planning step. (We are effectively assuming
758+
* that sub-Queries will get planned identically each time, or at least
759+
* that the impacts on their rangetables will be the same each time.)
760+
*
761+
* Another consideration is that when there are a lot of child tables, we
762+
* can eat a lot of memory this way. To fix that, we create a child
763+
* memory context that can be reset between steps to recover memory, at
764+
* the cost of having to copy the completed plan trees out to the parent
765+
* context.
766+
*/
767+
childcxt = AllocSetContextCreate(CurrentMemoryContext,
768+
"Inheritance child planning context",
769+
ALLOCSET_DEFAULT_MINSIZE,
770+
ALLOCSET_DEFAULT_INITSIZE,
771+
ALLOCSET_DEFAULT_MAXSIZE);
772+
746773
foreach(l, root->append_rel_list)
747774
{
748775
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
749776
Plan *subplan;
777+
MemoryContext oldcxt;
750778

751779
/* append_rel_list contains all append rels; ignore others */
752780
if (appinfo->parent_relid != parentRTindex)
753781
continue;
754782

755783
/*
756-
* Generate modified query with this rel as target.
784+
* Discard any cruft generated in previous loop iterations.
785+
*/
786+
MemoryContextReset(childcxt);
787+
oldcxt = MemoryContextSwitchTo(childcxt);
788+
789+
/*
790+
* Generate modified query (in childcxt) with this rel as target.
757791
*/
758792
memcpy(&subroot, root, sizeof(PlannerInfo));
759793
subroot.parse = (Query *)
@@ -767,24 +801,32 @@ inheritance_planner(PlannerInfo *root)
767801
/* and we haven't created PlaceHolderInfos, either */
768802
Assert(subroot.placeholder_list == NIL);
769803

770-
/* Generate plan */
804+
/* Generate plan (also in childcxt) */
771805
subplan = grouping_planner(&subroot, 0.0 /* retrieve all tuples */ );
772806

807+
MemoryContextSwitchTo(oldcxt);
808+
773809
/*
774810
* If this child rel was excluded by constraint exclusion, exclude it
775811
* from the plan.
776812
*/
777813
if (is_dummy_plan(subplan))
778814
continue;
779815

780-
/* Save rtable from first rel for use below */
816+
/*
817+
* Be sure to copy what we need out of the child context.
818+
*/
819+
subplan = copyObject(subplan);
820+
821+
/* Save rtable from first child to install in parent after the loop */
781822
if (subplans == NIL)
782-
rtable = subroot.parse->rtable;
823+
rtable = copyObject(subroot.parse->rtable);
783824

784825
subplans = lappend(subplans, subplan);
785826

786827
/* Make sure any initplans from this rel get into the outer list */
787-
root->init_plans = list_concat(root->init_plans, subroot.init_plans);
828+
root->init_plans = list_concat(root->init_plans,
829+
copyObject(subroot.init_plans));
788830

789831
/* Build target-relations list for the executor */
790832
resultRelations = lappend_int(resultRelations, appinfo->child_relid);
@@ -794,14 +836,18 @@ inheritance_planner(PlannerInfo *root)
794836
{
795837
List *rlist;
796838

839+
rlist = copyObject(subroot.parse->returningList);
797840
rlist = set_returning_clause_references(root->glob,
798-
subroot.parse->returningList,
841+
rlist,
799842
subplan,
800843
appinfo->child_relid);
801844
returningLists = lappend(returningLists, rlist);
802845
}
803846
}
804847

848+
/* Done with child context */
849+
MemoryContextDelete(childcxt);
850+
805851
root->resultRelations = resultRelations;
806852

807853
/* Mark result as unordered (probably unnecessary) */
@@ -824,15 +870,7 @@ inheritance_planner(PlannerInfo *root)
824870
}
825871

826872
/*
827-
* Planning might have modified the rangetable, due to changes of the
828-
* Query structures inside subquery RTEs. We have to ensure that this
829-
* gets propagated back to the master copy. But can't do this until we
830-
* are done planning, because all the calls to grouping_planner need
831-
* virgin sub-Queries to work from. (We are effectively assuming that
832-
* sub-Queries will get planned identically each time, or at least that
833-
* the impacts on their rangetables will be the same each time.)
834-
*
835-
* XXX should clean this up someday
873+
* Install modified rangetable from first child into parent parsetree.
836874
*/
837875
parse->rtable = rtable;
838876

0 commit comments

Comments
 (0)