Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 2d67342

Browse files
committed
Improve user-facing documentation for partial/parallel aggregation.
Add a section to xaggr.sgml, as we have done in the past for other extensions to the aggregation functionality. Assorted wordsmithing and other minor improvements. David Rowley and Tom Lane
1 parent 63ae052 commit 2d67342

File tree

2 files changed

+142
-29
lines changed

2 files changed

+142
-29
lines changed

doc/src/sgml/ref/create_aggregate.sgml

+44-27
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,8 @@ CREATE AGGREGATE <replaceable class="parameter">name</replaceable> ( [ [ <replac
5050
[ , FINALFUNC = <replaceable class="PARAMETER">ffunc</replaceable> ]
5151
[ , FINALFUNC_EXTRA ]
5252
[ , INITCOND = <replaceable class="PARAMETER">initial_condition</replaceable> ]
53-
[ , HYPOTHETICAL ]
5453
[ , PARALLEL = { SAFE | RESTRICTED | UNSAFE } ]
55-
54+
[ , HYPOTHETICAL ]
5655
)
5756

5857
<phrase>or the old syntax</phrase>
@@ -221,6 +220,17 @@ CREATE AGGREGATE <replaceable class="PARAMETER">name</replaceable> (
221220
aggregate-input rows as an additional <quote>hypothetical</> row.
222221
</para>
223222

223+
<para>
224+
An aggregate can optionally support <firstterm>partial aggregation</>,
225+
as described in <xref linkend="xaggr-partial-aggregates">.
226+
This requires specifying the <literal>COMBINEFUNC</> parameter.
227+
If the <replaceable class="PARAMETER">state_data_type</replaceable>
228+
is <type>internal</>, it's usually also appropriate to provide the
229+
<literal>SERIALFUNC</> and <literal>DESERIALFUNC</> parameters so that
230+
parallel aggregation is possible. Note that the aggregate must also be
231+
marked <literal>PARALLEL SAFE</> to enable parallel aggregation.
232+
</para>
233+
224234
<para>
225235
Aggregates that behave like <function>MIN</> or <function>MAX</> can
226236
sometimes be optimized by looking into an index instead of scanning every
@@ -408,12 +418,7 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
408418
<para>
409419
The <replaceable class="PARAMETER">combinefunc</replaceable> function
410420
may optionally be specified to allow the aggregate function to support
411-
partial aggregation. This is a prerequisite to allow the aggregate to
412-
participate in certain optimizations such as parallel aggregation.
413-
</para>
414-
415-
<para>
416-
If provided,
421+
partial aggregation. If provided,
417422
the <replaceable class="PARAMETER">combinefunc</replaceable> must
418423
combine two <replaceable class="PARAMETER">state_data_type</replaceable>
419424
values, each containing the result of aggregation over some subset of
@@ -422,20 +427,15 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
422427
represents the result of aggregating over both sets of inputs. This
423428
function can be thought of as
424429
an <replaceable class="PARAMETER">sfunc</replaceable>, where instead of
425-
acting upon individual input rows and adding these to the aggregate
426-
state, it adds another aggregate state to the aggregate state.
427-
Typically, it is not possible to define
428-
a <replaceable class="PARAMETER">combinefunc</replaceable> for aggregate
429-
functions that are sensitive to the order of the input values, since the
430-
relative ordering of the inputs that went into the subset states is
431-
indeterminate.
430+
acting upon an individual input row and adding it to the running
431+
aggregate state, it adds another aggregate state to the running state.
432432
</para>
433433

434434
<para>
435-
The <replaceable class="PARAMETER">combinefunc</replaceable> must accept
436-
two arguments of
435+
The <replaceable class="PARAMETER">combinefunc</replaceable> must be
436+
declared as taking two arguments of
437437
the <replaceable class="PARAMETER">state_data_type</replaceable> and
438-
return a value of
438+
returning a value of
439439
the <replaceable class="PARAMETER">state_data_type</replaceable>.
440440
Optionally this function may be <quote>strict</quote>. In this case the
441441
function will not be called when either of the input states are null;
@@ -446,11 +446,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
446446
For aggregate functions
447447
whose <replaceable class="PARAMETER">state_data_type</replaceable>
448448
is <type>internal</type>,
449-
the <replaceable class="PARAMETER">combinefunc</replaceable> must not be
450-
strict. In this scenario
451-
the <replaceable class="PARAMETER">combinefunc</replaceable> must ensure
452-
that null states are handled correctly and that the state being returned
453-
is properly stored in the aggregate memory context.
449+
the <replaceable class="PARAMETER">combinefunc</replaceable> must not
450+
be strict. In this case
451+
the <replaceable class="PARAMETER">combinefunc</replaceable> must
452+
ensure that null states are handled correctly and that the state being
453+
returned is properly stored in the aggregate memory context.
454454
</para>
455455
</listitem>
456456
</varlistentry>
@@ -586,6 +586,22 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
586586
</listitem>
587587
</varlistentry>
588588

589+
<varlistentry>
590+
<term><literal>PARALLEL</literal></term>
591+
<listitem>
592+
<para>
593+
The meanings of <literal>PARALLEL SAFE</>, <literal>PARALLEL
594+
RESTRICTED</>, and <literal>PARALLEL UNSAFE</> are the same as
595+
for <xref linkend="sql-createfunction">. An aggregate will not be
596+
considered for parallelization if it is marked <literal>PARALLEL
597+
UNSAFE</> (which is the default!) or <literal>PARALLEL RESTRICTED</>.
598+
Note that the parallel-safety markings of the aggregate's support
599+
functions are not consulted by the planner, only the marking of the
600+
aggregate itself.
601+
</para>
602+
</listitem>
603+
</varlistentry>
604+
589605
<varlistentry>
590606
<term><literal>HYPOTHETICAL</literal></term>
591607
<listitem>
@@ -686,10 +702,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
686702
</para>
687703

688704
<para>
689-
The meaning of <literal>PARALLEL SAFE</>, <literal>PARALLEL RESTRICTED</>,
690-
and <literal>PARALLEL UNSAFE</> is the same as for
691-
<xref linkend="sql-createfunction">.
692-
</para>
705+
Partial (including parallel) aggregation is currently not supported for
706+
ordered-set aggregates. Also, it will never be used for aggregate calls
707+
that include <literal>DISTINCT</> or <literal>ORDER BY</> clauses, since
708+
those semantics cannot be supported during partial aggregation.
709+
</para>
693710
</refsect1>
694711

695712
<refsect1>

doc/src/sgml/xaggr.sgml

+98-2
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
A <firstterm>final function</firstterm>
2424
can also be specified, in case the desired result of the aggregate
2525
is different from the data that needs to be kept in the running
26-
state value. The final function takes the last state value
26+
state value. The final function takes the ending state value
2727
and returns whatever is wanted as the aggregate result.
2828
In principle, the transition and final functions are just ordinary
2929
functions that could also be used outside the context of the
@@ -509,6 +509,102 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households;
509509
and therefore there is no need for them to support moving-aggregate mode.
510510
</para>
511511

512+
</sect2>
513+
514+
<sect2 id="xaggr-partial-aggregates">
515+
<title>Partial Aggregation</title>
516+
517+
<indexterm>
518+
<primary>aggregate function</primary>
519+
<secondary>partial aggregation</secondary>
520+
</indexterm>
521+
522+
<para>
523+
Optionally, an aggregate function can support <firstterm>partial
524+
aggregation</>. The idea of partial aggregation is to run the aggregate's
525+
state transition function over different subsets of the input data
526+
independently, and then to combine the state values resulting from those
527+
subsets to produce the same state value that would have resulted from
528+
scanning all the input in a single operation. This mode can be used for
529+
parallel aggregation by having different worker processes scan different
530+
portions of a table. Each worker produces a partial state value, and at
531+
the end those state values are combined to produce a final state value.
532+
(In the future this mode might also be used for purposes such as combining
533+
aggregations over local and remote tables; but that is not implemented
534+
yet.)
535+
</para>
536+
537+
<para>
538+
To support partial aggregation, the aggregate definition must provide
539+
a <firstterm>combine function</>, which takes two values of the
540+
aggregate's state type (representing the results of aggregating over two
541+
subsets of the input rows) and produces a new value of the state type,
542+
representing what the state would have been after aggregating over the
543+
combination of those sets of rows. It is unspecified what the relative
544+
order of the input rows from the two sets would have been. This means
545+
that it's usually impossible to define a useful combine function for
546+
aggregates that are sensitive to input row order.
547+
</para>
548+
549+
<para>
550+
As simple examples, <literal>MAX</> and <literal>MIN</> aggregates can be
551+
made to support partial aggregation by specifying the combine function as
552+
the same greater-of-two or lesser-of-two comparison function that is used
553+
as their transition function. <literal>SUM</> aggregates just need an
554+
addition function as combine function. (Again, this is the same as their
555+
transition function, unless the state value is wider than the input data
556+
type.)
557+
</para>
558+
559+
<para>
560+
The combine function is treated much like a transition function that
561+
happens to take a value of the state type, not of the underlying input
562+
type, as its second argument. In particular, the rules for dealing
563+
with null values and strict functions are similar. Also, if the aggregate
564+
definition specifies a non-null <literal>initcond</>, keep in mind that
565+
that will be used not only as the initial state for each partial
566+
aggregation run, but also as the initial state for the combine function,
567+
which will be called to combine each partial result into that state.
568+
</para>
569+
570+
<para>
571+
If the aggregate's state type is declared as <type>internal</>, it is
572+
the combine function's responsibility that its result is allocated in
573+
the correct memory context for aggregate state values. This means in
574+
particular that when the first input is <literal>NULL</> it's invalid
575+
to simply return the second input, as that value will be in the wrong
576+
context and will not have sufficient lifespan.
577+
</para>
578+
579+
<para>
580+
When the aggregate's state type is declared as <type>internal</>, it is
581+
usually also appropriate for the aggregate definition to provide a
582+
<firstterm>serialization function</> and a <firstterm>deserialization
583+
function</>, which allow such a state value to be copied from one process
584+
to another. Without these functions, parallel aggregation cannot be
585+
performed, and future applications such as local/remote aggregation will
586+
probably not work either.
587+
</para>
588+
589+
<para>
590+
A serialization function must take a single argument of
591+
type <type>internal</> and return a result of type <type>bytea</>, which
592+
represents the state value packaged up into a flat blob of bytes.
593+
Conversely, a deserialization function reverses that conversion. It must
594+
take two arguments of types <type>bytea</> and <type>internal</>, and
595+
return a result of type <type>internal</>. (The second argument is unused
596+
and is always zero, but it is required for type-safety reasons.) The
597+
result of the deserialization function should simply be allocated in the
598+
current memory context, as unlike the combine function's result, it is not
599+
long-lived.
600+
</para>
601+
602+
<para>
603+
Worth noting also is that for an aggregate to be executed in parallel,
604+
the aggregate itself must be marked <literal>PARALLEL SAFE</>. The
605+
parallel-safety markings on its support functions are not consulted.
606+
</para>
607+
512608
</sect2>
513609

514610
<sect2 id="xaggr-support-functions">
@@ -521,7 +617,7 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households;
521617

522618
<para>
523619
A function written in C can detect that it is being called as an
524-
aggregate transition or final function by calling
620+
aggregate support function by calling
525621
<function>AggCheckCallContext</>, for example:
526622
<programlisting>
527623
if (AggCheckCallContext(fcinfo, NULL))

0 commit comments

Comments
 (0)