Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 425a973

Browse files
vbwagnervbwagner
committed
Merge branch 'PGPROEE10-pgpro-791' into PGPROEE10
2 parents 8e9f49e + 34bb18e commit 425a973

File tree

19 files changed

+1164
-2
lines changed

19 files changed

+1164
-2
lines changed

‎doc/src/sgml/config.sgml

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2204,6 +2204,130 @@ include_dir 'conf.d'
22042204
</varlistentry>
22052205
</variablelist>
22062206
</sect2>
2207+
2208+
<sect2 id="runtime-config-resource-prioritization">
2209+
<title>Prioritization</title>
2210+
<variablelist>
2211+
<varlistentry id="guc-usage-tracking-interval" xreflabel="usage_tracking_interval">
2212+
<term><varname>usage_tracking_interval</varname> (<type>integer</type>)
2213+
<indexterm>
2214+
<primary><varname>usage_tracking_interval</> configuration parameter</primary>
2215+
</indexterm>
2216+
</term>
2217+
<listitem>
2218+
<para>
2219+
Sets the time interval, in seconds, for calculating usage statistics.
2220+
Based on this statistics, <productname>&productname;</productname>
2221+
can control resource usage for each session
2222+
in accordance with the prioritization policy configured by
2223+
<xref linkend="guc-session-cpu-weight">,
2224+
<xref linkend="guc-session-ioread-weight">,
2225+
and <xref linkend="guc-session-iowrite-weight"> parameters.
2226+
</para>
2227+
<para>
2228+
When set to a positive value, this parameter starts a
2229+
background worker that collects statistics on CPU time, the number of
2230+
local and shared blocks read by the backends, and the number of local
2231+
and shared blocks dirtied by the backends.
2232+
Avoid setting this parameter to a small value as frequent
2233+
statistic collection can cause overhead.
2234+
</para>
2235+
<para>
2236+
The default value is zero, which disables statistics collection
2237+
and resource prioritization.
2238+
</para>
2239+
<para>
2240+
This parameter can only be set in the <filename>postgresql.conf</>
2241+
file or on the server command line.
2242+
</para>
2243+
</listitem>
2244+
</varlistentry>
2245+
2246+
<varlistentry id="guc-session-cpu-weight" xreflabel="session_cpu_weight">
2247+
<term><varname>session_cpu_weight</varname> (<type>integer</type>)
2248+
<indexterm>
2249+
<primary><varname>session_cpu_weight</> configuration parameter</primary>
2250+
</indexterm>
2251+
</term>
2252+
<listitem>
2253+
<para>
2254+
Sets CPU usage weight for the current session.
2255+
Possible values are <literal>1</>, <literal>2</>, <literal>4</>, and
2256+
<literal>8</>.
2257+
The higher the value, the more resources the session can use
2258+
as compared to sessions with lower weights.
2259+
</para>
2260+
<para>
2261+
The <xref linkend="guc-usage-tracking-interval"> parameter must be set
2262+
to a positive value for this setting to take effect. Resource usage is
2263+
planned based on the statistics collected for the previous interval
2264+
defined by <xref linkend="guc-usage-tracking-interval">. If all sessions
2265+
have the same weight, <productname>&productname;</productname>
2266+
does not prioritize resource usage.
2267+
</para>
2268+
<para>
2269+
Default: <literal>4</literal>
2270+
</para>
2271+
</listitem>
2272+
</varlistentry>
2273+
2274+
<varlistentry id="guc-session-ioread-weight" xreflabel="session_ioread_priority">
2275+
<term><varname>session_ioread_priority</varname> (<type>integer</type>)
2276+
<indexterm>
2277+
<primary><varname>session_ioread_priority</> configuration parameter</primary>
2278+
</indexterm>
2279+
</term>
2280+
<listitem>
2281+
<para>
2282+
Sets the weight for reading local and shared blocks for the current
2283+
session. Possible values are <literal>1</>, <literal>2</>,
2284+
<literal>4</>, and <literal>8</>.
2285+
The higher the value, the more resources the session can use
2286+
as compared to sessions with lower weights.
2287+
</para>
2288+
<para>
2289+
The <xref linkend="guc-usage-tracking-interval"> parameter must be set
2290+
to a positive value for this setting to take effect. Resource usage is
2291+
planned based on the statistics collected for the previous interval
2292+
defined by <xref linkend="guc-usage-tracking-interval">. If all sessions
2293+
have the same weight, <productname>&productname;</productname>
2294+
does not prioritize resource usage.
2295+
</para>
2296+
<para>
2297+
Default: <literal>4</literal>
2298+
</para>
2299+
</listitem>
2300+
</varlistentry>
2301+
2302+
<varlistentry id="guc-session-iowrite-weight" xreflabel="session_iowrite_weight">
2303+
<term><varname>session_iowrite_weight</varname> (<type>integer</type>)
2304+
<indexterm>
2305+
<primary><varname>session_iowrite_weight</> configuration parameter</primary>
2306+
</indexterm>
2307+
</term>
2308+
<listitem>
2309+
<para>
2310+
Sets the weight for writing to local and shared blocks for the current
2311+
session. Possible values are <literal>1</>, <literal>2</>,
2312+
<literal>4</>, and <literal>8</>.
2313+
The higher the value, the more resources the session can use
2314+
as compared to sessions with lower weights.
2315+
</para>
2316+
<para>
2317+
The <xref linkend="guc-usage-tracking-interval"> parameter must be set
2318+
to a positive value for this setting to take effect. Resource usage is
2319+
planned based on the statistics collected for the previous interval
2320+
defined by <xref linkend="guc-usage-tracking-interval">. If all sessions
2321+
have the same weight, <productname>&productname;</productname>
2322+
does not prioritize resource usage.
2323+
</para>
2324+
<para>
2325+
Default: <literal>4</literal>
2326+
</para>
2327+
</listitem>
2328+
</varlistentry>
2329+
</variablelist>
2330+
</sect2>
22072331
</sect1>
22082332

22092333
<sect1 id="runtime-config-wal">

‎doc/src/sgml/runtime.sgml

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1538,6 +1538,114 @@ $ <userinput>grep Huge /proc/meminfo</userinput>
15381538
</para>
15391539

15401540
</sect2>
1541+
1542+
<sect2 id="resource-prioritization">
1543+
<title>Resource Prioritization</title>
1544+
1545+
<para>
1546+
On systems with limited resources or under heavy load, you may
1547+
need to prioritize transaction execution, so that some transactions
1548+
are executed more quickly than the other. For example, you may want to
1549+
execute simple user queries as fast as possible, even if it delays
1550+
less urgent tasks, such as complex <acronym>OLAP</acronym>
1551+
queries that may be running at the same time.
1552+
<productname>&productname;</productname> enables you to configure
1553+
resource prioritization policy, which can slow down a particular
1554+
session based on the amount of CPU, I/O read, and I/O write
1555+
resources this session consumes as compared to other sessions.
1556+
</para>
1557+
1558+
<para>
1559+
By default, resource prioritization is disabled, so all backends
1560+
have equal access to all the available resources. You can assign weight
1561+
to each backend to control the amount of resources each session
1562+
can use within the specified time interval. Depending on the current
1563+
resource consumption, <productname>&productname;</productname>
1564+
may suspend backends with lower weight from time to time to ensure that
1565+
high-priority sessions have more resources available.
1566+
</para>
1567+
1568+
<para>
1569+
To enable prioritization in your database cluster:
1570+
</para>
1571+
<orderedlist>
1572+
<listitem>
1573+
<para>
1574+
Configure the time interval for collecting usage statistics for all
1575+
active backends using the <xref linkend="guc-usage-tracking-interval">
1576+
parameter. For example, to set this interval to 20 seconds, run:
1577+
<programlisting>
1578+
ALTER SYSTEM SET usage_tracking_interval = 20;
1579+
SELECT pg_reload_conf();
1580+
</programlisting>
1581+
</para>
1582+
<para>
1583+
Once the <xref linkend="guc-usage-tracking-interval"> parameter is set,
1584+
<productname>&productname;</productname> starts collecting statistics
1585+
on resource usage at the specified interval.
1586+
</para>
1587+
<tip>
1588+
<para>
1589+
Avoid setting <xref linkend="guc-usage-tracking-interval"> to small
1590+
values as frequent statistics collection can cause overhead.
1591+
</para>
1592+
</tip>
1593+
</listitem>
1594+
<listitem>
1595+
<para>
1596+
Modify weight values for one or more clients to prioritize
1597+
the corresponding sessions by the resources you would like to control
1598+
using the following parameters:
1599+
<itemizedlist>
1600+
<listitem>
1601+
<para>
1602+
<xref linkend="guc-session-cpu-weight"> &mdash; CPU usage.
1603+
</para>
1604+
</listitem>
1605+
<listitem>
1606+
<para>
1607+
<xref linkend="guc-session-ioread-weight"> &mdash;
1608+
I/O read throughput.
1609+
</para>
1610+
</listitem>
1611+
<listitem>
1612+
<para>
1613+
<xref linkend="guc-session-iowrite-weight"> &mdash;
1614+
I/O write throughput.
1615+
</para>
1616+
</listitem>
1617+
</itemizedlist>
1618+
</para>
1619+
<para>
1620+
Possible weight values are <literal>1</>, <literal>2</>, <literal>4</>,
1621+
and <literal>8</>. The higher the value, the more resources
1622+
the session can use. Sessions with the same weight
1623+
have the same priority for resource usage, so if you assign
1624+
equal weights to all sessions, performance is not affected, regardless of the weight value.
1625+
By default, all sessions have weight <literal>4</> for all types
1626+
of resources.
1627+
</para>
1628+
</listitem>
1629+
</orderedlist>
1630+
1631+
<para>
1632+
Once you change the weight of one or more sessions,
1633+
<productname>&productname;</productname> enables
1634+
prioritization policy based on the assigned weight values and
1635+
the usage statistics measured for the previous
1636+
<xref linkend="guc-usage-tracking-interval">. Thus, session
1637+
activity is adjusted for each usage tracking interval, if required.
1638+
</para>
1639+
1640+
<note>
1641+
<para>
1642+
Even though weights for each resource are assigned separately,
1643+
prioritizing a session by one resource can indirectly affect the
1644+
session performance with regard to other resources.
1645+
</para>
1646+
</note>
1647+
1648+
</sect2>
15411649
</sect1>
15421650

15431651

‎src/backend/postmaster/bgworker.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
#include "storage/procsignal.h"
3535
#include "storage/shmem.h"
3636
#include "storage/cfs.h"
37+
#include "tcop/priority.h"
3738
#include "tcop/tcopprot.h"
3839
#include "utils/ascii.h"
3940
#include "utils/ps_status.h"
@@ -137,6 +138,9 @@ static const struct
137138
},
138139
{
139140
"cfs_gc_bgworker_main", cfs_gc_bgworker_main
141+
},
142+
{
143+
"PriorityWorkerMain", PriorityWorkerMain
140144
}
141145
};
142146

‎src/backend/postmaster/pgstat.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3497,6 +3497,9 @@ pgstat_get_wait_activity(WaitEventActivity w)
34973497
case WAIT_EVENT_PGSTAT_MAIN:
34983498
event_name = "PgStatMain";
34993499
break;
3500+
case WAIT_EVENT_PRIORITY_MAIN:
3501+
event_name = "PriorityMain";
3502+
break;
35003503
case WAIT_EVENT_RECOVERY_WAL_ALL:
35013504
event_name = "RecoveryWalAll";
35023505
break;

‎src/backend/storage/ipc/ipci.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
#include "storage/sinvaladt.h"
5050
#include "storage/spin.h"
5151
#include "storage/cfs.h"
52+
#include "tcop/priority.h"
5253
#include "utils/backend_random.h"
5354
#include "utils/snapmgr.h"
5455

@@ -158,6 +159,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
158159
size = add_size(size, AsyncShmemSize());
159160
size = add_size(size, BackendRandomShmemSize());
160161
size = add_size(size, cfs_shmem_size());
162+
size = add_size(size, PriorityWorkerShmemSize());
161163
#ifdef EXEC_BACKEND
162164
size = add_size(size, ShmemBackendArraySize());
163165
#endif
@@ -280,6 +282,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
280282
AsyncShmemInit();
281283
BackendRandomShmemInit();
282284
cfs_initialize();
285+
PriorityWorkerShmemInit();
283286

284287
/*
285288
* GUC will be here later. --stas

‎src/backend/storage/ipc/procarray.c

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4086,3 +4086,45 @@ KnownAssignedXidsReset(void)
40864086

40874087
LWLockRelease(ProcArrayLock);
40884088
}
4089+
4090+
/*
4091+
* Return sum of ru_diff and cpu_priority_value.
4092+
*/
4093+
void
4094+
ProcArrayGetPriorityUsageTotals(uint64 *spent_sum, uint64 *blks_read_sum,
4095+
uint64 *blks_write_sum,
4096+
uint32 *cpu_priority_sum,
4097+
uint32 *ioread_priority_sum,
4098+
uint32 *iowrite_priority_sum)
4099+
{
4100+
ProcArrayStruct *arrayP = procArray;
4101+
int i;
4102+
4103+
*spent_sum = 0;
4104+
*blks_read_sum = 0;
4105+
*blks_write_sum = 0;
4106+
*cpu_priority_sum = 0;
4107+
*ioread_priority_sum = 0;
4108+
*iowrite_priority_sum = 0;
4109+
4110+
LWLockAcquire(ProcArrayLock, LW_SHARED);
4111+
4112+
for (i = 0; i < arrayP->numProcs; i++)
4113+
{
4114+
int pgprocno = arrayP->pgprocnos[i];
4115+
volatile PGPROC *proc = &allProcs[pgprocno];
4116+
4117+
*spent_sum += pg_atomic_read_u64(&proc->priorityStats.ru_spent);
4118+
*blks_read_sum += pg_atomic_read_u64(&proc->priorityStats.blks_read);
4119+
*blks_write_sum += pg_atomic_read_u64(&proc->priorityStats.blks_write);
4120+
4121+
*cpu_priority_sum += pg_atomic_read_u32(
4122+
&proc->priorityStats.cpu_priority_value);
4123+
*ioread_priority_sum += pg_atomic_read_u32(
4124+
&proc->priorityStats.ioread_priority_value);
4125+
*iowrite_priority_sum += pg_atomic_read_u32(
4126+
&proc->priorityStats.iowrite_priority_value);
4127+
}
4128+
4129+
LWLockRelease(ProcArrayLock);
4130+
}

‎src/backend/storage/lmgr/proc.c

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -372,6 +372,20 @@ InitProcess(void)
372372
MyPgXact->xmin = InvalidTransactionId;
373373
MyPgXact->parent = NULL;
374374
MyProc->pid = MyProcPid;
375+
376+
/* Initialize statistics for prioritizing */
377+
MemSet(&MyProc->priorityStats, 0, sizeof(PGPriorityStats));
378+
pg_atomic_init_u64(&MyProc->priorityStats.ru_spent, 0);
379+
pg_atomic_init_u64(&MyProc->priorityStats.blks_read, 0);
380+
pg_atomic_init_u64(&MyProc->priorityStats.blks_write, 0);
381+
pg_atomic_init_u32(&MyProc->priorityStats.cpu_priority_value, 0);
382+
pg_atomic_init_u32(&MyProc->priorityStats.ioread_priority_value, 0);
383+
pg_atomic_init_u32(&MyProc->priorityStats.iowrite_priority_value, 0);
384+
385+
MyProc->isAutovacuumLauncher = false;
386+
if (IsAutoVacuumLauncherProcess())
387+
MyProc->isAutovacuumLauncher = true;
388+
375389
/* backendId, databaseId and roleId will be filled in later */
376390
MyProc->backendId = InvalidBackendId;
377391
MyProc->databaseId = InvalidOid;

‎src/backend/tcop/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ subdir = src/backend/tcop
1212
top_builddir = ../../..
1313
include $(top_builddir)/src/Makefile.global
1414

15-
OBJS= dest.o fastpath.o postgres.o pquery.o utility.o
15+
OBJS= dest.o fastpath.o postgres.o pquery.o priority.o utility.o
1616

1717
ifneq (,$(filter $(PORTNAME),cygwin win32))
1818
override CPPFLAGS += -DWIN32_STACK_RLIMIT=$(WIN32_STACK_RLIMIT)

‎src/backend/tcop/postgres.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@
6969
#include "storage/procsignal.h"
7070
#include "storage/sinval.h"
7171
#include "tcop/fastpath.h"
72+
#include "tcop/priority.h"
7273
#include "tcop/pquery.h"
7374
#include "tcop/tcopprot.h"
7475
#include "tcop/utility.h"
@@ -3948,6 +3949,9 @@ PostgresMain(int argc, char *argv[],
39483949
pqsignal(SIGQUIT, die); /* cancel current query and exit */
39493950
InitializeTimeouts(); /* establishes SIGALRM handler */
39503951

3952+
if (IsUnderPostmaster)
3953+
RegisterTimeout(SESSION_USAGE_TIMEOUT, SessionUsageTimeoutHandler);
3954+
39513955
/*
39523956
* Ignore failure to write to frontend. Note: if frontend closes
39533957
* connection, we will notice it and exit cleanly when control next

0 commit comments

Comments
 (0)