From d1e027221d0243b7b57eabb0e482923dd7d1c8eb Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Tue, 16 Feb 2010 22:34:57 +0000 Subject: Replace the pg_listener-based LISTEN/NOTIFY mechanism with an in-memory queue. In addition, add support for a "payload" string to be passed along with each notify event. This implementation should be significantly more efficient than the old one, and is also more compatible with Hot Standby usage. There is not yet any facility for HS slaves to receive notifications generated on the master, although such a thing is possible in future. Joachim Wieland, reviewed by Jeff Davis; also hacked on by me. --- doc/src/sgml/catalogs.sgml | 69 +---------------------- doc/src/sgml/func.sgml | 18 +++++- doc/src/sgml/libpq.sgml | 52 +++++++++--------- doc/src/sgml/protocol.sgml | 23 ++++---- doc/src/sgml/ref/listen.sgml | 38 +++++++++---- doc/src/sgml/ref/notify.sgml | 121 +++++++++++++++++++++++++++++------------ doc/src/sgml/ref/unlisten.sgml | 19 ++++--- doc/src/sgml/storage.sgml | 7 ++- 8 files changed, 184 insertions(+), 163 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 3503bc852cd..e5bef18d7d8 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -1,4 +1,4 @@ - + @@ -168,11 +168,6 @@ metadata for large objects - - pg_listener - asynchronous notification support - - pg_namespace schemas @@ -3253,68 +3248,6 @@ - - <structname>pg_listener</structname> - - - pg_listener - - - - The catalog pg_listener supports the - and - - commands. A listener creates an entry in - pg_listener for each notification name - it is listening for. A notifier scans pg_listener - and updates each matching entry to show that a notification has occurred. - The notifier also sends a signal (using the PID recorded in the table) - to awaken the listener from sleep. - - - - <structname>pg_listener</> Columns - - - - - Name - Type - Description - - - - - - relname - name - - Notify condition name. (The name need not match any actual - relation in the database; the name relname is historical.) - - - - - listenerpid - int4 - PID of the server process that created this entry - - - - notification - int4 - - Zero if no event is pending for this listener. If an event is - pending, the PID of the server process that sent the notification - - - - -
- - - - <structname>pg_namespace</structname> diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 71952ee1fc4..54369711155 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -1,4 +1,4 @@ - + Functions and Operators @@ -11529,6 +11529,12 @@ postgres=# select * from unnest2(array[[1,2],[3,4]]); + + pg_listening_channels() + setof text + channel names that the session is currently listening on + + inet_client_addr() inet @@ -11674,6 +11680,16 @@ SET search_path TO schema , schema, .. + + pg_listening_channels + + + + pg_listening_channels returns a set of names of + channels that the current session is listening to. See for more information. + + inet_client_addr diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 4972a8c2592..0bdb6401c34 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -1,4 +1,4 @@ - + <application>libpq</application> - C Library @@ -307,28 +307,28 @@ Description - + - + disable only try a non-SSL connection - + allow first try a non-SSL connection; if that fails, try an SSL connection - + prefer (default) first try an SSL connection; if that fails, try a non-SSL connection - + require only try an SSL connection @@ -481,7 +481,7 @@ - If expand_dbname is non-zero and + If expand_dbname is non-zero and dbname contains an = sign, it is taken as a conninfo string in exactly the same way as if it had been passed to PQconnectdb(see below). Previously @@ -4111,50 +4111,48 @@ typedef struct { PostgreSQL offers asynchronous notification via the LISTEN and NOTIFY commands. A client session registers its interest in a particular - notification condition with the LISTEN command (and + notification channel with the LISTEN command (and can stop listening with the UNLISTEN command). All - sessions listening on a particular condition will be notified + sessions listening on a particular channel will be notified asynchronously when a NOTIFY command with that - condition name is executed by any session. No additional information - is passed from the notifier to the listener. Thus, typically, any - actual data that needs to be communicated is transferred through a - database table. Commonly, the condition name is the same as the - associated table, but it is not necessary for there to be any associated - table. + channel name is executed by any session. A payload string can + be passed to communicate additional data to the listeners. libpq applications submit - LISTEN and UNLISTEN commands as + LISTEN, UNLISTEN, + and NOTIFY commands as ordinary SQL commands. The arrival of NOTIFY messages can subsequently be detected by calling PQnotifies.PQnotifies - The function PQnotifies - returns the next notification from a list of unhandled - notification messages received from the server. It returns a null pointer if - there are no pending notifications. Once a notification is - returned from PQnotifies, it is considered handled and will be - removed from the list of notifications. + The function PQnotifies returns the next notification + from a list of unhandled notification messages received from the server. + It returns a null pointer if there are no pending notifications. Once a + notification is returned from PQnotifies, it is considered + handled and will be removed from the list of notifications. + PGnotify *PQnotifies(PGconn *conn); typedef struct pgNotify { - char *relname; /* notification condition name */ + char *relname; /* notification channel name */ int be_pid; /* process ID of notifying server process */ - char *extra; /* notification parameter */ + char *extra; /* notification payload string */ } PGnotify; + After processing a PGnotify object returned by PQnotifies, be sure to free it with PQfreemem. It is sufficient to free the PGnotify pointer; the relname and extra - fields do not represent separate allocations. (At present, the - extra field is unused and will always point - to an empty string.) + fields do not represent separate allocations. (The names of these fields + are historical; in particular, channel names need not have anything to + do with relation names.) diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index e4364ec305f..f0a2aeba03a 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -1,4 +1,4 @@ - + Frontend/Backend Protocol @@ -354,7 +354,7 @@ This message contains the response data from the previous step of GSSAPI or SSPI negotiation (AuthenticationGSS, AuthenticationSSPI - or a previous AuthenticationGSSContinue). If the GSSAPI + or a previous AuthenticationGSSContinue). If the GSSAPI or SSPI data in this message indicates more data is needed to complete the authentication, the frontend must send that data as another PasswordMessage. If @@ -992,7 +992,7 @@ In the event of a backend-detected error during copy-in mode (including - receipt of a CopyFail message), the backend will issue an ErrorResponse + receipt of a CopyFail message), the backend will issue an ErrorResponse message. If the COPY command was issued via an extended-query message, the backend will now discard frontend messages until a Sync message is received, then it will issue ReadyForQuery and return to normal @@ -1117,7 +1117,7 @@ backend will send a NotificationResponse message (not to be confused with NoticeResponse!) whenever a NOTIFY command is executed for the same - notification name. + channel name. @@ -1340,7 +1340,7 @@ This section describes the base data types used in messages. value that will appear, otherwise the value is variable. Eg. String, String("user"). - + There is no predefined limit on the length of a string @@ -1951,7 +1951,7 @@ Bind (F) (denoted R below). This can be zero to indicate that there are no result columns or that the result columns should all use the default format - (text); + (text); or one, in which case the specified format code is applied to all result columns (if any); or it can equal the actual number of result columns of the query. @@ -2500,7 +2500,7 @@ CopyOutResponse (B) separated by separator characters, etc). 1 indicates the overall copy format is binary (similar to DataRow format). See for more information. + endterm="sql-copy-title"> for more information. @@ -3187,7 +3187,7 @@ NotificationResponse (B) - The name of the condition that the notify has been raised on. + The name of the channel that the notify has been raised on. @@ -3197,9 +3197,7 @@ NotificationResponse (B) - Additional information passed from the notifying process. - (Currently, this feature is unimplemented so the field - is always an empty string.) + The payload string passed from the notifying process. @@ -4353,7 +4351,7 @@ the backend. The NotificationResponse ('A') message has an additional string -field, which is presently empty but might someday carry additional data passed +field, which can carry a payload string passed from the NOTIFY event sender. @@ -4364,5 +4362,4 @@ string parameter; this has been removed. - diff --git a/doc/src/sgml/ref/listen.sgml b/doc/src/sgml/ref/listen.sgml index 93ca74c5722..57577c1f6ac 100644 --- a/doc/src/sgml/ref/listen.sgml +++ b/doc/src/sgml/ref/listen.sgml @@ -1,5 +1,5 @@ @@ -21,7 +21,7 @@ PostgreSQL documentation -LISTEN name +LISTEN channel @@ -30,24 +30,23 @@ LISTEN name LISTEN registers the current session as a - listener on the notification condition name. + listener on the notification channel named channel. If the current session is already registered as a listener for - this notification condition, nothing is done. + this notification channel, nothing is done. Whenever the command NOTIFY name is invoked, either + class="PARAMETER">channel is invoked, either by this session or another one connected to the same database, all - the sessions currently listening on that notification condition are + the sessions currently listening on that notification channel are notified, and each will in turn notify its connected client - application. See the discussion of NOTIFY for - more information. + application. - A session can be unregistered for a given notify condition with the + A session can be unregistered for a given notification channel with the UNLISTEN command. A session's listen registrations are automatically cleared when the session ends. @@ -78,16 +77,31 @@ LISTEN name - name + channel - Name of a notify condition (any identifier). + Name of a notification channel (any identifier). + + Notes + + + LISTEN takes effect at transaction commit. + If LISTEN or UNLISTEN is executed + within a transaction that later rolls back, the set of notification + channels being listened to is unchanged. + + + A transaction that has executed LISTEN cannot be + prepared for two-phase commit. + + + Examples diff --git a/doc/src/sgml/ref/notify.sgml b/doc/src/sgml/ref/notify.sgml index 563fbbe9638..b612bb4cb2a 100644 --- a/doc/src/sgml/ref/notify.sgml +++ b/doc/src/sgml/ref/notify.sgml @@ -1,5 +1,5 @@ @@ -21,7 +21,7 @@ PostgreSQL documentation -NOTIFY name +NOTIFY channel [ , payload ] @@ -29,35 +29,39 @@ NOTIFY name Description - The NOTIFY command sends a notification event to each - client application that has previously executed - LISTEN name - for the specified notification name in the current database. + The NOTIFY command sends a notification event together + with an optional payload string to each client application that + has previously executed + LISTEN channel + for the specified channel name in the current database. - NOTIFY provides a simple form of signal or + NOTIFY provides a simple interprocess communication mechanism for a collection of processes accessing the same PostgreSQL database. - Higher-level mechanisms can be built by using tables in the database to - pass additional data (beyond a mere notification name) from notifier to - listener(s). + A payload string can be sent along with the notification, and + higher-level mechanisms for passing structured data can be built by using + tables in the database to pass additional data from notifier to listener(s). - The information passed to the client for a notification event includes the notification - name and the notifying session's server process PID. It is up to the - database designer to define the notification names that will be used in a given - database and what each one means. + The information passed to the client for a notification event includes the + notification channel + name, the notifying session's server process PID, and the + payload string, which is an empty string if it has not been specified. - Commonly, the notification name is the same as the name of some table in + It is up to the database designer to define the channel names that will + be used in a given database and what each one means. + Commonly, the channel name is the same as the name of some table in the database, and the notify event essentially means, I changed this table, take a look at it to see what's new. But no such association is enforced by the NOTIFY and LISTEN commands. For - example, a database designer could use several different notification names - to signal different sorts of changes to a single table. + example, a database designer could use several different channel names + to signal different sorts of changes to a single table. Alternatively, + the payload string could be used to differentiate various cases. @@ -89,19 +93,22 @@ NOTIFY name - NOTIFY behaves like Unix signals in one important - respect: if the same notification name is signaled multiple times in quick - succession, recipients might get only one notification event for several executions - of NOTIFY. So it is a bad idea to depend on the number - of notifications received. Instead, use NOTIFY to wake up - applications that need to pay attention to something, and use a database - object (such as a sequence) to keep track of what happened or how many times - it happened. + If the same channel name is signaled multiple times from the same + transaction with identical payload strings, the + database server can decide to deliver a single notification only. + On the other hand, notifications with distinct payload strings will + always be delivered as distinct notifications. Similarly, notifications from + different transactions will never get folded into one notification. + Except for dropping later instances of duplicate notifications, + NOTIFY guarantees that notifications from the same + transaction get delivered in the order they were sent. It is also + guaranteed that messages from different transactions are delivered in + the order in which the transactions committed. It is common for a client that executes NOTIFY - to be listening on the same notification name itself. In that case + to be listening on the same notification channel itself. In that case it will get back a notification event, just like all the other listening sessions. Depending on the application logic, this could result in useless work, for example, reading a database table to @@ -111,12 +118,7 @@ NOTIFY name notification event message) is the same as one's own session's PID (available from libpq). When they are the same, the notification event is one's own work bouncing - back, and can be ignored. (Despite what was said in the preceding - paragraph, this is a safe technique. - PostgreSQL keeps self-notifications - separate from notifications arriving from other sessions, so you - cannot miss an outside notification by ignoring your own - notifications.) + back, and can be ignored. @@ -125,16 +127,61 @@ NOTIFY name - name + channel - Name of the notification to be signaled (any identifier). + Name of the notification channel to be signaled (any identifier). + + + + + payload + + + The payload string to be communicated along with the + notification. This string must be shorter than 8000 bytes, and + is treated as text. + (If binary data or large amounts of information need to be communicated, + it's best to put it in a database table and send the key of the record.) + + Notes + + + pg_notify + + + + To send a notification you can also use the function + pg_notify(text, + text). The function takes the channel name as the + first argument and the payload as the second. The function is much easier + to use than the NOTIFY command if you need to work with + non-constant channel names and payloads. + + + There is a queue that holds notifications that have been sent but not + yet processed by all listening sessions. If this queue becomes full, + transactions calling NOTIFY will fail at commit. + The queue is quite large (8GB in a standard installation) and should be + sufficiently sized for almost every use case. However, no cleanup can take + place if a session executes LISTEN and then enters a + transaction for a very long time. Once the queue is half full you will see + warnings in the log file pointing you to the session that is preventing + cleanup. In this case you should make sure that this session ends its + current transaction so that cleanup can proceed. + + + A transaction that has executed NOTIFY cannot be + prepared for two-phase commit. + + + Examples @@ -146,6 +193,12 @@ NOTIFY name LISTEN virtual; NOTIFY virtual; Asynchronous notification "virtual" received from server process with PID 8448. +NOTIFY virtual, 'This is the payload'; +Asynchronous notification "virtual" with payload "This is the payload" received from server process with PID 8448. + +LISTEN foo; +SELECT pg_notify('fo' || 'o', 'pay' || 'load'); +Asynchronous notification "foo" with payload "payload" received from server process with PID 14728. diff --git a/doc/src/sgml/ref/unlisten.sgml b/doc/src/sgml/ref/unlisten.sgml index 7cdd7394cb9..9e22ea4edc4 100644 --- a/doc/src/sgml/ref/unlisten.sgml +++ b/doc/src/sgml/ref/unlisten.sgml @@ -1,5 +1,5 @@ @@ -21,7 +21,7 @@ PostgreSQL documentation -UNLISTEN { name | * } +UNLISTEN { channel | * } @@ -33,8 +33,8 @@ UNLISTEN { name | * } registration for NOTIFY events. UNLISTEN cancels any existing registration of the current PostgreSQL session as a - listener on the notification name. The special wildcard + listener on the notification channel named channel. The special wildcard * cancels all listener registrations for the current session. @@ -52,10 +52,10 @@ UNLISTEN { name | * } - name + channel - Name of a notification (any identifier). + Name of a notification channel (any identifier). @@ -83,6 +83,11 @@ UNLISTEN { name | * } At the end of each session, UNLISTEN * is automatically executed. + + + A transaction that has executed UNLISTEN cannot be + prepared for two-phase commit. + @@ -100,7 +105,7 @@ Asynchronous notification "virtual" received from server process with PID 8448. Once UNLISTEN has been executed, further NOTIFY - commands will be ignored: + messages will be ignored: UNLISTEN virtual; diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index e0cef7dd7b0..c4b38ddb6c8 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -1,4 +1,4 @@ - + @@ -77,6 +77,11 @@ Item (used for shared row locks) + + pg_notify + Subdirectory containing LISTEN/NOTIFY status data + + pg_stat_tmp Subdirectory containing temporary files for the statistics -- cgit v1.2.3