Fix assorted issues in backend's GSSAPI encryption support.

Unrecoverable errors detected by GSSAPI encryption can't just be
reported with elog(ERROR) or elog(FATAL), because attempting to
send the error report to the client is likely to lead to infinite
recursion or loss of protocol sync.  Instead make this code do what
the SSL encryption code has long done, which is to just report any
such failure to the server log (with elevel COMMERROR), then pretend
we've lost the connection by returning errno = ECONNRESET.

Along the way, fix confusion about whether message translation is done
by pg_GSS_error() or its callers (the latter should do it), and make
the backend version of that function work more like the frontend
version.

Avoid allocating the port->gss struct until it's needed; we surely
don't need to allocate it in the postmaster.

Improve logging of "connection authorized" messages with GSS enabled.
(As part of this, I back-patched the code changes from dc11f31.)

Make BackendStatusShmemSize() account for the GSS-related space that
will be allocated by CreateSharedBackendStatus().  This omission
could possibly cause out-of-shared-memory problems with very high
max_connections settings.

Remove arbitrary, pointless restriction that only GSS authentication
can be used on a GSS-encrypted connection.

Improve documentation; notably, document the fact that libpq now
prefers GSS encryption over SSL encryption if both are possible.

Per report from Mikael Gustavsson.  Back-patch to v12 where
this code was introduced.

Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se

Author: tglsfdc

doc/src/sgml/client-auth.sgml

@@ -196,14 +196,6 @@ hostnogssenc  <replaceable>database</replaceable>  <replaceable>user</replaceabl
        the <literal>hostgssenc</literal> record is ignored except for logging
        a warning that it cannot match any connections.
       </para>
-
-      <para>
-        Note that the only supported
-        <link linkend="auth-methods">authentication methods</link> for use
-        with <acronym>GSSAPI</acronym> encryption
-        are <literal>gss</literal>, <literal>reject</literal>,
-        and <literal>trust</literal>.
-      </para>
      </listitem>
     </varlistentry>
 
@@ -1196,15 +1188,15 @@ omicron         bryanh                  guest1
 
    <para>
     <productname>GSSAPI</productname> is an industry-standard protocol
-    for secure authentication defined in RFC 2743.
-
+    for secure authentication defined in
+    <ulink url="https://tools.ietf.org/html/rfc2743">RFC 2743</ulink>.
     <productname>PostgreSQL</productname>
-    supports <productname>GSSAPI</productname> for use as either an encrypted,
-    authenticated layer, or for authentication only.
+    supports <productname>GSSAPI</productname> for authentication,
+    communications encryption, or both.
     <productname>GSSAPI</productname> provides automatic authentication
     (single sign-on) for systems that support it. The authentication itself is
     secure.  If <productname>GSSAPI</productname> encryption
-    (see <literal>hostgssenc</literal>) or <acronym>SSL</acronym> encryption are
+    or <acronym>SSL</acronym> encryption is
     used, the data sent along the database connection will be encrypted;
     otherwise, it will not.
    </para>
@@ -1216,37 +1208,46 @@ omicron         bryanh                  guest1
 
    <para>
     When <productname>GSSAPI</productname> uses
-    <productname>Kerberos</productname>, it uses a standard principal
-    in the format
+    <productname>Kerberos</productname>, it uses a standard service
+    principal (authentication identity) name in the format
     <literal><replaceable>servicename</replaceable>/<replaceable>hostname</replaceable>@<replaceable>realm</replaceable></literal>.
-    The PostgreSQL server will accept any principal that is included in the keytab used by
-    the server, but care needs to be taken to specify the correct principal details when
-    making the connection from the client using the <literal>krbsrvname</literal> connection parameter. (See
-    also <xref linkend="libpq-paramkeywords"/>.) The installation default can be
-    changed from the default <literal>postgres</literal> at build time using
-    <literal>./configure --with-krb-srvnam=</literal><replaceable>whatever</replaceable>.
-    In most environments,
-    this parameter never needs to be changed.
-    Some Kerberos implementations might require a different service name,
-    such as Microsoft Active Directory which requires the service name
-    to be in upper case (<literal>POSTGRES</literal>).
-   </para>
-   <para>
-    <replaceable>hostname</replaceable> is the fully qualified host name of the
-    server machine. The service principal's realm is the preferred realm
-    of the server machine.
-   </para>
-
-   <para>
-    Client principals can be mapped to different <productname>PostgreSQL</productname>
-    database user names with <filename>pg_ident.conf</filename>.  For example,
+    The principal name used by a particular installation is not encoded in
+    the <productname>PostgreSQL</productname> server in any way; rather it
+    is specified in the <firstterm>keytab</firstterm> file that the server
+    reads to determine its identity.  If multiple principals are listed in
+    the keytab file, the server will accept any one of them.
+    The server's realm name is the preferred realm specified in the Kerberos
+    configuration file(s) accessible to the server.
+   </para>
+
+   <para>
+    When connecting, the client must know the principal name of the server
+    it intends to connect to.  The <replaceable>servicename</replaceable>
+    part of the principal is ordinarily <literal>postgres</literal>,
+    but another value can be selected via <application>libpq</application>'s
+    <xref linkend="libpq-connect-krbsrvname"/> connection parameter.
+    The <replaceable>hostname</replaceable> part is the fully qualified
+    host name that <application>libpq</application> is told to connect to.
+    The realm name is the preferred realm specified in the Kerberos
+    configuration file(s) accessible to the client.
+   </para>
+
+   <para>
+    The client will also have a principal name for its own identity
+    (and it must have a valid ticket for this principal).  To
+    use <productname>GSSAPI</productname> for authentication, the client
+    principal must be associated with
+    a <productname>PostgreSQL</productname> database user name.
+    The <filename>pg_ident.conf</filename> configuration file can be used
+    to map principals to user names; for example,
     <literal>pgusername@realm</literal> could be mapped to just <literal>pgusername</literal>.
     Alternatively, you can use the full <literal>username@realm</literal> principal as
     the role name in <productname>PostgreSQL</productname> without any mapping.
    </para>
 
    <para>
-    <productname>PostgreSQL</productname> also supports a parameter to strip the realm from
+    <productname>PostgreSQL</productname> also supports mapping
+    client principals to user names by just stripping the realm from
     the principal.  This method is supported for backwards compatibility and is
     strongly discouraged as it is then impossible to distinguish different users
     with the same user name but coming from different realms.  To enable this,
@@ -1260,39 +1261,34 @@ omicron         bryanh                  guest1
    </para>
 
    <para>
-    Make sure that your server keytab file is readable (and preferably
-    only readable, not writable) by the <productname>PostgreSQL</productname>
-    server account.  (See also <xref linkend="postgres-user"/>.) The location
-    of the key file is specified by the <xref
+    The location of the server's keytab file is specified by the <xref
     linkend="guc-krb-server-keyfile"/> configuration
     parameter. The default is
-    <filename>/usr/local/pgsql/etc/krb5.keytab</filename> (or whatever
-    directory was specified as <varname>sysconfdir</varname> at build time).
+    <filename>FILE:/usr/local/pgsql/etc/krb5.keytab</filename>
+    (where the directory part is whatever was specified
+    as <varname>sysconfdir</varname> at build time).
     For security reasons, it is recommended to use a separate keytab
     just for the <productname>PostgreSQL</productname> server rather
-    than opening up permissions on the system keytab file.
+    than allowing the server to read the system keytab file.
+    Make sure that your server keytab file is readable (and preferably
+    only readable, not writable) by the <productname>PostgreSQL</productname>
+    server account.  (See also <xref linkend="postgres-user"/>.)
    </para>
+
    <para>
-    The keytab file is generated by the Kerberos software; see the
-    Kerberos documentation for details. The following example is
-   for MIT-compatible Kerberos 5 implementations:
+    The keytab file is generated using the Kerberos software; see the
+    Kerberos documentation for details. The following example shows
+    doing this using the <application>kadmin</application> tool of
+    MIT-compatible Kerberos 5 implementations:
 <screen>
-<prompt>kadmin% </prompt><userinput>ank -randkey postgres/server.my.domain.org</userinput>
+<prompt>kadmin% </prompt><userinput>addprinc -randkey postgres/server.my.domain.org</userinput>
 <prompt>kadmin% </prompt><userinput>ktadd -k krb5.keytab postgres/server.my.domain.org</userinput>
 </screen>
    </para>
 
    <para>
-    When connecting to the database make sure you have a ticket for a
-    principal matching the requested database user name. For example, for
-    database user name <literal>fred</literal>, principal
-    <literal>fred@EXAMPLE.COM</literal> would be able to connect. To also allow
-    principal <literal>fred/users.example.com@EXAMPLE.COM</literal>, use a user name
-    map, as described in <xref linkend="auth-username-maps"/>.
-   </para>
-
-   <para>
-    The following configuration options are supported for <productname>GSSAPI</productname>:
+    The following authentication options are supported for
+    the <productname>GSSAPI</productname> authentication method:
     <variablelist>
      <varlistentry>
       <term><literal>include_realm</literal></term>
@@ -1315,7 +1311,7 @@ omicron         bryanh                  guest1
       <term><literal>map</literal></term>
       <listitem>
        <para>
-        Allows for mapping between system and database user names. See
+        Allows mapping from client principals to database user names. See
         <xref linkend="auth-username-maps"/> for details.  For a GSSAPI/Kerberos
         principal, such as <literal>username@EXAMPLE.COM</literal> (or, less
         commonly, <literal>username/hostbased@EXAMPLE.COM</literal>), the
@@ -1342,6 +1338,15 @@ omicron         bryanh                  guest1
      </varlistentry>
     </variablelist>
    </para>
+
+   <para>
+    In addition to these settings, which can be different for
+    different <filename>pg_hba.conf</filename> entries, there is the
+    server-wide <xref linkend="guc-krb-caseins-users"/> configuration
+    parameter.  If that is set to true, client principals are matched to
+    user map entries case-insensitively.  <literal>krb_realm</literal>, if
+    set, is also matched case-insensitively.
+   </para>
   </sect1>
 
   <sect1 id="sspi-auth">

doc/src/sgml/libpq.sgml

@@ -1474,6 +1474,16 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
         connection.<indexterm><primary>SSL</primary><secondary
         sortas="libpq">with libpq</secondary></indexterm>
        </para>
+
+       <para>
+        Note that if <acronym>GSSAPI</acronym> encryption is possible,
+        that will be used in preference to <acronym>SSL</acronym>
+        encryption, regardless of the value of <literal>sslmode</literal>.
+        To force use of <acronym>SSL</acronym> encryption in an
+        environment that has working <acronym>GSSAPI</acronym>
+        infrastructure (such as a Kerberos server), also
+        set <literal>gssencmode</literal> to <literal>disable</literal>.
+       </para>
       </listitem>
      </varlistentry>
 
@@ -1615,6 +1625,15 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
         This must match the service name specified in the server
         configuration for Kerberos authentication to succeed. (See also
         <xref linkend="gssapi-auth"/>.)
+        The default value is normally <literal>postgres</literal>,
+        but that can be changed when
+        building <productname>PostgreSQL</productname> via
+        the <option>--with-krb-srvnam</option> option
+        of <application>configure</application>.
+        In most environments, this parameter never needs to be changed.
+        Some Kerberos implementations might require a different service name,
+        such as Microsoft Active Directory which requires the service name
+        to be in upper case (<literal>POSTGRES</literal>).
        </para>
       </listitem>
      </varlistentry>

doc/src/sgml/protocol.sgml

@@ -1454,6 +1454,10 @@ SELCT 1/0;<!-- this typo is intentional -->
     <acronym>SSL</acronym>-encrypted.  To continue after
     <literal>N</literal>, send the usual StartupMessage and proceed without
     encryption.
+    (Alternatively, it is permissible to issue a GSSENCRequest message
+    after an <literal>N</literal> response to try to
+    use <acronym>GSSAPI</acronym> encryption instead
+    of <acronym>SSL</acronym>.)
    </para>
 
    <para>
@@ -1508,20 +1512,14 @@ SELCT 1/0;<!-- this typo is intentional -->
     result from the server, until it returns no output.  When sending the
     results of <function>gss_init_sec_context()</function> to the server,
     prepend the length of the message as a four byte integer in network byte
-    order.  If this is successful, then use <function>gss_wrap()</function> to
-    encrypt the usual StartupMessage and all subsequent data, prepending the
-    length of the result from <function>gss_wrap()</function> as a four byte
-    integer in network byte order to the actual encrypted payload.  Note that
-    the server will only accept encrypted packets from the client which are less
-    than 16kB; <function>gss_wrap_size_limit()</function> should be used by the
-    client to determine the size of the unencrypted message which will fit
-    within this limit and larger messages should be broken up into multiple
-    <function>gss_wrap()</function> calls.  Typical segments are 8kB of
-    unencrypted data, resulting in encrypted packets of slightly larger than 8kB
-    but well within the 16kB maximum.  The server can be expected to not send
-    encrypted packets of larger than 16kB to the client.  To continue after
+    order.
+    To continue after
     <literal>N</literal>, send the usual StartupMessage and proceed without
     encryption.
+    (Alternatively, it is permissible to issue an SSLRequest message
+    after an <literal>N</literal> response to try to
+    use <acronym>SSL</acronym> encryption instead
+    of <acronym>GSSAPI</acronym>.)
    </para>
 
    <para>
@@ -1531,16 +1529,30 @@ SELCT 1/0;<!-- this typo is intentional -->
     support to <productname>PostgreSQL</productname>.  In this case the
     connection must be closed, but the frontend might choose to open a fresh
     connection and proceed without requesting <acronym>GSSAPI</acronym>
-    encryption.  Given the length limits specified above, the ErrorMessage can
-    not be confused with a proper response from the server with an appropriate
-    length.
+    encryption.
    </para>
 
    <para>
     An initial GSSENCRequest can also be used in a connection that is being
     opened to send a CancelRequest message.
    </para>
 
+   <para>
+    Once <acronym>GSSAPI</acronym> encryption has been successfully
+    established, use <function>gss_wrap()</function> to
+    encrypt the usual StartupMessage and all subsequent data, prepending the
+    length of the result from <function>gss_wrap()</function> as a four byte
+    integer in network byte order to the actual encrypted payload.  Note that
+    the server will only accept encrypted packets from the client which are less
+    than 16kB; <function>gss_wrap_size_limit()</function> should be used by the
+    client to determine the size of the unencrypted message which will fit
+    within this limit and larger messages should be broken up into multiple
+    <function>gss_wrap()</function> calls.  Typical segments are 8kB of
+    unencrypted data, resulting in encrypted packets of slightly larger than 8kB
+    but well within the 16kB maximum.  The server can be expected to not send
+    encrypted packets of larger than 16kB to the client.
+   </para>
+
    <para>
     While the protocol itself does not provide a way for the server to
     force <acronym>GSSAPI</acronym> encryption, the administrator can

doc/src/sgml/runtime.sgml

@@ -2632,7 +2632,7 @@ openssl x509 -req -in server.csr -text -days 365 \
    <productname>PostgreSQL</productname> also has native support for
    using <acronym>GSSAPI</acronym> to encrypt client/server communications for
    increased security.  Support requires that a <acronym>GSSAPI</acronym>
-   implementation (such as MIT krb5) is installed on both client and server
+   implementation (such as MIT Kerberos) is installed on both client and server
    systems, and that support in <productname>PostgreSQL</productname> is
    enabled at build time (see <xref linkend="installation"/>).
   </para>
@@ -2651,6 +2651,16 @@ openssl x509 -req -in server.csr -text -days 365 \
     some or all connections.
    </para>
 
+   <para>
+    When using <acronym>GSSAPI</acronym> for encryption, it is common to
+    use <acronym>GSSAPI</acronym> for authentication as well, since the
+    underlying mechanism will determine both client and server identities
+    (according to the <acronym>GSSAPI</acronym> implementation) in any
+    case.  But this is not required;
+    another <productname>PostgreSQL</productname> authentication method
+    can be chosen to perform additional verification.
+   </para>
+
    <para>
     Other than configuration of the negotiation
     behavior, <acronym>GSSAPI</acronym> encryption requires no setup beyond

src/backend/libpq/auth.c

@@ -381,17 +381,6 @@ ClientAuthentication(Port *port)
 					 errmsg("connection requires a valid client certificate")));
 	}
 
-#ifdef ENABLE_GSS
-	if (port->gss->enc && port->hba->auth_method != uaReject &&
-		port->hba->auth_method != uaImplicitReject &&
-		port->hba->auth_method != uaTrust &&
-		port->hba->auth_method != uaGSS)
-	{
-		ereport(FATAL, (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
-						errmsg("GSSAPI encryption can only be used with gss, trust, or reject authentication methods")));
-	}
-#endif
-
 	/*
 	 * Now proceed to do the actual authentication check
 	 */
@@ -532,7 +521,17 @@ ClientAuthentication(Port *port)
 
 		case uaGSS:
 #ifdef ENABLE_GSS
+			/* We might or might not have the gss workspace already */
+			if (port->gss == NULL)
+				port->gss = (pg_gssinfo *)
+					MemoryContextAllocZero(TopMemoryContext,
+										   sizeof(pg_gssinfo));
 			port->gss->auth = true;
+
+			/*
+			 * If GSS state was set up while enabling encryption, we can just
+			 * check the client's principal.  Otherwise, ask for it.
+			 */
 			if (port->gss->enc)
 				status = pg_GSS_checkauth(port);
 			else
@@ -547,6 +546,10 @@ ClientAuthentication(Port *port)
 
 		case uaSSPI:
 #ifdef ENABLE_SSPI
+			if (port->gss == NULL)
+				port->gss = (pg_gssinfo *)
+					MemoryContextAllocZero(TopMemoryContext,
+										   sizeof(pg_gssinfo));
 			sendAuthRequest(port, AUTH_REQ_SSPI, NULL, 0);
 			status = pg_SSPI_recvauth(port);
 #else
@@ -1189,9 +1192,9 @@ pg_GSS_recvauth(Port *port)
 		if (maj_stat != GSS_S_COMPLETE && maj_stat != GSS_S_CONTINUE_NEEDED)
 		{
 			gss_delete_sec_context(&lmin_s, &port->gss->ctx, GSS_C_NO_BUFFER);
-			pg_GSS_error(ERROR,
-						 _("accepting GSS security context failed"),
+			pg_GSS_error(_("accepting GSS security context failed"),
 						 maj_stat, min_stat);
+			return STATUS_ERROR;
 		}
 
 		if (maj_stat == GSS_S_CONTINUE_NEEDED)
@@ -1228,9 +1231,11 @@ pg_GSS_checkauth(Port *port)
 	 */
 	maj_stat = gss_display_name(&min_stat, port->gss->name, &gbuf, NULL);
 	if (maj_stat != GSS_S_COMPLETE)
-		pg_GSS_error(ERROR,
-					 _("retrieving GSS user name failed"),
+	{
+		pg_GSS_error(_("retrieving GSS user name failed"),
 					 maj_stat, min_stat);
+		return STATUS_ERROR;
+	}
 
 	/*
 	 * Copy the original name of the authenticated principal into our backend