Allow SCRAM authentication, when pg_hba.conf says 'md5'.

If a user has a SCRAM verifier in pg_authid.rolpassword, there's no reason
we cannot attempt to perform SCRAM authentication instead of MD5. The worst
that can happen is that the client doesn't support SCRAM, and the
authentication will fail. But previously, it would fail for sure, because
we would not even try. SCRAM is strictly more secure than MD5, so there's
no harm in trying it. This allows for a more graceful transition from MD5
passwords to SCRAM, as user passwords can be changed to SCRAM verifiers
incrementally, without changing pg_hba.conf.

Refactor the code in auth.c to support that better. Notably, we now have to
look up the user's pg_authid entry before sending the password challenge,
also when performing MD5 authentication. Also simplify the concept of a
"doomed" authentication. Previously, if a user had a password, but it had
expired, we still performed SCRAM authentication (but always returned error
at the end) using the salt and iteration count from the expired password.
Now we construct a fake salt, like we do when the user doesn't have a
password or doesn't exist at all. That simplifies get_role_password(), and
we can don't need to distinguish the  "user has expired password", and
"user does not exist" cases in auth.c.

On second thoughts, also rename uaSASL to uaSCRAM. It refers to the
mechanism specified in pg_hba.conf, and while we use SASL for SCRAM
authentication at the protocol level, the mechanism should be called SCRAM,
not SASL. As a comparison, we have uaLDAP, even though it looks like the
plain 'password' authentication at the protocol level.

Discussion: https://www.postgresql.org/message-id/6425.1489506016@sss.pgh.pa.us
Reviewed-by: Michael Paquier

Author: hlinnaka

doc/src/sgml/client-auth.sgml

@@ -412,23 +412,22 @@ hostnossl  <replaceable>database</replaceable>  <replaceable>user</replaceable>
        </varlistentry>
 
        <varlistentry>
-        <term><literal>md5</></term>
+        <term><literal>scram</></term>
         <listitem>
          <para>
-          Require the client to supply a double-MD5-hashed password for
-          authentication.
-          See <xref linkend="auth-password"> for details.
+          Perform SCRAM-SHA-256 authentication to verify the user's
+          password. See <xref linkend="auth-password"> for details.
          </para>
         </listitem>
        </varlistentry>
 
        <varlistentry>
-        <term><literal>scram</></term>
+        <term><literal>md5</></term>
         <listitem>
          <para>
-          Perform SCRAM-SHA-256 authentication to verify the user's
-          password.
-          See <xref linkend="auth-password"> for details.
+          Perform SCRAM-SHA-256 or MD5 authentication to verify the
+          user's password. See <xref linkend="auth-password">
+          for details.
          </para>
         </listitem>
        </varlistentry>
@@ -689,13 +688,12 @@ host    postgres        all             192.168.12.10/32        scram
 # Allow any user from hosts in the example.com domain to connect to
 # any database if the user's password is correctly supplied.
 #
-# Most users use SCRAM authentication, but some users use older clients
-# that don't support SCRAM authentication, and need to be able to log
-# in using MD5 authentication. Such users are put in the @md5users
-# group, everyone else must use SCRAM.
+# Require SCRAM authentication for most users, but make an exception
+# for user 'mike', who uses an older client that doesn't support SCRAM
+# authentication.
 #
 # TYPE  DATABASE        USER            ADDRESS                 METHOD
-host    all             @md5users       .example.com            md5
+host    all             mike            .example.com            md5
 host    all             all             .example.com            scram
 
 # In the absence of preceding "host" lines, these two lines will
@@ -949,12 +947,13 @@ omicron         bryanh                  guest1
    </para>
 
    <para>
-    In <literal>md5</>, the client sends a hash of a random challenge,
-    generated by the server, and the password. It prevents password sniffing,
-    but is less secure than <literal>scram</>, and provides no protection
-    if an attacker manages to steal the password hash from the server.
-    <literal>md5</> cannot be used with the <xref
-    linkend="guc-db-user-namespace"> feature.  
+    <literal>md5</> allows falling back to a less secure challenge-response
+    mechanism for those users with an MD5 hashed password.
+    The fallback mechanism also prevents password sniffing, but provides no
+    protection if an attacker manages to steal the password hash from the
+    server, and it cannot be used with the <xref
+    linkend="guc-db-user-namespace"> feature. For all other users,
+    <literal>md5</> works the same as <literal>scram</>.
    </para>
 
    <para>

src/backend/libpq/auth-scram.c

@@ -130,79 +130,91 @@ static char *scram_MockSalt(const char *username);
  * after the beginning of the exchange with verifier data.
  *
  * 'username' is the provided by the client.  'shadow_pass' is the role's
- * password verifier, from pg_authid.rolpassword.  If 'doomed' is true, the
- * authentication must fail, as if an incorrect password was given.
- * 'shadow_pass' may be NULL, when 'doomed' is set.
+ * password verifier, from pg_authid.rolpassword.  If 'shadow_pass' is NULL, we
+ * still perform an authentication exchange, but it will fail, as if an
+ * incorrect password was given.
  */
 void *
-pg_be_scram_init(const char *username, const char *shadow_pass, bool doomed)
+pg_be_scram_init(const char *username, const char *shadow_pass)
 {
 	scram_state *state;
-	int			password_type;
+	bool		got_verifier;
 
 	state = (scram_state *) palloc0(sizeof(scram_state));
 	state->state = SCRAM_AUTH_INIT;
 	state->username = username;
 
 	/*
-	 * Perform sanity checks on the provided password after catalog lookup.
-	 * The authentication is bound to fail if the lookup itself failed or if
-	 * the password stored is MD5-encrypted.  Authentication is possible for
-	 * users with a valid plain password though.
+	 * Parse the stored password verifier.
 	 */
+	if (shadow_pass)
+	{
+		int			password_type = get_password_type(shadow_pass);
 
-	if (shadow_pass == NULL || doomed)
-		password_type = -1;
-	else
-		password_type = get_password_type(shadow_pass);
+		if (password_type == PASSWORD_TYPE_SCRAM)
+		{
+			if (parse_scram_verifier(shadow_pass, &state->salt, &state->iterations,
+									 state->StoredKey, state->ServerKey))
+				got_verifier = true;
+			else
+			{
+				/*
+				 * The password looked like a SCRAM verifier, but could not be
+				 * parsed.
+				 */
+				elog(LOG, "invalid SCRAM verifier for user \"%s\"", username);
+				got_verifier = false;
+			}
+		}
+		else if (password_type == PASSWORD_TYPE_PLAINTEXT)
+		{
+			/*
+			 * The stored password is in plain format.  Generate a fresh SCRAM
+			 * verifier from it, and proceed with that.
+			 */
+			char	   *verifier;
 
-	if (password_type == PASSWORD_TYPE_SCRAM)
-	{
-		if (!parse_scram_verifier(shadow_pass, &state->salt, &state->iterations,
-								  state->StoredKey, state->ServerKey))
+			verifier = scram_build_verifier(username, shadow_pass, 0);
+
+			(void) parse_scram_verifier(verifier, &state->salt, &state->iterations,
+										state->StoredKey, state->ServerKey);
+			pfree(verifier);
+
+			got_verifier = true;
+		}
+		else
 		{
 			/*
-			 * The password looked like a SCRAM verifier, but could not be
-			 * parsed.
+			 * The user doesn't have SCRAM verifier, nor could we generate
+			 * one. (You cannot do SCRAM authentication with an MD5 hash.)
 			 */
-			elog(LOG, "invalid SCRAM verifier for user \"%s\"", username);
-			doomed = true;
+			state->logdetail = psprintf(_("User \"%s\" does not have a valid SCRAM verifier."),
+										state->username);
+			got_verifier = false;
 		}
 	}
-	else if (password_type == PASSWORD_TYPE_PLAINTEXT)
+	else
 	{
-		char	   *verifier;
-
 		/*
-		 * The password provided is in plain format, in which case a fresh
-		 * SCRAM verifier can be generated and used for the rest of the
-		 * processing.
+		 * The caller requested us to perform a dummy authentication.  This is
+		 * considered normal, since the caller requested it, so don't set log
+		 * detail.
 		 */
-		verifier = scram_build_verifier(username, shadow_pass, 0);
-
-		(void) parse_scram_verifier(verifier, &state->salt, &state->iterations,
-									state->StoredKey, state->ServerKey);
-		pfree(verifier);
+		got_verifier = false;
 	}
-	else
-		doomed = true;
 
-	if (doomed)
+	/*
+	 * If the user did not have a valid SCRAM verifier, we still go through
+	 * the motions with a mock one, and fail as if the client supplied an
+	 * incorrect password.  This is to avoid revealing information to an
+	 * attacker.
+	 */
+	if (!got_verifier)
 	{
-		/*
-		 * We don't have a valid SCRAM verifier, nor could we generate one, or
-		 * the caller requested us to perform a dummy authentication.
-		 *
-		 * The authentication is bound to fail, but to avoid revealing
-		 * information to the attacker, go through the motions with a fake
-		 * SCRAM verifier, and fail as if the password was incorrect.
-		 */
-		state->logdetail = psprintf(_("User \"%s\" does not have a valid SCRAM verifier."),
-									state->username);
 		mock_scram_verifier(username, &state->salt, &state->iterations,
 							state->StoredKey, state->ServerKey);
+		state->doomed = true;
 	}
-	state->doomed = doomed;
 
 	return state;
 }