Use SASLprep to normalize passwords for SCRAM authentication.

An important step of SASLprep normalization, is to convert the string to
Unicode normalization form NFKC. Unicode normalization requires a fairly
large table of character decompositions, which is generated from data
published by the Unicode consortium. The script to generate the table is
put in src/common/unicode, as well test code for the normalization.
A pre-generated version of the tables is included in src/include/common,
so you don't need the code in src/common/unicode to build PostgreSQL, only
if you wish to modify the normalization tables.

The SASLprep implementation depends on the UTF-8 functions from
src/backend/utils/mb/wchar.c. So to use it, you must also compile and link
that. That doesn't change anything for the current users of these
functions, the backend and libpq, as they both already link with wchar.o.
It would be good to move those functions into a separate file in
src/commmon, but I'll leave that for another day.

No documentation changes included, because there is no details on the
SCRAM mechanism in the docs anyway. An overview on that in the protocol
specification would probably be good, even though SCRAM is documented in
detail in RFC5802. I'll write that as a separate patch. An important thing
to mention there is that we apply SASLprep even on invalid UTF-8 strings,
to support other encodings.

Patch by Michael Paquier and me.

Discussion: https://www.postgresql.org/message-id/CAB7nPqSByyEmAVLtEf1KxTRh=PWNKiWKEKQR=e1yGehz=wbymQ@mail.gmail.com

1 files changed, 59 insertions, 4 deletions

diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 14ddc8bd542..5077ff33b16 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -11,13 +11,43 @@
  *
  * - Username from the authentication exchange is not used. The client
  *	 should send an empty string as the username.
- * - Password is not processed with the SASLprep algorithm.
+ *
+ * - If the password isn't valid UTF-8, or contains characters prohibited
+ *	 by the SASLprep profile, we skip the SASLprep pre-processing and use
+ *	 the raw bytes in calculating the hash.
+ *
  * - Channel binding is not supported yet.
  *
+ *
  * The password stored in pg_authid consists of the salt, iteration count,
  * StoredKey and ServerKey.
  *
- * On error handling:
+ * SASLprep usage
+ * --------------
+ *
+ * One notable difference to the SCRAM specification is that while the
+ * specification dictates that the password is in UTF-8, and prohibits
+ * certain characters, we are more lenient.  If the password isn't a valid
+ * UTF-8 string, or contains prohibited characters, the raw bytes are used
+ * to calculate the hash instead, without SASLprep processing.  This is
+ * because PostgreSQL supports other encodings too, and the encoding being
+ * used during authentication is undefined (client_encoding isn't set until
+ * after authentication).  In effect, we try to interpret the password as
+ * UTF-8 and apply SASLprep processing, but if it looks invalid, we assume
+ * that it's in some other encoding.
+ *
+ * In the worst case, we misinterpret a password that's in a different
+ * encoding as being Unicode, because it happens to consists entirely of
+ * valid UTF-8 bytes, and we apply Unicode normalization to it.  As long
+ * as we do that consistently, that will not lead to failed logins.
+ * Fortunately, the UTF-8 byte sequences that are ignored by SASLprep
+ * don't correspond to any commonly used characters in any of the other
+ * supported encodings, so it should not lead to any significant loss in
+ * entropy, even if the normalization is incorrectly applied to a
+ * non-UTF-8 password.
+ *
+ * Error handling
+ * --------------
  *
  * Don't reveal user information to an unauthenticated client.  We don't
  * want an attacker to be able to probe whether a particular username is
@@ -37,6 +67,7 @@
  * to the encoding being used, whatever that is.  We cannot avoid that in
  * general, after logging in, but let's do what we can here.
  *
+ *
  * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
  * Portions Copyright (c) 1994, Regents of the University of California
  *
@@ -52,6 +83,7 @@
 #include "catalog/pg_authid.h"
 #include "catalog/pg_control.h"
 #include "common/base64.h"
+#include "common/saslprep.h"
 #include "common/scram-common.h"
 #include "common/sha2.h"
 #include "libpq/auth.h"
@@ -344,6 +376,17 @@ scram_build_verifier(const char *username, const char *password,
 	char		salt[SCRAM_SALT_LEN];
 	char	   *encoded_salt;
 	int			encoded_len;
+	char	   *prep_password = NULL;
+	pg_saslprep_rc rc;
+
+	/*
+	 * Normalize the password with SASLprep.  If that doesn't work, because
+	 * the password isn't valid UTF-8 or contains prohibited characters, just
+	 * proceed with the original password.  (See comments at top of file.)
+	 */
+	rc = pg_saslprep(password, &prep_password);
+	if (rc == SASLPREP_SUCCESS)
+		password = (const char *) prep_password;
 
 	if (iterations <= 0)
 		iterations = SCRAM_ITERATIONS_DEFAULT;
@@ -373,6 +416,9 @@ scram_build_verifier(const char *username, const char *password,
 	(void) hex_encode((const char *) keybuf, SCRAM_KEY_LEN, serverkey_hex);
 	serverkey_hex[SCRAM_KEY_LEN * 2] = '\0';
 
+	if (prep_password)
+		pfree(prep_password);
+
 	return psprintf("scram-sha-256:%s:%d:%s:%s", encoded_salt, iterations, storedkey_hex, serverkey_hex);
 }
 
@@ -392,13 +438,14 @@ scram_verify_plain_password(const char *username, const char *password,
 	uint8		stored_key[SCRAM_KEY_LEN];
 	uint8		server_key[SCRAM_KEY_LEN];
 	uint8		computed_key[SCRAM_KEY_LEN];
+	char	   *prep_password = NULL;
+	pg_saslprep_rc rc;
 
 	if (!parse_scram_verifier(verifier, &encoded_salt, &iterations,
 							  stored_key, server_key))
 	{
 		/*
-		 * The password looked like a SCRAM verifier, but could not be
-		 * parsed.
+		 * The password looked like a SCRAM verifier, but could not be parsed.
 		 */
 		elog(LOG, "invalid SCRAM verifier for user \"%s\"", username);
 		return false;
@@ -412,10 +459,18 @@ scram_verify_plain_password(const char *username, const char *password,
 		return false;
 	}
 
+	/* Normalize the password */
+	rc = pg_saslprep(password, &prep_password);
+	if (rc == SASLPREP_SUCCESS)
+		password = prep_password;
+
 	/* Compute Server key based on the user-supplied plaintext password */
 	scram_ClientOrServerKey(password, salt, saltlen, iterations,
 							SCRAM_SERVER_KEY_NAME, computed_key);
 
+	if (prep_password)
+		pfree(prep_password);
+
 	/*
 	 * Compare the verifier's Server Key with the one computed from the
 	 * user-supplied password.