Fix broken collation-aware searches in SP-GiST text opclass.

spg_text_leaf_consistent() supposed that it should compare only
Min(querylen, entrylen) bytes of the two strings, and then deal with
any excess bytes in one string or the other by assuming the longer
string is greater if the prefixes are equal.  Quite aside from the
fact that that's just wrong in some locales (e.g., 'ch' is not less
than 'd' in cs_CZ), it also risked passing incomplete multibyte
characters to strcoll(), with ensuing bad results.

Instead, just pass the full strings to varstr_cmp, and let it decide
what to do about unequal-length strings.

Fortunately, this error doesn't imply any index corruption, it's just
that searches might return the wrong set of entries.

Per report from Emre Hasegeli, though this is not his patch.
Thanks to Peter Geoghegan for review and discussion.

This code was born broken, so back-patch to all supported branches.
In HEAD, I failed to resist the temptation to do a bit of cosmetic
cleanup/pgindent'ing on 710d90da1, too.

Discussion: https://postgr.es/m/CAE2gYzzb6K51VnTq5i5p52z+j9p2duEa-K1T3RrC_GQEynAKEg@mail.gmail.com

diff --git a/src/backend/access/spgist/spgtextproc.c b/src/backend/access/spgist/spgtextproc.c
index 1c7d0a3775a2f896d6d7b84fd6fac35b11f8985f..f3783b324e85c604ecb21abff4bc1958956d74ea 100644 (file)
--- a/src/backend/access/spgist/spgtextproc.c
+++ b/src/backend/access/spgist/spgtextproc.c
@@ -603,22 +603,22 @@ spg_text_leaf_consistent(PG_FUNCTION_ARGS)
            /* If asserts enabled, verify encoding of reconstructed string */
            Assert(pg_verifymbstr(fullValue, fullLen, false));
 
-           r = varstr_cmp(fullValue, Min(queryLen, fullLen),
-                          VARDATA_ANY(query), Min(queryLen, fullLen),
+           r = varstr_cmp(fullValue, fullLen,
+                          VARDATA_ANY(query), queryLen,
                           PG_GET_COLLATION());
        }
        else
        {
            /* Non-collation-aware comparison */
            r = memcmp(fullValue, VARDATA_ANY(query), Min(queryLen, fullLen));
-       }
 
-       if (r == 0)
-       {
-           if (queryLen > fullLen)
-               r = -1;
-           else if (queryLen < fullLen)
-               r = 1;
+           if (r == 0)
+           {
+               if (queryLen > fullLen)
+                   r = -1;
+               else if (queryLen < fullLen)
+                   r = 1;
+           }
        }
 
        switch (strategy)