Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit aad663a

Browse files
committed
Reduce number of bytes examined by convert_one_string_to_scalar().
Previously, convert_one_string_to_scalar() would examine up to 20 bytes of the input string, producing a scalar conversion with theoretical precision far greater than is of any possible use considering the other limitations on the accuracy of the resulting selectivity estimate. (I think this choice might pre-date the caller-level logic that strips any common prefix of the strings; before that, there could have been value in scanning the strings far enough to use all the precision available in a double.) Aside from wasting cycles to little purpose, this choice meant that the "denom" variable could grow to as much as 256^21 = 3.74e50, which could overflow in some non-IEEE float arithmetics. While we don't really support any machines with non-IEEE arithmetic anymore, this still seems like quite an unnecessary platform dependency. Limit the scan to 12 bytes instead, thus limiting "denom" to 256^13 = 2.03e31, a value more likely to be computable everywhere. Per testing by Greg Stark, which showed overflow failures in our standard regression tests on VAX.
1 parent 44ed65a commit aad663a

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

src/backend/utils/adt/selfuncs.c

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3861,10 +3861,16 @@ convert_one_string_to_scalar(char *value, int rangelo, int rangehi)
38613861
return 0.0; /* empty string has scalar value 0 */
38623862

38633863
/*
3864-
* Since base is at least 10, need not consider more than about 20 chars
3864+
* There seems little point in considering more than a dozen bytes from
3865+
* the string. Since base is at least 10, that will give us nominal
3866+
* resolution of at least 12 decimal digits, which is surely far more
3867+
* precision than this estimation technique has got anyway (especially in
3868+
* non-C locales). Also, even with the maximum possible base of 256, this
3869+
* ensures denom cannot grow larger than 256^13 = 2.03e31, which will not
3870+
* overflow on any known machine.
38653871
*/
3866-
if (slen > 20)
3867-
slen = 20;
3872+
if (slen > 12)
3873+
slen = 12;
38683874

38693875
/* Convert initial characters to fraction */
38703876
base = rangehi - rangelo + 1;

0 commit comments

Comments
 (0)