Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit e54a758

Browse files
committed
Fix corner-case loss of precision in numeric_power().
This fixes a loss of precision that occurs when the first input is very close to 1, so that its logarithm is very small. Formerly, during the initial low-precision calculation to estimate the result weight, the logarithm was computed to a local rscale that was capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be as close as 1e-16383 to 1, hence its logarithm may be as small as 1e-16383, and so the local rscale needs to be allowed to exceed 16383, otherwise all precision is lost, leading to a poor choice of rscale for the full-precision calculation. Fix this by removing the cap on the local rscale during the initial low-precision calculation, as we already do in the full-precision calculation. This doesn't change the fact that the initial calculation is a low-precision approximation, computing the logarithm to around 8 significant digits, which is very fast, especially when the base is very close to 1. Patch by me, reviewed by Alvaro Herrera. Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
1 parent ba216d3 commit e54a758

File tree

3 files changed

+12
-1
lines changed

3 files changed

+12
-1
lines changed

src/backend/utils/adt/numeric.c

+5-1
Original file line numberDiff line numberDiff line change
@@ -10266,9 +10266,13 @@ power_var(const NumericVar *base, const NumericVar *exp, NumericVar *result)
1026610266
*/
1026710267
ln_dweight = estimate_ln_dweight(base);
1026810268

10269+
/*
10270+
* Set the scale for the low-precision calculation, computing ln(base) to
10271+
* around 8 significant digits. Note that ln_dweight may be as small as
10272+
* -SHRT_MAX, so the scale may exceed NUMERIC_MAX_DISPLAY_SCALE here.
10273+
*/
1026910274
local_rscale = 8 - ln_dweight;
1027010275
local_rscale = Max(local_rscale, NUMERIC_MIN_DISPLAY_SCALE);
10271-
local_rscale = Min(local_rscale, NUMERIC_MAX_DISPLAY_SCALE);
1027210276

1027310277
ln_var(base, &ln_base, local_rscale);
1027410278

src/test/regress/expected/numeric.out

+6
Original file line numberDiff line numberDiff line change
@@ -2483,6 +2483,12 @@ select coalesce(nullif(0.9999999999 ^ 23300000000000, 0), 0) as rounds_to_zero;
24832483
0
24842484
(1 row)
24852485

2486+
select round(((1 - 1.500012345678e-1000) ^ 1.45e1003) * 1e1000);
2487+
round
2488+
----------------------------------------------------------
2489+
25218976308958387188077465658068501556514992509509282366
2490+
(1 row)
2491+
24862492
-- cases that used to error out
24872493
select 0.12 ^ (-25);
24882494
?column?

src/test/regress/sql/numeric.sql

+1
Original file line numberDiff line numberDiff line change
@@ -1148,6 +1148,7 @@ select 1.2 ^ 345;
11481148
select 0.12 ^ (-20);
11491149
select 1.000000000123 ^ (-2147483648);
11501150
select coalesce(nullif(0.9999999999 ^ 23300000000000, 0), 0) as rounds_to_zero;
1151+
select round(((1 - 1.500012345678e-1000) ^ 1.45e1003) * 1e1000);
11511152

11521153
-- cases that used to error out
11531154
select 0.12 ^ (-25);

0 commit comments

Comments
 (0)