allow numeric rounding to be 3...7

[MichaelChirico]

Closes #1898

It's trivial enough to implement, I'm not 100% sure the best way to test it's WAI, esp. lacking MRE from original issue. Here's what I came up with:

DT = data.table(x = pi*(1 + 2^((-128):0)))

options(digits=17L)
for (ii in 0:7) { setNumericRounding(ii); print(DT[, .N, by=x]) }

with output:

                     x     N
                 <num> <int>
 1: 3.1415926535897931    76
 2: 3.1415926535897940     1
 3: 3.1415926535897944     1
 4: 3.1415926535897958     1
 5: 3.1415926535897989     1
 6: 3.1415926535898042     1
 7: 3.1415926535898153     1
 8: 3.1415926535898380     1
 9: 3.1415926535898824     1
10: 3.1415926535899716     1
11: 3.1415926535901502     1
12: 3.1415926535905072     1
13: 3.1415926535912218     1
14: 3.1415926535926504     1
15: 3.1415926535955077     1
16: 3.1415926536012222     1
17: 3.1415926536126513     1
18: 3.1415926536355094     1
19: 3.1415926536812253     1
20: 3.1415926537726580     1
21: 3.1415926539555228     1
22: 3.1415926543212520     1
23: 3.1415926550527113     1
24: 3.1415926565156291     1
25: 3.1415926594414656     1
26: 3.1415926652931376     1
27: 3.1415926769964826     1
28: 3.1415927004031716     1
29: 3.1415927472165501     1
30: 3.1415928408433071     1
31: 3.1415930280968216     1
32: 3.1415934026038497     1
33: 3.1415941516179062     1
34: 3.1415956496460193     1
35: 3.1415986457022460     1
36: 3.1416046378146985     1
37: 3.1416166220396038     1
38: 3.1416405904894145     1
39: 3.1416885273890358     1
40: 3.1417844011882790     1
41: 3.1419761487867643     1
42: 3.1423596439837360     1
43: 3.1431266343776789     1
44: 3.1446606151655643     1
45: 3.1477285767413359     1
46: 3.1538644998928782     1
47: 3.1661363461959633     1
48: 3.1906800388021335     1
49: 3.2397674240144743     1
50: 3.3379421944391554     1
51: 3.5342917352885173     1
52: 3.9269908169872414     1
53: 4.7123889803846897     1
54: 6.2831853071795862     1
                     x     N
                     x     N
                 <num> <int>
 1: 3.1415926535897931    83
 2: 3.1415926535898824     1
 3: 3.1415926535899716     1
 4: 3.1415926535901502     1
 5: 3.1415926535905072     1
 6: 3.1415926535912218     1
 7: 3.1415926535926504     1
 8: 3.1415926535955077     1
 9: 3.1415926536012222     1
10: 3.1415926536126513     1
11: 3.1415926536355094     1
12: 3.1415926536812253     1
13: 3.1415926537726580     1
14: 3.1415926539555228     1
15: 3.1415926543212520     1
16: 3.1415926550527113     1
17: 3.1415926565156291     1
18: 3.1415926594414656     1
19: 3.1415926652931376     1
20: 3.1415926769964826     1
21: 3.1415927004031716     1
22: 3.1415927472165501     1
23: 3.1415928408433071     1
24: 3.1415930280968216     1
25: 3.1415934026038497     1
26: 3.1415941516179062     1
27: 3.1415956496460193     1
28: 3.1415986457022460     1
29: 3.1416046378146985     1
30: 3.1416166220396038     1
31: 3.1416405904894145     1
32: 3.1416885273890358     1
33: 3.1417844011882790     1
34: 3.1419761487867643     1
35: 3.1423596439837360     1
36: 3.1431266343776789     1
37: 3.1446606151655643     1
38: 3.1477285767413359     1
39: 3.1538644998928782     1
40: 3.1661363461959633     1
41: 3.1906800388021335     1
42: 3.2397674240144743     1
43: 3.3379421944391554     1
44: 3.5342917352885173     1
45: 3.9269908169872414     1
46: 4.7123889803846897     1
47: 6.2831853071795862     1
                     x     N
                     x     N
                 <num> <int>
 1: 3.1415926535897931    90
 2: 3.1415926536012222     2
 3: 3.1415926536355094     1
 4: 3.1415926536812253     1
 5: 3.1415926537726580     1
 6: 3.1415926539555228     1
 7: 3.1415926543212520     1
 8: 3.1415926550527113     1
 9: 3.1415926565156291     1
10: 3.1415926594414656     1
11: 3.1415926652931376     1
12: 3.1415926769964826     1
13: 3.1415927004031716     1
14: 3.1415927472165501     1
15: 3.1415928408433071     1
16: 3.1415930280968216     1
17: 3.1415934026038497     1
18: 3.1415941516179062     1
19: 3.1415956496460193     1
20: 3.1415986457022460     1
21: 3.1416046378146985     1
22: 3.1416166220396038     1
23: 3.1416405904894145     1
24: 3.1416885273890358     1
25: 3.1417844011882790     1
26: 3.1419761487867643     1
27: 3.1423596439837360     1
28: 3.1431266343776789     1
29: 3.1446606151655643     1
30: 3.1477285767413359     1
31: 3.1538644998928782     1
32: 3.1661363461959633     1
33: 3.1906800388021335     1
34: 3.2397674240144743     1
35: 3.3379421944391554     1
36: 3.5342917352885173     1
37: 3.9269908169872414     1
38: 4.7123889803846897     1
39: 6.2831853071795862     1
                     x     N
                     x     N
                 <num> <int>
 1: 3.1415926535897931    98
 2: 3.1415926565156291     2
 3: 3.1415926652931376     1
 4: 3.1415926769964826     1
 5: 3.1415927004031716     1
 6: 3.1415927472165501     1
 7: 3.1415928408433071     1
 8: 3.1415930280968216     1
 9: 3.1415934026038497     1
10: 3.1415941516179062     1
11: 3.1415956496460193     1
12: 3.1415986457022460     1
13: 3.1416046378146985     1
14: 3.1416166220396038     1
15: 3.1416405904894145     1
16: 3.1416885273890358     1
17: 3.1417844011882790     1
18: 3.1419761487867643     1
19: 3.1423596439837360     1
20: 3.1431266343776789     1
21: 3.1446606151655643     1
22: 3.1477285767413359     1
23: 3.1538644998928782     1
24: 3.1661363461959633     1
25: 3.1906800388021335     1
26: 3.2397674240144743     1
27: 3.3379421944391554     1
28: 3.5342917352885173     1
29: 3.9269908169872414     1
30: 4.7123889803846897     1
31: 6.2831853071795862     1
                     x     N
                     x     N
                 <num> <int>
 1: 3.1415926535897931   105
 2: 3.1415930280968216     2
 3: 3.1415941516179062     1
 4: 3.1415956496460193     1
 5: 3.1415986457022460     1
 6: 3.1416046378146985     1
 7: 3.1416166220396038     1
 8: 3.1416405904894145     1
 9: 3.1416885273890358     1
10: 3.1417844011882790     1
11: 3.1419761487867643     1
12: 3.1423596439837360     1
13: 3.1431266343776789     1
14: 3.1446606151655643     1
15: 3.1477285767413359     1
16: 3.1538644998928782     1
17: 3.1661363461959633     1
18: 3.1906800388021335     1
19: 3.2397674240144743     1
20: 3.3379421944391554     1
21: 3.5342917352885173     1
22: 3.9269908169872414     1
23: 4.7123889803846897     1
24: 6.2831853071795862     1
                     x     N
                     x     N
                 <num> <int>
 1: 3.1415926535897931   110
 2: 3.1416046378146985     6
 3: 3.1423596439837360     1
 4: 3.1431266343776789     1
 5: 3.1446606151655643     1
 6: 3.1477285767413359     1
 7: 3.1538644998928782     1
 8: 3.1661363461959633     1
 9: 3.1906800388021335     1
10: 3.2397674240144743     1
11: 3.3379421944391554     1
12: 3.5342917352885173     1
13: 3.9269908169872414     1
14: 4.7123889803846897     1
15: 6.2831853071795862     1
                    x     N
                <num> <int>
1: 3.1415926535897931   124
2: 3.3379421944391554     1
3: 3.5342917352885173     1
4: 3.9269908169872414     1
5: 4.7123889803846897     1
6: 6.2831853071795862     1
                    x     N
                <num> <int>
1: 3.1415926535897931   129

Looks vaguely OK.

Q before proceeding to round out the PR is (1) if there's a good reason to limit setNumericRounding<=2 (2) whether we should be much more careful about designing tests, e.g. working backwards from bit representations.

[github-actions]

Generated via commit 7e0c88f

Download link for the artifact containing the test results: ↓ atime-results.zip

Time taken to finish the standard R installation steps: 11 minutes and 52 seconds

Time taken to run atime::atime_pkg on the tests: 3 minutes and 16 seconds

[jangorecki]

Looks ok, maybe user who requested that could test it...

[MichaelChirico]

cc @rmcgehee

[MichaelChirico]

Not as easy as I'd hoped -- here's taken from test 1147.*:

https://github.com/Rdatatable/data.table/blob/014c82fd7b04e6e41da9680d41b25620a11f46b6/inst/tests/tests.Rraw#L4123-L4124

The behavior for setNumericRounding(n!=4) seems reasonable enough, no idea what's going on for n=4 though. Makes me think we need to examine things a lot more carefully & construct better examples to understand what's going on to proceed:

names(x) = x
m = sapply(0:7, \(d) { setNumericRounding(d); names(x)[forderv(x)]})
rownames(m) = 1:nrow(m)
colnames(m) = 0:7
format(m, just='left')
#    0                  1                  2                  3                  4                  5                  6                  7                 
# 1  "NA              " "NA              " "NA              " "NA              " "NA              " "NA              " "NA              " "NA              "
# 2  "NA              " "NA              " "NA              " "NA              " "NA              " "NA              " "NA              " "NA              "
# 3  "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             "
# 4  "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             " "NaN             "
# 5  "-Inf            " "-Inf            " "-Inf            " "-Inf            " "-Inf            " "-Inf            " "-Inf            " "-Inf            "
# 6  "-7.18918111     " "-7.18918111     " "-7.18918111     " "-7.18918111     " "5.18909000000002" "-7.18918        " "-7.18918        " "-7.18918        "
# 7  "-7.18918        " "-7.18918        " "-7.18918        " "-7.18918        " "5.18909         " "-7.18918111     " "-7.18918111     " "-7.18918111     "
# 8  "5.18908999999964" "5.18908999999964" "5.18909000000002" "5.18909000000002" "5.18908999999964" "5.18909000000002" "5.18909000000002" "8               "
# 9  "5.18908999999999" "5.18909000000002" "5.18909         " "5.18909         " "5.18908999999999" "5.18909         " "5.18909         " "5.18909000000002"
# 10 "5.18909         " "5.18909         " "5.18908999999964" "5.18908999999964" "8               " "5.18908999999964" "5.18908999999964" "5.18909         "
# 11 "5.18909000000002" "5.18908999999999" "5.18908999999999" "5.18908999999999" "-7.18918111     " "5.18908999999999" "5.18908999999999" "5.18908999999964"
# 12 "8               " "8               " "8               " "8               " "-7.18918        " "8               " "8               " "5.18908999999999"
# 13 "Inf             " "Inf             " "Inf             " "Inf             " "Inf             " "Inf             " "Inf             " "Inf             "

[MichaelChirico]

Closed the issue with comment:

#1898 (comment)