Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KSampleADTest fails on 32-bit with longer data arrays #244

Open
oschulz opened this issue Sep 17, 2021 · 3 comments
Open

KSampleADTest fails on 32-bit with longer data arrays #244

oschulz opened this issue Sep 17, 2021 · 3 comments

Comments

@oschulz
Copy link

oschulz commented Sep 17, 2021

On 32-bit systems (tested on 32-bit Linux), KSampleADTest starts to fail when the data arrays grow larger (not really all that large):

julia> using HypothesisTests

julia> pvalue(KSampleADTest(randn(100), randn(100)))
0.4684450603589013

julia> pvalue(KSampleADTest(randn(1000), randn(1000)))
ERROR: DomainError with -0.008755839260454446:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
[...]

julia> pvalue(KSampleADTest(randn(50000), randn(50000)))
NaN

julia> versioninfo()
Julia Version 1.6.2
Commit 1b93d53fc4 (2021-07-14 15:36 UTC)
Platform Info:
  OS: Linux (i686-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

On 64-bit, the above runs fine.

@oschulz
Copy link
Author

oschulz commented Sep 17, 2021

Same happens with ApproximateTwoSampleKSTest. I think for the KS-test it's due to this:

function pvalue(x::ApproximateTwoSampleKSTest; tail=:both)
    n = x.n_x*x.n_y/(x.n_x+x.n_y)
    #...
end

We could use float(x.n_x) * float(x.n_y) or so to prevent the integer overflow in the product.

@nalimilan
Copy link
Member

Or maybe just doing n = x.n_x*(x.n_y/(x.n_x+x.n_y))? Anyway feel free to make a PR.

@oschulz
Copy link
Author

oschulz commented Sep 27, 2021

I'll put it on my to-do list. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants