mass_absolute returns NaN for some negative time-series

[plodocus]

I found this while playing with the PAMAP dataset. Haven't been able to reproduce this with random numbers, so here's the relevant data and code.

Q = np.array([-13.09, -14.1 , -15.08, -16.31, -17.13, -17.5 , -18.07, -18.07,
       -17.48, -16.24, -14.88, -13.56, -12.65, -11.93, -11.48, -11.06,
       -10.83, -10.67, -10.59, -10.81, -10.92, -11.15, -11.37, -11.53,
       -11.19, -11.08, -10.48, -10.14,  -9.92,  -9.99, -10.11,  -9.92,
        -9.7 ,  -9.47,  -9.06,  -9.01,  -8.79,  -8.67,  -8.33,  -8.  ,
        -8.26,  -8.  ,  -7.54,  -7.32,  -7.13,  -7.24,  -7.43,  -7.93,
        -8.8 ,  -9.71])
print(stumpy.core.mass_absolute(Q, Q))

I get nan, but of course it should be 0. The problem is some float imprecision in _mass_absolute that leads to negative values that can't be properly np.sqrted.

Easy to fix, of course. A test for this edge case should be added as well.

[seanlaw]

@DanBenHa Thank you for this reproducer! You're right. It looks like it is trying to take the square root of a very small negative number so maybe we should just replace with:

D = Q_squared + T_squared - 2 * QT
D[D < 0] = 0.0
return np.sqrt(D)

And then, as you said, add this to our test suite. Would you like to submit a PR for this? Otherwise, I can do it

[plodocus]

Sure, I'll do it. My ad hoc solution was using more of numpy's syntactic sugar, i.e.
np.sqrt((Q_squared + T_squared - 2 * QT).clip(min=0))
Not sure if this is less maintainable, but I liked the one-liner ;)

[seanlaw]

Oooh, I like that (please go for it)! I learned something new too

[plodocus]

Ah, unfortunately numba doesn't support clip yet: numba/numba#3468

[seanlaw]

Here are some reference discussion (but no solution):

1. Numerical precision of euclidean_distances with float32 scikit-learn/scikit-learn#9354
2. https://www.robots.ox.ac.uk/~albanie/notes/Euclidean_distance_trick.pdf
3. https://cs.stackexchange.com/questions/106402/stable-and-fast-computation-of-the-squared-euclidean-distance-matrix
4. Catastrophic cancellation with Euclidean distance calculation spotify/annoy#314

From what I can tell, the issue may be coming from imprecisions in the sliding dot product

[plodocus]

Thanks for posting the references!