Variance and Std Deviation of a collection of Quantities..

[benhutchison]

Erik, have you ever needed to compute the variance and standard deviation of some Quantity[V, U]s?

The variance ends up in units U * U. I'm not sure if that's useful. Theoretically, taking the square root will mean the std dev has the right units, but there isn't a sqr root function ATM that will cleanly turn U * U into U, AFAICT?

My inclination, for now, is to coerce variance to type U.

[erikerlandson]

It's a good question - If I had collections for Quantity, then I'd probably (under the hood) compute stdv in unitless raw value types, but return the final output as raw_stdv.withUnit[U], and similarly for variance raw_var.withUnit[U^2]

[erikerlandson]

The thing that's doing the square root should be this:
https://github.com/erikerlandson/coulomb/blob/scala3/spire/src/main/scala/coulomb/ops/algebra/spire/bigdecimal.scala#L27

[erikerlandson]

My recollection is, that I was assuming if people wanted roots, they were ok with floating point jitter, and I was trying to leverage the fpow "standard", which you can see forces the exponent from rational to fp, which is likely going to work against exact answers to rational powers.

We might reconsider this, it would require a redesign of the algebras I used to define powers.

[erikerlandson]

or we could do something for q.sqrt and maybe specialized algebras for that particular case

[erikerlandson]

I'm still slightly puzzled by why this is happening. 1/2 has an exact fp representation, even if other fractions don't.

[benhutchison]

"I was assuming if people wanted roots, they were ok with floating point jitter" <- on consideration I think this is a decent first assumption. While Im working with financial data where to-the-cent accuracy is needed, the only use of square root shows up in stats over the transactions amounts, ie stddev.

The only reason I noticed this was a unit test where there was a contrived sqrt(4) that failed to eql 2. If I use epsilon equality like I probably ought to have been, the problem goes away.