Feature request: confidence intervals for bar charts

[jaanli]

We have been using Mosaic for reporting on hundreds of thousands of hospital prices (demos here: https://beta.payless.health/examples/stlukes-bethlehem.html & https://beta.payless.health/examples/mount-sinai.html).

These prices are often listed according to minimum and maximum negotiated rates across several insurance products.

To accurately relay this information in a visualization, confidence intervals are necessary.

Would this be possible in Mosaic?

(In the docs I only found confidence intervals mentioned for the regressionY mark here: https://uwdata.github.io/mosaic/vgplot/#connected-marks)

[jheer]

There are two pieces to this: 1. Compute a confidence interval, and 2. Draw the interval as a mark.

For part 1, there are many possibilities. You could compute a standard parametric CI via an expression (e.g., MEAN(x) +/- 1.96 * STDDEV(x) / SQRT(COUNT()) for a 95% CI), though this bakes in a number of assumptions. Other methods, such as bootstraped CIs or model specific CIs would need to be computed through other means.

For part 2, given a calculated interval, you could use the link mark (for example) to visually represent the interval.

Something like this should work for a horizontally-oriented interval:

link(from(data), {
 x1: agg`AVG(x) - 1.96 * STDDEV(x) / SQRT(COUNT(*) FILTER (WHERE x IS NOT NULL))`,
 x2: agg`AVG(x) + 1.96 * STDDEV(x) / SQRT(COUNT(*) FILTER (WHERE x IS NOT NULL))`,
 y: 'your_y_variable'
})

[domoritz]

#380 adds support for variance.

[jheer]

Variance aggregates were always supported in general, but in the next release, variance, stddev, etc also have data cube indexing support. However, we don't yet support parsing of aggregate expressions to identify "indexable" aggregates, and so for now only top-level aggregate functions will be indexed. That said, if you have a client that queries directly for avg, stddev, and count as separate columns, they will be indexed and then you can put them together to form a client-side adjustable range of parametric CIs.