Explanations Table

[wags-1314]

This PR adds functionality and some more stats to the explanations algorithm:

• Adds a table showing explanations for each erroneous subexpression along with count
• Compares the algorithm to total error, and then calculates the confusion matrix, and precision recall for our predictions
• Shows root cause of overflows and underflows for rescues

[pavpanchekha]

We should merge, even if there are more cleanups possible, just to keep this in sync with the rest of Herbie.

[pavpanchekha]

Issue with pow(x, 1000). I think we need:

if G_x > 100 && ! exact(x)
if G_y > 100 && ! exact(y)

[pavpanchekha]

So I think list? is not quite right, because it can be a constant but not one exactly representable in float, like 1/3. There's gotta be some way to do this exactly right.

[wags-1314]

I have been thinking about this. I think it should be a composition of (a) expression that's a product or quotient of constants and (b) representable in floating point. (a) is easy enough not sure how to do (b).

[pavpanchekha]

I think we should write a little wrapper around eval-application and it can check something like:

(define (exact-constant? expr)
  (let ([val (eval-application-recursively expr)])
    (and val (= val (exact->inexact val)))))

Would be even better to use real->representation instead of just exact->inexact but that is more complex.

[pavpanchekha]

I think we want this to be (compile-progs exprs ctx) but there might be issues with different ctxs but maybe we're ok with some expressions erroring for now?

[wags-1314]

Will it crash on booleans and if? If yes, then I'd rather not use it because right now we don't get crashes on booleans and if.

[pavpanchekha]

Let's not mess with this for now.

[pavpanchekha]

It would be nice to make all-subexpressions-rev return the same kind of thing as all-subexpressions, in other words have it return what is called subexprs-list here. Then the code could be a little nicer.

[pavpanchekha]

Fine to do in another PR

[pavpanchekha]

You also wouldn't need repr-hash if we changed all-sub...s-rev

[pavpanchekha]

Proposal: let's not fix the exact-expr thing in this PR, let's do that in another PR, and merge this PR now.

[pavpanchekha]

There are two extra crashes, otherwise let's merge

[wags-1314]

Closing this in favor of #824

[pavpanchekha]

https://nightly.cs.washington.edu/reports/herbie/1716983316:nightly:explanations-table:18ecdfd05b/regression/1-xusedtobehardtosample/graph.html

[wags-1314]

Forgot to push the fix, sorry