Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH add confidence intervals to plotly results #339

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jolars
Copy link
Contributor

@jolars jolars commented Apr 28, 2022

This pull request adds support for confidence intervals to the result plots, but just for the plotly versions of the plots. The idea is to have intervals that show the confidence for the median in addition to quantile plots, so that the width of the intervals decrease as the number of repetitions increase. I've changed the slider to a drop-down list instead. The default is still quantiles.

Here's for instance a benchmark using quantiles and confidence intervals, which was run for 50 repetitions:

image

image

I've used standard percentile bootstrap CIs here because they are always positive and need less babysitting that bias-controlled intervals do. The bootstrapping does add a bit of extra overhead, but I've found it to be largely neglible. But if you would prefer an interval that's faster to compute then there are alternatives.

This functionality could be added to the CLI interface too of course, although it would require a new argument.

My proficiency in javascript is horrible by the way, so I hope I didn't screw anything up.

Copy link
Collaborator

@agramfort agramfort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems it can make the plots more readable 👍

@codecov-commenter
Copy link

Codecov Report

Merging #339 (c26bf60) into main (640c38c) will decrease coverage by 0.15%.
The diff coverage is 29.41%.

@@            Coverage Diff             @@
##             main     #339      +/-   ##
==========================================
- Coverage   55.55%   55.40%   -0.16%     
==========================================
  Files          42       42              
  Lines        2403     2415      +12     
  Branches      443      445       +2     
==========================================
+ Hits         1335     1338       +3     
- Misses        981      990       +9     
  Partials       87       87              

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants