Adds automatic display of bechmark results #90
Conversation
|
Thanks for the contribution @aa25desh What do you mean by "relative precision"? |
|
This function measures the relative precision of the result in a more informative way than |
i see, you'd like to store additional columns in the results table. i had a quick look and i'm not sure that feature is available in the user interface of PkgBenchmark.jl |
|
@lbenet should we find some workaround or is it ok for now? |
There was a problem hiding this comment.
I have added few general remarks. I guess the most important is how this works. Can you show it? Perhaps @dpsanders has other comments on this PR.
| # Only trigger the benchmark job when you add `run benchmark` label to the PR | ||
| jobs: | ||
| Benchmark: | ||
| runs-on: ubuntu-latest | ||
| if: contains(github.event.pull_request.labels.*.name, 'run benchmark') |
There was a problem hiding this comment.
Can you show/describe how this works? It seems to me from these lines that you need to add run benchmark somewhere to see this working.
There was a problem hiding this comment.
Can you show/describe how this works?
BenchmarkCI crates Benchmark-result branch and saves results there.
For example look at this PR, github-actions bot comments the results there, and saves them in Benchmark-result branch here (In this repositories case we need not mention run benchmark explicitly)
There are many ways one can trigger Benchmarking. In our case when some one adds run benchmark label to the comment of PR. We can also have configuration that trigger benchmarking for each PR without mentioning it explicitly.
| /.benchmarkci | ||
| /benchmark/*.json |
There was a problem hiding this comment.
Are these files appearing? How are they generated?
There was a problem hiding this comment.
Locally these files appear for me.
They are generated while saving the results of benchmark to the file look here
benchmark/Manifest.toml
Outdated
| @@ -0,0 +1,488 @@ | |||
| # This file is machine-generated - editing it directly is not advised | |||
There was a problem hiding this comment.
This file should be excluded from the PR; it is not needed.
There was a problem hiding this comment.
I have been following BenchmarkCIExample.jl there they have kept Manifest.toml in the benchmarking folder.
But now I see the BenchmarkCI.jl working without Manifest.toml, looks like we don't need it.
lbenet
changed the title
lbenet
changed the title
|
I think this PR is quite interesting; yet, I am not able to see any results; while I see that something is being triggered by github actions, there are no details since it is simply skipped. @dpsanders @mforets What do you think about this? Shall we give it a try? |
|
from https://github.com/JuliaIntervals/TaylorModels.jl/pull/90/files#r490853817 we shall try |
Yes as a comment in PR. |
Look at issue #47
With this commit I am getting required results locally.
As
relative precisionis stored inRELPRECdictionary, which is not generated by PkgBenchmark. we would not get it automatically.@mforets Is there way to get
relative precision?Still there might be few things to be done by repository owner