Benchmarks only track minimum number of allocations and memory usage

[Arkoniak]

It seems, that BenchmarkTools.Trial uses only single value for storing number of allocations and memory usage during trial runs. And judging from the results it track only minimum allocations and memory. Somewhat exaggerated example:

using BenchmarkTools

function f()
    n = rand() < 0.5 ? 1 : 100000
    return rand(n)
end

and result

julia> @benchmark f() evals = 1
BenchmarkTools.Trial:
  memory estimate:  64 bytes
  allocs estimate:  1
  --------------
  minimum time:     55.000 ns (0.00% GC)
  median time:      86.357 μs (0.00% GC)
  mean time:        56.463 μs (3.11% GC)
  maximum time:     573.892 μs (45.10% GC)
  --------------
  samples:          10000
  evals/sample:     1

which doesn't feel right.

[gdalle]

There are several reasons why the minimum is a good choice for benchmarking, but I agree it should be better documented