Problem: Rust nightly after 2017-06-20 affects benchmarks negatively

[yrashk]

Running cargo bench acker in the pumpkindb_engine crate:

nightly-2017-06-20: (last known "good" nightly)

test script::tests::ackermann                                                  ... bench:  32,562,056 ns/iter (+/- 4,313,209)
test script::tests::ackermann_stack                                            ... bench:  24,391,250 ns/iter (+/- 4,758,534)

---

Most likely "first offender"

nightly-2017-06-21: (rust-lang/rust@445077963)

test script::tests::ackermann                                                  ... bench:  99,398,843 ns/iter (+/- 5,556,874)
test script::tests::ackermann_stack                                            ... bench:  51,625,752 ns/iter (+/- 3,971,108)

Random builds:

2017-07-24:

test script::tests::ackermann                                                  ... bench:  49,617,433 ns/iter (+/- 6,028,955)
test script::tests::ackermann_stack                                            ... bench:  36,797,438 ns/iter (+/- 4,442,438)

2017-07-01:

test script::tests::ackermann                                                  ... bench:  97,953,353 ns/iter (+/- 5,756,554)
test script::tests::ackermann_stack                                            ... bench:  51,599,955 ns/iter (+/- 3,756,541)

2017-07-12:

test script::tests::ackermann                                                  ... bench:  50,183,660 ns/iter (+/- 7,364,549)
test script::tests::ackermann_stack                                            ... bench:  39,244,258 ns/iter (+/- 6,607,862)

2017-06-23:

test script::tests::ackermann                                                  ... bench: 111,698,229 ns/iter (+/- 17,495,313)
test script::tests::ackermann_stack                                            ... bench:  58,807,694 ns/iter (+/- 4,591,336)

Last build:

2017-08-02:

test script::tests::ackermann                                                  ... bench:  49,253,683 ns/iter (+/- 9,255,754)
test script::tests::ackermann_stack                                            ... bench:  35,636,600 ns/iter (+/- 5,089,613)

Proposed solution: figure out what happened in the first place, why did it get so much slower?

[bstrie]

If you can narrow it down at all (and possibly even if you can't), please file a performance regression bug at https://github.com/rust-lang/rust/issues .

[yrashk]

Ben, for sure!