[In-Progress] Transition redis-benchmark to memtier_benchmark benchmarks for result stabilization

[filipecosta90]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6b52674) 78.38% compared to head (72976ad) 78.02%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1020      +/-   ##
==========================================
- Coverage   78.38%   78.02%   -0.37%     
==========================================
  Files          15       15              
  Lines        3873     3891      +18     
==========================================
  Hits         3036     3036              
- Misses        837      855      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[filipecosta90]

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

• Detected a total of 25 stable tests between versions.
• Detected a total of 4 improvements above the improvement water line.
• Detected a total of 9 regressions bellow the regression water line 5.0.

You can check a comparison in detail via the grafana link

Comparison between master and redis_benchmark.to.memtier.

Time Period from 30 days ago. (environment used: oss-standalone)

Test Case	Baseline master (median obs. +- std.dev)	Comparison redis_benchmark.to.memtier (median obs. +- std.dev)	% change (higher-better)	Note
json_arrappend_geojson	66653 +- 5.6% (7 datapoints)	67783	1.7%	waterline=5.6%. No Change
json_get_ResultSet.totalResultsAvailable_jsonsl-yahoo2_json	121942 +- 4.7% (7 datapoints)	122353	0.3%	No Change
json_get_[0]_jsonsl-1	115599 +- 8.3% (7 datapoints)	116883	1.1%	waterline=8.3%. No Change
json_get_[7]_jsonsl-1	120476 +- 5.8% (7 datapoints)	127686	6.0%	waterline=5.8%. IMPROVEMENT
json_get_[8].zero_jsonsl-1	109880 +- 7.7% (7 datapoints)	112798	2.7%	waterline=7.7%. No Change
json_get_[web-app].servlet[0][servlet-name]_json-parser-0000	87900 +- 5.8% (7 datapoints)	114739	30.5%	waterline=5.8%. IMPROVEMENT
json_get_[web-app].servlet[0]_json-parser-0000	82977 +- 5.9% (7 datapoints)	88617	6.8%	waterline=5.9%. IMPROVEMENT
json_get_[web-app].servlet_json-parser-0000	75747 +- 4.7% (7 datapoints)	77775	2.7%	No Change
json_get_array_of_docs[1]_pass_100_json	107518 +- 7.3% (7 datapoints)	105220	-2.1%	waterline=7.3%. No Change
json_get_array_of_docs[1]sclr_pass_100_json	117639 +- 7.2% (7 datapoints)	108127	-8.1%	waterline=7.2%. REGRESSION
json_get_array_of_docs_pass_100_json	116954 +- 7.6% (7 datapoints)	119298	2.0%	waterline=7.6%. No Change
json_get_fulldoc_json-parser-0000	83325 +- 4.6% (7 datapoints)	81541	-2.1%	No Change
json_get_fulldoc_jsonsl-1	92157 +- 3.8% (7 datapoints)	95754	3.9%	potential IMPROVEMENT
json_get_fulldoc_jsonsl-yahoo2_json	25798 +- 2.8% (7 datapoints)	27033	4.8%	potential IMPROVEMENT
json_get_fulldoc_jsonsl-yelp_json	13880 +- 1.9% (7 datapoints)	14025	1.0%	No Change
json_get_fulldoc_pass_100_json	125780 +- 3.2% (7 datapoints)	114619	-8.9%	REGRESSION
json_get_key_empty	140837 +- 6.9% (7 datapoints)	124246	-11.8%	waterline=6.9%. REGRESSION
json_get_message.code_jsonsl-yelp_json	120473 +- 6.7% (7 datapoints)	116414	-3.4%	waterline=6.7%. potential REGRESSION
json_get_sclr_pass_100_json	131572 +- 5.4% (7 datapoints)	120626	-8.3%	waterline=5.4%. REGRESSION
json_get_sub_doc.sclr_pass_100_json	121203 +- 4.4% (7 datapoints)	121889	0.6%	No Change
json_get_sub_doc_pass_100_json	119036 +- 6.1% (7 datapoints)	118243	-0.7%	waterline=6.1%. No Change
json_numincrby_num_1	124214 +- 5.1% (7 datapoints)	118580	-4.5%	waterline=5.1%. potential REGRESSION
json_nummultby_num_2	124991 +- 7.0% (7 datapoints)	125547	0.4%	waterline=7.0%. No Change
json_recursive_descent_with_filter_uid_issue674	146 +- 0.5% (7 datapoints)	145	-1.0%	No Change
json_set_ResultSet.totalResultsAvailable_1_jsonsl-yahoo2_json	114278 +- 5.8% (7 datapoints)	116749	2.2%	waterline=5.8%. No Change
json_set_[0]foo_jsonsl-1	115596 +- 7.0% (7 datapoints)	104594	-9.5%	waterline=7.0%. REGRESSION
json_set_fulldoc_api_replies_q1_google_autocomplete	48066 +- 2.9% (7 datapoints)	49190	2.3%	No Change
json_set_fulldoc_api_replies_q2_gmaps_areatraffic	75605 +- 4.4% (7 datapoints)	81458	7.7%	IMPROVEMENT
json_set_fulldoc_api_replies_q3_gmaps_passiveassist	41141 +- 2.2% (7 datapoints)	39514	-4.0%	potential REGRESSION
json_set_fulldoc_api_replies_q4_gmaps_assist	97553 +- 4.6% (7 datapoints)	97078	-0.5%	No Change
json_set_fulldoc_pass-json-parser-0000	40722 +- 2.6% (7 datapoints)	39204	-3.7%	potential REGRESSION
json_set_fulldoc_pass_100_json	88098 +- 6.2% (7 datapoints)	79671	-9.6%	waterline=6.2%. REGRESSION
json_set_fulldoc_yahoo2	11065 +- 4.0% (7 datapoints)	11480	3.7%	potential IMPROVEMENT
json_set_key_empty	150367 +- 5.1% (7 datapoints)	133753	-11.0%	waterline=5.1%. REGRESSION
json_set_message.code_1_jsonsl-yelp_json	112352 +- 4.3% (7 datapoints)	101515	-9.6%	REGRESSION
json_set_num_0	150367 +- 6.6% (7 datapoints)	132987	-11.6%	waterline=6.6%. REGRESSION
json_set_sclr_1_pass_100_json	117639 +- 7.7% (7 datapoints)	119912	1.9%	waterline=7.7%. No Change
json_set_sclr_pass_100_json	123451 +- 5.1% (7 datapoints)	118981	-3.6%	waterline=5.1%. potential REGRESSION