[In-Progress] Extending ingestion benchmarks. Included a 10M docs single field benchmark

[filipecosta90]

Bellow, here's a quick sample of each added test:

• JSON.SET of 10M docs with a single text field within an array
  sample command:

"JSON.SET" "doc:single:bb90e0afaf4b4232968016756d5755ad:0" "." "[{\"field1\": \"03247JPN5JWWG00T\"}]"

• JSON.SET of 10M docs with a single numeric int field within an array
  sample command:

"JSON.SET" "doc:single:bb90e0afaf4b4232968016756d5755ad:0" "." "[{\"field1\": 3979443}]"

• JSON.SET of 10M docs with a single nested array with a text field within it
  sample command:

"JSON.SET" "doc:single:a96a2f91fa0b42a18739c466cd57a633:266100" "." "[{\"field1\": [\"HNQDDNUFELI\"]}]

• JSON.SET of 10M docs with a single nested array with a dict with text field within it
  sample command:

"JSON.SET" "doc:single:8f2d70bd81cd4b4691c60408d3dac364:849454" "." "[{\"field1\": {\"field2\": \"BM8RR8UVTS8TD\"}}]"

• JSON.SET of 10M docs with a single nested array with a dict with an array with a text field within it
  sample command:

"JSON.SET" "doc:single:c00101ca14fd4caeace60bb74067bb3d:761505" "." "[{\"field1\": {\"field2\": [\"IV9UE15ZYUPXSX\"]}}]"

• JSON.SET of 10M docs with a single nested array with a dict with numeric field within it
  sample command:

"JSON.SET" "doc:single:8f2d70bd81cd4b4691c60408d3dac364:849454" "." "[{\"field1\": {\"field2\": 1111}}]"

• JSON.SET of 10M docs with a single text field
  sample command:

"JSON.SET" "doc:single:36005850af32483692828d52805ba519:0" "." "{\"field1\": \"03247JPN5JWWG00T\"}"

• JSON.SET of 10M docs with a single numeric float field

"JSON.SET" "doc:single:56ac1c98413d4027a93f53ebf86101f5:99" "." "{\"field1\": 0.7904532831270685}"

• JSON.SET of 10M docs with a single numeric int field

"JSON.SET" "doc:single:ec2dfc5c975d4f82887a32741f1dc32b:0" "." "{\"field1\": 3979443}"

• JSON.SET of 10M docs with a single bool field

 "JSON.SET" "doc:single:80698057e4cd4abeb035f317607cf1af:0" "." "{\"field1\": false}"

[filipecosta90]

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

• Detected a total of 21 stable tests between versions.
• Detected a total of 3 highly unstable benchmarks.
• Detected a total of 5 improvements above the improvement water line.
• Detected a total of 12 regressions bellow the regression water line 5.0.

You can check a comparison in detail via the grafana link

Comparison between master and extend.ingestion.benchmarks.

Time Period from 30 days ago. (environment used: oss-standalone)

Test Case	Baseline master (median obs. +- std.dev)	Comparison extend.ingestion.benchmarks (median obs. +- std.dev)	% change (higher-better)	Note
json_arrappend_geojson	75455 +- 3.2% (5 datapoints)	70166	-7.0%	REGRESSION
json_get_ResultSet.totalResultsAvailable_jsonsl-yahoo2_json	132440 +- 8.0% (5 datapoints)	119039	-10.1%	waterline=8.0%. REGRESSION
json_get_[0]_jsonsl-1	122690 +- 6.9% (5 datapoints)	123451	0.6%	waterline=6.9%. No Change
json_get_[7]_jsonsl-1	135124 +- 7.6% (5 datapoints)	112984	-16.4%	waterline=7.6%. REGRESSION
json_get_[8].zero_jsonsl-1	125777 +- 4.3% (5 datapoints)	128199	1.9%	No Change
json_get_[web-app].servlet[0][servlet-name]_json-parser-0000	98756 +- 5.5% (5 datapoints)	105252	6.6%	waterline=5.5%. IMPROVEMENT
json_get_[web-app].servlet[0]_json-parser-0000	81623 +- 4.4% (5 datapoints)	77211	-5.4%	REGRESSION
json_get_[web-app].servlet_json-parser-0000	80636 +- 6.1% (5 datapoints)	71674	-11.1%	waterline=6.1%. REGRESSION
json_get_array_of_docs[1]_pass_100_json	112352 +- 4.9% (5 datapoints)	119039	6.0%	IMPROVEMENT
json_get_array_of_docs[1]sclr_pass_100_json	108686 +- 2.4% (5 datapoints)	113629	4.5%	potential IMPROVEMENT
json_get_array_of_docs_pass_100_json	128195 +- 6.5% (5 datapoints)	119752	-6.6%	waterline=6.5%. REGRESSION
json_get_fulldoc_json-parser-0000	80961 +- 5.8% (5 datapoints)	84380	4.2%	waterline=5.8%. potential IMPROVEMENT
json_get_fulldoc_jsonsl-1	101513 +- 3.6% (5 datapoints)	99000	-2.5%	No Change
json_get_fulldoc_jsonsl-yahoo2_json	27201 +- 2.7% (5 datapoints)	25797	-5.2%	REGRESSION
json_get_fulldoc_jsonsl-yelp_json	14175 +- 3.7% (5 datapoints)	13594	-4.1%	potential REGRESSION
json_get_fulldoc_pass_100_json	125780 +- 6.6% (5 datapoints)	125777	-0.0%	waterline=6.6%. No Change
json_get_key_empty	145977 +- 8.8% (5 datapoints)	132443	-9.3%	waterline=8.8%. REGRESSION
json_get_message.code_jsonsl-yelp_json	121945 +- 9.6% (5 datapoints)	135124	10.8%	waterline=9.6%. IMPROVEMENT
json_get_sclr_pass_100_json	138877 +- 6.4% (5 datapoints)	116271	-16.3%	waterline=6.4%. REGRESSION
json_get_sub_doc.sclr_pass_100_json	121200 +- 8.5% (5 datapoints)	138881	14.6%	waterline=8.5%. IMPROVEMENT
json_get_sub_doc_pass_100_json	135124 +- 8.8% (5 datapoints)	125777	-6.9%	waterline=8.8%. potential REGRESSION
json_numincrby_num_1	121942 +- 6.3% (5 datapoints)	120476	-1.2%	waterline=6.3%. No Change
json_nummultby_num_2	114278 +- 12.7% UNSTABLE (5 datapoints)	122690	7.4%	UNSTABLE (very high variance)
json_recursive_descent_with_filter_uid_issue674	159 +- 1.0% (5 datapoints)	158	-0.6%	No Change
json_set_ResultSet.totalResultsAvailable_1_jsonsl-yahoo2_json	124217 +- 4.4% (5 datapoints)	123448	-0.6%	No Change
json_set_[0]foo_jsonsl-1	110487 +- 8.6% (5 datapoints)	114932	4.0%	waterline=8.6%. potential IMPROVEMENT
json_set_[web-app].servlet[0][servlet-name]_bar_json-parser-0000	94778 +- 2.2% (5 datapoints)	97550	2.9%	No Change
json_set_fulldoc_api_replies_q1_google_autocomplete	48473 +- 5.2% (5 datapoints)	46072	-5.0%	waterline=5.2%. potential REGRESSION
json_set_fulldoc_api_replies_q2_gmaps_areatraffic	72983 +- 10.5% UNSTABLE (5 datapoints)	79356	8.7%	UNSTABLE (very high variance)
json_set_fulldoc_api_replies_q3_gmaps_passiveassist	41268 +- 2.9% (5 datapoints)	41742	1.1%	No Change
json_set_fulldoc_api_replies_q4_gmaps_assist	105254 +- 7.8% (5 datapoints)	91108	-13.4%	waterline=7.8%. REGRESSION
json_set_fulldoc_pass-json-parser-0000	39671 +- 2.6% (5 datapoints)	38083	-4.0%	potential REGRESSION
json_set_fulldoc_pass_100_json	93449 +- 5.1% (5 datapoints)	91316	-2.3%	waterline=5.1%. No Change
json_set_fulldoc_yahoo2	11350 +- 1.6% (5 datapoints)	10945	-3.6%	potential REGRESSION
json_set_key_empty	156240 +- 5.3% (5 datapoints)	128195	-17.9%	waterline=5.3%. REGRESSION
json_set_message.code_1_jsonsl-yelp_json	116951 +- 6.1% (5 datapoints)	122690	4.9%	waterline=6.1%. potential IMPROVEMENT
json_set_num_0	153322 +- 4.4% (5 datapoints)	152662	-0.4%	No Change
json_set_sclr_1_pass_100_json	115599 +- 7.4% (5 datapoints)	130712	13.1%	waterline=7.4%. IMPROVEMENT
json_set_sclr_pass_100_json	117639 +- 10.7% UNSTABLE (5 datapoints)	115596	-1.7%	UNSTABLE (very high variance)
json_vs_hashes_hset_key_simple	154182 +- 4.7% (4 datapoints)	148690	-3.6%	potential REGRESSION
json_vs_hashes_json.set_key_simple	135128 +- 5.2% (5 datapoints)	123449	-8.6%	waterline=5.2%. REGRESSION

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.20%. Comparing base (79c7497) to head (ee55fbc).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1180   +/-   ##
=======================================
  Coverage   78.20%   78.20%           
=======================================
  Files          14       14           
  Lines        3895     3895           
=======================================
  Hits         3046     3046           
  Misses        849      849           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.