Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generic optimizations #1688

Open
tjerman opened this issue Feb 15, 2024 · 23 comments
Open

Generic optimizations #1688

tjerman opened this issue Feb 15, 2024 · 23 comments
Assignees
@tjerman
Milestone
2024.3.x

Comments

@tjerman
Copy link
Member

tjerman commented Feb 15, 2024

No description provided.

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Running the benchmarking scripts we get this:

Image

{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3530,"max":4252,"avg":3812.55,"total":76251,"runs":[3845,3814,4035,3546,3530,4252,3667,3891,3708,3748,3654,3863,3626,3953,4120,3823,4090,3678,3553,3855]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3542,"max":4268,"avg":3819.15,"total":76383,"runs":[3901,3757,4021,3589,3542,4268,3688,3842,3757,3769,3671,3891,3637,3963,4137,3745,4067,3657,3600,3881]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":46,"avg":14.38,"total":1438,"runs":[24,15,17,20,13,14,13,15,14,15,12,16,15,16,39,16,13,15,26,13,14,19,15,13,14,13,14,13,13,12,14,22,14,14,12,11,13,17,18,21,15,14,15,12,14,13,21,15,13,12,13,13,13,13,12,12,46,13,12,12,13,12,14,15,14,15,16,13,12,11,10,11,11,12,12,11,12,13,11,13,14,13,11,11,14,13,13,12,11,12,13,13,12,12,11,14,11,12,12,13]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":24,"avg":10.21,"total":1021,"runs":[15,12,11,22,13,15,11,10,16,9,9,8,10,10,15,10,10,9,13,9,11,11,10,9,9,9,9,9,9,10,8,9,10,9,10,10,10,13,12,12,9,9,9,10,9,10,14,10,9,11,9,9,9,8,9,9,24,12,11,10,8,10,9,16,10,9,9,11,9,9,9,9,9,9,9,9,8,8,8,8,13,9,10,9,9,8,9,9,9,11,11,9,9,9,9,9,9,8,9,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":13,"max":37,"avg":17.42,"total":1742,"runs":[18,25,19,37,22,19,20,20,22,17,19,18,17,21,18,15,16,20,16,18,17,19,18,17,17,16,16,16,17,16,18,17,18,19,17,15,19,25,24,17,17,18,17,15,15,16,21,18,17,15,19,16,15,15,15,14,20,15,16,14,17,16,16,17,15,18,15,16,17,15,15,13,15,15,15,14,15,16,16,18,17,17,33,18,17,17,18,14,17,17,20,15,17,15,16,15,15,14,15,18]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":9,"max":22,"avg":11.49,"total":1149,"runs":[11,15,13,20,15,12,12,12,14,11,12,13,11,10,13,11,12,13,11,11,19,13,22,10,11,9,11,10,10,11,11,10,10,11,11,12,14,13,14,12,15,12,11,12,11,10,11,11,13,12,12,11,9,9,10,9,14,11,12,12,9,11,10,12,10,16,10,11,11,9,9,10,9,11,11,11,10,9,9,10,10,9,15,13,11,11,9,11,12,11,10,10,11,11,9,11,10,11,12,11]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":56,"avg":14.24,"total":1424,"runs":[21,15,35,13,15,14,18,13,14,15,15,14,14,16,19,13,19,16,13,13,13,12,12,14,13,12,14,14,13,15,19,17,17,13,12,14,14,14,13,12,16,14,11,11,56,13,12,12,12,14,14,15,11,14,13,14,10,12,12,11,11,12,14,14,12,13,13,13,13,11,13,12,14,11,13,12,12,13,18,13,11,22,12,12,13,13,13,12,12,13,16,13,13,13,13,16,13,12,13,13]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":10,"max":88,"avg":13.16,"total":1316,"runs":[88,17,12,17,12,13,13,12,11,13,15,12,11,13,12,11,14,13,12,11,11,13,14,12,12,12,12,11,13,13,15,13,12,14,13,13,17,15,13,13,13,11,11,12,16,11,13,12,12,11,14,14,13,11,12,10,12,11,13,12,11,10,12,12,11,15,12,13,10,12,12,11,11,12,14,11,11,11,11,12,12,14,13,12,12,12,12,12,12,13,13,12,12,11,12,12,13,11,12,13]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":19,"max":47,"avg":23.57,"total":2357,"runs":[30,36,26,24,27,22,23,21,24,21,35,24,22,23,21,23,27,21,21,22,22,19,21,23,29,22,22,24,24,28,32,25,23,23,23,23,30,23,23,22,22,20,21,19,28,21,20,21,21,22,24,26,20,21,21,20,20,22,21,20,20,21,25,21,43,23,21,22,22,21,26,21,21,22,21,21,20,25,21,47,23,23,23,24,22,25,26,22,23,23,23,22,23,22,27,25,26,23,21,23]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":14,"max":28,"avg":17.1,"total":1710,"runs":[18,27,23,16,17,24,16,16,16,16,28,16,15,25,15,19,17,27,18,15,15,15,14,16,18,15,16,17,19,21,21,19,19,17,15,17,20,16,16,17,16,16,14,17,17,18,14,16,18,24,15,21,19,16,14,15,14,15,16,17,15,17,20,15,16,18,18,16,19,18,16,16,15,15,16,17,14,19,16,14,19,16,15,14,15,16,17,16,15,19,18,15,15,16,19,15,15,16,15,15]}},"meta":{}}

Looks like RBAC resource building has a gigantic memory footprint

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Drafting a fix where we cache RBAC resources to hashmaps, we get to this

Image

{"special":"delimiter run line","ts":"2024-02-09T09:13:34.069Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3058,"max":6592,"avg":3688.9,"total":73778,"runs":[3607,3221,3339,3237,3082,3614,3102,3793,4850,3076,3058,6592,3549,3070,3231,3516,3256,5184,4301,3100]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3077,"max":6610,"avg":3701.6,"total":74032,"runs":[3623,3173,3323,3222,3130,3643,3113,3805,4867,3086,3077,6610,3566,3081,3241,3532,3269,5198,4322,3151]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":35,"avg":13.21,"total":1321,"runs":[22,15,14,15,14,22,14,15,19,13,12,17,14,16,16,17,13,13,12,15,13,13,12,13,14,35,12,14,12,13,12,12,13,12,14,12,15,17,13,12,13,11,12,11,11,10,11,11,13,13,12,10,11,11,11,10,11,11,12,12,21,11,11,11,12,11,11,12,11,11,12,11,12,14,15,12,11,12,11,11,12,11,11,11,12,13,10,12,20,11,12,14,30,13,13,12,10,11,11,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":7,"max":24,"avg":9.65,"total":965,"runs":[12,12,11,9,9,10,10,11,15,10,14,14,9,9,12,9,8,9,9,8,11,9,9,9,7,21,8,11,10,9,9,9,9,10,10,9,24,9,9,10,9,10,9,7,7,8,8,9,8,9,8,8,7,9,7,9,9,8,8,8,10,11,9,9,9,9,8,8,9,7,8,9,8,13,11,9,10,8,9,9,9,9,8,9,9,8,9,8,22,9,8,9,16,12,9,8,9,8,7,9]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":13,"max":54,"avg":16.83,"total":1683,"runs":[18,21,19,17,16,18,20,18,19,16,16,18,15,20,20,17,18,21,18,16,16,17,13,15,17,26,17,19,14,16,16,20,15,15,15,16,31,18,16,14,16,17,15,15,14,17,15,14,15,15,16,15,14,14,15,14,15,14,14,17,30,14,15,14,15,15,14,14,15,16,16,14,15,54,26,15,15,15,14,15,17,14,14,14,14,16,14,15,26,14,17,15,19,18,15,15,16,13,14,14]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":8,"max":45,"avg":11.64,"total":1164,"runs":[13,14,14,10,35,16,13,14,13,12,10,11,12,13,18,12,10,13,11,11,11,10,12,10,11,13,11,12,11,10,11,10,12,10,11,11,30,10,11,11,10,10,10,9,10,11,9,9,10,10,9,10,10,10,10,9,10,12,9,45,9,17,9,10,10,9,9,9,10,10,9,9,12,24,10,9,10,9,11,9,9,11,10,9,9,10,9,10,24,10,10,10,10,13,9,8,9,8,9,8]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":36,"avg":13.05,"total":1305,"runs":[17,15,16,36,18,17,17,13,17,14,15,19,13,15,15,14,13,12,12,12,12,15,13,14,11,12,12,12,14,14,12,12,12,12,12,11,12,11,11,12,12,12,12,13,11,13,11,15,11,12,11,14,12,12,12,12,12,13,17,12,13,12,11,11,12,12,11,12,12,12,28,12,12,15,14,11,12,10,10,13,10,11,11,17,11,10,11,12,11,11,14,12,12,10,10,11,12,12,15,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":9,"max":71,"avg":12.51,"total":1251,"runs":[71,17,10,18,13,13,15,12,13,12,14,18,11,14,11,12,11,11,11,30,12,15,12,12,12,12,12,11,28,13,13,13,12,11,10,11,11,10,10,10,11,10,10,10,11,10,11,22,12,11,11,10,11,11,10,11,11,13,13,10,11,10,11,11,10,11,10,10,10,10,10,10,10,12,12,10,11,11,10,9,10,10,10,23,10,11,10,10,10,11,11,10,10,11,10,10,10,12,15,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":17,"max":45,"avg":21.82,"total":2182,"runs":[24,26,20,24,25,27,21,25,24,22,26,23,21,25,23,24,22,21,20,41,21,23,20,21,23,21,22,21,43,21,21,21,22,19,20,21,19,22,21,19,19,20,19,20,19,19,32,23,22,19,20,19,18,20,19,19,20,45,31,19,19,19,20,21,18,21,19,19,20,31,20,21,28,22,22,19,17,17,18,18,21,19,19,24,19,20,22,21,18,19,20,20,19,20,21,20,18,22,23,21]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":12,"max":42,"avg":16.02,"total":1602,"runs":[18,17,15,18,19,23,16,16,17,17,22,18,15,15,13,16,16,16,17,18,17,15,15,18,18,15,16,15,25,17,15,15,14,16,14,15,14,16,15,12,14,13,14,15,15,14,42,15,15,15,13,15,14,14,14,15,17,37,13,13,13,15,14,14,13,13,14,13,15,31,14,17,28,20,16,13,13,15,18,14,13,13,21,14,12,13,13,13,12,19,15,13,14,13,13,13,14,17,15,13]}},"meta":{}}

Much better memory wise, performance wise, about the same.
Regardless -- less memory, gooder 👌

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Drafting a more proper solution (using a trie -- pkg/ds)

Image

{"special":"delimiter run line","ts":"2024-02-15T14:30:39.610Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":2788,"max":5352,"avg":3380.85,"total":67617,"runs":[4326,3926,3171,3191,3352,3036,3055,3409,3033,5352,3077,2990,3237,2788,2888,3681,2896,2954,4386,2869]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":2803,"max":5366,"avg":3392.9,"total":67858,"runs":[4340,3929,3185,3199,3366,3042,3065,3419,3044,5366,3092,3000,3244,2803,2893,3690,2934,2991,4336,2920]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":38,"avg":14.05,"total":1405,"runs":[19,13,15,13,16,23,34,14,14,13,15,15,14,15,12,14,15,12,15,20,14,13,14,12,16,14,13,14,15,14,13,13,14,15,15,20,13,14,14,12,16,14,14,12,16,12,16,12,13,16,11,12,12,17,15,13,11,12,11,13,24,12,22,13,15,12,13,14,12,11,11,11,12,11,11,12,14,14,13,11,12,14,12,11,12,12,12,12,38,12,13,11,12,11,11,13,12,12,11,13]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":17,"avg":9.69,"total":969,"runs":[14,9,10,10,10,11,9,10,10,11,10,12,9,9,9,9,9,11,9,12,9,10,9,9,9,11,11,9,9,11,10,12,11,10,10,10,10,10,10,9,10,9,9,11,11,8,9,9,8,11,9,10,10,9,9,11,11,9,8,11,12,11,17,8,8,10,12,9,8,8,9,10,10,8,9,9,8,8,9,9,9,8,9,9,9,9,8,9,17,8,8,8,10,8,9,8,10,9,8,10]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":14,"max":36,"avg":17.34,"total":1734,"runs":[20,18,15,21,17,36,27,20,22,19,23,33,16,17,18,22,17,16,15,18,20,18,15,18,17,19,14,15,17,16,21,16,16,18,19,14,16,16,18,18,18,15,16,17,17,14,16,15,20,16,17,16,17,17,16,14,15,16,16,20,14,27,26,16,16,15,16,14,17,16,15,15,16,16,15,14,15,17,17,15,14,15,17,15,14,18,17,16,25,15,14,16,17,15,15,15,17,18,15,15]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":10,"max":49,"avg":12.69,"total":1269,"runs":[12,12,21,12,13,26,12,10,10,11,16,19,13,11,12,16,18,11,12,10,18,10,13,14,11,16,12,13,12,12,16,10,15,13,11,12,13,12,13,10,11,11,15,13,12,12,11,12,13,11,11,11,12,15,11,11,11,11,11,15,10,49,11,25,11,10,10,12,10,12,11,10,11,12,11,10,12,10,10,12,10,11,11,11,11,11,10,25,10,11,11,10,11,11,11,11,10,10,10,11]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":35,"avg":13.37,"total":1337,"runs":[17,14,15,12,35,13,13,14,21,18,12,12,21,17,14,16,18,13,11,11,16,12,13,15,15,12,15,14,13,13,13,12,14,12,12,14,13,12,14,13,12,12,15,13,12,13,14,16,11,21,12,13,12,14,12,14,12,12,11,12,11,12,12,12,11,11,12,11,11,11,25,12,12,11,13,12,13,13,13,11,12,12,11,11,11,12,12,12,11,12,12,11,12,12,12,11,13,11,19,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":10,"max":46,"avg":12.93,"total":1293,"runs":[46,13,13,13,18,15,12,14,15,13,13,14,13,11,12,14,17,12,12,14,14,12,12,13,14,12,14,13,13,12,12,13,13,13,14,13,13,12,11,13,11,14,16,12,10,10,13,21,11,24,11,11,13,11,12,11,10,12,12,12,12,11,12,11,13,10,10,10,12,12,20,12,10,12,11,11,12,12,10,17,22,11,11,10,11,10,10,11,11,11,11,11,11,11,10,11,11,14,16,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":18,"max":36,"avg":21.92,"total":2192,"runs":[20,21,21,32,33,25,27,24,27,21,21,22,22,22,21,21,21,21,25,22,23,22,24,22,22,22,24,24,25,20,22,23,23,20,23,23,21,19,24,19,23,21,28,21,21,23,21,24,35,27,36,20,21,19,20,19,20,22,21,18,20,21,19,20,19,22,20,20,18,23,19,22,21,21,20,21,20,20,20,21,21,20,19,20,22,20,20,20,23,20,20,21,21,20,21,20,21,23,19,20]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":12,"max":50,"avg":17.7,"total":1770,"runs":[17,32,19,32,26,17,18,19,29,21,17,19,16,17,23,20,18,17,21,18,17,17,16,18,21,19,19,17,19,19,17,19,16,20,21,15,16,17,22,22,16,19,17,15,16,14,19,15,50,16,17,17,15,16,17,15,15,14,16,15,16,18,17,13,16,15,17,17,16,48,14,14,14,14,15,15,16,15,17,15,14,15,15,12,16,13,14,15,14,14,16,15,15,15,15,15,15,15,13,15]}},"meta":{}}

The package performance looks like this:

goos: darwin
goarch: arm64
pkg: github.com/cortezaproject/corteza/server/pkg/ds
BenchmarkTrie-12    	 4795262	       248.2 ns/op	       0 B/op	       0 allocs/op

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Next up, this looks like a sizeable chunk

Image

{"special":"delimiter run line","ts":"2024-02-15T12:47:03.370Z"}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3236,"max":4508,"avg":3600.3,"total":72006,"runs":[3937,3902,3569,3386,3644,3497,3539,3328,3449,4508,3491,3507,3347,3647,3236,3290,3612,3389,4372,3356]}},"meta":{}}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3192,"max":4522,"avg":3608.7,"total":72174,"runs":[3915,3952,3579,3396,3651,3496,3549,3282,3488,4522,3502,3523,3356,3659,3192,3283,3664,3398,4388,3379]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":48,"avg":13.83,"total":1383,"runs":[19,13,11,11,16,14,48,15,34,12,11,13,16,14,11,13,13,11,11,14,15,13,12,14,13,13,14,13,12,13,19,16,13,12,13,12,11,13,12,13,13,14,13,13,13,15,13,12,42,18,13,15,14,12,15,13,16,14,12,12,13,11,17,12,13,12,12,13,12,13,11,13,12,10,12,12,12,13,11,12,13,12,13,14,12,12,15,11,13,13,11,11,12,13,12,12,13,11,14,12]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":37,"avg":10.14,"total":1014,"runs":[16,10,9,11,37,22,14,10,11,10,8,9,10,10,9,9,10,9,10,12,8,8,9,10,9,8,10,8,11,10,10,10,8,10,9,9,10,10,9,9,11,11,10,9,9,9,8,10,17,11,11,11,10,10,11,20,13,9,10,8,9,9,11,8,10,9,10,9,9,9,9,9,9,9,9,8,9,9,9,9,9,10,11,10,8,10,10,9,9,8,10,8,10,9,10,9,9,8,8,8]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":14,"max":169,"avg":19.34,"total":1934,"runs":[21,18,17,18,21,33,21,16,16,15,19,16,16,16,18,16,16,17,18,22,17,16,18,18,17,19,17,18,21,20,19,16,17,22,16,15,16,19,16,18,21,15,16,18,18,16,16,169,27,18,18,18,17,18,17,24,20,17,15,16,15,22,34,17,15,17,15,18,17,16,21,17,16,17,16,18,17,14,17,18,18,20,16,16,17,15,14,18,17,18,18,16,16,16,16,17,16,19,17,18]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":10,"max":62,"avg":13.61,"total":1361,"runs":[12,13,12,12,24,57,14,13,11,11,12,10,13,12,11,11,13,12,12,13,12,13,12,14,14,12,11,13,13,13,13,11,12,11,12,12,12,13,12,11,12,12,12,13,12,11,12,62,20,14,13,33,12,15,16,14,12,12,12,12,12,18,14,11,14,11,11,12,11,11,14,12,10,12,13,12,11,13,12,13,13,14,12,12,12,12,12,11,11,11,13,11,11,13,12,11,13,12,11,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":66,"avg":13.9,"total":1390,"runs":[15,12,32,30,15,12,12,15,13,14,11,12,13,11,16,11,13,13,12,14,12,12,15,14,12,13,14,12,12,12,12,15,11,12,13,12,11,66,18,15,13,14,15,12,16,13,12,11,12,19,13,11,14,14,12,11,18,14,12,13,13,12,13,11,13,12,14,13,12,12,13,11,13,12,11,13,13,12,12,13,12,13,15,12,11,11,11,12,13,11,13,11,11,12,11,15,17,18,18,17]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":7,"max":101,"avg":10.79,"total":1079,"runs":[101,10,12,19,9,30,8,9,9,9,9,8,8,9,10,7,8,10,9,8,9,10,9,9,9,10,8,8,7,10,8,10,9,11,8,9,9,38,10,9,26,8,11,21,11,9,8,9,8,18,9,9,8,8,9,9,9,8,9,8,10,9,8,15,7,9,8,9,9,8,8,9,8,8,8,7,9,8,8,7,9,9,9,9,8,8,8,8,7,9,8,9,9,8,8,8,14,12,11,9]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":18,"max":97,"avg":23.43,"total":2343,"runs":[26,25,35,97,29,27,22,23,26,23,23,22,24,24,27,23,24,27,25,23,24,26,26,26,23,24,22,22,23,22,24,22,23,23,25,22,21,42,25,26,30,22,24,26,25,21,21,22,26,32,21,20,22,20,23,20,22,21,22,20,20,20,22,20,20,20,21,19,20,19,20,20,21,19,20,20,20,18,19,19,19,20,20,18,20,21,21,28,19,19,20,19,19,22,18,24,24,25,24,20]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":13,"max":168,"avg":19.47,"total":1947,"runs":[19,20,25,31,19,18,18,19,20,19,19,19,19,22,20,18,19,21,19,21,19,20,24,19,19,17,18,19,18,19,19,23,20,20,23,19,168,28,19,19,19,18,22,22,18,19,18,16,22,16,19,16,16,16,17,17,18,16,17,15,17,16,16,16,16,19,17,16,15,16,14,17,13,16,15,16,15,19,14,14,15,16,15,16,16,15,17,15,15,15,16,15,14,15,17,18,18,19,17,14]}},"meta":{}}

We're spending quite a bunch of time in loading the module/namespace combo.
Drafting a solution to cache namespaces/modules would get us to this:

Image

{"special":"delimiter run line","ts":"2024-02-15T14:13:36.115Z"}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":1957,"max":2356,"avg":2154.5,"total":43090,"runs":[2148,2127,2275,2288,2356,2249,2167,2133,2180,2262,2257,2068,2023,1982,2129,1957,2316,2038,2058,2077]}},"meta":{}}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":1972,"max":2343,"avg":2157.45,"total":43149,"runs":[2134,2118,2264,2275,2343,2240,2204,2146,2195,2221,2243,2113,2036,1992,2136,1972,2327,2050,2010,2130]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":8,"max":44,"avg":11.21,"total":1121,"runs":[16,12,11,13,11,11,10,10,10,44,21,13,11,10,11,11,10,10,10,11,11,10,11,16,11,13,13,13,11,12,13,12,16,11,12,11,11,11,12,10,11,10,10,12,10,11,9,10,10,10,9,11,9,9,11,9,10,9,19,9,10,9,9,9,13,10,11,10,10,10,8,8,11,9,9,9,9,10,9,9,10,12,10,11,12,13,9,9,10,10,11,9,10,10,11,17,10,9,10,12]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":6,"max":33,"avg":7.52,"total":752,"runs":[9,7,8,8,8,8,8,6,7,33,8,9,7,7,7,8,7,7,8,7,7,7,7,7,8,8,8,7,8,8,7,7,6,8,7,7,8,8,7,7,9,7,7,7,7,8,6,6,6,8,7,7,7,7,6,6,7,7,10,7,6,9,7,7,8,7,6,8,7,8,7,7,6,7,8,8,7,7,8,9,7,7,7,7,7,7,6,7,6,7,6,6,7,7,7,8,7,8,8,7]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":11,"max":43,"avg":14.52,"total":1452,"runs":[14,15,16,16,15,17,16,16,12,43,15,31,16,14,13,12,15,13,14,13,15,17,13,15,13,17,15,15,23,13,18,15,14,14,14,14,15,15,15,14,16,13,14,13,15,13,13,15,15,13,18,14,14,13,13,13,14,13,14,13,12,13,14,13,13,17,15,11,13,13,13,12,13,13,13,12,12,15,12,14,13,13,12,13,12,15,13,13,14,13,13,12,13,14,15,16,15,12,14,15]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":5,"max":11,"avg":7.35,"total":735,"runs":[8,8,9,7,8,8,7,8,11,10,7,8,8,9,7,9,8,8,6,8,7,7,7,6,8,10,6,9,11,7,7,8,7,8,8,9,8,9,8,6,8,10,8,8,7,8,7,6,7,7,6,6,6,6,7,6,8,5,7,7,5,8,8,7,7,6,7,7,6,6,6,7,6,6,7,8,7,7,6,7,6,6,6,6,7,9,7,7,8,8,7,8,6,8,9,8,7,7,6,7]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":9,"max":20,"avg":10.77,"total":1077,"runs":[14,11,11,12,11,10,12,20,10,12,11,10,11,10,10,12,15,13,10,16,11,13,11,13,12,10,10,11,10,12,11,11,13,11,10,11,9,10,11,9,9,11,9,10,9,10,15,10,10,11,12,10,10,11,10,9,11,10,10,11,11,10,11,9,11,10,9,11,11,11,10,11,11,10,12,10,10,10,12,9,9,9,9,11,9,10,10,10,11,10,11,11,9,12,10,10,10,10,9,10]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":31,"avg":10.3,"total":1030,"runs":[31,13,10,10,11,12,13,15,13,10,11,9,10,11,9,10,11,9,10,13,10,19,11,10,10,11,9,11,14,10,10,11,10,10,11,9,10,9,9,8,8,8,9,11,10,9,10,9,10,11,9,8,11,9,9,11,9,10,10,9,9,10,9,11,10,10,10,9,11,9,11,11,11,9,10,9,9,11,9,11,8,9,9,9,9,9,13,11,8,9,11,10,9,10,8,9,9,9,9,10]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":16,"max":79,"avg":19.64,"total":1964,"runs":[19,25,19,20,20,20,79,20,32,21,18,20,21,18,19,19,20,20,24,21,21,24,22,19,20,19,19,20,20,17,20,20,21,21,17,18,19,17,19,17,21,16,18,22,17,19,17,18,19,18,17,19,17,17,18,17,18,19,18,17,18,19,18,18,21,19,19,20,19,17,18,21,20,19,18,22,19,20,18,20,18,18,19,16,17,18,21,17,17,18,20,17,18,19,18,17,16,16,17,17]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":9,"max":32,"avg":11.77,"total":1177,"runs":[15,13,13,12,11,11,32,14,13,11,12,14,12,13,14,12,11,13,13,13,14,11,12,15,11,11,15,12,13,12,11,12,13,13,12,11,10,11,11,11,11,11,10,16,12,11,11,10,10,10,11,10,9,11,11,11,13,11,11,12,10,10,12,10,12,11,11,10,12,10,11,17,12,9,12,11,14,10,11,10,11,10,10,11,9,11,12,10,9,13,12,12,12,11,10,11,10,11,10,12]}},"meta":{}}

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Caching namespaces/modules (other top level resources) seems to bring the biggest performance increase so far

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

Next up might be worth checking contextual role bits?

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

I just removed RBAC and we get to this

Image

{"special":"delimiter run line","ts":"2024-02-15T14:58:18.933Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":1856,"max":3040,"avg":2084.75,"total":41695,"runs":[3040,2090,2421,1976,1862,1863,2249,1856,1942,1969,1995,1997,2120,2163,1897,1947,2120,1904,2075,2209]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":1859,"max":3046,"avg":2087.75,"total":41755,"runs":[3046,2104,2435,1982,1871,1874,2262,1859,1951,1980,2015,1956,2108,2152,1888,1937,2108,1898,2068,2261]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":8,"max":31,"avg":9.84,"total":984,"runs":[12,10,11,9,10,9,8,10,9,11,11,11,10,9,10,9,9,9,10,13,9,13,9,9,9,9,11,21,13,10,11,8,10,10,9,9,9,9,9,9,10,9,8,10,10,9,9,8,9,8,9,9,12,9,8,12,9,12,14,9,9,9,9,9,9,9,9,9,9,10,31,8,8,8,8,9,8,8,8,8,8,8,9,8,8,10,11,11,12,13,12,9,9,9,8,10,10,9,8,8]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":4,"max":15,"avg":6.23,"total":623,"runs":[8,7,6,6,6,6,10,6,7,6,8,7,6,6,6,6,7,7,6,6,6,7,6,6,7,7,7,10,7,7,6,6,6,5,6,6,6,6,6,6,5,5,5,6,6,7,6,7,6,5,6,8,15,7,5,5,6,7,7,6,5,6,5,6,5,7,6,6,5,6,7,6,6,5,6,5,6,5,5,5,5,5,5,7,5,6,7,8,8,6,7,5,5,6,5,6,6,7,4,5]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":10,"max":119,"avg":13.74,"total":1374,"runs":[13,12,12,12,13,13,13,15,12,14,13,13,12,14,11,13,11,11,119,12,12,12,14,11,14,13,12,13,14,12,12,14,12,14,13,16,12,12,12,15,12,14,12,14,11,12,11,13,12,13,11,14,21,13,13,15,13,24,15,12,12,12,11,11,12,11,13,12,13,22,20,10,11,10,10,11,10,11,10,13,11,11,11,10,12,13,14,20,14,11,12,11,11,10,10,13,11,11,10,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":5,"max":21,"avg":6.8,"total":680,"runs":[8,7,8,8,6,6,7,7,8,7,8,7,7,7,7,7,7,5,7,7,8,6,6,7,6,6,7,6,8,7,7,9,6,6,8,7,7,7,6,7,6,7,7,8,7,6,6,7,6,7,6,7,7,6,6,7,7,9,9,7,7,6,7,6,7,7,6,5,7,21,6,5,5,6,5,6,6,6,5,7,6,6,6,6,6,14,7,11,6,6,6,5,6,6,5,6,5,5,5,5]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":7,"max":31,"avg":9.52,"total":952,"runs":[14,9,10,9,10,9,10,11,8,9,10,10,9,10,12,11,9,9,11,10,21,12,9,9,10,12,11,10,9,10,9,11,9,11,9,8,9,8,8,9,9,9,9,14,11,8,9,9,9,10,9,9,8,31,8,8,7,9,9,8,8,8,7,8,8,9,11,14,10,11,8,8,9,8,8,9,8,9,9,8,8,8,8,8,7,8,8,8,7,7,8,8,7,8,8,8,8,18,8,9]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":6,"max":34,"avg":8.72,"total":872,"runs":[27,34,9,8,8,8,10,9,9,8,7,8,8,8,9,10,7,8,8,10,11,9,9,8,8,8,9,9,9,8,8,8,9,8,8,7,8,9,8,10,10,8,8,10,11,8,8,8,8,7,8,8,8,24,8,7,9,7,7,8,7,7,9,7,7,9,10,12,9,7,7,7,8,8,7,7,7,8,7,7,8,7,9,8,6,7,8,6,7,8,7,8,7,6,7,6,7,15,7,7]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":12,"max":112,"avg":16.98,"total":1698,"runs":[19,17,16,18,18,18,17,18,16,18,17,17,17,112,16,19,18,17,17,19,17,18,17,19,16,16,20,17,18,18,16,17,19,17,16,16,15,16,16,28,15,18,17,32,19,15,17,17,15,15,16,15,17,23,14,14,14,15,13,13,16,14,13,16,14,21,18,22,15,16,15,14,14,16,15,12,16,14,12,12,13,13,13,13,12,12,13,12,13,13,12,13,15,12,13,13,12,19,13,14]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":7,"max":19,"avg":9.93,"total":993,"runs":[11,11,11,13,12,11,12,11,10,11,10,11,11,12,11,11,10,11,12,11,13,11,10,12,10,12,10,11,11,11,11,10,11,10,10,10,10,10,12,13,10,15,10,15,11,10,9,11,10,10,10,12,19,9,9,9,8,9,10,9,9,8,10,8,10,11,16,10,11,11,9,8,9,10,10,9,11,7,7,7,7,7,7,7,7,8,7,7,8,7,7,7,7,8,7,9,7,7,7,8]}},"meta":{}}

@tjerman
Copy link
Member Author

tjerman commented Feb 15, 2024

If we campare this to the original numbers:

Image

This would then reduce the memory footprint by ~30GB for this scenario.

From ~39GB to ~9GB which is ~9.3 times better
Time wise; from roughly ~3800ms to ~2000ms which is ~1.9 times better

So the performance gain is not as great, but efficiency would be much much better (~77% less memory consumption (9/39*100)).

@Fajfa Fajfa added this to the 2023.9.5 milestone Mar 1, 2024
@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

TL;DR

The most impactful bits are:

Caching RBAC resource keys

We're working a bunch with strings when working with any resource.
In tight loops, all these strings rack up a bunch of memory.
If we cache all these keys, we'd be golden.

We can utilize a trie-like structure with usage counters to figure out the most used indexes, we can implement a cache with optimal lookup time and constraint it's memory consumption to not grow out of control.

Caching top-level resources

Things like namespaces and modules are commonly read as they are involved in other operations (like looking up and creating records).
There were plans to introduce some caching layer but it never became a thing.

Should we implement some simplified thing (potentially similar to what's noted for RBAC keys) as a workaround until we do something more proper?

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Caching top-level resources

Improving this, memory consumption goes down for about ~4GB.
A bunch but might opt-in to ignore it for now (in favour to a proper cache layer).

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Caching top-level resources would be quite a bit of an improvement but...

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Caching RBAC resources

Before:

    	 4533682	       253.2 ns/op	     204 B/op	       9 allocs/op

After:

    	26023652	        46.57 ns/op	       0 B/op	       0 allocs/op

Comparing the numbers:

time 253.2/46.57 = 5.43 times better
space 204/0 = optimal space
allocations 9/0 = optimal allocs

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Coming back to reality; cleaning up a stable working version; we get to this:

Image

{"special":"delimiter run line","ts":"2024-03-04T11:46:17.124Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3051,"max":6254,"avg":3530.25,"total":70605,"runs":[3207,3277,3192,3051,3178,3156,3745,3376,3276,3285,3103,3099,3182,4701,3561,3504,6254,3685,3348,3425]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3073,"max":6290,"avg":3552.15,"total":71043,"runs":[3224,3295,3204,3073,3192,3184,3754,3391,3291,3301,3123,3114,3206,4737,3598,3548,6290,3710,3367,3441]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":13,"max":25,"avg":16.7,"total":167,"runs":[25,23,14,13,15,14,18,14,14,17]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":9,"max":23,"avg":11.4,"total":114,"runs":[23,10,10,11,12,10,10,9,10,9]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":17,"max":21,"avg":18.5,"total":185,"runs":[18,19,17,19,19,21,18,18,18,18]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":10,"max":18,"avg":13.2,"total":132,"runs":[16,11,13,11,11,18,16,10,14,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":12,"max":21,"avg":15.3,"total":153,"runs":[21,14,16,17,14,12,17,17,12,13]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":12,"max":95,"avg":21.5,"total":215,"runs":[95,13,12,16,16,12,12,12,14,13]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":20,"max":29,"avg":23.9,"total":239,"runs":[25,24,25,29,20,21,23,25,24,23]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":15,"max":46,"avg":20.3,"total":203,"runs":[46,15,18,21,17,19,17,16,17,17]}},"meta":{}}

So... XSS is quite a thing but there isn't much we can do about it.
Considering some parallel work.

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024 •
edited

Running sanitizers and validators a in parallel, we get this:

{"special":"delimiter run line","ts":"2024-03-04T13:24:16.075Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3016,"max":5938,"avg":3312.5,"total":66250,"runs":[3348,3136,3153,3255,3061,3060,3121,3035,3046,3176,3076,3016,3060,3519,3088,3043,3038,3026,5938,4055]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3027,"max":6836,"avg":3337.3,"total":66746,"runs":[3365,3151,3167,3269,3080,3084,3154,3055,3097,3181,3067,3027,3073,3509,3093,3041,3051,3054,6836,3392]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":33,"avg":13.24,"total":1324,"runs":[21,13,14,13,14,12,13,12,12,14,12,13,13,14,12,15,13,11,13,11,17,12,14,33,11,12,14,18,14,12,13,13,14,11,13,12,14,13,12,12,14,13,14,11,12,13,12,11,13,12,12,13,14,12,11,28,16,30,12,11,14,13,11,12,12,13,11,12,12,11,12,14,12,12,12,11,13,13,12,12,13,11,14,12,11,11,12,12,13,12,12,12,12,12,12,11,12,16,15,12]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":7,"max":98,"avg":10.26,"total":1026,"runs":[15,11,10,9,9,10,10,9,12,12,10,10,9,12,8,11,9,9,8,10,9,9,10,10,9,9,9,10,9,8,9,10,9,8,9,10,8,9,10,8,9,10,10,8,8,8,9,7,8,9,10,8,9,10,98,21,10,9,9,8,8,8,8,9,8,9,8,8,8,10,9,9,9,9,9,9,9,8,8,9,8,8,8,9,9,8,10,16,9,10,9,9,10,10,9,10,11,10,9,9]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":12,"max":123,"avg":17.89,"total":1789,"runs":[18,23,15,18,19,14,16,15,19,16,17,15,18,16,16,19,15,15,17,15,19,15,17,19,18,17,30,15,15,17,16,20,16,17,17,17,15,17,16,18,14,18,17,16,14,15,15,17,15,15,15,16,16,15,123,29,17,16,16,15,15,14,17,16,15,14,15,15,17,15,18,12,16,18,16,16,16,15,19,17,14,17,15,17,14,16,29,26,16,16,21,15,16,16,18,15,23,17,15,16]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":9,"max":35,"avg":11.49,"total":1149,"runs":[12,11,11,11,10,12,11,11,14,11,10,11,13,9,9,10,11,9,9,11,10,11,11,11,12,10,28,11,12,12,12,9,11,10,13,10,12,9,10,9,12,11,12,11,11,11,11,11,10,10,11,10,9,11,35,15,10,11,12,10,9,11,10,11,10,10,11,13,10,18,12,10,11,10,15,10,12,11,11,9,11,16,11,10,11,9,18,11,11,10,9,12,10,12,9,12,17,11,10,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":159,"avg":14.45,"total":1445,"runs":[16,12,16,14,14,13,15,13,12,15,10,14,11,11,10,16,13,15,16,13,15,12,12,12,14,13,12,15,12,12,12,14,14,13,11,11,11,12,13,13,13,13,11,159,23,15,12,12,12,13,12,12,12,12,11,12,14,11,13,13,13,15,13,11,11,18,12,13,13,24,12,11,14,11,12,12,13,20,13,12,12,12,12,11,11,12,23,13,12,11,11,12,11,13,12,11,11,12,11,12]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":96,"avg":13.21,"total":1321,"runs":[96,12,11,12,12,10,13,12,12,14,12,14,12,10,11,12,11,11,11,10,12,14,12,12,12,11,11,15,13,12,12,11,11,13,13,12,11,11,10,11,11,11,11,74,17,23,11,12,11,11,12,12,11,10,11,12,13,10,11,11,11,12,10,13,12,11,11,10,10,12,12,11,14,13,11,11,11,14,13,11,13,11,8,12,11,11,13,11,12,12,12,11,11,11,11,12,11,11,11,11]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":17,"max":46,"avg":21.63,"total":2163,"runs":[27,22,20,21,20,23,22,21,20,24,20,21,19,19,19,21,20,33,20,23,39,19,19,23,23,21,21,20,19,22,20,21,23,23,18,20,20,19,20,20,20,19,20,46,23,21,20,20,20,20,19,19,19,20,21,27,22,21,21,24,20,21,25,18,19,21,21,20,21,32,20,21,18,21,21,21,21,26,22,20,17,23,19,24,21,23,22,20,20,21,21,22,23,23,23,23,21,20,21,20]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":13,"max":32,"avg":15.45,"total":1545,"runs":[15,16,14,16,15,18,15,15,19,15,15,15,14,16,16,14,17,20,16,13,28,17,16,15,14,14,14,17,16,16,14,14,17,15,17,15,14,13,14,14,15,15,14,32,18,16,15,13,13,15,13,15,14,17,15,15,14,15,15,15,16,13,15,15,17,15,13,13,22,18,13,14,17,17,16,13,15,17,15,16,13,14,14,15,16,17,16,14,13,15,15,13,14,16,14,15,15,15,14,15]}},"meta":{}}

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Looking at the CRUD benchmarks, we get this

Image

if I remove RBAC, I get this:

{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":53,"avg":12.34,"total":1234,"runs":[53,15,13,12,14,14,14,13,11,12,15,14,12,13,11,14,11,11,13,11,13,12,11,12,11,12,12,11,13,14,13,11,15,14,13,13,12,12,11,12,11,12,12,12,12,11,11,12,11,14,12,12,11,12,11,12,11,13,12,12,10,10,11,12,12,12,12,11,13,12,12,11,13,11,11,10,10,15,11,14,12,13,13,10,11,10,10,11,11,11,10,10,10,12,11,12,12,11,12,10]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":7,"max":22,"avg":8.61,"total":861,"runs":[16,11,8,8,10,15,9,8,8,11,9,8,9,8,9,9,9,8,8,8,7,8,22,10,8,10,10,9,8,7,8,9,8,8,8,8,8,8,9,8,9,8,8,8,8,8,8,8,8,9,9,8,8,9,7,8,9,8,9,8,8,8,9,9,9,8,8,9,8,8,7,8,8,8,8,7,9,19,7,8,7,7,7,7,7,7,8,8,7,8,9,9,9,7,7,8,8,8,7,8]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":12,"max":41,"avg":15.73,"total":1573,"runs":[19,16,15,18,17,22,19,17,15,18,19,16,15,15,20,18,16,15,16,17,18,14,23,15,14,16,16,15,14,14,14,14,14,14,14,15,14,15,17,15,22,15,17,17,14,14,16,15,15,16,17,16,15,15,17,24,25,17,14,15,15,16,14,14,14,13,13,13,14,14,13,14,14,15,18,15,16,13,13,13,15,13,12,14,15,41,14,14,16,15,14,14,15,13,14,14,13,13,13,13]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":9,"max":18,"avg":10.68,"total":1068,"runs":[13,13,11,12,12,11,11,9,9,12,12,10,11,11,13,13,10,11,10,10,12,12,11,11,10,12,10,10,10,11,11,10,10,10,11,12,11,13,10,10,11,11,9,10,10,11,11,10,11,11,10,10,10,11,9,18,17,10,10,12,9,10,10,10,10,10,10,10,11,11,10,10,11,11,10,9,10,10,10,9,9,11,11,9,10,17,10,10,9,10,12,9,10,10,10,9,9,9,10,10]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":43,"avg":12.18,"total":1218,"runs":[43,13,12,16,12,12,14,12,12,13,13,14,12,11,11,12,11,13,12,12,13,11,10,13,11,11,12,11,13,12,20,11,11,12,11,13,13,12,10,12,13,11,11,12,12,12,12,11,13,12,11,12,11,11,12,11,12,11,22,11,11,10,10,11,11,12,11,13,14,12,11,10,12,12,11,12,11,11,10,11,11,12,11,11,11,11,12,11,11,11,10,11,11,11,13,10,10,11,18,10]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":9,"max":52,"avg":11.77,"total":1177,"runs":[52,12,11,14,13,11,11,13,11,11,12,12,12,12,13,14,11,12,11,12,11,12,11,11,12,12,12,10,12,11,11,11,11,11,11,11,11,12,11,12,11,13,24,11,11,11,10,11,11,11,12,11,11,11,12,12,11,10,10,11,11,11,11,10,29,11,11,10,11,10,11,10,10,10,11,10,10,11,10,10,11,10,9,11,10,10,11,10,10,11,10,11,10,11,10,10,10,11,11,11]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":18,"max":40,"avg":21.42,"total":2142,"runs":[29,20,25,29,23,21,24,24,22,22,24,25,21,22,21,22,40,22,20,24,22,21,21,23,20,22,23,20,22,23,21,20,22,20,22,20,24,23,22,21,23,33,26,21,20,19,20,21,19,19,19,20,20,19,20,20,20,21,20,20,20,22,19,21,29,21,19,19,21,22,20,21,18,20,20,19,22,21,19,20,20,20,20,19,21,20,19,21,20,20,20,20,20,19,21,21,18,19,20,19]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":13,"max":37,"avg":16.22,"total":1622,"runs":[15,18,20,23,18,17,21,18,17,16,21,18,18,17,17,15,18,16,20,16,18,17,17,18,18,17,16,17,17,16,17,16,16,16,17,17,17,16,16,17,16,14,16,16,17,18,15,15,16,15,15,16,15,16,16,18,14,20,16,16,16,15,16,14,17,15,17,17,15,16,14,14,14,15,17,14,15,15,14,14,13,14,15,13,15,13,13,15,14,15,14,14,16,13,15,13,13,37,13,15]}},"meta":{}}

Not that big of a bump so won't look at it much further.

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Throwing some ideas to see if we can find the hardest blockers...

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024

Removing DAL for CRUD:

{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":40,"avg":12.29,"total":1229,"runs":[40,13,12,13,13,13,12,12,14,11,12,11,12,12,13,12,13,12,30,13,19,13,12,14,12,13,12,11,11,13,11,13,13,12,12,11,12,10,11,11,10,11,13,11,11,12,13,11,21,11,13,11,11,11,12,11,12,13,12,11,11,12,11,11,11,11,10,11,11,12,10,11,11,12,11,11,11,12,13,12,10,11,15,11,11,10,12,11,10,10,11,11,11,11,11,11,11,11,12,13]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":6,"max":17,"avg":8.05,"total":805,"runs":[12,9,7,8,8,7,8,7,8,8,8,9,10,7,8,8,8,9,17,9,8,9,8,8,7,8,8,8,9,8,8,9,9,9,9,9,9,8,8,8,9,10,10,8,9,9,8,9,7,8,7,7,7,7,7,8,7,7,9,8,8,8,10,9,8,8,8,8,6,7,7,8,8,7,7,7,8,9,10,6,8,7,8,7,8,9,6,7,9,8,8,7,6,7,7,6,7,6,7,7]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":11,"max":77,"avg":15.16,"total":1516,"runs":[17,14,13,12,13,14,13,13,11,15,12,13,25,29,13,14,13,77,36,27,15,15,15,14,16,16,15,16,14,14,14,15,13,13,13,13,14,13,15,13,15,14,14,13,14,15,14,13,16,14,14,13,14,14,13,14,12,25,13,16,13,15,11,12,12,13,12,13,12,14,13,14,13,13,15,13,14,14,14,13,29,35,14,14,15,13,13,13,13,12,12,11,11,11,11,11,11,12,11,11]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":7,"max":35,"avg":9.28,"total":928,"runs":[10,8,8,10,8,9,9,8,12,8,10,9,10,15,10,8,9,35,25,24,9,8,9,8,7,9,9,8,9,9,9,8,8,9,8,10,9,9,9,10,8,9,8,10,9,9,9,17,8,9,8,8,11,9,8,8,8,13,9,8,8,8,9,9,8,9,8,9,8,9,9,10,8,8,8,8,7,8,7,9,8,12,7,7,8,8,8,9,8,7,7,8,7,8,7,7,8,7,8,7]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":10,"max":43,"avg":12.4,"total":1240,"runs":[43,13,11,12,14,12,13,13,14,12,12,12,12,11,13,13,13,11,30,12,19,13,13,13,12,12,11,12,12,13,12,13,13,12,12,12,12,11,11,11,11,12,13,11,12,13,13,11,21,12,12,11,10,11,12,11,12,13,12,11,12,11,12,12,11,10,12,12,11,10,11,12,12,12,11,11,11,11,12,11,10,11,14,10,11,11,12,11,12,11,11,10,11,11,11,11,11,11,11,13]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":6,"max":20,"avg":7.95,"total":795,"runs":[13,9,7,8,6,8,8,7,7,8,8,7,9,8,8,7,7,10,20,8,8,9,7,9,7,8,8,8,9,8,9,9,9,8,10,9,8,9,9,9,9,9,9,9,8,7,9,9,8,7,6,7,7,7,8,7,7,7,8,10,8,8,8,8,8,8,7,9,7,8,7,7,8,7,7,7,7,9,10,8,7,7,8,8,7,7,7,7,7,6,8,6,6,7,6,7,6,7,7,7]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":11,"max":78,"avg":15.45,"total":1545,"runs":[18,16,13,14,14,13,13,12,13,15,13,13,25,28,14,14,14,78,35,28,16,14,17,14,15,17,15,15,13,13,14,15,13,15,14,12,13,13,14,13,15,15,14,12,15,15,14,13,16,15,15,13,13,15,14,13,13,25,14,14,13,16,13,12,13,13,13,13,12,14,14,15,15,14,15,14,13,14,14,13,30,36,14,14,15,14,12,13,14,14,12,12,11,12,11,11,13,11,11,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":6,"max":32,"avg":9.08,"total":908,"runs":[9,8,7,9,8,8,7,9,10,9,8,9,9,15,9,9,9,32,24,24,9,9,8,8,8,10,9,8,9,9,9,8,9,8,8,10,9,8,9,8,9,8,9,9,9,10,8,17,8,10,7,8,12,8,8,9,7,15,8,8,8,8,8,8,8,8,7,7,9,8,7,8,7,7,8,8,9,9,8,8,8,12,7,7,8,8,8,8,8,7,8,8,7,7,7,7,6,8,8,11]}},"meta":{}}

Not that big of a boost so won't pursue further atm

@tjerman
Copy link
Member Author

tjerman commented Mar 4, 2024 •
edited

I yanked out DAL for record searching (so iterator stuff) and we go from this

{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":19,"max":41,"avg":22.69,"total":2269,"runs":[36,22,21,22,22,23,21,21,20,20,21,22,21,21,23,20,21,22,23,24,24,23,21,25,23,24,24,22,22,23,20,23,24,21,22,24,21,23,24,27,24,25,22,23,21,23,21,21,25,20,24,24,24,41,24,22,24,23,22,21,23,28,23,24,22,20,22,22,22,21,23,22,23,23,22,22,24,21,23,22,23,25,21,21,21,22,24,22,22,20,23,23,20,22,23,21,23,20,22,19]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":53,"max":86,"avg":60.16,"total":6016,"runs":[86,61,60,60,64,62,65,64,62,67,62,64,60,61,69,64,65,61,68,74,66,64,64,66,61,64,63,61,67,65,65,62,64,62,61,59,55,61,59,55,57,54,55,57,55,57,63,57,58,55,56,68,79,57,59,62,57,56,55,56,54,57,58,59,76,66,60,56,57,54,56,55,57,57,56,60,57,61,79,61,56,57,58,55,56,56,55,56,54,53,53,55,57,56,55,56,54,55,53,54]}},"meta":{}}

to this:

{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":11,"max":25,"avg":13.18,"total":1318,"runs":[25,17,13,13,15,13,13,13,12,14,13,13,12,11,12,12,13,11,12,12,12,15,13,14,14,13,14,12,13,14,13,13,14,12,13,12,11,14,12,14,13,12,13,12,13,13,12,13,12,12,14,11,14,12,13,12,12,12,13,13,13,11,12,12,11,13,13,14,14,14,14,14,13,12,13,12,12,23,21,13,12,14,12,12,12,11,13,12,12,17,20,13,13,12,12,12,13,14,13,13]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":27,"max":74,"avg":30.78,"total":3078,"runs":[74,40,31,32,30,31,30,31,33,33,32,34,32,33,33,32,34,34,32,31,32,32,31,32,34,32,33,33,32,33,40,31,33,32,34,42,30,32,32,32,30,30,30,31,29,29,29,28,29,28,30,29,29,28,28,28,29,27,29,27,28,29,29,29,29,28,28,29,29,28,29,29,28,28,28,29,28,28,28,28,27,27,29,29,31,28,28,28,29,29,28,29,28,29,28,35,32,28,29,27]}},"meta":{}}

Making it quite a bit faster; continuing...

@tjerman
Copy link
Member Author

tjerman commented Mar 5, 2024

If I remove the thing to generate page navigation we get to...

{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":10,"max":39,"avg":12.76,"total":1276,"runs":[39,13,12,13,15,13,14,12,13,13,13,13,11,12,18,14,12,12,12,12,10,12,13,11,15,14,12,13,13,13,12,11,11,12,12,11,11,11,12,11,13,11,15,23,13,14,13,12,11,12,10,15,12,13,12,12,11,11,11,11,12,11,12,12,11,16,16,12,13,27,11,11,12,11,12,13,12,11,11,12,10,11,12,12,10,12,10,12,11,11,13,13,11,11,10,20,12,11,13,12]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":26,"max":106,"avg":30.36,"total":3036,"runs":[106,40,30,30,33,33,33,31,38,31,32,30,30,31,30,35,30,31,31,34,30,30,29,30,32,30,44,30,29,31,31,30,31,30,30,30,40,33,30,26,28,29,27,32,30,28,29,28,27,27,27,27,27,28,27,27,26,27,26,26,26,26,28,28,27,27,27,27,28,28,28,30,30,30,28,27,27,27,28,27,27,27,50,30,29,29,29,29,29,28,28,28,27,27,27,26,28,26,28,32]}},"meta":{}}

So about same difference

@tjerman
Copy link
Member Author

tjerman commented Mar 5, 2024 •
edited

looking into reusing some bits;

before

Image

{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":19,"max":38,"avg":22.84,"total":2284,"runs":[38,24,25,21,29,22,22,22,21,19,29,32,23,20,22,21,19,19,19,22,23,20,20,22,21,38,22,25,21,23,21,22,21,22,21,22,23,23,21,23,21,21,22,21,20,23,28,28,30,28,25,36,26,25,23,21,20,21,22,20,20,21,23,24,23,22,20,22,23,24,25,23,20,22,20,20,21,20,19,22,20,20,21,22,22,25,23,26,25,22,22,21,20,21,22,24,23,23,22,22]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":51,"max":134,"avg":61.17,"total":6117,"runs":[121,70,63,72,62,64,59,67,76,65,61,64,67,62,66,75,69,74,61,63,62,67,63,65,64,61,60,63,63,64,70,62,65,65,59,61,62,60,63,73,56,53,55,55,53,55,57,54,54,55,55,56,54,56,56,53,56,57,56,61,57,56,56,56,57,53,55,57,59,59,62,57,58,56,53,54,54,52,53,70,53,53,51,52,52,134,104,60,60,55,55,57,55,55,55,56,55,54,54,53]}},"meta":{}}

after

Image

{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":18,"max":46,"avg":22.84,"total":2284,"runs":[38,22,22,21,22,22,22,21,21,20,26,19,21,23,20,19,24,21,21,21,20,18,22,24,32,30,22,26,20,20,22,19,46,32,24,25,34,31,23,22,21,22,21,21,22,22,24,22,32,24,21,24,25,24,33,21,21,24,21,20,23,21,21,21,21,19,22,21,24,21,22,23,23,21,27,21,23,21,21,19,23,24,19,22,21,20,24,21,20,21,22,20,23,21,21,19,20,22,22,21]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":52,"max":122,"avg":59.21,"total":5921,"runs":[122,67,62,63,61,64,63,74,68,60,81,68,70,65,63,60,71,65,63,65,62,63,64,61,62,64,68,61,64,63,64,62,61,60,60,53,56,59,58,60,59,56,55,57,56,54,57,56,54,56,52,59,59,54,55,54,53,55,55,55,53,56,54,55,57,53,54,53,53,55,54,53,53,52,55,55,55,54,59,57,57,56,54,59,58,53,55,57,57,54,53,56,58,54,58,56,56,57,55,57]}},"meta":{}}

Performance is about the same; memory is somewhat smaller (only for some cases!!)

@tjerman
Copy link
Member Author

tjerman commented Mar 5, 2024

For now; will leave it at this:

Pre:

{"special":"delimiter run line","ts":"2024-03-05T09:38:41.978Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3518,"max":6516,"avg":3833.5,"total":76670,"runs":[3909,3695,3721,3619,3518,3602,3578,3673,3571,3676,3727,3687,3639,3776,3866,3728,3767,3631,3771,6516]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3534,"max":6569,"avg":3851.55,"total":77031,"runs":[3953,3709,3729,3630,3534,3610,3599,3686,3582,3691,3736,3706,3678,3774,3882,3737,3793,3640,3793,6569]}},"meta":{}}
{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":34,"max":190,"avg":43.99,"total":4399,"runs":[190,97,78,87,91,71,47,46,47,48,52,45,37,39,40,42,47,39,43,40,39,39,38,42,52,54,36,38,40,37,43,43,41,44,49,43,45,38,41,42,37,38,35,36,39,40,39,36,37,38,46,35,37,39,46,37,38,37,34,38,39,40,36,45,46,37,42,36,51,35,37,35,34,39,40,38,37,36,35,36,39,48,36,36,38,40,39,38,35,40,37,36,36,61,47,40,38,38,37,34]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":12,"max":97,"avg":19.26,"total":1926,"runs":[97,54,53,34,30,18,26,23,22,18,24,42,18,18,18,23,24,23,18,21,19,17,21,17,17,19,18,19,19,19,21,20,24,18,17,22,15,20,18,19,23,18,15,19,18,19,18,30,18,16,14,19,18,19,17,16,12,15,15,13,12,14,14,13,15,14,12,14,24,14,13,12,17,13,14,15,13,15,15,13,16,18,14,16,17,13,14,17,16,16,14,20,12,14,14,19,13,15,13,14]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":93,"avg":13.65,"total":1365,"runs":[93,32,18,16,19,12,15,18,24,14,13,14,12,13,17,15,20,17,12,14,12,13,12,12,21,12,11,11,13,16,12,18,12,12,12,11,14,15,14,13,18,11,12,15,12,16,10,13,14,10,14,11,15,12,11,10,10,9,10,20,10,10,38,8,10,9,10,10,10,10,9,12,10,12,8,11,10,10,8,10,9,17,11,11,10,9,9,14,10,13,8,11,11,10,10,10,10,10,9,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":15,"max":87,"avg":23.89,"total":2389,"runs":[87,83,34,31,40,24,26,23,24,25,24,39,25,24,32,27,34,23,29,27,21,25,26,24,30,21,31,21,23,29,24,34,25,29,20,22,25,24,22,18,35,19,23,25,31,22,21,22,20,22,22,20,21,25,21,15,21,16,17,21,17,18,19,17,17,19,15,19,23,18,18,16,18,16,20,17,19,45,17,19,19,18,18,16,18,17,18,36,16,18,23,20,21,20,19,21,18,15,19,18]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":10,"max":54,"avg":15.93,"total":1593,"runs":[52,54,20,25,17,14,26,20,19,16,15,20,16,13,18,13,15,15,19,18,15,18,13,18,18,18,17,14,14,18,15,17,17,34,13,14,16,17,15,18,19,18,21,13,21,14,53,14,17,15,13,11,14,14,12,13,11,12,12,12,10,13,13,12,14,13,12,12,12,11,13,14,10,12,12,14,12,25,11,10,16,14,11,11,13,11,11,13,11,13,11,14,13,12,11,15,11,12,10,12]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":68,"max":440,"avg":82.12,"total":8212,"runs":[440,141,96,91,98,81,84,87,84,82,90,94,77,80,90,83,94,81,81,84,78,83,82,84,80,87,75,79,80,75,84,77,88,75,80,78,79,79,83,82,79,78,82,75,98,82,78,76,72,75,87,73,75,70,73,79,70,91,68,73,70,73,72,69,74,72,69,72,69,87,72,75,77,70,69,71,73,76,70,74,75,74,78,80,73,70,70,69,71,78,76,73,70,74,71,78,69,70,69,70]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":187,"avg":19.9,"total":1990,"runs":[187,57,28,32,20,19,24,20,20,23,19,17,22,30,21,24,15,17,22,16,27,15,22,17,20,22,25,19,29,21,16,21,20,20,21,19,18,21,17,17,17,19,20,16,18,17,14,12,14,16,18,15,14,13,13,16,16,24,16,15,14,12,13,16,14,43,11,15,18,13,13,15,13,16,12,14,29,18,17,13,13,16,15,16,14,20,13,20,13,12,15,16,13,13,15,13,14,13,15,19]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":10,"max":183,"avg":17.31,"total":1731,"runs":[183,21,20,30,14,26,15,15,16,30,17,17,17,22,16,16,16,15,16,16,23,17,16,14,16,16,30,16,24,15,18,17,17,16,15,13,17,14,13,56,15,16,18,13,17,16,13,11,12,10,12,13,39,11,13,10,12,14,13,12,14,11,10,12,14,23,10,13,14,12,12,12,12,32,13,14,16,13,11,13,14,11,12,14,10,14,13,13,11,11,11,13,11,11,10,11,12,11,11,15]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":19,"max":105,"avg":27.11,"total":2711,"runs":[105,39,40,29,31,37,39,35,41,32,26,38,30,37,27,31,27,31,29,34,28,28,23,24,33,25,33,28,29,27,28,28,25,31,25,25,26,32,26,38,27,24,25,21,25,24,21,24,23,20,19,22,22,22,21,21,21,24,23,21,23,22,24,21,24,22,25,23,23,22,25,21,22,26,22,22,25,25,22,28,20,22,24,23,24,33,26,25,23,22,25,35,24,24,22,23,23,22,26,28]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":13,"max":62,"avg":20.08,"total":2008,"runs":[62,27,32,19,24,21,23,24,36,22,20,23,18,25,21,26,21,25,19,20,20,30,19,19,21,21,21,22,18,17,23,22,20,33,20,32,20,21,19,18,21,17,13,21,22,19,17,14,14,24,18,17,20,18,15,14,15,14,15,17,16,16,17,15,17,17,13,21,18,18,18,16,15,20,20,22,18,18,19,22,17,18,18,19,16,23,26,19,20,17,22,17,18,17,15,13,16,16,19,17]}},"meta":{}}

Post:

{"special":"delimiter run line","ts":"2024-03-05T09:38:41.978Z"}
{"scenario":"x100: basic build regular","operation":"unmaker","numbers":{"100-10000":{"min":3518,"max":6516,"avg":3833.5,"total":76670,"runs":[3909,3695,3721,3619,3518,3602,3578,3673,3571,3676,3727,3687,3639,3776,3866,3728,3767,3631,3771,6516]}},"meta":{}}
{"scenario":"x100: basic build physical","operation":"unmaker","numbers":{"100-10000":{"min":3534,"max":6569,"avg":3851.55,"total":77031,"runs":[3953,3709,3729,3630,3534,3610,3599,3686,3582,3691,3736,3706,3678,3774,3882,3737,3793,3640,3793,6569]}},"meta":{}}
{"scenario":"x100: list plain regular","operation":"search","numbers":{"100-10000":{"min":34,"max":190,"avg":43.99,"total":4399,"runs":[190,97,78,87,91,71,47,46,47,48,52,45,37,39,40,42,47,39,43,40,39,39,38,42,52,54,36,38,40,37,43,43,41,44,49,43,45,38,41,42,37,38,35,36,39,40,39,36,37,38,46,35,37,39,46,37,38,37,34,38,39,40,36,45,46,37,42,36,51,35,37,35,34,39,40,38,37,36,35,36,39,48,36,36,38,40,39,38,35,40,37,36,36,61,47,40,38,38,37,34]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-create","numbers":{"100-10000":{"min":12,"max":97,"avg":19.26,"total":1926,"runs":[97,54,53,34,30,18,26,23,22,18,24,42,18,18,18,23,24,23,18,21,19,17,21,17,17,19,18,19,19,19,21,20,24,18,17,22,15,20,18,19,23,18,15,19,18,19,18,30,18,16,14,19,18,19,17,16,12,15,15,13,12,14,14,13,15,14,12,14,24,14,13,12,17,13,14,15,13,15,15,13,16,18,14,16,17,13,14,17,16,16,14,20,12,14,14,19,13,15,13,14]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-lookup","numbers":{"100-10000":{"min":8,"max":93,"avg":13.65,"total":1365,"runs":[93,32,18,16,19,12,15,18,24,14,13,14,12,13,17,15,20,17,12,14,12,13,12,12,21,12,11,11,13,16,12,18,12,12,12,11,14,15,14,13,18,11,12,15,12,16,10,13,14,10,14,11,15,12,11,10,10,9,10,20,10,10,38,8,10,9,10,10,10,10,9,12,10,12,8,11,10,10,8,10,9,17,11,11,10,9,9,14,10,13,8,11,11,10,10,10,10,10,9,11]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-update","numbers":{"100-10000":{"min":15,"max":87,"avg":23.89,"total":2389,"runs":[87,83,34,31,40,24,26,23,24,25,24,39,25,24,32,27,34,23,29,27,21,25,26,24,30,21,31,21,23,29,24,34,25,29,20,22,25,24,22,18,35,19,23,25,31,22,21,22,20,22,22,20,21,25,21,15,21,16,17,21,17,18,19,17,17,19,15,19,23,18,18,16,18,16,20,17,19,45,17,19,19,18,18,16,18,17,18,36,16,18,23,20,21,20,19,21,18,15,19,18]}},"meta":{}}
{"scenario":"x100: basic CRUD regular","operation":"crud-delete","numbers":{"100-10000":{"min":10,"max":54,"avg":15.93,"total":1593,"runs":[52,54,20,25,17,14,26,20,19,16,15,20,16,13,18,13,15,15,19,18,15,18,13,18,18,18,17,14,14,18,15,17,17,34,13,14,16,17,15,18,19,18,21,13,21,14,53,14,17,15,13,11,14,14,12,13,11,12,12,12,10,13,13,12,14,13,12,12,12,11,13,14,10,12,12,14,12,25,11,10,16,14,11,11,13,11,11,13,11,13,11,14,13,12,11,15,11,12,10,12]}},"meta":{}}
{"scenario":"x100: list plain physical","operation":"search","numbers":{"100-10000":{"min":68,"max":440,"avg":82.12,"total":8212,"runs":[440,141,96,91,98,81,84,87,84,82,90,94,77,80,90,83,94,81,81,84,78,83,82,84,80,87,75,79,80,75,84,77,88,75,80,78,79,79,83,82,79,78,82,75,98,82,78,76,72,75,87,73,75,70,73,79,70,91,68,73,70,73,72,69,74,72,69,72,69,87,72,75,77,70,69,71,73,76,70,74,75,74,78,80,73,70,70,69,71,78,76,73,70,74,71,78,69,70,69,70]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-create","numbers":{"100-10000":{"min":11,"max":187,"avg":19.9,"total":1990,"runs":[187,57,28,32,20,19,24,20,20,23,19,17,22,30,21,24,15,17,22,16,27,15,22,17,20,22,25,19,29,21,16,21,20,20,21,19,18,21,17,17,17,19,20,16,18,17,14,12,14,16,18,15,14,13,13,16,16,24,16,15,14,12,13,16,14,43,11,15,18,13,13,15,13,16,12,14,29,18,17,13,13,16,15,16,14,20,13,20,13,12,15,16,13,13,15,13,14,13,15,19]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-lookup","numbers":{"100-10000":{"min":10,"max":183,"avg":17.31,"total":1731,"runs":[183,21,20,30,14,26,15,15,16,30,17,17,17,22,16,16,16,15,16,16,23,17,16,14,16,16,30,16,24,15,18,17,17,16,15,13,17,14,13,56,15,16,18,13,17,16,13,11,12,10,12,13,39,11,13,10,12,14,13,12,14,11,10,12,14,23,10,13,14,12,12,12,12,32,13,14,16,13,11,13,14,11,12,14,10,14,13,13,11,11,11,13,11,11,10,11,12,11,11,15]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-update","numbers":{"100-10000":{"min":19,"max":105,"avg":27.11,"total":2711,"runs":[105,39,40,29,31,37,39,35,41,32,26,38,30,37,27,31,27,31,29,34,28,28,23,24,33,25,33,28,29,27,28,28,25,31,25,25,26,32,26,38,27,24,25,21,25,24,21,24,23,20,19,22,22,22,21,21,21,24,23,21,23,22,24,21,24,22,25,23,23,22,25,21,22,26,22,22,25,25,22,28,20,22,24,23,24,33,26,25,23,22,25,35,24,24,22,23,23,22,26,28]}},"meta":{}}
{"scenario":"x100: basic CRUD physical","operation":"crud-delete","numbers":{"100-10000":{"min":13,"max":62,"avg":20.08,"total":2008,"runs":[62,27,32,19,24,21,23,24,36,22,20,23,18,25,21,26,21,25,19,20,20,30,19,19,21,21,21,22,18,17,23,22,20,33,20,32,20,21,19,18,21,17,13,21,22,19,17,14,14,24,18,17,20,18,15,14,15,14,15,17,16,16,17,15,17,17,13,21,18,18,18,16,15,20,20,22,18,18,19,22,17,18,18,19,16,23,26,19,20,17,22,17,18,17,15,13,16,16,19,17]}},"meta":{}}

Comparing all these numbers (based on the jsons above):

## Logging x100: basic build physical;unmaker;100-10000
min: 1.13
max: 1.30
avg: 1.10
total: 1.10
## Logged x100: basic build physical;unmaker;100-10000


## Logging x100: basic build regular;unmaker;100-10000
min: 1.13
max: 1.30
avg: 1.09
total: 1.09
## Logged x100: basic build regular;unmaker;100-10000


## Logging x100: basic CRUD physical;crud-create;100-10000
min: 1.00
max: 5.34
avg: 1.26
total: 1.26
## Logged x100: basic CRUD physical;crud-create;100-10000


## Logging x100: basic CRUD physical;crud-delete;100-10000
min: 1.08
max: 2.07
avg: 1.07
total: 1.07
## Logged x100: basic CRUD physical;crud-delete;100-10000


## Logging x100: basic CRUD physical;crud-lookup;100-10000
min: 1.00
max: 2.51
avg: 1.17
total: 1.17
## Logged x100: basic CRUD physical;crud-lookup;100-10000


## Logging x100: basic CRUD physical;crud-update;100-10000
min: 1.06
max: 1.94
avg: 1.07
total: 1.07
## Logged x100: basic CRUD physical;crud-update;100-10000


## Logging x100: basic CRUD regular;crud-create;100-10000
min: 1.09
max: 1.45
avg: 1.14
total: 1.14
## Logged x100: basic CRUD regular;crud-create;100-10000


## Logging x100: basic CRUD regular;crud-delete;100-10000
min: 1.11
max: 2.35
avg: 1.17
total: 1.17
## Logged x100: basic CRUD regular;crud-delete;100-10000


## Logging x100: basic CRUD regular;crud-lookup;100-10000
min: 1.00
max: 2.51
avg: 1.13
total: 1.13
## Logged x100: basic CRUD regular;crud-lookup;100-10000


## Logging x100: basic CRUD regular;crud-update;100-10000
min: 1.15
max: 2.02
avg: 1.13
total: 1.13
## Logged x100: basic CRUD regular;crud-update;100-10000


## Logging x100: list plain physical;search;100-10000
min: 1.05
max: 3.73
avg: 1.09
total: 1.09
## Logged x100: list plain physical;search;100-10000


## Logging x100: list plain regular;search;100-10000
min: 1.03
max: 2.41
avg: 1.14
total: 1.14
## Logged x100: list plain regular;search;100-10000

And heap allocations: 39503.92/17870.83 ~ 2.21 times better

Performance-wise not that much better, but memory-wise, quite a bit better.
Some extra bits where we could get additional performance is caching top-level resources and potentially looking into optimizing some extra DB-level interactions

@tjerman
Copy link
Member Author

tjerman commented Mar 5, 2024

EOF

@tjerman
Copy link
Member Author

tjerman commented Mar 5, 2024

Introduced an easter egg -- tests may fail due to the RBAC index being reused globally in the process.

@Fajfa Fajfa modified the milestones: 2023.9.5, 2024.3.x Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tjerman tjerman

Labels
None yet
Projects
None yet
Milestone
2024.3.x
Development

No branches or pull requests
2 participants
@tjerman @Fajfa