Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update tests for spring-data-commons to test v3+ #6826

Closed
wants to merge 3 commits into from
Closed

Update tests for spring-data-commons to test v3+ #6826

wants to merge 3 commits into from

Conversation

zarirhamza
Copy link
Contributor

What Does This Do

Update tests for spring-data-commons to test v3+

Motivation

Part of larger initiative for our tests to cover versions most used by customers

Additional Notes

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Mar 19, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master zarir/upgrade-v3-spring-data-tests
git_commit_date 1710833793 1710868074
git_commit_sha f97f632 055ffc7
release_version 1.32.0-SNAPSHOT~f97f63248a 1.32.0-SNAPSHOT~055ffc770d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710871224 1710871224
ci_job_id 463621104 463621104
ci_pipeline_id 30386275 30386275
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 13 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.077 s) : 0, 1076515
Total [baseline] (8.538 s) : 0, 8537541
Agent [candidate] (1.077 s) : 0, 1077438
Total [candidate] (8.576 s) : 0, 8575951
section iast
Agent [baseline] (1.208 s) : 0, 1208465
Total [baseline] (9.057 s) : 0, 9056537
Agent [candidate] (1.203 s) : 0, 1203428
Total [candidate] (9.046 s) : 0, 9046385
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.2 s) : 0, 1200012
Total [baseline] (9.011 s) : 0, 9011284
Agent [candidate] (1.207 s) : 0, 1206898
Total [candidate] (9.02 s) : 0, 9019969
section iast_TELEMETRY_OFF
Agent [baseline] (1.193 s) : 0, 1193186
Total [baseline] (9.011 s) : 0, 9011380
Agent [candidate] (1.22 s) : 0, 1219843
Total [candidate] (9.121 s) : 0, 9120867
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.208 s 131.95 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.2 s 123.497 ms (11.5%)
Agent iast_TELEMETRY_OFF 1.193 s 116.671 ms (10.8%)
Total tracing 8.538 s -
Total iast 9.057 s 518.996 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.011 s 473.743 ms (5.5%)
Total iast_TELEMETRY_OFF 9.011 s 473.839 ms (5.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.203 s 125.989 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.207 s 129.46 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.22 s 142.404 ms (13.2%)
Total tracing 8.576 s -
Total iast 9.046 s 470.434 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.02 s 444.019 ms (5.2%)
Total iast_TELEMETRY_OFF 9.121 s 544.916 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (694.745 ms) : 0, 694745
BytebuddyAgent [candidate] (694.75 ms) : 0, 694750
GlobalTracer [baseline] (289.957 ms) : 0, 289957
GlobalTracer [candidate] (290.971 ms) : 0, 290971
AppSec [baseline] (49.135 ms) : 0, 49135
AppSec [candidate] (48.971 ms) : 0, 48971
Remote Config [baseline] (1.093 ms) : 0, 1093
Remote Config [candidate] (1.062 ms) : 0, 1062
Telemetry [baseline] (7.359 ms) : 0, 7359
Telemetry [candidate] (7.484 ms) : 0, 7484
section iast
BytebuddyAgent [baseline] (803.56 ms) : 0, 803560
BytebuddyAgent [candidate] (799.491 ms) : 0, 799491
GlobalTracer [baseline] (289.117 ms) : 0, 289117
GlobalTracer [candidate] (288.666 ms) : 0, 288666
AppSec [baseline] (51.139 ms) : 0, 51139
AppSec [candidate] (48.855 ms) : 0, 48855
IAST [baseline] (22.842 ms) : 0, 22842
IAST [candidate] (24.039 ms) : 0, 24039
Remote Config [baseline] (567.545 µs) : 0, 568
Remote Config [candidate] (572.183 µs) : 0, 572
Telemetry [baseline] (6.623 ms) : 0, 6623
Telemetry [candidate] (7.437 ms) : 0, 7437
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (797.835 ms) : 0, 797835
BytebuddyAgent [candidate] (802.828 ms) : 0, 802828
GlobalTracer [baseline] (287.296 ms) : 0, 287296
GlobalTracer [candidate] (289.09 ms) : 0, 289090
AppSec [baseline] (50.174 ms) : 0, 50174
AppSec [candidate] (48.828 ms) : 0, 48828
IAST [baseline] (23.391 ms) : 0, 23391
IAST [candidate] (24.502 ms) : 0, 24502
Remote Config [baseline] (554.266 µs) : 0, 554
Remote Config [candidate] (574.64 µs) : 0, 575
Telemetry [baseline] (6.558 ms) : 0, 6558
Telemetry [candidate] (6.568 ms) : 0, 6568
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (790.286 ms) : 0, 790286
BytebuddyAgent [candidate] (806.618 ms) : 0, 806618
GlobalTracer [baseline] (287.869 ms) : 0, 287869
GlobalTracer [candidate] (295.366 ms) : 0, 295366
AppSec [baseline] (50.925 ms) : 0, 50925
AppSec [candidate] (48.808 ms) : 0, 48808
IAST [baseline] (22.147 ms) : 0, 22147
IAST [candidate] (26.956 ms) : 0, 26956
Remote Config [baseline] (572.23 µs) : 0, 572
Remote Config [candidate] (588.282 µs) : 0, 588
Telemetry [baseline] (7.228 ms) : 0, 7228
Telemetry [candidate] (6.72 ms) : 0, 6720
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.075 s) : 0, 1075206
Total [baseline] (9.159 s) : 0, 9158761
Agent [candidate] (1.077 s) : 0, 1077357
Total [candidate] (9.175 s) : 0, 9175304
section appsec
Agent [baseline] (1.197 s) : 0, 1196967
Total [baseline] (9.257 s) : 0, 9256616
Agent [candidate] (1.215 s) : 0, 1215216
Total [candidate] (9.349 s) : 0, 9348645
section iast
Agent [baseline] (1.209 s) : 0, 1208910
Total [baseline] (9.35 s) : 0, 9349610
Agent [candidate] (1.209 s) : 0, 1209014
Total [candidate] (9.376 s) : 0, 9375646
section profiling
Agent [baseline] (1.27 s) : 0, 1270071
Total [baseline] (9.32 s) : 0, 9319677
Agent [candidate] (1.28 s) : 0, 1280009
Total [candidate] (9.327 s) : 0, 9326852
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent appsec 1.197 s 121.761 ms (11.3%)
Agent iast 1.209 s 133.704 ms (12.4%)
Agent profiling 1.27 s 194.865 ms (18.1%)
Total tracing 9.159 s -
Total appsec 9.257 s 97.855 ms (1.1%)
Total iast 9.35 s 190.849 ms (2.1%)
Total profiling 9.32 s 160.916 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent appsec 1.215 s 137.859 ms (12.8%)
Agent iast 1.209 s 131.657 ms (12.2%)
Agent profiling 1.28 s 202.652 ms (18.8%)
Total tracing 9.175 s -
Total appsec 9.349 s 173.341 ms (1.9%)
Total iast 9.376 s 200.342 ms (2.2%)
Total profiling 9.327 s 151.548 ms (1.7%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.635 ms) : 0, 693635
BytebuddyAgent [candidate] (694.401 ms) : 0, 694401
GlobalTracer [baseline] (290.248 ms) : 0, 290248
GlobalTracer [candidate] (291.239 ms) : 0, 291239
AppSec [baseline] (48.793 ms) : 0, 48793
AppSec [candidate] (49.103 ms) : 0, 49103
Remote Config [baseline] (1.095 ms) : 0, 1095
Remote Config [candidate] (1.06 ms) : 0, 1060
Telemetry [baseline] (7.255 ms) : 0, 7255
Telemetry [candidate] (7.339 ms) : 0, 7339
section appsec
BytebuddyAgent [baseline] (693.993 ms) : 0, 693993
BytebuddyAgent [candidate] (705.021 ms) : 0, 705021
GlobalTracer [baseline] (290.093 ms) : 0, 290093
GlobalTracer [candidate] (295.311 ms) : 0, 295311
AppSec [baseline] (153.432 ms) : 0, 153432
AppSec [candidate] (154.484 ms) : 0, 154484
IAST [baseline] (17.805 ms) : 0, 17805
IAST [candidate] (18.089 ms) : 0, 18089
Remote Config [baseline] (606.893 µs) : 0, 607
Remote Config [candidate] (626.416 µs) : 0, 626
Telemetry [baseline] (6.821 ms) : 0, 6821
Telemetry [candidate] (6.992 ms) : 0, 6992
section iast
BytebuddyAgent [baseline] (804.478 ms) : 0, 804478
BytebuddyAgent [candidate] (801.937 ms) : 0, 801937
GlobalTracer [baseline] (288.589 ms) : 0, 288589
GlobalTracer [candidate] (291.046 ms) : 0, 291046
AppSec [baseline] (49.53 ms) : 0, 49530
AppSec [candidate] (51.317 ms) : 0, 51317
IAST [baseline] (23.839 ms) : 0, 23839
IAST [candidate] (23.154 ms) : 0, 23154
Remote Config [baseline] (575.324 µs) : 0, 575
Remote Config [candidate] (574.484 µs) : 0, 574
Telemetry [baseline] (7.406 ms) : 0, 7406
Telemetry [candidate] (6.659 ms) : 0, 6659
section profiling
BytebuddyAgent [baseline] (687.106 ms) : 0, 687106
BytebuddyAgent [candidate] (694.204 ms) : 0, 694204
GlobalTracer [baseline] (374.951 ms) : 0, 374951
GlobalTracer [candidate] (376.448 ms) : 0, 376448
AppSec [baseline] (49.981 ms) : 0, 49981
AppSec [candidate] (49.885 ms) : 0, 49885
Remote Config [baseline] (902.875 µs) : 0, 903
Remote Config [candidate] (877.419 µs) : 0, 877
Telemetry [baseline] (7.245 ms) : 0, 7245
Telemetry [candidate] (7.267 ms) : 0, 7267
ProfilingAgent [baseline] (93.901 ms) : 0, 93901
ProfilingAgent [candidate] (94.705 ms) : 0, 94705
Profiling [baseline] (93.925 ms) : 0, 93925
Profiling [candidate] (94.728 ms) : 0, 94728

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.569 µs) : 340, 381
.   : milestone, 361,
iast (465.492 µs) : 445, 486
.   : milestone, 465,
iast_FULL (534.94 µs) : 514, 556
.   : milestone, 535,
iast_GLOBAL (493.296 µs) : 473, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (467.112 µs) : 446, 488
.   : milestone, 467,
iast_INACTIVE (439.324 µs) : 419, 460
.   : milestone, 439,
iast_TELEMETRY_OFF (467.335 µs) : 446, 489
.   : milestone, 467,
tracing (439.646 µs) : 419, 461
.   : milestone, 440,
section candidate
no_agent (362.33 µs) : 342, 382
.   : milestone, 362,
iast (468.119 µs) : 448, 489
.   : milestone, 468,
iast_FULL (531.015 µs) : 511, 552
.   : milestone, 531,
iast_GLOBAL (493.467 µs) : 472, 515
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (467.524 µs) : 447, 488
.   : milestone, 468,
iast_INACTIVE (446.255 µs) : 425, 467
.   : milestone, 446,
iast_TELEMETRY_OFF (474.565 µs) : 453, 496
.   : milestone, 475,
tracing (442.823 µs) : 422, 464
.   : milestone, 443,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.569 µs [340.456 µs, 380.681 µs] -
iast 465.492 µs [445.198 µs, 485.786 µs] 104.923 µs (29.1%)
iast_FULL 534.94 µs [514.004 µs, 555.876 µs] 174.371 µs (48.4%)
iast_GLOBAL 493.296 µs [472.579 µs, 514.013 µs] 132.727 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED 467.112 µs [446.235 µs, 487.988 µs] 106.543 µs (29.5%)
iast_INACTIVE 439.324 µs [419.029 µs, 459.619 µs] 78.755 µs (21.8%)
iast_TELEMETRY_OFF 467.335 µs [446.083 µs, 488.587 µs] 106.766 µs (29.6%)
tracing 439.646 µs [418.541 µs, 460.751 µs] 79.077 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.33 µs [342.49 µs, 382.169 µs] -
iast 468.119 µs [447.653 µs, 488.585 µs] 105.79 µs (29.2%)
iast_FULL 531.015 µs [510.505 µs, 551.525 µs] 168.686 µs (46.6%)
iast_GLOBAL 493.467 µs [472.23 µs, 514.703 µs] 131.137 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED 467.524 µs [447.13 µs, 487.918 µs] 105.194 µs (29.0%)
iast_INACTIVE 446.255 µs [425.165 µs, 467.344 µs] 83.925 µs (23.2%)
iast_TELEMETRY_OFF 474.565 µs [453.304 µs, 495.826 µs] 112.235 µs (31.0%)
tracing 442.823 µs [421.66 µs, 463.986 µs] 80.493 µs (22.2%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~055ffc770d, baseline=1.32.0-SNAPSHOT~f97f63248a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.34 ms) : 1321, 1359
.   : milestone, 1340,
appsec (1.748 ms) : 1724, 1772
.   : milestone, 1748,
iast (1.509 ms) : 1485, 1533
.   : milestone, 1509,
profiling (1.546 ms) : 1521, 1570
.   : milestone, 1546,
tracing (1.483 ms) : 1460, 1506
.   : milestone, 1483,
section candidate
no_agent (1.331 ms) : 1312, 1350
.   : milestone, 1331,
appsec (1.763 ms) : 1739, 1787
.   : milestone, 1763,
iast (1.505 ms) : 1482, 1528
.   : milestone, 1505,
profiling (1.56 ms) : 1535, 1585
.   : milestone, 1560,
tracing (1.529 ms) : 1505, 1553
.   : milestone, 1529,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.321 ms, 1.359 ms] -
appsec 1.748 ms [1.724 ms, 1.772 ms] 407.989 µs (30.4%)
iast 1.509 ms [1.485 ms, 1.533 ms] 168.986 µs (12.6%)
profiling 1.546 ms [1.521 ms, 1.57 ms] 205.527 µs (15.3%)
tracing 1.483 ms [1.46 ms, 1.506 ms] 142.984 µs (10.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.331 ms [1.312 ms, 1.35 ms] -
appsec 1.763 ms [1.739 ms, 1.787 ms] 431.871 µs (32.4%)
iast 1.505 ms [1.482 ms, 1.528 ms] 173.89 µs (13.1%)
profiling 1.56 ms [1.535 ms, 1.585 ms] 228.831 µs (17.2%)
tracing 1.529 ms [1.505 ms, 1.553 ms] 197.866 µs (14.9%)

@zarirhamza zarirhamza closed this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@zarirhamza