Execute performance and stress tests with K6 for gas throttling

[kselveliev]

Problem

After implementing the new throttling mechanism we need to check how it affects performance

Solution

Execute performance and stress tests with K6 with large data size and high gas consumption (30M, 50M per second) and monitor the changes with the new approach

Alternatives

No response

[bilyana-gospodinova]

The results before the gas throttling are the following (executed with 10 VUs for 5 seconds with token freeze and token transfer transactions):

• Test 1 (with sleep 0.1s):
  ✅ 415 ❌ 0
• Test 2 (with sleep 0.2s):
  ✅ 193 ❌ 0
• Test 3 (with sleep 0.3s):
  ✅ 152 ❌ 0

The results after the gas throttling changes in #8266 executed with 10 VUs for 5 seconds with token freeze and token transfer transactions:

• Test 1 (with 0.3s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 15_000_000):
  ✅ 24 ❌ 135
• Test 2 (with 0.3s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 5_000_000):
  ✅ 71 ❌ 78
• Test 3 (with 0.3s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 500_000):
  ✅ 160 ❌ 0
• Test 4 (with 0.2s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 500_000):
  ✅ 223 ❌ 0
• Test 5 (with 0.2s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 5_000_000):
  ✅ 73 ❌ 167
• Test 6 (with 0.2s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 1_000_000):
  ✅ 235 ❌ 0
• Test 7 (with 0.2s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 15_000_000):
  ✅ 24 ❌ 221
• Test 8 (with 0.1s sleep, throttleGasLimit = 50_000_000, gasLimit per request = 500_000):
  ✅ 441 ❌ 0

Comparing the results before and after the changes we can see that the gas throttle mechanism can effectively regulate the number of requests that are being processed. Also, the additional synchronization from the gas throttle bucket does not affect the performance.

[zhpetkov]

I compared the results before and after the gas throttling changes. I executed an estimate contract deploy with 4 VUs for 3s, 0.1s sleep, and data size about 24 kB.

The results before the changes:

• Test 1 (with a gas limit of 653 000 per request)
  ✅ 112 ❌ 0 🟩 100%
• Test 2 (with a gas limit of 933 000 per request )
  ✅ 112 ❌ 0 🟩 100%
• Test 3 (with a gas limit of 1 404 000 per request)
  ✅ 112 ❌ 0 🟩 100%
• Test 4 (with a gas limit of 1 875 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 5 (with a gas limit of 2 813 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 6 (with a gas limit of 3 750 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 7 (with a gas limit of 5 625 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 8 (with a gas limit of 7 500 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 9 (with a gas limit of 11 250 000 per request)
  ✅ 108 ❌ 0 🟩 100%
• Test 10 (with a gas limit of 15 000 000 per request)
  ✅ 108 ❌ 0 🟩 100%

The results after the gas throttling changes in #8266:

• Test 1 (with a gas limit of 653 000 per request and throttle gas limit of 50 000 000L)
  ✅ 108 ❌ 0 🟩 100%
• Test 2 (with a gas limit of 933 000 per request and throttle gas limit of 50 000 000L)
  ✅ 104 ❌ 0 🟩 100%
• Test 3 (with a gas limit of 1 404 000 per request and throttle gas limit of 50 000 000L)
  ✅ 108 ❌ 0 🟩 100%
• Test 4 (with a gas limit of 1 875 000 per request and throttle gas limit of 50 000 000L)
  ✅ 108 ❌ 0 🟩 100%
• Test 5 (with a gas limit of 2 813 000 per request and throttle gas limit of 50 000 000L)
  ✅ 87 ❌ 21 🟧 80%
• Test 6 (with a gas limit of 3 750 000 per request and throttle gas limit of 50 000 000L)
  ✅ 66 ❌ 45 🟧 59%
• Test 7 (with a gas limit of 5 625 000 per request and throttle gas limit of 50 000 000L)
  ✅ 43 ❌ 69 🟨 38%
• Test 8 (with a gas limit of 7 500 000 per request and throttle gas limit of 50 000 000L)
  ✅ 33 ❌ 79 🟨 29%
• Test 9 (with a gas limit of 11 250 000 per request and throttle gas limit of 50 000 000L)
  ✅ 21 ❌ 91 🟥 18%
• Test 10 (with a gas limit of 15 000 000 per request and throttle gas limit of 50 000 000L)
  ✅ 16 ❌ 96 🟥 14%

After changes, the gas throttle mechanism regulates the number of requests that are being processed. It works effectively and doesn't drastically reduce performance.