utubettl: slow take #228

oleg-jukovec · 2024-04-12T09:54:59Z

Our clients reported that utubettl:take() loads CPU too much and takes too much time when there is a utube with a big amount of tasks (+100.000).

We need a benchmark for the case in t/benchmark. It could be a queue with utubettl driver and 5-10 utubes with ~50.000 tasks + 5-10 parallel clients with take() + ack(). It will help to identify the problem.
We need an another benchmark in t/benchmark. With a big number of utubes (10.000) with a small number of tasks on each (4-5) + 5-10 parallel clients with take() + ack(). This way we can make sure that our fix doesn't degrade performance.
We need to fix the problem.

I suggest to make the task in the order:

Create a benchmark for the problem and ensure that we have the problem.
You offer solutions to the problem in the issue, which we will discuss.
Fix the problem after discussion. Ensure that it helps.
Create a regression benchmark and ensure that we don't degrade performance.

The text was updated successfully, but these errors were encountered:

If some of the utube for tasks at the start of the tube were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, but by the cost of slowing the `put` method (it ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, v2 = true}) ``` Part of #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, v2 = true}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, v2 = true}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, v2 = true}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, v2 = true}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, v2 = true}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `v2 = true` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, v2 = true}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `policy = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, policy = "utube_ready_space"}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `policy = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, policy = "utube_ready_space"}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `policy = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, policy = "utube_ready_space"}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `policy = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, policy = "utube_ready_space"}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = "utube_ready_space"}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = "utube_ready_space"` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = "utube_ready_space"}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = queue.driver.utube.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method (it is ~3 times slower). Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE}) ``` Closes #228

If some of the utube for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready_buffer`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method. Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utube.STORAGE_MODE_READY_BUFFER` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utube', {temporary = true, storage_mode = queue.driver.utube.STORAGE_MODE_READY_BUFFER}) ``` Part of #228

If some of the utubettl for tasks at the top of the queue were busy most of the time, `take` would slow down for every other task. This problem is fixed by creating a new space `space_ready_buffer`. It contains first task with `READY` status from each utube. This solution shows great results for the stated problem, with the cost of slowing the `put` method. Thus, this workaround is disabled by default. To enable it, user should set the `storage_mode = queue.driver.utubettl.STORAGE_MODE_READY_BUFFER` as an option while creating the tube. As example: ```lua local test_queue = queue.create_tube('test_queue', 'utubettl', {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_READY_BUFFER}) ``` Closes #228