Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time spent in CI #3975

Open
5 tasks
romac opened this issue Apr 25, 2024 · 1 comment
Open
5 tasks

Reduce time spent in CI #3975

romac opened this issue Apr 25, 2024 · 1 comment

Comments

@romac
Copy link
Member

romac commented Apr 25, 2024

Summary

Test Time taken Action
test_atomic_ibc_forward_hop_transfer > 720s Reduce time
test_misspelled_memo_content_ibc_forward_transfer > 420s Reduce time
test_misspelled_memo_fields_ibc_forward_transfer > 420s Reduce time
test_filter_incentivized_fees_relayer > 300s Reduce time
test_pay_packet_fee_async > 300s Reduce time
test_ternary_ibc_transfer > 420s Reduce time
python_end_to_end_tests > 240s Remove test

Total time: 47min (2'820s)

If we can get all of those under 120s we'd save ~35min (2'100s)


For Admin Use

  • Not duplicate issue
  • Appropriate labels applied
  • Appropriate milestone (priority) applied
  • Appropriate contributors tagged
  • Contributor assigned/self-assigned
@ljoss17
Copy link
Contributor

ljoss17 commented Jun 7, 2024

Here are some thoughts on the mentioned tests, let me know what you think.

Tests with more than 2 chains

The time for these tests is mostly due to the Connection and Channel handshakes:

  • test_atomic_ibc_forward_hop_transfer need 4 chains to run, as it will test forwarding a packet from A to D, passing by B and C. The bootstrapping will fully connect chains A, B, C and D
  • test_misspelled_memo_content_ibc_forward_transfer and test_misspelled_memo_fields_ibc_forward_transfer bootstrap chains A, B and C
  • test_ternary_ibc_transfer bootstraps chains A, B and C

For this reducing the handshake times should decrease the run time of all tests.

ICS29 tests

  • test_filter_incentivized_fees_relayer
  • test_pay_packet_fee_async

Some ICS29 tests take longer due to a change introduced in v8.1.0+, where only the highest value between recv+ack and timeout fee is escrowed. First verifies if the max(recv+ack, timeout) is escrowed and if it fails after 90 seconds, verifies if recv+ack+timeout is escrowed. This allows the test to be compatible with both implementation.

For this solution we could either verify if the amount a or b is escrowed at each iteration, or add a feature flag to determine if the chain is running v8.1.0+ or not.

CLI tests

The python_end_to_end_tests is testing many CLIs. This takes a long time as it contains many tests grouped in a single one.
Not sure if there is any way to easily reduce the time for this one, but it might be a good idea to migrate the tests to use the test framework so that we can remove the python dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🩹 Triage
Development

No branches or pull requests

2 participants