Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add transaction package integration tests #146

Merged
merged 11 commits into from
May 14, 2024

Conversation

djordon
Copy link
Contributor

@djordon djordon commented May 13, 2024

Description

Address part of #67.

This PR mainly sets up integration tests with bitcoin-core. Specifically it:

  1. Makes sure bitcoin-core is running in CI in regtest mode.
  2. Add helper functions and structs for running integration tests with bitcoin-core in regtest mode.
  3. Adds an integration test that makes sure that the signed version of the UnsignedTransactions returned in the utxo module successfully moves deposits to the signer's UTXO.
  4. Updates the Schnorr public keys in the utxo module to use XOnlyPublicKey since bitcoin-core expects Schnorr public keys to be 32-bit x-coordinate only keys.
  5. Change tests to run on every push instead of only when the PR is open and changes have been made to it. I don't mind changing this back.

Testing information

This PR mainly adds tests but also changes how testing is done. The biggest difference is that cargo test now runs our integration tests and will fail if bitcoin-core is not running locally. To get around that do cargo test --lib or make test if you don't want to run integration tests. Running make integration-test now starts a docker container with bitcoind and stops the container if tests succeed.

Checklist

  • Code is commented where needed
  • Unit test coverage for new or modified code paths
  • make integration-test passes

@djordon djordon added the utxo consolidation Tickets related to how we consolidate deposit and withdrawal BTC transactions label May 13, 2024
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be here? It showed up after the rebase, I'm not sure if I need to gitignore it something.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is certianly my fault, and this makes me realize I don't think I understand the idioms around lock files. Do these need to be in each package, or is pnpm meant to make package-lock.json files within the workspace root and not each sub package.

Copy link
Contributor Author

@djordon djordon May 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohhh I see. I thought that it's supposed to be committed to the repo in this directory, but I'm not entirely sure. I'll find out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know about that particular lock file, but when it comes to committing lock files or not the rule of thumb I'm following is that it should be committed for binaries but not for libraries.

Since this is primarily a binary (or a set of binaries) we're developing and not a generic library to be imported by other people, I believe we should generally commit lock files.

Though this is a Cargo convention, and I don't know what the convention is in JavaScript land.

Copy link
Contributor Author

@djordon djordon May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think this was generated from my .generated-sources/emily/build.rs file, which calls npm still. Should I just delete that file, is it still relevant?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this PR, I would like to see it deleted because it's irrelevant for #67. But I'd be open to seeing it committed as a follow-up depending on what convention we'd like to follow for this file. But it should either be committed or explicitly ignored in .gitignore.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we still need it because it adds the clippy labels to to prevent the clippy warnings.

But we could certainly have it call pnpm, I'll change that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait I'm confused, the lock file doesn't add any Clippy labels right? I mean, I frequently delete it from my local repo and that seems to be alright.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait I'm confused, the lock file doesn't add any Clippy labels right? I mean, I frequently delete it from my local repo and that seems to be alright.

I think he was referring to .generated-sources/emily/build.rs. Also, should that file be unignored?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should figure this all out in a separate PR. I'll be removing this and adding it back later.

@djordon djordon changed the title 67 add transaction package integration tests ci: add transaction package integration tests May 13, 2024
.github/workflows/unit-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/unit-tests.yaml Show resolved Hide resolved
.github/workflows/unit-tests.yaml Outdated Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is certianly my fault, and this makes me realize I don't think I understand the idioms around lock files. Do these need to be in each package, or is pnpm meant to make package-lock.json files within the workspace root and not each sub package.

signer/src/utxo.rs Show resolved Hide resolved
signer/tests/regtest/mod.rs Outdated Show resolved Hide resolved
@djordon djordon force-pushed the 67-add-transaction-package-integration-tests branch 2 times, most recently from 6f2556c to 96cd0da Compare May 13, 2024 18:41
@djordon djordon changed the base branch from main to 142-rename-sbtc-signer-package-name-to-signer May 13, 2024 18:42
@djordon djordon force-pushed the 142-rename-sbtc-signer-package-name-to-signer branch from 70a7ba3 to 910c03d Compare May 14, 2024 04:16
@netrome netrome linked an issue May 14, 2024 that may be closed by this pull request
5 tasks
Copy link
Contributor

@netrome netrome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great to see these integration tests! I just have some nits and style opinions.

Makefile Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this PR, I would like to see it deleted because it's irrelevant for #67. But I'd be open to seeing it committed as a follow-up depending on what convention we'd like to follow for this file. But it should either be committed or explicitly ignored in .gitignore.

emily/api-definition/package-lock.json Outdated Show resolved Hide resolved
signer/src/utxo.rs Outdated Show resolved Hide resolved
signer/tests/regtest/mod.rs Outdated Show resolved Hide resolved
signer/tests/regtest/mod.rs Outdated Show resolved Hide resolved
signer/tests/regtest/mod.rs Outdated Show resolved Hide resolved
signer/tests/utxo_construction.rs Outdated Show resolved Hide resolved
signer/tests/utxo_construction.rs Outdated Show resolved Hide resolved
signer/tests/utxo_construction.rs Outdated Show resolved Hide resolved
Base automatically changed from 142-rename-sbtc-signer-package-name-to-signer to main May 14, 2024 12:50
@djordon djordon force-pushed the 67-add-transaction-package-integration-tests branch from e1d3fe3 to 502e1ca Compare May 14, 2024 14:07
@djordon djordon merged commit 2778c63 into main May 14, 2024
3 checks passed
@djordon djordon deleted the 67-add-transaction-package-integration-tests branch May 14, 2024 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
utxo consolidation Tickets related to how we consolidate deposit and withdrawal BTC transactions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add integration tests for our transaction package
3 participants