Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking] Pre-release of the mainnet branch #3160

Draft
wants to merge 686 commits into
base: testnet3
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
686 commits
Select commit Hold shift + click to select a range
ef8b522
Merge branch 'mainnet-staging' into min-proposal-time
raychu86 Apr 2, 2024
6ff3343
Track the proposal timestamp in signed_proposals
raychu86 Apr 2, 2024
11c4618
Expire proposals after PROPOSAL_EXPIRATION_IN_SECS
raychu86 Apr 2, 2024
2dc94f5
Cleanup and add tests
raychu86 Apr 2, 2024
ab93cd6
cleanup
raychu86 Apr 3, 2024
0400e76
Lower priority of warnings on invalid solutions
d0cd Apr 3, 2024
7ac2c09
Clean up and document
d0cd Apr 3, 2024
3f8cf7c
Reduce aggressive disconnect
raychu86 Apr 3, 2024
e26c608
Merge pull request #3193 from AleoHQ/min-proposal-time
howardwu Apr 3, 2024
ed30f9e
nit: reorder cli flags, improve order
howardwu Apr 3, 2024
b17ab57
Switches storage_path to storage
howardwu Apr 3, 2024
a5dbf77
Merge pull request #3189 from AleoHQ/optional_node_ip
howardwu Apr 3, 2024
f038b45
Merge branch 'mainnet-staging' into ensure_validator_connects_to_vali…
howardwu Apr 3, 2024
bbd5aaa
Merge pull request #3198 from AleoHQ/ensure_validator_connects_to_val…
howardwu Apr 3, 2024
cd6f71f
Merge mainnet-staging
raychu86 Apr 3, 2024
1b6386f
Remove unused derivations
raychu86 Apr 3, 2024
7fec580
Unify timestamp checks to a helper method
raychu86 Apr 3, 2024
830a8b8
cleanup
raychu86 Apr 3, 2024
2e75635
Fix tests
raychu86 Apr 3, 2024
982b017
Bump snarkVM rev - 39282a1
raychu86 Apr 3, 2024
1848688
Silence instead of tracing solution errors within 10 blocks of the in…
d0cd Apr 3, 2024
080ee55
Fix
d0cd Apr 3, 2024
9c2a8d1
Merge pull request #3202 from AleoHQ/feat/expire-proposal
howardwu Apr 4, 2024
f6ca75d
Merge pull request #3203 from AleoHQ/log/deprior-spurious-solution-logs
howardwu Apr 4, 2024
6e88e9e
Merge pull request #3205 from AleoHQ/bump/snarkVM-39282a1
howardwu Apr 4, 2024
d389dd5
Update snarkVM rev - 36a1e0f
raychu86 Apr 4, 2024
2913459
Update snarkVM rev - ed20562
d0cd Apr 5, 2024
ec943b3
Merge pull request #3206 from AleoHQ/fix/tx-ordering
howardwu Apr 5, 2024
c620ab4
Merge branch 'AleoHQ:mainnet-staging' into feat/unconfirmed-transacti…
miazn Apr 8, 2024
8357eae
nit: reorder
howardwu Apr 8, 2024
348b21e
Include the aborted solution IDs and transaction IDs
howardwu Apr 8, 2024
53bd65d
Parallelize potentially slow for loop
howardwu Apr 8, 2024
1ebf4d5
remove register histogram
miazn Apr 8, 2024
8e79de5
Add aborted_transmissions to find_missing_transmissions
raychu86 Apr 8, 2024
bf07a92
Track aborted transactions in insert_certificate and check_certificate
raychu86 Apr 8, 2024
190cd43
Add test for inserting certificate with aborted transmissions
raychu86 Apr 8, 2024
e753a81
Add ledger check for aborted transmissions
raychu86 Apr 8, 2024
6235461
Merge pull request #3169 from miazn/feat/unconfirmed-transaction-late…
howardwu Apr 9, 2024
f812c26
move some metrics to CoreLedgerService, out of consensus
miazn Apr 9, 2024
2646592
add func
miazn Apr 9, 2024
19ba95e
remove unconfirmed transmissions
miazn Apr 9, 2024
234f11e
fmt
miazn Apr 9, 2024
5a3f5b0
Optimize check
raychu86 Apr 9, 2024
003227a
add more granular transaction metrics
miazn Apr 10, 2024
7f22a8e
Merge pull request #3210 from miazn/feat/move-metrics-from-consensus-…
howardwu Apr 10, 2024
380bc89
add aborted transmissions
miazn Apr 11, 2024
19c77c7
Choose random set of transmissions in worker ping
raychu86 Apr 11, 2024
d48f75c
Remove transaction minimum in proposal
raychu86 Apr 11, 2024
ebbbd17
Reinsert transmission to worker after failure to create proposal
raychu86 Apr 11, 2024
5287579
Update MAX_TRANSMISSIONS_TOLERANCE to 1024
raychu86 Apr 11, 2024
08361b4
Removes request_timeouts from BlockSync
howardwu Apr 11, 2024
cced2c2
Adds a shuffle on the sync_peers
howardwu Apr 11, 2024
42094a2
Remove timeout vec
howardwu Apr 11, 2024
27aea25
deps(bft): add `pea2pea` to dev deps
niklaslong Mar 6, 2024
2c67849
tests: implement test peer tooling
niklaslong Mar 6, 2024
816fbf1
tests: implement gateway handshake cases
niklaslong Mar 6, 2024
b9f11e1
tests: `deadline!` assertions to fix CI flakiness
niklaslong Mar 7, 2024
4cb0478
tests: fix post-rebase
niklaslong Mar 19, 2024
88dfdc6
deps: update `pea2pea`
niklaslong Mar 25, 2024
e01a329
chore: update lock file
niklaslong Mar 19, 2024
1fb1448
chore: clean up a little lingering noise
niklaslong Apr 12, 2024
f9c4703
Construct the requests based on the largest valid range of blocks
raychu86 Apr 12, 2024
7a5f073
Increase block request range to 5
raychu86 Apr 12, 2024
2d5ab54
Randomly select sync ips per chunk
raychu86 Apr 12, 2024
24d0cfa
Find the first start height that is valid
raychu86 Apr 12, 2024
4511ac4
Increase expected end height to match MAXIMUM_NUMBER_OF_BLOCKS
raychu86 Apr 12, 2024
0628714
Change MAXIMUM_NUMBER_OF_BLOCKS to 10
raychu86 Apr 12, 2024
d6589a2
Update DataBlocks::MAXIMUM_NUMBER_OF_BLOCKS to 10
raychu86 Apr 12, 2024
49a97ab
Merge pull request #3212 from niklaslong/chore/cleanup
howardwu Apr 12, 2024
ff93a40
Update MAX_EVENT_SIZE to 256mb
raychu86 Apr 12, 2024
bf0ada3
Merge branch 'mainnet-staging' into feat/sync-updates
howardwu Apr 12, 2024
3eb2d67
Merge pull request #3208 from AleoHQ/skip-aborted-transmissions
howardwu Apr 12, 2024
af0b74a
Merge pull request #3213 from AleoHQ/update-max-num-blocks
howardwu Apr 12, 2024
0148db2
Remove transmission minimum in proposal
raychu86 Apr 12, 2024
aae1e50
Update MAX_TRANSMISSIONS_TOLERANCE
raychu86 Apr 12, 2024
f42209e
Merge pull request #3164 from niklaslong/tests/gateway-no-noise
howardwu Apr 12, 2024
86b706e
Allow proposals with no transmissions
raychu86 Apr 12, 2024
f5051d0
Merge pull request #3214 from AleoHQ/feat/sync-updates
howardwu Apr 12, 2024
29cf16a
Bump snarkVM rev - d48f6fb
raychu86 Apr 13, 2024
5a3ed41
Merge bump/snarkVM-d48f6fb
raychu86 Apr 13, 2024
1c952d7
Reinsert transmission to worker after failure to create header or pro…
raychu86 Apr 13, 2024
b9c7f23
Use proper Network::ID in match cases
raychu86 Apr 13, 2024
a499325
Merge pull request #3211 from AleoHQ/worker-ping-updates
howardwu Apr 13, 2024
1577b7b
Merge branch 'mainnet-staging' into feat/testnetv0
howardwu Apr 13, 2024
51ae568
Fix flaky test
raychu86 Apr 13, 2024
ebfe993
Merge pull request #3199 from AleoHQ/feat/testnetv0
howardwu Apr 13, 2024
a1c1286
Bump snarkVM rev - 69c25a8
raychu86 Apr 14, 2024
58b10da
Add aborted transmissions ID to BFTMemoryService
raychu86 Apr 14, 2024
5a0e235
Add aborted transmissions ID to BFTPersistentStorage
raychu86 Apr 14, 2024
a497580
Update tests and add check that aborted transmisssion IDs exist in st…
raychu86 Apr 14, 2024
ff6161c
make standalone ledger func, counters atomic
miazn Apr 15, 2024
2ac4cea
Merge branch 'mainnet-staging' into feat/add-transaction-types
miazn Apr 15, 2024
ba9a7b6
nit
raychu86 Apr 15, 2024
7e6d229
Add transactions and solutions from the pending queue to the rest end…
raychu86 Apr 15, 2024
8f403d4
Add stateRoot endpoints
raychu86 Apr 15, 2024
51127d9
change to cfg_iter macro
miazn Apr 16, 2024
b69fef6
Reintroduce IS_SYNCING const generic
raychu86 Apr 16, 2024
fffaa67
Add get_delegators_for_validator
vicsn Apr 16, 2024
e7d4720
Nit log
raychu86 Apr 16, 2024
d9c3826
Update snarkVM rev - da3d78a
raychu86 Apr 16, 2024
dde9e17
Merge pull request #3217 from AleoHQ/feat/aborted-transmissions-map
howardwu Apr 16, 2024
e908b39
Update endpoint name
raychu86 Apr 16, 2024
c0af3fb
Rename getter functions
raychu86 Apr 16, 2024
d87fdec
nit
raychu86 Apr 16, 2024
1ac9997
add optional to cargo.toml
miazn Apr 16, 2024
1064bbb
make rayon install optional
miazn Apr 17, 2024
bf77e9c
Merge branch 'mainnet-staging' of https://github.com/AleoHQ/snarkOS i…
raychu86 Apr 17, 2024
06dec10
restructure metrics functions to metrics crate for optional dependencies
miazn Apr 18, 2024
7bbc708
Add seen_inbound_block_requests to cache
raychu86 Apr 18, 2024
276fc17
Add is_block_synced to Outbound
raychu86 Apr 18, 2024
e7b0e43
Do not peer refresh a sync peer
raychu86 Apr 18, 2024
4ae5926
Add is_block_synced to Outbound
raychu86 Apr 18, 2024
0eef8ed
Perform clone after seen_before check
raychu86 Apr 18, 2024
bdfb225
Do not process unconfirmed solution if node is syncing
raychu86 Apr 18, 2024
877bf7f
Verify the solution prior to broadcasting
raychu86 Apr 18, 2024
67a5fdd
Do not process unconfirmed transaction if node is syncing
raychu86 Apr 18, 2024
244fc00
Use tokio task for REST solution verification
raychu86 Apr 18, 2024
043b97e
Merge branch 'mainnet-staging' into feat/rest-endpoints
raychu86 Apr 18, 2024
cabf7b4
Update MAXIMUM_NUMBER_OF_BLOCKS to 5
raychu86 Apr 19, 2024
282d1c6
Merge branch 'mainnet-staging' into feat/add-transaction-types
miazn Apr 19, 2024
3a87e73
Merge pull request #3223 from AleoHQ/feat/no-syncing-peer-refresh
howardwu Apr 19, 2024
3c5bba1
Fix ordering
raychu86 Apr 19, 2024
cca3c44
Update ordering for find/blockHeight
raychu86 Apr 19, 2024
588c284
Merge pull request #3218 from AleoHQ/feat/rest-endpoints
howardwu Apr 19, 2024
06c874b
Bump snarkVM rev - 43abe1b
raychu86 Apr 19, 2024
e26d174
Update node/metrics/Cargo.toml
miazn Apr 19, 2024
98e2189
import reordering
miazn Apr 19, 2024
2e42c41
Cleanup
raychu86 Apr 19, 2024
02cc077
Merge pull request #3228 from AleoHQ/feat/sync-range
howardwu Apr 19, 2024
f38883f
Update node/metrics/src/lib.rs
howardwu Apr 19, 2024
f6176ac
Merge pull request #3216 from miazn/feat/add-transaction-types
howardwu Apr 19, 2024
999aa17
Add SYNCED metric
d0cd Apr 20, 2024
e94b8b0
Update SYNCED metric for validators
d0cd Apr 20, 2024
d9bd0db
Update metric to IS_SYNCED
d0cd Apr 22, 2024
8c983e0
Perform block advancement after quorum check
raychu86 Apr 22, 2024
da40a7b
Cleanup and add tracker for latest block
raychu86 Apr 23, 2024
14eadb4
Nit
raychu86 Apr 23, 2024
aebc090
nit comment
raychu86 Apr 23, 2024
e7e7c5a
Use last block response
raychu86 Apr 23, 2024
a95e6c7
Update logs and add check
raychu86 Apr 23, 2024
3769541
Use availability threshold
raychu86 Apr 23, 2024
88a70b9
Add is_linked checks during sync
raychu86 Apr 24, 2024
f464991
Clear latest block responses on faulty blocks
raychu86 Apr 24, 2024
455078c
Add logs
raychu86 Apr 24, 2024
eedbd31
nits
raychu86 Apr 24, 2024
6157b45
Merge pull request #3231 from AleoHQ/feat/sync-metrics
howardwu Apr 24, 2024
7d34635
Add is_block_synced checks to rest endpoints
raychu86 Apr 24, 2024
55c8a6e
nits
raychu86 Apr 24, 2024
a83a767
Merge pull request #3225 from AleoHQ/optimize/solution-processing
howardwu Apr 24, 2024
a83c4e4
Merge pull request #3233 from AleoHQ/feat/syncing-is-linked
howardwu Apr 24, 2024
d3f14a2
Merge branch 'mainnet-staging' into fix/check-validator-sync
raychu86 Apr 24, 2024
4669e3d
Bump snarkVM rev - aef07da
raychu86 Apr 24, 2024
040631f
Add tracker for num_blocks_behind
raychu86 Mar 14, 2024
baf3d10
Add num_blocks_behind to Outbound
raychu86 Apr 25, 2024
546773d
Add sync leniency for rest processing
raychu86 Apr 25, 2024
89a5f24
Update function documentation
raychu86 Apr 25, 2024
5c2804f
Use SYNC_LENIENCY in inbound processing
raychu86 Apr 25, 2024
08a40be
Fix check
raychu86 Apr 25, 2024
3095707
Add unit test for syncing is_linked blocks
raychu86 Apr 25, 2024
190e125
Merge pull request #3232 from AleoHQ/fix/check-validator-sync
howardwu Apr 26, 2024
9be1039
Merge pull request #3234 from AleoHQ/rest/sync-leniency
howardwu Apr 26, 2024
aa89be6
Merge branch 'mainnet-staging' into adjust-timeouts
raychu86 Apr 26, 2024
8426d35
Decrement counter
ghostant-1017 Apr 28, 2024
fe8fb4b
Resolve merge
raychu86 Apr 29, 2024
7c43fe2
Use read instead of write
raychu86 Apr 29, 2024
ad0d54c
Reduce instances of fetching current timestamp
raychu86 Apr 29, 2024
042c94d
feat: store current batch proposal to disk on shutdown
ljedrz Apr 2, 2024
535d63a
Merge pull request #3238 from ghostant-1017/fix/peer_response
howardwu Apr 29, 2024
b8a586d
Introduce dedicated SignedProposals object
raychu86 Apr 29, 2024
6088bc1
Minor fix
raychu86 Apr 29, 2024
8a1ab3b
nit
raychu86 Apr 29, 2024
ebb54d8
Merge branch 'mainnet-staging' into get_delegators_for_validator
raychu86 Apr 29, 2024
296020e
Update node/bft/src/helpers/pending.rs
raychu86 Apr 30, 2024
a39996d
Address comments
raychu86 Apr 30, 2024
964599b
Use singluar lock and remove collect
raychu86 Apr 30, 2024
f380d6f
Add checks for MAX_TRANSACTION_SIZE
niklaslong Apr 29, 2024
c0bb346
tests: add large execution transaction helper
niklaslong Apr 21, 2024
899bfb5
feat: check transaction length before decoding
niklaslong Apr 21, 2024
e3f7f3c
tests: proptest codec with unconfirmed transactions
niklaslong Apr 21, 2024
a92a0fe
ref: move byte length check into `Message`
niklaslong Apr 23, 2024
fe7cbfa
deps: update snarkVM rev
niklaslong Apr 29, 2024
79efabe
chore: update lock file
niklaslong Apr 29, 2024
ec4946d
ref: move message size check to `UnconfirmedTransaction`
niklaslong Apr 30, 2024
778030e
Resolve flaky test with sleep
raychu86 Apr 30, 2024
dc6a4f5
Introduce ProposalCache that is stored and loaded
raychu86 Apr 30, 2024
10a8bc2
Remove proposal expiration
raychu86 Apr 30, 2024
c1af74f
Nit
raychu86 Apr 30, 2024
72dc8d2
Add additional safety check
raychu86 Apr 30, 2024
bd5ea75
Merge branch 'mainnet-staging' into feat/store_proposals
raychu86 Apr 30, 2024
ceb5a81
ref: revert move message size check to `UnconfirmedTransaction`
niklaslong May 1, 2024
bf37794
ref: import ordering
niklaslong May 1, 2024
9208c5d
deps: update snarkVM rev
niklaslong May 1, 2024
96c756f
chore: update lock file
niklaslong May 1, 2024
5876dec
feat: check tx length in `ensure_transmission_is_well_formed`
niklaslong May 1, 2024
7aaf276
Address comments
raychu86 May 1, 2024
53eca2f
nit
raychu86 May 1, 2024
e4d8175
Add periodic round increments if quorum is met
raychu86 May 1, 2024
d3dc630
Store the latest storage round to ProposalCache
raychu86 May 2, 2024
600e863
Bump snarkVM rev - 57641ca
raychu86 May 2, 2024
c4e1c89
Merge pull request #3135 from AleoHQ/adjust-timeouts
howardwu May 2, 2024
a5a9c3a
Use propose lock for latest round
raychu86 May 2, 2024
011bb60
Add unit tests
raychu86 May 2, 2024
052b287
Update filename in dev instances
raychu86 May 2, 2024
9220bba
Merge branch 'mainnet-staging' into feat/store_proposals
raychu86 May 2, 2024
d038389
Add documentation
raychu86 May 2, 2024
c53f5fd
nits
raychu86 May 2, 2024
6b54869
Merge pull request #3239 from AleoHQ/feat/store_proposals
howardwu May 2, 2024
a2b887f
Disable transaction loop for production environments
raychu86 May 2, 2024
2b75824
ref: adjust the buffer size in `transaction_broadcast`
niklaslong May 2, 2024
8d3a58b
feat: limit rest body size to 512KiB
niklaslong May 1, 2024
edb846a
Update logs and add minor optimization
raychu86 May 2, 2024
99a8efc
Merge branch 'mainnet-staging' into get_delegators_for_validator
iamalwaysuncomfortable May 3, 2024
10305c5
Merge pull request #3244 from niklaslong/feat/rest-max-request-size
howardwu May 3, 2024
63489ca
Merge pull request #3246 from AleoHQ/update/logs
howardwu May 3, 2024
c28949a
Merge pull request #3245 from AleoHQ/dev-transaction-loop
howardwu May 3, 2024
b2bc20e
Merge branch 'mainnet-staging' into feat/transaction-byte-limit
howardwu May 3, 2024
2cd1b3b
Merge pull request #3229 from niklaslong/feat/transaction-byte-limit
howardwu May 3, 2024
913db14
Update block_sync.rs
howardwu May 3, 2024
fc340c6
Bump snarkVM rev - 5c57a48
raychu86 May 4, 2024
c34a5e6
Merge branch 'mainnet-staging' into get_delegators_for_validator
iamalwaysuncomfortable May 4, 2024
1c74880
Put delegator retrieval into a blocking task + return an error if the…
iamalwaysuncomfortable May 4, 2024
8a127c7
feat: dim logs on shutdown
joske Apr 25, 2024
c0c4b23
feat: add warning to wait until shutdown complete
joske Apr 25, 2024
abf19f6
fix: avoid lifetime annotations
joske Apr 30, 2024
1534c87
fix: review comments
joske Apr 30, 2024
f022ea8
chore: update lock file
joske Apr 30, 2024
dad784f
fix: improve shutdown warning
joske May 2, 2024
eae0034
Move proposal cache loading
raychu86 May 7, 2024
f24e9f6
Store and load the pending certificates
raychu86 May 7, 2024
6faa05a
Sync storage with ledger at bootup prior to loading proposal cache
raychu86 May 7, 2024
3df364e
add committee endpoint for specific block
miazn May 8, 2024
dd26919
Add --network to snarkos clean in devnet script
raychu86 May 8, 2024
4970677
Adds unit test for pending certificates
mdelle1 May 8, 2024
89a3b32
cargo fmt
raychu86 May 8, 2024
dfe074f
Remove usage of current commitee round if the round is in the future
raychu86 May 8, 2024
209d0c6
Bump snarkVM rev - 3ebe60c
raychu86 May 9, 2024
d6a441b
Merge pull request #3219 from AleoHQ/get_delegators_for_validator
howardwu May 9, 2024
ac1bf32
Resolve merge conflict
howardwu May 9, 2024
ec2d7fa
Merge pull request #3248 from miazn/feat/add-committee-block-endpoint
howardwu May 9, 2024
7ed91da
Add check for number of certificates
raychu86 May 9, 2024
9a6ac11
Merge pull request #3247 from AleoHQ/feat/store-pending-certificates
howardwu May 9, 2024
9a69a82
Merge branch 'mainnet-staging' into feat/dim_logs
howardwu May 9, 2024
05a3fa4
Add spawn_blocking around the get_blocks GET request
raychu86 May 9, 2024
75e57e4
Update shutdown message
howardwu May 9, 2024
8836506
Merge pull request #3235 from eqlabs/feat/dim_logs
howardwu May 9, 2024
c26f719
Merge pull request #3252 from AleoHQ/optimize/spawn-blocking-blocks
howardwu May 9, 2024
b1f8930
Merge branch 'mainnet-staging' into update/get_committee_for_round
raychu86 May 9, 2024
996b3ba
Add grace period before starting block sync loop
raychu86 May 10, 2024
3e09ad2
Merge pull request #3250 from AleoHQ/update/get_committee_for_round
howardwu May 10, 2024
6ac2fe5
Merge pull request #3249 from AleoHQ/fix/update-is-block-synced
howardwu May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
44 changes: 22 additions & 22 deletions .circleci/config.yml
Expand Up @@ -160,7 +160,7 @@ commands:
jobs:
integration:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial_long:
Expand All @@ -169,7 +169,7 @@ jobs:

snarkos:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial:
Expand All @@ -178,7 +178,7 @@ jobs:

account:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -187,7 +187,7 @@ jobs:

cli:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -196,7 +196,7 @@ jobs:

display:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -205,7 +205,7 @@ jobs:

node:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial:
Expand All @@ -214,7 +214,7 @@ jobs:

node-bft:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial:
Expand All @@ -223,7 +223,7 @@ jobs:

node-bft-events:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial:
Expand All @@ -232,7 +232,7 @@ jobs:

node-bft-ledger-service:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -241,7 +241,7 @@ jobs:

node-bft-storage-service:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -250,7 +250,7 @@ jobs:

node-cdn:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -259,7 +259,7 @@ jobs:

node-consensus:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- run_serial:
Expand All @@ -268,7 +268,7 @@ jobs:

node-rest:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -277,7 +277,7 @@ jobs:

node-router:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -286,7 +286,7 @@ jobs:

node-router-messages:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -295,7 +295,7 @@ jobs:

node-sync:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -304,7 +304,7 @@ jobs:

node-sync-communication-service:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -313,7 +313,7 @@ jobs:

node-sync-locators:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -322,7 +322,7 @@ jobs:

node-tcp:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- run_serial:
Expand All @@ -331,7 +331,7 @@ jobs:

check-fmt:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: xlarge
steps:
- checkout
Expand All @@ -347,7 +347,7 @@ jobs:

check-clippy:
docker:
- image: cimg/rust:1.74.1
- image: cimg/rust:1.76.0 # Attention - Change the MSRV in Cargo.toml and rust-toolchain as well
resource_class: 2xlarge
steps:
- checkout
Expand Down Expand Up @@ -420,6 +420,6 @@ workflows:
filters:
branches:
only:
- testnet3
- mainnet
jobs:
- integration
118 changes: 91 additions & 27 deletions .devnet/.analytics/analytics.js
Expand Up @@ -6,6 +6,44 @@ const yargs = require('yargs');
const dimStart = "\x1b[2m";
const dimEnd = "\x1b[0m";

// Function to get the IP address of a given AWS node
async function getIPAddress(awsNodeName) {
// Read the ~/.ssh/config file
const sshConfigFile = fs.readFileSync(`${process.env.HOME}/.ssh/config`, 'utf8');

// Use regular expressions to extract the associated IP address
const regex = new RegExp(`Host\\s+${awsNodeName}[\\s\\S]*?HostName\\s+(\\S+)`);
const match = sshConfigFile.match(regex);

if (match && match[1]) {
return match[1];
} else {
console.error(`No IP address found for ${awsNodeName} in ~/.ssh/config`);
}
}

// Function to get the count of AWS nodes based on the naming convention aws-nXX in the SSH config file
async function getAWSNodeCount() {
// Read the ~/.ssh/config file
const sshConfigFile = fs.readFileSync(`${process.env.HOME}/.ssh/config`, 'utf8');

// Regular expression to match all aws-nXX formats
const regex = /Host\s+(aws-n\d+)/g;
let match;
let highestNumber = -1;

// Iterate over all matches and find the highest number
while ((match = regex.exec(sshConfigFile)) !== null) {
const nodeNumber = parseInt(match[1].replace('aws-n', ''), 10);
if (nodeNumber > highestNumber) {
highestNumber = nodeNumber;
}
}

// Return the count of nodes, adding 1 because it starts from 0
return highestNumber >= 0 ? highestNumber + 1 : 0;
}

// Function to fetch block data
async function fetchBlockData(baseUrl, height) {
try {
Expand Down Expand Up @@ -77,10 +115,32 @@ async function calculateRoundsInBlocks(baseUrl, latestHeight) {
}
}

async function checkBlockHash(blockHeight) {
const numNodes = await getAWSNodeCount();
console.log(`Detected ${numNodes} AWS nodes... \n`);

for (let i = 0; i < numNodes; i++) {
// Define the AWS node name to search for (e.g., aws-n1)
const awsNodeName = `aws-n${i}`;
// Get the IP address of the AWS node
const ipAddress = await getIPAddress(awsNodeName);
// Define the base URL for the node
const baseUrl = `http://${ipAddress}:3030/mainnet/block`;

// Fetch the block data
const blockData = await fetchBlockData(baseUrl, blockHeight);
if (blockData && blockData.block_hash) {
console.log(`${awsNodeName} - Block ${blockHeight} - ${blockData.block_hash}`);
} else {
console.log(`${awsNodeName} - Block ${blockHeight} - No block hash found`);
}
}
}

// Main function to fetch block metrics
async function fetchBlockMetrics(baseUrl, metricType) {
async function fetchBlockMetrics(metricType, optionalBlockHeight) {
// Function to get the latest block height
async function getLatestBlockHeight() {
async function getLatestBlockHeight(baseUrl) {
try {
const response = await axios.get(`${baseUrl}/height/latest`);
const latestHeight = response.data;
Expand All @@ -92,7 +152,17 @@ async function fetchBlockMetrics(baseUrl, metricType) {
}
}

const latestHeight = await getLatestBlockHeight();
// Define the AWS node name to search for (e.g., aws-n1)
const awsNodeName = 'aws-n1';
// Get the IP address of the AWS node
const ipAddress = await getIPAddress(awsNodeName);
// Define the base URL for the node.
const baseUrl = `http://${ipAddress}:3030/mainnet/block`;

console.log(`${dimStart}IP Address: ${ipAddress}${dimEnd}`);
console.log(`${dimStart}Base URL: ${baseUrl}${dimEnd}`);

const latestHeight = await getLatestBlockHeight(baseUrl);
if (latestHeight === null) {
console.error('Unable to fetch latest block height, try again...');
return;
Expand All @@ -104,6 +174,8 @@ async function fetchBlockMetrics(baseUrl, metricType) {
calculateAverageBlockTime(baseUrl, latestHeight);
} else if (metricType === 'roundsInBlocks') {
calculateRoundsInBlocks(baseUrl, latestHeight);
} else if (metricType === 'checkBlockHash' && optionalBlockHeight) {
checkBlockHash(optionalBlockHeight);
} else {
console.error('Invalid metric type. Supported types: "averageBlockTime" or "roundsInBlocks".');
}
Expand All @@ -115,35 +187,27 @@ async function main() {
.options({
'metric-type': {
alias: 'm',
describe: 'Metric type to fetch (averageBlockTime or roundsInBlocks)',
describe: 'Metric type to fetch (averageBlockTime, roundsInBlocks, or checkBlockHash)',
demandOption: true,
choices: ['averageBlockTime', 'roundsInBlocks'],
choices: ['averageBlockTime', 'roundsInBlocks', 'checkBlockHash'],
},
'block-height': {
alias: 'b',
describe: 'Block height to examine for checkBlockHash metric',
type: 'number',
},
})
.check((argv) => {
// Check if metric-type is checkBlockHash and block-height is provided
if (argv['metric-type'] === 'checkBlockHash' && (isNaN(argv['block-height']) || argv['block-height'] == null)) {
throw new Error('Block height is required when metric-type is checkBlockHash');
}
return true; // Indicate that the arguments passed the check
})
.argv;

// Read the ~/.ssh/config file
const sshConfigFile = fs.readFileSync(`${process.env.HOME}/.ssh/config`, 'utf8');

// Define the AWS node name to search for (e.g., aws-n1)
const awsNodeName = 'aws-n1';

// Use regular expressions to extract the IP address associated with aws-n0
const regex = new RegExp(`Host\\s+${awsNodeName}[\\s\\S]*?HostName\\s+(\\S+)`);
const match = sshConfigFile.match(regex);

if (match && match[1]) {
const ipAddress = match[1];
const baseUrl = `http://${ipAddress}:3033/testnet3/block`;

console.log(`${dimStart}IP Address: ${ipAddress}${dimEnd}`);
console.log(`${dimStart}Base URL: ${baseUrl}${dimEnd}`);

// Fetch and output the specified block metric
fetchBlockMetrics(baseUrl, argv['metric-type']);
} else {
console.error(`No IP address found for ${awsNodeName} in ~/.ssh/config`);
}
// Fetch and output the specified block metric
fetchBlockMetrics(argv['metric-type'], argv['block-height']);
}

// Run the main function
Expand Down
2 changes: 0 additions & 2 deletions .devnet/.analytics/package.json
Expand Up @@ -4,8 +4,6 @@
"description": "",
"main": "analytics.js",
"scripts": {
"averageBlockTime": "node analytics.js --metric-type averageBlockTime",
"roundsInBlocks": "node analytics.js --metric-type roundsInBlocks",
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "The Aleo Team",
Expand Down
6 changes: 3 additions & 3 deletions .devnet/README.md
Expand Up @@ -6,11 +6,11 @@ Start by creating EC2 instances in the AWS console.
- Ubuntu 22.04 LTS (not Amazon Linux)
- Security Group - Inbound Policy
- SSH - Port 22 - 0.0.0.0/0 (or your IP)
- Custom TCP - Port 3033 - 0.0.0.0/0 (or your IP)
- Custom TCP - Port 4133 - 0.0.0.0/0
- Custom TCP - Port 3030 - 0.0.0.0/0 (or your IP)
- Custom TCP - Port 4130 - 0.0.0.0/0
- Custom TCP - Port 5000 - 0.0.0.0/0

Be sure the give the EC2 instances a name tag, i.e. `testnet3`.
Be sure the give the EC2 instances a name tag, i.e. `devnet`.

Make sure you set the correct SSH `.pem` and have the `.pem` in your `~/.ssh` directory.

Expand Down