Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup record access #2792

Merged
merged 4 commits into from
Jan 25, 2022
Merged

Cleanup record access #2792

merged 4 commits into from
Jan 25, 2022

Conversation

tinker-michaelj
Copy link
Collaborator

@tinker-michaelj tinker-michaelj commented Jan 24, 2022

Description:

  • Removes a vestigial code path in GetTxnRecordAnswer that looks for the the TransactionRecord in the TransactionID's payer account when it is missing from the txnHistories map.
    • After we stopped creating 25-hour threshold records, the payer account will have the record iff it is still in the txnHistories map. (Both live for 180 seconds, and are removed simultaneously during record expiration.)
  • Limits the payer records returned for a CryptoGetAccountRecordsQuery by the global/dynamic property ledger.records.maxQueryableByAccount=180.
    • Avoids retrieving these records twice (once for fee calculation, once for query response) by using the Map<String, Object> queryCtx set by the StakedAnswerFlow.

tinker-michaelj added 3 commits January 23, 2022 15:08
…ords this is pointless)

Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
…unt=180

Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
@codecov
Copy link

codecov bot commented Jan 24, 2022

Codecov Report

Merging #2792 (33100d0) into master (da7cdb9) will increase coverage by 0.00%.
The diff coverage is 98.03%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #2792   +/-   ##
=========================================
  Coverage     95.76%   95.77%           
- Complexity     9656     9667   +11     
=========================================
  Files           699      699           
  Lines         26377    26395   +18     
  Branches       2710     2714    +4     
=========================================
+ Hits          25260    25279   +19     
+ Misses          581      580    -1     
  Partials        536      536           
Impacted Files Coverage Δ
...crypto/queries/GetAccountRecordsResourceUsage.java 92.85% <80.00%> (-7.15%) ⬇️
...rvices/context/properties/BootstrapProperties.java 98.42% <100.00%> (+<0.01%) ⬆️
...es/context/properties/GlobalDynamicProperties.java 100.00% <100.00%> (ø)
...tion/crypto/queries/GetTxnRecordResourceUsage.java 100.00% <100.00%> (ø)
...ra/services/queries/answering/AnswerFunctions.java 100.00% <100.00%> (ø)
...rvices/queries/crypto/GetAccountRecordsAnswer.java 100.00% <100.00%> (ø)
...dera/services/queries/meta/GetTxnRecordAnswer.java 100.00% <100.00%> (ø)
...om/hedera/services/state/merkle/MerkleAccount.java 94.26% <100.00%> (+0.04%) ⬆️
...ava/com/hedera/services/state/submerkle/TxnId.java 95.08% <0.00%> (-1.64%) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da7cdb9...33100d0. Read the comment docs.

Copy link
Member

@Neeharika-Sompalli Neeharika-Sompalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@rbair23
Copy link
Member

rbair23 commented Jan 24, 2022

I'm taking a deeper look at the PR. If we are going to add a mechanism to limit the number of returned records (and we really do need this), then we should also talk about pagination or watermarking. Since the number of transactions change all the time, I don't think we can do pagination (it would be basically worthless). Instead we should allow the user to say "get me the next N number of records newer than time T" or "newer than record R", or something along those lines. And we have a cap (as proposed in this PR) for the number of records returned. We can do that as a second PR, but we must recognize that this PR does introduce a breaking change in our API behavior and we need to make sure we document it as such as well.

@SimiHunjan SimiHunjan added this to the 0.22.3 milestone Jan 24, 2022
@tinker-michaelj
Copy link
Collaborator Author

I'm taking a deeper look at the PR. If we are going to add a mechanism to limit the number of returned records (and we really do need this), then we should also talk about pagination or watermarking. Since the number of transactions change all the time, I don't think we can do pagination (it would be basically worthless). Instead we should allow the user to say "get me the next N number of records newer than time T" or "newer than record R", or something along those lines. And we have a cap (as proposed in this PR) for the number of records returned. We can do that as a second PR, but we must recognize that this PR does introduce a breaking change in our API behavior and we need to make sure we document it as such as well.

Created a PR in hedera-protobufs to support this watermarking.

@@ -112,6 +153,6 @@ public HederaFunctionality canonicalFunction() {
@Override
public Optional<SignedTxnAccessor> extractPaymentFrom(final Query query) {
final var paymentTxn = query.getCryptoGetAccountRecords().getHeader().getPayment();
return Optional.ofNullable(SignedTxnAccessor.uncheckedFrom(paymentTxn));
return Optional.of(SignedTxnAccessor.uncheckedFrom(paymentTxn));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the significance of this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gRPC accessors never return null, so IntelliJ was marking this Optional.ofNullable() as a code smell.

@tinker-michaelj tinker-michaelj merged commit 9c66950 into master Jan 25, 2022
@tinker-michaelj tinker-michaelj deleted the cleanup-record-access branch January 25, 2022 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants