Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checksum verification not working when uploading using stageBlock/commitBlockList or when using blobClient.getBlockBlobClient().getBlobOutputStream #40065

Open
ihudedi opened this issue May 7, 2024 · 8 comments
Assignees
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Storage Storage Service (Queues, Blobs, Files)

Comments

@ihudedi
Copy link

ihudedi commented May 7, 2024

Hi,
I would like to perform checksum verfication after I upload large file via stageBlock/commitBlockList or via blobClient.getBlockBlobClient().getBlobOutputStream(BlockBlobOutputStreamOptions)
I tried to set the checksum in commitBlock but the checksum verfication doesn't affect only set the md5 in the blob.
I tried to set the md5 in blobHttpHeaders of the BlockBlobOutputStreamOptions without success.
Is there any option do perfrom checksum verfication on server side when uploading via stageBlock/commitBlockList or via getBlobOutputStream
The checksum is saved in the blob but no verification is done.
Thanks,
Itay

@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Storage Storage Service (Queues, Blobs, Files) labels May 7, 2024
Copy link

github-actions bot commented May 7, 2024

Copy link

github-actions bot commented May 7, 2024

Thank you for your feedback. Tagging and routing to the team member best able to assist.

@ihudedi ihudedi changed the title Checksum verification not working when uploading using stageBlock/commitBlockList Checksum verification not working when uploading using stageBlock/commitBlockList or when using blobClient.getBlockBlobClient().getBlobOutputStream May 7, 2024
@ihudedi
Copy link
Author

ihudedi commented May 9, 2024

Hi,
Is there any solution to my issue?
Thanks,
Itay

@ihudedi
Copy link
Author

ihudedi commented May 15, 2024

Any updates?

1 similar comment
@ihudedi
Copy link
Author

ihudedi commented May 19, 2024

Any updates?

@ibrahimrabab
Copy link
Contributor

Hi @ihudedi
Thanks for reaching out! Apologies for not being able to respond to your question earlier. You can use checksum verification when uploading using stageBlock() or upload() while setting the contentMd5 parameter on either BlockBlobStageBlockOptions for stageBlock or BlockBlobSimpleUploadOptions for upload. How are you trying to trying to upload and verify the checksum? Can you provide a code snippet of how you are doing the checksum verification? Also, what SDK version are you using?

@ihudedi
Copy link
Author

ihudedi commented May 22, 2024

Hi @ibrahimrabab
When uploading using stageblock and commitblocklist the checksum is being verified per block uploading and not per all file.when I upload the file as one chunk and not in chunks ( bliboutputoptions or stage block ) I can't verify checksum for whole file.its working fine when I upload file via upload()
Thanks,
Itay

@ibrahimrabab
Copy link
Contributor

@ihudedi
By design, the way stageBlock and commitBlockList work is that it will verify per chunk uploaded. MD5 cannot be calculated for the entire file, it would be calculated for uploaded chunk. For your scenario, using upload() will be the best option so that you will be able to upload the entire file and calculate the checksum together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

2 participants