[Storage] MD5 missing when uploading large blobs

[wahyuen]

• Package Name: @azure/storage-blob
• Package Version: 10.3.0
• Operating system:
• nodejs
  • version:
• browser
  • name/version: Chrome (Version 74.0.3729.131)
• typescript
  • version: 3.1.6
• Is the bug related to documentation in
  • README.md
  • source code documentation
  • SDK API docs on https://docs.microsoft.com

Describe the bug
When uploading blobs that are larger than 256Mb, it appears that the blob is missing the MD5 value.

To Reproduce
Steps to reproduce the behavior:

1. Upload file larger than 256Mb using the uploadBrowserDataToBlockBlob function call.

Expected behavior
Blob is uploaded and the ContentMD5 value populated for the blob

Screenshots

Additional context
Unsure if related to this recent issue, MD5 is missing from uploaded files

[kurtzeborn]

Thank you for opening this issue! We are routing it to the appropriate team for follow up.

CC: @jeremymeng, @XiaoningLiu, @vinjiang

[XiaoningLiu]

@wahyuen Please refer to this thread Azure/azure-storage-js#40

[wahyuen]

@XiaoningLiu we are not programatically setting the MD5 currently. We were expecting the value to be set for us once the blob has been uploaded.

This appears to work fine for blobs under 256Mb (as i believe at that point it goes into a block list implementation)

[XiaoningLiu]

It's an expected behavior. For blobs smaller than 256MB, Azure Storage server will calculate the MD5 in background. For larger blobs, you need calculate and provide MD5.

[wahyuen]

This seems to be relatively inconsistent when performing the same upload via azcopy. Does that then imply that azcopy is programatically calculating MD5 prior to be being sent to storage? Is this behaviour only a exhibited when performing block list transfer and might it go away if we were to increase the maxSingleShotSize?

[XiaoningLiu]

Yes, you are right, AzCopy is a tool designed for high performance data transfer, it will calculate MD5 no matter the blob size.

Azure Storage server will create MD5 for blobs uploaded via only one PUT BLOB request. In SDK, blob smaller than maxSingleShotSize will be uploaded via one PUT BLOB request.

Upper limit for maxSingleShotSize is 256MB, this upper limitation is defined by Azure Storage Server. See here https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob#remarks

[zhusijia26]

Close the item based on last comment and idle time.