Request options to affect the order in which AZCopy processes files

[rozeboosje]

Which version of the AzCopy was used?

10.24.0

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What command did you run?

AZCopy sync

What problem was encountered?

No problem.... but have a suggestion to make it possible to process files in certain order

How can we reproduce the problem in the simplest way?

n/a

Have you found a mitigation/solution?

n/a

We use AZCopy as part of a process to migrate files (documents) from a server, through a blob container, to a much faster server and then upload to a new document storage.

It would be really useful if we could have a new flag that would allow us to tell AZCopy to process documents in a specific order. For example, for us it would be really useful to have AZCopy process the files in a folder by Date Modified Descending so that the most recently added or modified documents are brought across first. That way the most current migrated documents would appear in the target location first so that our clients can immediately continue working in the migrated environment.

Other people may have other use cases that might be helped with a flag that lets you instruct AZCopy to process files in a bespoke order.

[tanyasethi-msft]

Thanks for your request @rozeboosje, we're discussing your request and will get back on this thread asap.

[seanmcc-msft]

@rozeboosje unfortunately we are not going to be able to take this feature request. The underlying List Blob REST API does not allow applications to specify the order blobs should be listed, and it is not feasible to implement this functionality client side, as we would need to handle edge cases with potentially billions of blobs.

[rozeboosje]

@seanmcc-msft That makes a lot of sense, it's a pity but I understand. Thanks for looking into it all the same.