No way to download everything including subfolders into workspace

[g-ericso]

I am having trouble to download everything under a bucket with subfolders.

gsutil cp -r does not work in this case.

[agoulti]

I'm sorry, I don't completely understand your use case.
Are you using the Jenkins Storage plugin? Classic or pipeline?
Where does "gsutil -r" come in?

In the docs for the Download Step, it points out that in Objects to Download:

Currently only a single asterisk at the lowest level of the object name is supported.

If you are using gsutil directly, isn't the "-r" option specifically to disable wildcard handling?
Do you want to use it without the "-r"?
See https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames#directory-by-directory-vs-recursive-wildcards

Does this help? I admit, I may be misunderstanding what you are trying to do.

[g-ericso]

I want to download everything in a subfolder under a bucket. I am using Jenkins Cloud Storage plugin.

If I do:

gs://.../*

which only downloads all files under the specific directory but not the files inside subdirectory.

I cannot do:

• gs://.../**/*
• gs://.../**

I don't have any solution except putting all subdirectories into a different sections.

[agoulti]

Yes, if you are using gsutil you would do gs://.../**
Unfortunately the plugin does not yet support this syntax.

This sounds like a feature request to implement support for the double asterisk.
It would be great to support it! Unfortunately, I don't have time to jump on it.

I will leave this open and will gladly review a PR if anyone wants to send one.

[daegalus]

Any progress on this? I just hit this same problem when trying to download a path and everything in it.