New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
getSignedUrl for multiple files #969
Comments
@rakyesh22 sorry for the trouble, but would you be able to provide more context around what you're trying to do? |
there is folders in firestore lets say images, excel, json. so now i want to get some files ( a_1.png, a_14.png....a_55.png ) from images folder in single api call. .. |
Here's the understanding I'm working from, so let me know if I messed it up: you would like to download multiple files from Firestore into memory, using the
I don't believe this is an option that the upstream API offers. You can use const [files] = await bucket.getFiles({ prefix: 'a_' })
const fileContents = await Promise.all(files.map(file => file.download())) |
Thanks for answer... const [files] = await bucket.getFiles({ prefix: 'a_' }) above code will give me All file links I just want to get Specific Files link. |
I hope I'm understanding your requirements correctly, but it appears you'd like for the client library to generate pre-signed URLs for specific Cloud Storage files with known file names, not directly downloading the file. If that's the case, you would use const fileNames = ['a_5.png', 'a_11.png', ...];
const bucket = storage.bucket('bucket_name');
for (const fileName of fileNames) {
const [signedUrl] = await bucket.file(fileName).getSignedUrl({
version: 'v4',
expires: new Date('2020-12-31T11:59:59'),
action: 'read'
});
console.log(`The signed URL for ${fileName} is ${signedUrl}`);
} |
@jkwlui Thanks buddy . so i am looking for firebase APi/function, where i can pass whole array of file-names and which returns list of all signedUrl . |
Yes I have the same issue for getting multiple files url in one api hit. Please suggest better solution . Because by using foreach loop to get all files links. with signed url one by one slow downs the application, |
To summarise what people want here I think: Imagines having The actual process imply that you have to generate a People are requesting something like |
I managed to kind of solve my issue, so I'll explain what I did here maybe it will help peoples there. Since I use the My Google Cloud function looks like this:
By doing so, you're only generating SignedURL once, and it is fast. I know it's not the best approach since it's not completly resilient, but that's how I solved my problem now, waiting for a better approach. |
Hi folks, apologies for the delay. The Cloud Storage API doesn't support a prefix based signed URL to access a group of objects. At the moment the signing process is going to be slow because each signed URL needs to be signed individually either through IAM API or a local service account private key. The main solution that I'm aware of is what @jkwlui provided using a foreach. I also do like your solution @sboudouk, but definitely only see two options. I just noticed that there was a performance improvement released last week with the updated IAM signBlob endpoint https://github.com/googleapis/google-auth-library-nodejs/releases/tag/v6.0.0 in PR: googleapis/google-auth-library-nodejs#908. In https://github.com/googleapis/nodejs-common/releases/tag/v3.1.0. This is being updated when this PR is released: #1157. The performance issue is for the case that you're signing from a location far away from the West Cost data centers. The new IAM SignBlob API does not have this performance hit and may help improve performance when signing a group of blobs, but it will still take time. @jkwlui could you help cut the release? |
@frankyn we have released 5.0.0 👍 |
Thanks @stephenplusplus! For folks who were seeing a slow turn around time, this may help improve performance as it did for our Java library but don't expect it to be a solution for all. We may want to close out this issue. I'll forward this feedback to the Cloud Storage team for now, but don't have a turn around time at this time. Leaving it open in case folks have feedback. |
There should really be a way to create signed URLs in bulk or be able to create a single URLs that will work for multiple objects. I have a use case where a user needs to upload hundreds of files. With the current SDK I need to create hundreds of URLs, one by one, or create a signed URL for the whole bucket. Neither of these solutions is optimal. I could create a bucket just for this upload, and then move the files to the final bucket, but it's a lot of work that should be unnecessary. |
The
https://cloud.google.com/storage/docs/gsutil/commands/signurl |
Hi @PierBover , can you please open a new issue so we can track this request? |
So there's no official option to generate one presingedURL for multiple files? |
getSignedUrl works only for single file.
need to work with multiple files, so there is no option to pass and retrieve multiple files signed url.
please suggest the fastest option to create multiple file signedUrl.
The text was updated successfully, but these errors were encountered: