Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: delete folder without iterator #10025

Open
Auxority opened this issue Apr 23, 2024 · 1 comment
Open

storage: delete folder without iterator #10025

Auxority opened this issue Apr 23, 2024 · 1 comment
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: question Request for information or clarification. Not an issue.

Comments

@Auxority
Copy link

Auxority commented Apr 23, 2024

Is your feature request related to a problem? Please describe.
Currently I am using a "transformations" bucket that contains modified versions of an original image stored in an "uploads" bucket. I store these transformed images in folders like so: /{originalSHA256Hash}/{nameOfTransformations}.{extension}. But if a user wants to delete their original upload, I now have to query the bucket with a prefix and delete each object individually.

Describe the solution you'd like
I'd like to be able to delete an entire folder with one request. A possible implementation could be something like this:

// bucket.go
func (b *Bucket) Folder(name string) *Folder {
    // add implementation here, no network requests yet.
}

// folder.go
func (f *Folder) Delete() error {
    // add implementation here, execute API request.
}

// usage:
client, err := storage.NewClient(ctx)
if err != nil {
	log.Fatalf("error creating cloud storage client: %w", err)
}

bucket := client.Bucket("NameOfBucket")

// new code:
folder := bucket.Folder("NameOfFolder")

err := folder.Delete()
if err != nil {
    log.Errorf("could not delete folder: %w", err)
}

Describe alternatives you've considered

  1. Delete all objects inside a folder using goroutines. This introduces a bunch of network traffic that could be prevented.

  2. Using lifecycle object management rules with a prefix to achieve this. Does the job, but your service needs to have permission to create these rules and might not delete the folder immediately.

  3. You can just call the existing endpoint of the Google API via any HTTP Client to achieve the desired end result.

Additional context
The Google API already supports this feature through the web interface.

Screenshot 2024-04-23 at 09 47 23
@Auxority Auxority added the triage me I really want to be triaged. label Apr 23, 2024
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Apr 23, 2024
@tritone tritone added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Apr 29, 2024
@tritone
Copy link
Contributor

tritone commented Apr 29, 2024

Hey @Auxority , thanks for the feature request.

Unfortunately Cloud Storage doesn't have a way to delete all objects under a prefix in a single API call. The web interface that you are looking at basically does your option (1) under the hood. If we offered this in the client we would have to do the same, which as you note might be slow and has the overhead of many network calls.

I would recommend using OLM for now if possible (option 2); you can also mention it to your GCP account manager to get a feature request for this filed with the Cloud Storage API team.

@tritone tritone added type: question Request for information or clarification. Not an issue. and removed type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

2 participants