Attach file directly to "store" storage?

[Tao-Galasse]

Hello !

First of all, here is my Shrine config :

Shrine.storages = {
  cache: Shrine::Storage::GoogleCloudStorage.new(bucket: ENV['GCS_BUCKET'], prefix: 'uploads/cache'),
  store: Shrine::Storage::GoogleCloudStorage.new(bucket: ENV['GCS_BUCKET'], prefix: 'uploads')
}

For most of my records with attachments on it, I'm using the backgrounding plugin to upload the files to a google cloud bucket.

However, there is one specific case where an "Attachment" record is created by the front-end, and my API should return a permanent URL of the file, but currently, it returns a URL to the cache storage which expire after the promotion is executed in the background.

I cannot use direct upload in this case, because I need to create a db record, not only have a URL to the file on the bucket.

Is it possible to do something like :

• specify when i want to use the store storage directly instead of the cache storage?
• get the "final" (and definitive) URL the file even if the promotion is not over yet? (for now, if I do something like attachment.create(file: params[:file]), then attachment.file.url returns the "/uploads/cache" URL, not the "/uploads" URL).
• wait for the promotion to be completed before sending a response to the front-end? (I really would like to avoid that, to keep my API as fast as possible)

I hope I've been clear enough :)

Thanks in advance for you help! 🙏

[davidwessman]

You could create a special uploader and add:
plugin(:model, cache: false) and then any assignment to the attachment is directly uploaded to the :store

[Tao-Galasse]

Thanks for your suggestion @davidwessman :)

With my current implementation, I use a specific uploader for each specific model (related to a db table / record). But my current issue consist of a model (let's say Attachment) which need to be uploaded with backgrounding most of the time, but in a specific moment & a specific controller, should not be cached.

Is it a way to perform this on the controller level and not only on the uploader config ?

Thanks in advance :)

[davidwessman]

Then I do not know how to solve it :/

[Tao-Galasse]

Your suggestion is still helpful and could be used by anyone going through this discussion, thanks for that 😊

[janko]

You can bypass cache by using the attacher directly – https://shrinerb.com/docs/attacher#attaching-stored.

photo.image_attacher.attach(file) # goes to store
photo.image_attacher.assign(file) # goes to cache (called by `photo.image = ...`)

[Tao-Galasse]

Indeed, it works like a charm :)

Thanks a lot @janko 🎉

just a question about this: to avoid the file to be uploaded in the cache, I have to run photo.image_attacher.attach(file) BEFORE the record "photo" is saved, right?

So il my controller, it could look like this :

photo = Photo.new(params.except(:file))
photo.image_attacher.attach(file) # goes to store
photo.save!

but what if "photo" is not saved? (for example, it didn't pass rails validations) Will the file be deleted on the bucket, or will it juste live in here forever, but being "inaccessible" from the app, because not linked to any record?

Thanks a lot! 🙏

[janko]

Yes, you should only attach the file if the record is valid, because it will not be deleted automatically. This doesn't cover file validations, though, because those are executed only after uploading to the storage. You can ensure the invalid file is removed in this case by enabling the remove_invalid plugin.

[jrochkind]

I have some very complicated custom code that I wrote, as a shrine plugin, to let me on-the-fly switch the promotion process from background to foreground -- on a model-by-model basis, at ingest time.

So I had this use case too... but I really am not fond of the code i wrote to make this happen, it is very convoluted and I don't even want to share it because I don't really recommend it. On the other hand... it works. I tried to do in retrospect too-complicated things with allowing the directive of whether promotion should be bg or fg to be set in various ways, including thread variables and global variables, etc. Perhaps there is a simplification that wouldn't be so ugly....