Error: "Cannot insert legacy ACL for an object when uniform bucket-level access is enabled."

[jywarren]

We're using this via Paperclip and seeing this error on upload:

Google::Apis::ClientError

invalid: Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access

https://sentry.io/share/issue/393f9e2786b543be9b2061a933268129/

Our config is:

Paperclip::Attachment.default_options[:storage] = :fog
Paperclip::Attachment.default_options[:fog_directory] = ENV["GOOGLE_STORAGE_BUCKET_NAME"] || ''
Paperclip::Attachment.default_options[:path] = ":rails_root/public/system/system/:class/:attachment/:id_partition/:style/:filename"
Paperclip::Attachment.default_options[:fog_credentials] = {
    provider: ENV["FOG_PROVIDER"] || "Local",
    local_root: "#{Rails.root}/public",
    google_project: 'public-lab' ,
    google_json_key_location: ENV["GOOGLE_JSON_KEY_FILE"] || '' ,
    google_storage_access_key_id: ENV["GOOGLE_STORAGE_KEY"] || '' ,
    google_storage_secret_access_key: ENV["GOOGLE_STORAGE_SECRET"] || ''
}
Paperclip::Attachment.default_options[:fog_public] = true # we tried adding this line or removing it
Paperclip::Attachment.default_options[:fog_host] = ""

Has anyone seen this error? I can't find any mention of uniform bucket level access in this repository.

https://cloud.google.com/storage/docs/uniform-bucket-level-access

Thank you very much!! cc @icarito

[Temikus]

@jywarren so this looks like something on server-side, at least to me, from the docs:

Caution: If you enable uniform bucket-level access, you revoke access from users 
who gain their access solely through object ACLs and prevent users from administrating
permissions using bucket ACLs. Be sure that you read considerations when migrating
an existing bucket prior to enabling uniform bucket-level access.

Can you post the output of:

gsutil uniformbucketlevelaccess get gs://BUCKET_NAME

[icarito]

Hi!
Yes we've been using gcsfuse with this bucket and have found uniform bucket-level access desirable.
The output of the command is:

$ gsutil uniformbucketlevelaccess get gs://production-public-files
Uniform bucket-level access setting for gs://production-public-files:
  Enabled: True
  LockedTime: 2021-05-20 16:23:28.446000+00:00

We can't change the Uniform Bucket Level access unless we clone the bucket, but that's our plan B.
Thanks for your assistance.

[Temikus]

@icarito then in this case the problem is indeed because of the Uniform Bucket Level access. You should still be able to manage ACL's for files in the bucket, but paperclip needs to use the new ACL model, not a legacy one.

In theory the support for fog_public that paperclip uses is there: https://github.com/fog/fog-google/pull/318/files

Can you try deleting the XML credentials from the config (thus forcing the JSON driver) and let me know how it goes?

    google_storage_access_key_id: ENV["GOOGLE_STORAGE_KEY"] || '' ,
    google_storage_secret_access_key: ENV["GOOGLE_STORAGE_SECRET"] || ''

[jywarren]

We tried deleting those creds but still got this, unfortunately:

invalid: Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access

Is there any way to turn off fog's setting of ACL, and just push the file up respecting whatever the bucket setting is?

[jywarren]

i.e. predefined_acl could be unset or left as default?

[Temikus]

Sorry, busy week.

@icco can you take a look into it? IIRC you know the storage part of things a bit better than me.

[icco]

Hmm, we currently don't have that functionality... but lemme see how hard it would be.

[github-actions]

This issue has been marked inactive and will be closed if no further activity occurs.

[Temikus]

@icco any luck?

[guppy0356]

@Temikus Hello, I have same problems when using CarrierWave. Do you have some good ideas to resolve this.

[matsudai]

@guppy0356 I had the same issue.

• Rails (7.0.3)
• CarrierWave (2.2.2)
• fog-google (1.18.0)

My solution is ...

1. Change GCS bucket ACL from "Uniform" (bucket-level) to "Fine-grained".
2. If config.fog_public is unset or true in CarrierWave.configure do |config| ..., change public access to "REMOVE PUBLIC ACCESS PREVENTION".
3. If config.fog_public = false in CarrierWave.configure do |config| ..., change public access to "PREVENT PUBLIC ACCESS" (for your security).

Thanks.

[div-co]

@Temikus

In SAP we are using fog-google CF Cloud Controller Blobstore to access Google storage buckets.

The missing uniformBucketLevelAccess support in fog-google has been known for some time but never has been a blocker for us. With the Sovereign Cloud Restrictions the uniformBucketLevelAccess becomes hot topic for everyone who would like to use the offering.

Therefore, could you please update the status of uniformBucketLevelAccess support.

Appreciate your support.

[harinigunabalan]

@Temikus @icco

As @div-co has brought to our attention, with the Restrictions and limitations in EU Regions Sovereignty Controls, fog-google's missing uniformBucketLevelAccess support is a blocker for those who would like to use Sovereign cloud offering.

Hence really appreciate your support, if you can provide an update here. Thanks in advance.

[Temikus]

@div-co @harinigunabalan is there a chance you folks can provide a PR for this or collaborate? Happy to review and help.

I'm asking since both myself and @icco are pretty short on time right now and don't have an active CarrierWave/PaperClip app we maintain, which I assume you do.

[cwjenkins]

@Temikus also kreeti/kt-paperclip#121

[Temikus]

Thanks @cwjenkins - merged and released 🙌

Do we need to wait for paperclip fork to change as well or is this enough to work around things? 🤔

[div-co]

Thanks @cwjenkins, @Temikus!
We will be testing the release in CF too and provide feedback.

[cwjenkins]

@Temikus this is enough to work around things.

If one uses paperclip, setting fog_options: { uniform: true } will set the new var to NOT set predefined_acls allowing the request to succeed against a bucket with uniform access configured.

Currently one can set fog_public: ->(x) { nil } to achieve the same, however I don't find this solution obvious hence the issue being raised to them to discuss it.

[nikolaydrm]

Hi @cwjenkins , @Temikus -

@Temikus this is enough to work around things.

If one uses paperclip, setting fog_options: { uniform: true } will set the new var to NOT set predefined_acls allowing the request to succeed against a bucket with uniform access configured.

⚠️ Not setting predefined_acls (<your fog option>: { uniform: true } of #600 workaround) seems to have the effect of also allowing the request to succeed against ACL buckets -- kinda confusing for a uniform option:

• is it an intended feature, I don't find it specified in Add uniform attr to inserting file #600 and related issues?
• what security concerns, if any?

Example meta of an ACL bucket that works with { uniform: true } fog-google:

$ gsutil ubla get gs://<bucket>
Uniform bucket-level access setting for gs://<bucket>:
 Enabled: False

$ gcloud storage ls -L -b gs://<bucket>
 Storage Class:               STANDARD
 Location Type:               region
 Location Constraint:         EUROPE-WEST1
 Versioning Enabled:          None
 Logging Configuration:       Present
 Website Configuration:       None
 CORS Configuration:          []
 Lifecycle Configuration:     None
 Requester Pays Enabled:      None
 Labels:  { <redacted> }
 Default KMS Key:             None
 Time Created:                2023-11-26T23:38:19Z
 Time Updated:                2023-11-29T11:14:46Z
 Metageneration:              33
 Bucket Policy Only Enabled:  False
 Public Access Prevention:    inherited
 ACL: [ <redacted> ]
 Default ACL: [ <redacted> ]

Detection activity: migration of CloudFoundry blobstore from ACL to IAM (UBLA). I must note that support-both-accesses effect nicely serves the zero-downtime migration of CF blobstore ACL <-> UBLA, hence the concern on the stability(⚠️) of the feature.

[cwjenkins]

Hey @nikolaydrm, did you read kreeti/kt-paperclip#121 referenced in the above comment?

Not sure I'm following the 'detection activity' comment. Could you elaborate?

[nikolaydrm]

Hi @cwjenkins -

Hey @nikolaydrm, did you read kreeti/kt-paperclip#121 referenced in the above comment?

Yes, however I don't use kreeti/kt-paperclip (no experience configuring, deploying and using it, sorry!) but fog-google for https://github.tools.sap/cloudfoundry/cloud_controller_ng#blobstore. PR #600 nicely supports the case of CF consuming Uniform buckets when uniform support is enabled at fog-google:

fog-google/lib/fog/storage/google_json/models/file.rb

Line 120 in 2081621

options[:predefined_acl] ||= @predefined_acl unless @uniform

I got surprised that enabling the access to uniform buckets (uniform: true in fog-google config) would accept ACL buckets as well. Hence the questions: "is it the expected behavior?" and "are there security concerns?". Can't tell if these concern kreeti/kt-paperclip, my understanding is the PR was driven by the piperclip usecase ...

Not sure I'm following the 'detection activity' comment. Could you elaborate?

The motivation to ask for clarification is the usecase of migrating existing CF buckets from ACL to uniform access. By setting uniform: true at fog-google side a zero-downtime migration could be achieved. Hence the concerns:

• is the current behavior stable, i.e. all are happy with it? (I personally am)
• does it compromise the security of the setup?

[cwjenkins]

Thanks @nikolaydrm for the clarification.

When you say 'would accept ACL buckets as well', are you meaning you'd expect the code to look something like...

if @uniform
  options[:predefined_acl] = nil
else
  options[:predefined_acl] ||= @predefined_acl
end

If so, I think that better aligns with the server side given they state they disable ACL when uniform is enabled.
https://cloud.google.com/storage/docs/uniform-bucket-level-access

If you are concerned from a security standpoint that the client, fog-google, can pass ACLs to a GCS bucket with uniform access control and ignore the uniform control then that isn't the case.
This ticket was created due to the fact that some gems, (e.g. kt-paperclip), configured fog-google in a way that would always pass ACL causing the exception mentioned in the title.

[nikolaydrm]

@cwjenkins - thanks!

If there are no concerns that fog-google configured for uniform access plays fine with ACL buckets as well (seems to be the case, per your input) then it's "case closed" for me. Otherwise I'd expect the psudo flow:

if uniform option
  if bucket not uniform
    raise Exception
  end
else
  set ACL
end