AccessDenied after upgrading from 1.3.0 to 1.4.0

[Lucas12138]

Hi, we are upgrading our ozone from 1.3.0 to 1.4.0 and we got
An error occurred (AccessDenied) when calling the ListBuckets operation: User doesn't have the right to access this resource.
error when we want to list the buckets.
We verified that the user and acl setup is correct and as before

[ {
 "metadata" : { },
 "name" : "s3v",
 "admin" : "om",
 "owner" : "om",
 "quotaInBytes" : -1,
 "quotaInNamespace" : -1,
 "usedNamespace" : 14,
 "creationTime" : "2023-06-08T06:06:15.324Z",
 "modificationTime" : "2024-03-25T17:00:20.727Z",
 "acls" : [ {
  "type" : "USER",
  "name" : "XXXXXXXXXXXXXXXXXXXX,
  "aclScope" : "ACCESS",
  "aclList" : [ "ALL" ]
 } ],
 "refCount" : 0
} ] 

Do you know what may be the cause? Thanks

[errose28]

Can you check the om audit log for operations that were done on your s3v volume shown here? I see a very recent modification time of 2024-03-25T17:00:20.727Z. This time is set when ACLs, owner, or quota for a volume are changed.

Can you confirm that you are using Native Ozone ACLs shown here in the JSON and not Ranger ACLS/Ranger integration. For Ranger integration the ACLs will be displayed in the Ranger UI.

Also can you confirm if you are using an S3 client to access the buckets? If so, does access also fail using ozone sh bucket list /s3v when you kinit as a user that should have permission?

[Lucas12138]

The setacl command output suggests it's successful:

2024-03-27 20:50:15 WARN  main NativeCodeLoader:60 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-03-27 20:50:16 INFO  main ClientTrustManager:148 - Loading certificates for client.
ACLs set successfully.

However there's no related audit logs and only:
2024-03-27 20:41:59 INFO Socket Reader #1 for port 9862 Server:2139 - Auth successful for xx/xx@XXX.COM (auth:KERBEROS) from 100.114.54.16:40260 / 100.114.54.16:40260

I got the acl json via the ozone getacl command and not via ranger.

I tired both s3 client and ozone sh bucket list command and they are showing the same error.

[Lucas12138]

BTW, we didn't enable ranger

[ChenSammi]

Hi Lucas,
One thing we have changed in 1.4.0 is that we use the KERBEROS short user name instead of whole principal during the ACL check. For example, user xx/xx@XXX.COM, "xx" will be the short user name to check against the ACLs.
So if "xx/xx@XXX.COM" is used as user in ACL rule, then it's maybe the reason that ACL check failure.
Can you make sure use "xx" in ACL as user name, and try again?

[Lucas12138]

It works. Thanks!
BTW, sorry for the inconvenience, would you mind editing the answer to hide some of the information that might be confidential?
For example, I updated it to xx/xx@CONT-DEV.XXX-DEV.SVC.KUBE.US-EAST-1E.K8S.CLOUD.XXX.COM
Thanks!

[ChenSammi]

Sure, it's updated.

[Lucas12138]

Sorry it looks like it still have some confidential info left. Would you mind updating to xx/xx@XXX.COM? Thanks!