What exactly is needed for the S3 setup?

[Magicloud]

I have a general private S3 bucket, and I want to try out mounting it as a volume to Kubernetes.

I have this dataset:

apiVersion: com.ie.ibm.hpsys/v1alpha1
kind: Dataset
metadata:
  name: xxx
spec:
  local:
    type: "COS"
    secret-name: "xxx"
    secret-namespace: "audit"
    endpoint: "https://s3.eu-west-3.amazonaws.com/abcde"
    bucket: "abcde"
    readonly: "true"
    region: "eu-west-3"

But the PVC is pending due to failed to provision volume with StorageClass "csi-s3": rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8.

I wonder if I need something set for the bucket, or the IAM permission. The secret has been tested, I can upload and download files.

[srikumar003]

@Magicloud could you try changing your endpoint from https://s3.eu-west-3.amazonaws.com/abcde to https://s3.eu-west-3.amazonaws.com ? However, this may not be the root cause of the problem. It could be that your bucket name cannot be converted to UTF-8. Try validating encoding of the bucket name, if the first fix does not work

[Magicloud]

I tried. No luck. And all masked values are alphanum and hyphen. I wonder if GRPC did not get a valid GRPC data but some error webpage. I have met this kind of issue on other project. Just a hint.

[srikumar003]

Could you paste the relevant logs of the csi-s3 container here using the following command :
kubectl logs -n dlf <csi-s3-daemon-pod> -c csi-s3

There is a daemon per node, so you would have to look at the logs from all the nodes to spot the error messages

[Magicloud]

Sorry, did not get time on this. And I have lost the original IAM permission setup. Right now, with a much loose permission (any operations on the bucket), it works well. But the problem remains, a clear statement in the doc on the permissions needed would be quite useful.