User does not have permission to create Istio Config on namespace: istio-system

[genfemme]

What is the reason of this error - "User does not have permission to create Istio Config on namespace: istio-system" on the Kiali UI?

[jmazzitelli]

That error message means either the user doesn't have read-write permissions for that namespace (see Namespace Access Control) or Kiali is in view-only mode.

This involves the Kiali Wizards... @hhovsepy might be able to provide more details. But in short, the above is the short reason why.

[genfemme]

@jmazzitelli
Thanks. I have checked again and kiali is not in view-only mode as I have this in the CR
view_only_mode: false

Now may be a silly question but thought to ask that I see this in the kiali UI -

This means that the user accessing kiali is kiali-service-account in the istio-system namespace. Is this not correct?
Can the clusterrole and clusterrolebinding be added to this service account to provide the read-write permissions?

To provide full access, following could be the role reference to use -

apiGroups: ["networking.istio.io", "security.istio.io"] resources: ["virtualservices", "destinationrules", "gateways", "serviceentries", "authorizationpolicies", "peerauthentications"] verbs: ["get", "list", "watch", "create", "update", "delete"]

[jmazzitelli]

IMO, you should not be allowing your users to log in using Kiali's own SA. You should create users / service accounts for your Kiali users to use. You can provide your own RBAC permissions for those service accounts. The Kiali SA is not meant for users to use for logins using the token-auth. (That docs probably show that only as an example - the docs do explicitly say Create different service accounts with different permissions for your users to use. Each user should only have access to their own service accounts and tokens.).

[genfemme]

@jmazzitelli
I created a new service account in the istio-system namespace with extended permissions to manage Istio resources.
The permissions include access to resources like VirtualServices, DestinationRules, Gateways, etc., across the networking and security API groups of Istio.
I've been using port forwarding to access the Kiali UI. However, when attempting to log in using the token from the kiali-user service account, I encounter an error stating insufficient privileges when using the token of the new service account.

kubectl auth can-i --list --as=system:serviceaccount:istio-system:kiali-user is confirming that the service account does have the necessary permissions.

Should I update the Kiali deployment to use the kiali-user service account by modifying the serviceAccountName in the deployment spec? Or is this even recommended? I think that all the updates should be done via the CR. Could you suggest the way to add this to the CR?
Would this change allow the Kiali UI to utilize the permissions assigned to kiali-user?

[jmazzitelli]

Do not modify anything that the operator created - the deployment, SA, configmap, roles, etc. Those are all correct and should never be touched. You are correct - you should only have to touch the Kiali CR.

You are logging in as your own user - so that's your own SA that you create. That should be the only thing you have to touch. Make sure your SA gives the proper namespace permissions. See https://kiali.io/docs/configuration/rbac/

I don't know why the Kiali Wizard gives you the error it does. I don't know anything about the wizards. @hhovsepy knows much more and may be able to help.

[genfemme]

@jmazzitelli thanks.
Kiali-user is another Service Account that I created to use with my account. The Cluster role example in the official doc does not provide permissions to amend the configurations of istio-config in Kiali UI. The goal is to provide all the permissions to the user to be able to make any additions and changes to the available options in the UI. I am not sure still why the token is not working. I tried to delete and recreate it several times now with no success so far.

@hhovsepy would you be able to provide an insight on this as well ?

[hhovsepy]

Hi @genfemme, Kiali wizards are using Istio client to create/modify the Istio Config resources, and they use the SA of logged in user. In case of access error, there should be a log in Kiali pod. Can you provide this error log please?
Similar way is done in Istio Config details YAML editor page. Can you confirm if your user can modify Istio configs from 'istio-system' namespace?

[genfemme]

Hi @hhovsepy
the logs suggests that it is using the kiali-service-account. Below are the part of the logs -

`W0501 09:32:02.911669       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231: failed to list *v1beta1.Sidecar: sidecars.networking.istio.io is forbidden: User "system:serviceaccount:istio-system:kiali-service-account" cannot list resource "sidecars" in API group "networking.istio.io" in the namespace "project-clients-portal"
E0501 09:32:02.911697       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231: Failed to watch *v1beta1.Sidecar: failed to list *v1beta1.Sidecar: sidecars.networking.istio.io is forbidden: User "system:serviceaccount:istio-system:kiali-service-account" cannot list resource "sidecars" in API group "networking.istio.io" in the namespace "project-clients-portal"
W0501 09:32:04.216024       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231: failed to list *v1.Service: services is forbidden: User "system:serviceaccount:istio-system:kiali-service-account" cannot list resource "services" in API group "" in the namespace "project-clients-portal"
E0501 09:32:04.216059       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:serviceaccount:istio-system:kiali-service-account" cannot list resource "services" in API group "" in the namespace "project-clients-portal"
`

Yes I can edit the crds in the istio-system namespace. We do not have virtualservice and destinationrules set up in the cluster in this namespace.

[jmazzitelli]

the logs suggests that it is using the kiali-service-account.

Because you were using the Kiali SA's token for your login - see your screenshot from this post: #7320 (reply in thread)

[genfemme]

@jmazzitelli @hhovsepy
I started the whole thing again from the start and resolved the error now.

Thanks