Failed to verify operator role for addons requiring additional credential requests for sts clusters

[KevFan]

When installing the RHOAM addon, it takes several runs of the install addon command to correctly apply the addon onto a ROSA sts cluster.

Below is a sample output (aws account id has been redacted):

➜  ~ rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true -
W: Addon 'managed-api-service' needs access to resources in account '0124567910'
I: Created role 'chfan-sts-y8x4-redhat-rhoam-cloud-resources-operator-sts-credent' with ARN 'arn:aws:iam::0124567910:role/chfan-sts-y8x4-redhat-rhoam-cloud-resources-operator-sts-credent'
E: Failed to add operator role to cluster 'chfan-sts': Failed to verify operator role for cluster '20jnic2pu9seuitl5ta0frmc8evn1tc9'
➜  ~ rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true --s3-access-key-id s3-key --s3-secret-access-key s3-secret
W: Addon 'managed-api-service' needs access to resources in account '0124567910'
E: Failed to add operator role to cluster 'chfan-sts': Failed to verify operator role for cluster '20jnic2pu9seuitl5ta0frmc8evn1tc9'
➜  ~ rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true 
W: Addon 'managed-api-service' needs access to resources in account '0124567910'
I: Created role 'chfan-sts-y8x4-redhat-rhoam-3scale-sts-s3-credentials' with ARN 'arn:aws:iam::0124567910:role/chfan-sts-y8x4-redhat-rhoam-3scale-sts-s3-credentials'
E: Failed to add operator role to cluster 'chfan-sts': Failed to verify operator role for cluster '20jnic2pu9seuitl5ta0frmc8evn1tc9'
➜  ~ rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true 
W: Addon 'managed-api-service' needs access to resources in account '0124567910'
E: Failed to add operator role to cluster 'chfan-sts': Failed to verify operator role for cluster '20jnic2pu9seuitl5ta0frmc8evn1tc9'
➜  ~ rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true -
W: Addon 'managed-api-service' needs access to resources in account '0124567910'
I: Enabling interactive mode
? Billing Model: standard
I: Add-on 'managed-api-service' is now installing. To check the status run 'rosa list addons -c chfan-sts'
I: To install this addOn again in the future, you can run:
   rosa install addon --cluster chfan-sts managed-api-service -y --rosa-cli-required true --cidr-range 10.1.0.0/26 --notification-email chfan@redhat.com --addon-managed-api-service 1 --addon-resource-required true --billing-model standard

It seems there is a delay between adding the required addon credenital requests and being available on the cluster api which fails some sort of validation.

This occurs on even the latest rosa version - 1.2.10 (at the time of writing) and is not the best user experience since the error looks to be transient

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[laurafitzgerald]

/remove-lifecycle stale

[laurafitzgerald]

wanting to raise this one again as we've faced it again.

[oharan2]

I've also faced this issue more then once - any updates? (version 1.2.22)

[KevFan]

I've also created this JIRA that tracks this - https://issues.redhat.com/browse/SDA-7568 and it looks like it has marked for an upcoming sprint

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[KevFan]

/reopen
/remove-lifecycle rotten
/lifecycle frozen

[openshift-ci]

@KevFan: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten
/lifecycle frozen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.