Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure CLI task fails with AADSTS700024 after 5 minutes #28737

Closed
jiasli opened this issue Apr 12, 2024 · 22 comments · Fixed by #28747
Closed

Azure CLI task fails with AADSTS700024 after 5 minutes #28737

jiasli opened this issue Apr 12, 2024 · 22 comments · Fixed by #28747
Assignees
Labels
AAD Auto-Assign Auto assign by bot bug This issue requires a change to an existing behavior in the product in order to be resolved. Service Attention This issue is responsible by Azure service team.

Comments

@jiasli
Copy link
Member

jiasli commented Apr 12, 2024

MSAL introduced a regression in 1.27.0 (AzureAD/microsoft-authentication-extensions-for-python#127, AzureAD/microsoft-authentication-library-for-python#644) which is adopted by Azure CLI 2.59.0 (#28556).

This regression makes MSAL's ConfidentialClientApplication bypass msal_extensions.token_cache.PersistedTokenCache, so access tokens are no longer retrieved from the token cache. Instead, every command now retrieves a new access token from the AAD Security Token Service (STS). Any commands fails with AADSTS700024 after the ID token expires (5 minutes on GitHub Actions, 10 minutes on Azure DevOps).

Originally posted by @jiasli in #28708 (comment)

@jiasli
Copy link
Member Author

jiasli commented Apr 12, 2024

Workarounds

Adopt either workarounds:

  1. Use service principal secret for authentication:

  2. [GitHub Actions only] In Azure CLI Action azure/cli@v2, specify azcliversion to use an older version of Azure CLI below 2.59.0, such as 2.58.0: https://github.com/marketplace/actions/azure-cli-action

@yonzhan yonzhan added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label Apr 12, 2024
@yonzhan yonzhan added this to the Backlog milestone Apr 12, 2024
@yonzhan yonzhan added the Service Attention This issue is responsible by Azure service team. label Apr 12, 2024
@jiasli jiasli pinned this issue Apr 12, 2024
@jiasli
Copy link
Member Author

jiasli commented Apr 16, 2024

We plan to fix this issue in the next Azure CLI release: https://github.com/Azure/azure-cli/milestone/141

Official Release: 04/30/2024
Azure CLI version: 2.60.0

For now, to get unblocked, please follow the instructions at #28737 (comment).

@andre-qumulo
Copy link

@jiasli Thank you for the update. Just for clarity, what, specifically, will get fixed? Will we still need the code to continue requesting the OID token in the background or will we just need to use Azure CLI 2.60.0?

@danwkennedy
Copy link

danwkennedy commented Apr 16, 2024

Is it possible to release this as a patch fix sooner? This potentially impacts every use of the azure/login GitHub Action.

Also, the workaround only works if you're using the azure/cli Action. If you use the az command directly in your scripts, it won't work. This use case is fairly common since hosted GitHub Actions runners come with the latest version of the az command preinstalled. Without this, users have to override the version and we have to hope they remember to undo their workarounds later.

@danwkennedy
Copy link

@jiasli would it be possible to release this in a patch fix?

@cloudcosmonaut
Copy link

@jiasli is there anyway to promote this as a hotfix so releases that rely on az cli can work again? The current time limit is braking a lot of builds :(

@yonzhan
Copy link
Collaborator

yonzhan commented Apr 24, 2024

@jiasli is working on the new build right now.

@cloudcosmonaut
Copy link

Awesome! Any ETA for this release?

@vaibbavisk20
Copy link

Is there any workaround for Azure/login@v2.1.0 while the hot fix makes it to production, since there doesn't seem to be a way to change the cli version this action uses?

@vaibbavisk20
Copy link

Service principal with a secret is not feasible for our case, due to issues transmitting and storing the value. Changing azure/cli version is not feasible due to using azure/login and azure/powershell only

@jiasli
Copy link
Member Author

jiasli commented Apr 25, 2024

Changing azure/cli version is not feasible due to using azure/login and azure/powershell only

This 5-minute-expiration issue only affects azure/cli action. azure/login and azure/powershell are not affected. If your task lasts longer than 60 minutes, this is currently a known limitation: #28708

@cloudcosmonaut
Copy link

@jiasli is working on the new build right now.

@yonzhan any update on this? When can we expect a release?

@yonzhan
Copy link
Collaborator

yonzhan commented Apr 26, 2024

Build to Cloud Shell: 04/25/2024
Official Release: 04/30/2024
Azure CLI version: 2.60.0

@nosalan
Copy link

nosalan commented Apr 26, 2024

This is quite a problematic issue for us and makes service connections based on Workload Identity federation unusable. Please add a test suite for regression so that it doesn't happen again.

@nosalan
Copy link

nosalan commented Apr 26, 2024

The issue also happens with Azure PowerShell on the latest MS Hosted Azure DevOps agents, example:

Set-AzFirewall -AzureFirewall $azfwAllocated | Select-Object Name,  …
     |    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | ClientAssertionCredential authentication failed: A configuration issue
     | is preventing authentication - check the error message from the server
     | for details. You can modify the configuration in the application
     | registration portal. See https://aka.ms/msal-net-invalid-client for
     | details.  Original exception: AADSTS700024: Client assertion is not
     | within its valid time range. Current time: 2024-04-26T07:47:14.2804194Z,
     | assertion valid from 2024-04-26T06:52:04.0000000Z, expiry time of
     | assertion 2024-04-26T07:02:04.0000000Z. Review the documentation at
     | https://docs.microsoft.com/azure/active-directory/develop/active-directory-certificate-credentials . Trace ID: <REDACTED> Correlation ID: <REDACTED>

Also AzureCli@2 task is affected.

@cloudcosmonaut
Copy link

Build to Cloud Shell: 04/25/2024 Official Release: 04/30/2024 Azure CLI version: 2.60.0

Can I fetch the Cloud build to ingest it in a pipeline decorator?

@jiasli
Copy link
Member Author

jiasli commented Apr 30, 2024

Azure CLI 2.60.0 has been released just now with this issue fixed:

https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/HISTORY.rst#2600

Fix #28737: Fix token cache for service principal authentication (#28747)

The rollout status for Azure CLI on GitHub Actions and Azure DevOps images can be found at https://github.com/actions/runner-images

@bcarthic
Copy link

bcarthic commented May 7, 2024

@jiasli, Even after upgrading to Azure CLI 2.60.0, I am facing the same issue. I am running the Azure CLI task from Azure devops and it expires after 10 mins and I get ERROR: AADSTS700024: Client assertion is not within its valid time range.

Since the images still uses 2.59.0, I do az upgrade --yes before running dotnet test.

@LiliDeng
Copy link

LiliDeng commented May 9, 2024

@jiasli, Even after upgrading to Azure CLI 2.60.0, I am facing the same issue. I am running the Azure CLI task from Azure devops and it expires after 10 mins and I get ERROR: AADSTS700024: Client assertion is not within its valid time range.

Since the images still uses 2.59.0, I do az upgrade --yes before running dotnet test.

+1

@jiasli
Copy link
Member Author

jiasli commented May 9, 2024

@bcarthic, are you requesting a data-plane access token? If so, please see #28708 (comment).

@m-soltani
Copy link

m-soltani commented May 9, 2024

@bcarthic, are you requesting a data-plane access token? If so, please see #28708 (comment).

@jiasli
What happens if I intend to kick off a long-running operation that starts by obtaining an access token and concludes five hours later? This scenario is typical for us, as we use Packer to create VM images for our build servers. Are there any plans to implement a process in Azure CLI that automatically exchanges a refresh token for an access token in the background? Packer is configured to use already logged in account via Azure CLi. With the current fix, the access token expires 1 Hour later and there is no chance to refreshing that :(

@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from Moazzem-Hossain-pixel May 10, 2024
@Azure Azure deleted a comment from MoazzemHossain-bot May 13, 2024
@jiasli
Copy link
Member Author

jiasli commented May 16, 2024

Azure CLI 2.60.0 has been deployed to GitHub Actions and Azure DevOps images: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md#cli-tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AAD Auto-Assign Auto assign by bot bug This issue requires a change to an existing behavior in the product in order to be resolved. Service Attention This issue is responsible by Azure service team.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants