Use AzureIdentity SDK or MSAL instead of ADAL

[adam-orosz]

Which version of the AzCopy was used?

V10.16.0

Which platform are you using? (ex: Windows, Mac, Linux)

Linux

What command did you run?

I'm trying to use azcopy with Azure AD Workload Identity without using Workload Identity's proxy sidecar. The proxy sidecar is used to intercept token requests to IMDS and acquire an AAD token on behalf of the user with federated identity credential. Unfortunately, our use case requires running azcopy in a k8s init-container therefore we can't rely on the proxy sidecar.

Based on this comment if a project uses the AzureIdentity SDK or MSAL then there is no need for the proxy sidecar. However, it seems azcopy is using ADAL, which is both deprecated and prevents azcopy to be used in an init-container.

The command below only works in conjunction with a proxy sidecar:
azcopy login --identity --identity-client-id < client ID of a managed identity >

What problem was encountered?

Not using Azure AD Workload Identity's proxy sidecar will make azcopy fail to acquire a token.

How can we reproduce the problem in the simplest way?

• follow the installation guide for Workload Identity
• create a pod without the proxy sidecar (this is the default behaviour)
• attempt to run azcopy login --identity --identity-client-id < client ID of a managed identity >

Have you found a mitigation/solution?

No

[jbpaux]

With ADAL being deprecated on June 2023, @zezha-msft how do you plan to migrate to MSAL or AzureIdentity library ?

[gapra-msft]

Hi @jbpaux

The plan is to migrate azcopy to the new Azure Identity library https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/azidentity This is work we have scheduled and will get to it soon.

[jbpaux]

Perfect @gapra-msft please target 1.3.0+ as it support Workload Identity 😉

[watahani]

Hi @gapra-msft Do you have any updates on this issue? It would be helpful if you could comment on the ETA if you have one.

[gapra-msft]

Hi, we are currently blocked on the MSAL and Azure Identity SDKs until they add support for token cache persistence. Azure/azure-sdk-for-go#16643

We have almost completed the bulk of the work on our end to upgrade, we are just waiting on this support.

[maikschulze]

@gapra-msft , will it be possible after migrating to MSAL and Azure Identity SDk to share a token cache with other applications? In my use case, a long-running service application already uses MSAL to obtain a valid token. Occasionally, azcopy is called to perform actions. At the moment, this requires re-authenticating in the azcopy child process due to the opaque token cache persistance, or abusing $env:AZCOPY_OAUTH_TOKEN_INFO.

[gapra-msft]

Hi @maikschulze I would recommend you create a new Github issue describing the details of the scenario you are interested in. Then, the team will internally discuss this and get back to you on our thoughts.

[randywallace]

I got tired of waiting on this, as its a massive blocker for our usecases, and only gets worse as time marches forward. So, for those of you in the same position we're in, I patched in azidentity 1.3.0 to replace the MSI implementation for ADAL, which works in our initial testing. https://github.com/Azure/azure-storage-azcopy/compare/main...randywallace:azure-storage-azcopy:add-default-azure-credential-msal?expand=1 . I'm not opening a PR, b/c I'm all but certain the implementation is not sustainably mergeable, but for those that need this, here it is, until this issue is resolved.

[andrewluotechnologies]

I am running into this issue also so hope it gets fixed soon. In the meantime, for us, we used the PowerShell module as a workaround. It can't replace all the use cases/functionality of azcopy, but for basic blob upload/download use cases it works perfectly with Workload Identity.

[jbpaux]

It seems to be available in next version 10.21.0 in preview

[gapra-msft]

Hi, yes we have upgraded most of the OAuth logic to the new azure identity SDK, with the exception of device code credential, as of 10.21.0

Closing this issue as this has been addressed in a recent release.

[LukaszRacon]

Hi @gapra-msft,
Original task was about using "azcopy with Azure AD Workload Identity", not about SDK upgrade.
When I tried 10.21.0 a couple weeks ago it wasn't working as expected.
Is there a config, environmental variable setup that I am missing?

Related issue: #2112

[junchen6072]

Hi @gapra-msft
Would also like to have some instructions on how to use workload identity/federated credentials, if it's already supported by azcopy. Thanks!

[gapra-msft]

To use workload identity, you can login via AzCLI login and then set azcopy's auto login environment variable to AZCLI

[gapra-msft]

Here are the official public docs https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-authorize-azure-active-directory#authorize-with-azure-cli

Closing as AzCLI and Powershell login is the mitigation for this item.

[rpahli]

so for docker images the workaround is to install a second cli which doubles the image size to solve this issue?

[gapra-msft]

@rpahli we are tracking adding native support to AzCopy for workload identity here. Please subscribe to that issue for updates.