Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

az ml environment create is not working in github actions. #28911

Open
Ioannis-Mykelos opened this issue May 7, 2024 · 4 comments
Open

az ml environment create is not working in github actions. #28911

Ioannis-Mykelos opened this issue May 7, 2024 · 4 comments
Labels
Auto-Assign Auto assign by bot Auto-Resolve Auto resolve by bot bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning az ml Service Attention This issue is responsible by Azure service team.

Comments

@Ioannis-Mykelos
Copy link

Describe the bug

Trying create a step in my GitHub action to:

  1. login in Azure and
  2. create an Environment to be registered in my Azure ML Studio.

Related command

The yml is the following:

All steps are working except the last part, when running the step (name: Run Azure CLI script) of creating the Environment.

# This workflow will build and push a Docker container to the Environments (in Azure ML studio).
name: Build and register an environment in Azure ML Studio.

# Enable a workflow to be triggered manually using the GitHub API, GitHub CLI, or GitHub browser interface.
on: workflow_dispatch

env:
  CONTAINER_NAME: "${{ secrets.RESOURCE_GROUP_NAME }}-env"

permissions:
  id-token: write
  contents: read

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    environment: dev # This is the name of the Environment you will create in this Repository (see README_Environment.md).
    name: Build and register environment
    steps:
      - uses: actions/checkout@v3

      - name: Log in to Azure container registry
        uses: azure/docker-login@v1
        with:
          username: ${{ secrets.REGISTRY_USERNAME }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
          login-server: "${{ vars.AZURE_CONTAINER_REGISTRY_NAME }}" # ensure your login-server matches the fully qualified path to your image

      - name: Build and push container image to registry
        run: |
          DOCKER_BUILDKIT=1 docker build \
          -t "${{env.CONTAINER_NAME}}" \
          -f docker/Dockerfile.environment \
          --build-arg ssh_prv_key="${{ secrets.SSH_KEY }}" \
          --build-arg ssh_prv_key_sub="${{ secrets.SSH_KEY }}" \
          --build-arg ssh_ml_utils="${{ secrets.SSH_KEY }}" \
          --build-arg git_branch_name="${{ github.ref_name }}" \
          --build-arg git_repository_name="${{ github.event.repository.name }}" .
          docker tag "${{env.CONTAINER_NAME}}" "${{ vars.AZURE_CONTAINER_REGISTRY_NAME }}/environments/${{env.CONTAINER_NAME}}:latest"
          docker push "${{ vars.AZURE_CONTAINER_REGISTRY_NAME }}/environments/${{env.CONTAINER_NAME}}:latest"
      - name: Log in with Azure
        uses: azure/login@v1
        with:
          tenant-id: ${{ secrets.TENANT_ID}}
          client-id: ${{ secrets.SERVICE_PRINCIPAL}}
          subscription-id: ${{ secrets.SUBSCRIPTION_ID}}
      - name: Run Azure CLI script
        uses: azure/CLI@v2
        with:
          azcliversion: 2.38.0
          inlineScript: |
            az extension add -n azure-cli-ml --yes
            if az ml environment list -w "${{ secrets.WORKSPACE_NAME }}" -g "${{ secrets.RESOURCE_GROUP_NAME }}" | grep -q "${{env.CONTAINER_NAME}}"; then echo "Environment already registered!!!"; else az ml environment create --name "${{env.CONTAINER_NAME}}" -i "${{ env.CONTAINER_NAME }}/environments/${{env.CONTAINER_NAME}}:latest" -w "${{ secrets.WORKSPACE_NAME }}" -g "${{ secrets.RESOURCE_GROUP_NAME }}"; fi

Errors

Run azure/CLI@v2
  with:
    azcliversion: [2](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:2).59.0
    inlineScript: az extension add -n ml --debug
  
  env:
    CONTAINER_NAME: ***-env
    DOCKER_CONFIG: /home/runner/work/_temp/docker_login_171[3](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:3)369238754
Starting script execution via docker image mcr.microsoft.com/azure-cli:2.59.0
DEBUG: cli.knack.cli: Command arguments: ['extension', 'add', '-n', 'ml', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f7e3269c0e0>, <function OutputProducer.on_global_arguments at 0x7f7e325a6[4](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:4)80>, <function CLIQuery.on_global_arguments at 0x7f7e32[5](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:5)dff[6](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:6)0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: extension                 0.002         1         [7](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:7)
DEBUG: cli.azure.cli.core: Total (1)                 0.002         1         7
DEBUG: cli.azure.cli.core: Loaded 1 groups, 7 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : extension add
DEBUG: cli.azure.cli.core: Command table: extension add
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f7e30964fe0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-04-17.15-56-13.extension_add.7.log'.
INFO: az_command_data_logger: command args: extension add -n {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f7e3099a[8](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:8)e0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f7e30[9](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:10)9d4e0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f7e3099d620>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f7e325a6520>, <function CLIQuery.handle_query_parameter at 0x7f7e32340040>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f7e3099d580>]
WARNING: cli.azure.cli.core.extension.operations: Default enabled including preview versions for extension installation now. Disabled in May 2024. Use '--allow-preview true' to enable it specifically if needed. Use '--allow-preview false' to install stable version only. 
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
DEBUG: urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 3932233
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.[10](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:11).0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl', 'ml-2.24.0-py3-none-any.whl', 'ml-2.25.0-py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.[11](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:12).0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.[12](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:13).1-py3-none-any.whl', 'ml-2.[13](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:14).0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.[14](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:15).1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl', 'ml-2.24.0-py3-none-any.whl', 'ml-2.25.0-py3-none-any.whl']

.
.
.
**************************************************************************************************************
* WARNING:                                                                                                   *
* Extension "azure-cli-ml" cannot be used along with extension "ml". This may result in unexpected behaviour.*
* Please remove azure-cli-ml extension by running  "az extension remove -n azure-cli-ml                      *
**************************************************************************************************************          
(ResourceGroupNotFound) Resource group 'RESOURCEGROUPNAME' could not be found.
Code: ResourceGroupNotFound
Message: Resource group 'RESOURCEGROUPNAME' could not be found.
.
.
.
.


Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux1_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/b2/bc/dd247bf682c5e463b4f263f36ec6f1de61c937138bb73716f4b42e2ef0b2/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux1_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
  Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux2010_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/4f/4d/7f21edf7e3c2bd1ed76e6fd8a99b272e304e12cf98e6b1e94b8c51d9d4d6/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux2010_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
  Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux2014_x86_64, pp39-pypy39_pp73-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/8a/83/178088e3b236b9b73a08057c49e9513fdd3185044754c0320bd73f58a127/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
Skipping link: not a file: https://pypi.org/simple/azureml-dataprep-rslex/
Given no hashes to check 0 links for project 'azureml-dataprep-rslex': discarding no candidates
INFO: pip is looking at multiple versions of ml to determine which version is compatible with other requirements. This could take a while.
Will try a different candidate, due to conflict:
    ml 2.25.0 depends on azureml-dataprep-rslex>=2.22.0
ERROR: Could not find a version that satisfies the requirement azureml-dataprep-rslex>=2.22.0 (from ml) (from versions: none)
ERROR: No matching distribution found for azureml-dataprep-rslex>=2.22.0
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 439, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('azureml-dataprep-rslex>=2.22.0'), parent=LinkCandidate('file:///tmp/tmpuquhizl5/ml-2.25.0-py3-none-any.whl'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: No matching distribution found for azureml-dataprep-rslex>=2.22.0
Removed build tracker: '/tmp/pip-build-tracker-v3m07nji'

DEBUG: cli.azure.cli.core.extension.operations: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/opt/az/azcliextensions/ml', '/tmp/tmpuquhizl5/ml-2.25.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
DEBUG: cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /opt/az/azcliextensions/ml
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 701, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 334, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 349, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

ERROR: cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
ERROR: az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f7e30965260>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.__main__: Command ran in 4.944 seconds (init: 0.299, invoke: 4.645)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3803 in cache
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/usr/local/bin/python /usr/local/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /root/.azure"
INFO: telemetry.process: Return from creating process
INFO: telemetry.main: Finish creating telemetry upload process.
Error: Error: az cli script failed.
cleaning up container...
MICROSOFT_AZURE_CLI_1713369353286_CONTAINER

Error: az cli script failed.

Issue script & Debug output

The results are :

Run azure/CLI@v2
  with:
    azcliversion: [2](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:2).59.0
    inlineScript: az extension add -n ml --debug
  
  env:
    CONTAINER_NAME: ***-env
    DOCKER_CONFIG: /home/runner/work/_temp/docker_login_171[3](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:3)369238754
Starting script execution via docker image mcr.microsoft.com/azure-cli:2.59.0
DEBUG: cli.knack.cli: Command arguments: ['extension', 'add', '-n', 'ml', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f7e3269c0e0>, <function OutputProducer.on_global_arguments at 0x7f7e325a6[4](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:4)80>, <function CLIQuery.on_global_arguments at 0x7f7e32[5](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:5)dff[6](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:6)0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: extension                 0.002         1         [7](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:7)
DEBUG: cli.azure.cli.core: Total (1)                 0.002         1         7
DEBUG: cli.azure.cli.core: Loaded 1 groups, 7 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : extension add
DEBUG: cli.azure.cli.core: Command table: extension add
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f7e30964fe0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-04-17.15-56-13.extension_add.7.log'.
INFO: az_command_data_logger: command args: extension add -n {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f7e3099a[8](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:8)e0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f7e30[9](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:10)9d4e0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f7e3099d620>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f7e325a6520>, <function CLIQuery.handle_query_parameter at 0x7f7e32340040>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f7e3099d580>]
WARNING: cli.azure.cli.core.extension.operations: Default enabled including preview versions for extension installation now. Disabled in May 2024. Use '--allow-preview true' to enable it specifically if needed. Use '--allow-preview false' to install stable version only. 
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
DEBUG: urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 3932233
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.[10](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:11).0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl', 'ml-2.24.0-py3-none-any.whl', 'ml-2.25.0-py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.[11](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:12).0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.[12](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:13).1-py3-none-any.whl', 'ml-2.[13](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:14).0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.[14](https://github.com/nuuday/ml-testenv/actions/runs/8738202540/job/23976757603#step:7:15).1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl', 'ml-2.24.0-py3-none-any.whl', 'ml-2.25.0-py3-none-any.whl']

.
.
.
.
.
.
.


Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux1_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/b2/bc/dd247bf682c5e463b4f263f36ec6f1de61c937138bb73716f4b42e2ef0b2/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux1_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
  Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux2010_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/4f/4d/7f21edf7e3c2bd1ed76e6fd8a99b272e304e12cf98e6b1e94b8c51d9d4d6/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux2010_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
  Skipping link: none of the wheel's tags (pp39-pypy39_pp73-manylinux2014_x86_64, pp39-pypy39_pp73-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/8a/83/178088e3b236b9b73a08057c49e9513fdd3185044754c0320bd73f58a127/azureml_dataprep_rslex-2.22.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/azureml-dataprep-rslex/)
Skipping link: not a file: https://pypi.org/simple/azureml-dataprep-rslex/
Given no hashes to check 0 links for project 'azureml-dataprep-rslex': discarding no candidates
INFO: pip is looking at multiple versions of ml to determine which version is compatible with other requirements. This could take a while.
Will try a different candidate, due to conflict:
    ml 2.25.0 depends on azureml-dataprep-rslex>=2.22.0
ERROR: Could not find a version that satisfies the requirement azureml-dataprep-rslex>=2.22.0 (from ml) (from versions: none)
ERROR: No matching distribution found for azureml-dataprep-rslex>=2.22.0
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 439, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('azureml-dataprep-rslex>=2.22.0'), parent=LinkCandidate('file:///tmp/tmpuquhizl5/ml-2.25.0-py3-none-any.whl'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: No matching distribution found for azureml-dataprep-rslex>=2.22.0
Removed build tracker: '/tmp/pip-build-tracker-v3m07nji'

DEBUG: cli.azure.cli.core.extension.operations: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/opt/az/azcliextensions/ml', '/tmp/tmpuquhizl5/ml-2.25.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
DEBUG: cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /opt/az/azcliextensions/ml
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 701, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 334, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 349, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

ERROR: cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
ERROR: az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f7e30965260>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.__main__: Command ran in 4.944 seconds (init: 0.299, invoke: 4.645)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3803 in cache
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/usr/local/bin/python /usr/local/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /root/.azure"
INFO: telemetry.process: Return from creating process
INFO: telemetry.main: Finish creating telemetry upload process.
Error: Error: az cli script failed.
cleaning up container...
MICROSOFT_AZURE_CLI_1713369353286_CONTAINER

Error: az cli script failed.

Expected behavior

I am expecting the GitHub action to check if there is a registered environment in the Azure ML studio for the Workspace.
If there is then echo a message, if not then create one (poetry environment registration), according to our poetry file.

Environment Summary

azure-cli:2.59.0
core : 2.59.0

Additional context

No response
@Ioannis-Mykelos Ioannis-Mykelos added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label May 7, 2024
@azure-client-tools-bot-prd azure-client-tools-bot-prd
Copy link

Hi @Ioannis-Mykelos,

2.59.0 is not the latest Azure CLI(2.60.0).

If you haven't already attempted to do so, please upgrade to the latest Azure CLI version by following https://learn.microsoft.com/en-us/cli/azure/update-azure-cli.

@azure-client-tools-bot-prd azure-client-tools-bot-prd bot added the Auto-Resolve Auto resolve by bot label May 7, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. Auto-Assign Auto assign by bot Service Attention This issue is responsible by Azure service team. Machine Learning az ml labels May 7, 2024
@yonzhan
Copy link
Collaborator

yonzhan commented May 7, 2024

Thank you for opening this issue, we will look into it.

@Ioannis-Mykelos
Copy link
Author

Ioannis-Mykelos commented May 22, 2024 •
edited

Any updates on our issue?

@subesokun
Copy link

I encountered a similar issue where the azureml-dataprep-rslex package was also missing after running the command az extension add -n ml.

azure-cli                         2.61.0

core                              2.61.0
telemetry                          1.1.0

Extensions:
ml                                2.26.0

+ az ml data mount --mount-point ./data --mode ro_mount --path azureml:data:1
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The command failed with an unexpected error. Here is the traceback:
Cannot mount since `azureml-dataprep-rslex` is not installed. It is required for `az ml data mount` and `az ml data mount` to work.To install, run: `$ pip install --target $(az extension show -n ml --query path) azureml-dataprep-rslex`
Traceback (most recent call last):
  File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 701, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 334, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/root/.azure/cliextensions/ml/azext_mlv2/manual/custom/data.py", line 273, in ml_data_mount
    modify_sys_path_for_rslex_mount(persistent)
  File "/root/.azure/cliextensions/ml/azext_mlv2/manual/custom/utils.py", line 463, in modify_sys_path_for_rslex_mount
    raise Exception(
Exception: Cannot mount since `azureml-dataprep-rslex` is not installed. It is required for `az ml data mount` and `az ml data mount` to work.To install, run: `$ pip install --target $(az extension show -n ml --query path) azureml-dataprep-rslex`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Auto-Assign Auto assign by bot Auto-Resolve Auto resolve by bot bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning az ml Service Attention This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@subesokun @yonzhan @Ioannis-Mykelos