Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SAM Does not build the application, returns "PythonPipBuilder:ResolveDependencies failed #7003

Closed
Hanna-Hinn-imVitro opened this issue Apr 30, 2024 · 6 comments
Labels
blocked/more-info-needed More info is needed from the requester. If no response in 14 days, it will become stale.

Comments

@Hanna-Hinn-imVitro
Copy link

Hanna-Hinn-imVitro commented Apr 30, 2024

I have a project where I upload a domain using SAM Build.
I have packages that are inside helpers that uses setuptools.
I am using a virtual environment using pyenv , python version 3.8.17 // sam version : 1.82.0
Note:** If I run the requirements.txt theirs no issue, even the pip install that is generated by the sam build it works fine

For each lambda I have a requirements file for example:
-e ./helpers/adapters
boto3==1.14.47
dataclasses-json==0.5.2
requests==2.23.0
python_dynamodb_lock==0.9.1
pycryptodome==3.16.0
numpy==1.20.1

If I keep the adapters I get error :

  error: subprocess-exited-with-error                                                                                                                                                                
                                                                                                                                                                                                     
  × python setup.py egg_info did not run successfully.                                                                                                                                               
  │ exit code: 1                                                                                                                                                                                     
  ╰─> [1 lines of output]                                                                                                                                                                            
      ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.                                                                                                  
      [end of output]                                                                                                                                                                                
                                                                                                                                                                                                     
  note: This error originates from a subprocess, and is likely not a problem with pip.                                                                                                               
error: metadata-generation-failed   

If I delete the adapter library I get :

2024-04-30 17:43:19,683 | PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute
    package_builder.build_dependencies(
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 156, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 256, in build_site_packages
    wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 280, in _download_dependencies
    deps = self._download_all_dependencies(requirements_filename, directory)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 362, in _download_all_dependencies
    self._pip.download_all_dependencies(requirements_filename, directory)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 710, in download_all_dependencies
    raise NoSuchPackageError(str(package_name))
aws_lambda_builders.workflows.python_pip.packager.NoSuchPackageError: Could not satisfy the requirement: dataclasses-json==0.5.2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 369, in run
    action.execute()
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 57, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: Could not satisfy the requirement: dataclasses-json==0.5.2

Can anyone please help?

@Hanna-Hinn-imVitro Hanna-Hinn-imVitro added the stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. label Apr 30, 2024
@mildaniel
Copy link
Contributor

Hi @Hanna-Hinn-imVitro, can you provide a minimal sample application for us to reproduce the issue?

@mildaniel mildaniel added blocked/more-info-needed More info is needed from the requester. If no response in 14 days, it will become stale. and removed stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. labels Apr 30, 2024
@Hanna-Hinn
Copy link

Hanna-Hinn commented May 1, 2024

Hey, here's the minimal sample application :
Here's the project structure

|── helpers
|  ├── adapters
|  |  ├── adapters
|  |  ├── adapters.egg-info
|  |  ├── setup.py
|  |  |
|── domain
|  ├── bin
|  ├── tests
|  ├── src
|  |  ├── functions
|  |   ├── function1
|  |   | ├── __init__.py
|  |   | ├── handler.py
|  |   | ├── requirements.txt
|  |   ├── __init__.py
|  |  ├── __init__.py
|  |
|  ├── tests
|  ├── template.yaml
|  ├── ... other files

example on the requirements:

-e ../helpers/adapters
dataclasses-json==0.5.2
pycryptodome==3.16.0

Here's the overall requirements.txt for the whole project:

dataclasses-json==0.5.2
pytest==6.1.2
-e helpers/adapters
numpy==1.20.1
boto3==1.16.25
moto==1.3.16
dynamodb-encryption-sdk==1.2.0
pytest-mock==3.5.1
requests-mock==1.8.0
python_dynamodb_lock==0.9.1
pandas==1.3.4
opencv-python-headless==4.5.5.62
pillow==9.0.1
pycryptodome==3.16.0
aiohttp==3.8.1
aiohttp-retry==2.5.2
requests==2.25.1
aws-lambda-powertools[all]==2.8.0
pytest-cov==4.0.0
pydantic==1.10.13
asynctest==0.13.0
pytest-asyncio==0.20

The project is a bit old. I have a theory regarding some dependency version between 2 libraries but I can not figure it out :(. tbh I do not know how exactly can I help you reproduce the issue.

@mildaniel
Copy link
Contributor

Hi @Hanna-Hinn, I did some more digging and this is what I found.

  1. For the local dependencies, make sure that the library is being passed into the CodeURI. SAM CLI will move the CodeURI into a temporary scratch directory to build, and if the local dependency is outside of that, relative paths outside of the CodeURI won't be found. You can see an example of local dependencies being used in the Lambda Builders integration tests.
  2. For the rest of the dependencies, there was a bug fixed in version 1.95.0 of SAM CLI that should address the issue you're having, try updating to the latest version and seeing if that helps. However, there does seem to be an issue with one of the dependencies of dataclasses-json, depending on the version of pip you are using. stringcase==1.2.0 uses a deprecated pip feature which prevents Lambda Builders from building a wheel. There isn't much we can do for it since it doesn't contain any wheels we can use and we can't change the pip behaviour.

Let us know if this helps.

@Hanna-Hinn
Copy link

Okay thank you so much 🙏

@Hanna-Hinn
Copy link

Hey, So I migrated the project to python3.11 and upgraded some packages versions and the issue was gone, but I do not know how or why it was gone.

Copy link
Contributor

github-actions bot commented Jun 4, 2024

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked/more-info-needed More info is needed from the requester. If no response in 14 days, it will become stale.
Projects
None yet
Development

No branches or pull requests

3 participants