Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I specify differing sets of dependency extras for each of my project's extras? #834

Open
2 tasks done
kurtiss opened this issue Jan 25, 2019 · 11 comments · May be fixed by python-poetry/poetry-core#613
Open
2 tasks done
Labels
kind/feature Feature requests/implementations

Comments

@kurtiss
Copy link

kurtiss commented Jan 25, 2019

  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Question

How can I specify differing sets of dependency extras for each of my project's extras? This is something that is relatively trivial to do with setup.py. To clarify, I would like to be able to specify the following dependency graph:

myproject[feature-a] depends on [dependency-one, dependency-two[feature-a]]
myproject[feature-b] depends on [dependency-one, dependency-two[feature-b]]

I'll propose a couple ways that pyproject.toml might model these graphs, but the core value of these examples is to highlight an apparent deficiency with the current model.

  1. By allowing dependency aliases that specify distinct sets of extras. For example:
[tool.poetry.dependencies]
…
dependency-two-a = { git = "git+ssh://git@company.githost.io/projects/dependency-two.git",optional = true,extras = ["feature-a"], provides="dependency-two" }
dependency-two-b = { git = "git+ssh://git@company.githost.io/projects/dependency-two.git",optional = true,extras = ["feature-b"], provides="dependency-two" }
…

[tool.poetry.extras]
feature-a = ["dependency-one", "dependency-two-a"]
feature-b = ["dependency-one", "dependency-two-b"]
  1. By treating the set of extras in the dependencies section as a superset that can be further filtered down when the dependency is referenced in the extras section. For example:
[tool.poetry.dependencies]
…
dependency-two = { git = "git+ssh://git@company.githost.io/projects/dependency-two.git",optional = true,extras = ["feature-a", "feature-b"]}
…

[tool.poetry.extras]
feature-a = ["dependency-one", "dependency-two[feature-a]"]
feature-b = ["dependency-one", "dependency-two[feature-b]"]

Thanks for your time!

@kurtiss
Copy link
Author

kurtiss commented Oct 7, 2019

@sdispater Gently bumping this – is this use case compelling enough to warrant some attention?

@stale
Copy link

stale bot commented Dec 6, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Dec 6, 2019
@kurtiss
Copy link
Author

kurtiss commented Dec 6, 2019

Bumping again, because the issue is outstanding and marks a flaw in the way dependencies are modeled, preventing reasonable use of this application. Hope somebody can take a look at this.

@stale stale bot removed the stale label Dec 6, 2019
@wshayes
Copy link
Contributor

wshayes commented Aug 27, 2020

Looks like you can install multiple dependencies in a single set using the following approach:

poetry add -E api fastapi
poetry add -E api gunicorn
poetry add -E api uvicorn

resulting in the following changes to the pyproject.toml file:

fastapi = {version = "^0.61.0", extras = ["api"]}
gunicorn = {version = "^20.0.4", extras = ["api"]}
uvicorn = {version = "^0.11.8", extras = ["api"]}

But running poetry update; poetry install -E API doesn't work yet.

You need to add the following entry to the pyproject.toml file:

[tool.poetry.extras]
api = ["fastapi^0.61.0", "gunicorn^20.0.4", "uvicorn^0.11.8"]

and then poetry update; poetry install -E API works fine.

@micuda
Copy link

micuda commented Feb 10, 2021

Thanks for creating this issue, @kurtiss. We have the same problem. I would prefer option (2) proposed by @kurtiss, because it feels more intuitive - this was the first thing I tried.

What do you think, @sdispater?

@wshayes you are describing a different use-case here.

@finswimmer finswimmer added the kind/feature Feature requests/implementations label Mar 7, 2021
@graingert
Copy link
Contributor

I'm looking to port celery from setuptools to poetry and have this problem also

See #4313

@Qsi1984
Copy link

Qsi1984 commented Dec 6, 2021

I reported a possible solution using markers here: #4313 (comment).
Maybe this would be a (temporary) solution in your cases too.

@DaanRademaker
Copy link

Is this still being worken on? It'quite important to decrease size of package installs for other packages that depend on packages build with poetry.

@blaise17
Copy link

blaise17 commented Jul 14, 2022

I have a subtly different use case, which I'm not sure works with the currently proposed solutions without some modification.
Specifically, I need the dependency without extras to not be optional, but the dependency with extras to be optional. I also need one or more of the extras for the dependency to not be optional, while one or more still are optional.
Would it be possible to specify the optional flag per extra?

For example:

[tool.poetry.dependencies]
…
dependency-one = {version = "^1.2.0", extras = [{extra="feature-a", optional = true}, "feature-b"] }
dependency-two = {version = "^1.2.0", extras = [{extra="feature-a", optional = true}] }
…

[tool.poetry.extras]
feature-a = ["dependency-one[feature-a]", "dependency-two[feature-a]"]

@sorasful
Copy link

Hi, do we have workarounds this ?

We do have the same problem, we want to be able to specify extras that depends on each other to avoid redundancy but they are not getting installed.

Thanks!

@sorasful
Copy link

sorasful commented Sep 1, 2023

So basically, what we did is we created another pyproject.toml which we edit and we use the syntax :

[tool.poetry.extras]
mylib1 = [
    "oauthlib",
    "requests_oauthlib",
]
mylib2 = [
    "mylib1 ",
    "Flask",
    "Fastapi",
]

And we created a script that generates a pyproject.toml file and when it does, it repaces "mylib1" by all the requirements in it. For example we would have something like that :

[tool.poetry.extras]
mylib1 = [
    "oauthlib",
    "requests_oauthlib",
]
mylib2 = [
    "oauthlib",
    "requests_oauthlib",
    "Flask",
    "Fastapi",
]

We added this in a pre-commit, and it's done.
It's not ideal, but it's a workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Feature requests/implementations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants