Use micropipenv for installing dependencies

[fridex]

No description provided.

[centos-ci]

Can one of the admins verify this patch?

[rhscl-bot]

Can one of the admins verify this patch?

[rhscl-bot]

Can one of the admins verify this patch?

[rhscl-bot]

Can one of the admins verify this patch?

[pkubatrh]

Pretty sure this change will affect more than Fedora though...

[frenzymadness]

Yes, the process here is different because the content in version folders is generated via distgen. Nevertheless, I understand what you are trying to say and will take a look at this possibility.

[fridex]

Yes, the process here is different because the content in version folders is generated via distgen. Nevertheless, I understand what you are trying to say and will take a look at this possibility.

Cool, thanks!

My few bits here if this approach would be accepted.

• we should take a look at the installation process as done in the s2i build. The relative ordering in the installed dependencies is relevant in Python - see for example [1]. We have created a patch to address this issue in [2] for micropipenv. If users use pip-tools style requirements.txt/requirements.in to lock dependencies, they can encounter similar issues (also with unpinned requirements.txt but that should be strongly discouraged anyway). We can include this logic in micropipenv if you agree with this in s2i (we can rename micropipenv :) )

• it would be probably viable to package micropipenv and install it via rpm, rather than using PyPI as its source - its tooling layer

[1] thoth-station/s2i-thoth#39
[2] thoth-station/micropipenv#13

[frenzymadness]

Have micropipenv as RPM would be very good!

I am afraid about one thing — the lack of Python 2 compatibility. That would mean that we would be able to use micropipenv only for Python 3 images which would make our situation a little bit more complicated and resulting images different in this aspect. On the other hand, micropipenv is only ~500 lines of code and I have a lot of experience with writing code compatible with multiple Pythons. Would you be interested in adding support for EOL Python?

I'm willing to help you with packaging and add support for more Python versions.

[fridex]

Have micropipenv as RPM would be very good!

I am afraid about one thing — the lack of Python 2 compatibility. That would mean that we would be able to use micropipenv only for Python 3 images which would make our situation a little bit more complicated and resulting images different in this aspect. On the other hand, micropipenv is only ~500 lines of code and I have a lot of experience with writing code compatible with multiple Pythons. Would you be interested in adding support for EOL Python?

If python2 support helps here, definitely.

I'm willing to help you with packaging and add support for more Python versions.

Awesome, combining efforts is always a good idea! 👍 I just shooted first possible fixes in [2]. I would appreciate your input.

Before we dive into this - what about considering "promoting" micropipenv to a tool that can handle different file formats (pip-tools/poetry/pipenv) and serve as a layer for installing python packages with possible error handling? Python s2i builds would not need to bundle other tools as pipenv/poetry.

[1] thoth-station/micropipenv#18

[fridex]

Before we dive into this - what about considering "promoting" micropipenv to a tool that can handle different file formats (pip-tools/poetry/pipenv) and serve as a layer for installing python packages with possible error handling? Python s2i builds would not need to bundle other tools as pipenv/poetry.

There would be also benefits on our (Thoth) side. We automatically parse build error logs and having one type of stdout/stderr logging (that would be pip output in this case) would definitely make our lives easier for data aggregation.

[frenzymadness]

Let's close it for now. Our possibilities are summarized in #368