Publish to PyPi #42
Publish to PyPi #42
Conversation
publish on release and test on tag
| @@ -1,5 +0,0 @@ | |||
| beautifulsoup4 | |||
There was a problem hiding this comment.
Can we please keep the requirements.txt file? This makes it far easier for downstream project and also versions are more relaxed than with poetry. Also I am lazy and don't want another package manager.
There was a problem hiding this comment.
The point of having stricter dependency management is to ensure that repeated builds of the project (without code changes) will always be the same. Without a “lockfile” it is entirely possible for the dependencies to change and therefore produce different builds despite making no code changes ourselves. Also enforced good security practises.
I believe the pros of poetry (including virtual environments to separate the local packages from global ones) outweigh the cons of having another package manager (curious to know which other package managers you recommend)
There was a problem hiding this comment.
That is a real problem for projects that are not actively maintained which then download old outdated versions. Also distributions can't always package other package manager.
I only use docker so I already have a virtual environment and in my system I only install distribution packages and I am not sure if I can pack poetry.
Also pip can lock versions numbers and can be used with virtualenv if you like to.
So this gains nothing for me other than needing to look how peotry works and increase the size of docker images.
Edit: this does the exact same
beautifulsoup4==4.8.1
There was a problem hiding this comment.
We should assume that all open source projects are going to sit unmaintained for a long time, because most do.
Docker can be accomplished without increasing the size of the image just by using multi-stage builds python-poetry/poetry#1178
Pip can lock the version number, but can’t verifiably lock the actual version. Poetry can do this by hashing the actual version to ensure that it is the correct code being bundled (good for security, just good practise overall). Other languages include a package manager which locks versions properly (NPM, Go Modules etc)
Including a pyproject.toml with information about the project seems good practise anyway, especially specifying the build requirements: https://www.python.org/dev/peps/pep-0518/
There was a problem hiding this comment.
Using poetry and multi stage builds adds a lot of complexity if you can just install using pip and the requirements.txt. The features of poetry are nice but really they should be included with pip and not force me to install yet another package manager just to try or this project.
There was a problem hiding this comment.
Those changes kinda break this.
There was a problem hiding this comment.
Could you explain how please? I just don’t understand your point
There was a problem hiding this comment.
I built the required python packages as wheels and uploaded them to my local pypi repository and the tried installing it with pip3 install -r https://raw.githubusercontent.com/jamesjarvis/geo-heatmap/master/requirements.txt and got
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; som
eone may have tampered with them.
ijson==2.5.1 from https://www.piwheels.org/simple/ijson/ijson-2.5.1-py3-none-any.whl#sha256=df48777ae9de5e78de8c989c724c9bc7f446b834034fb4c29695576cab79be26 (from -r https://raw.githubu
sercontent.com/jamesjarvis/geo-heatmap/master/requirements.txt (line 21)):
Expected sha256 92ee4d4d3d0fd5ad59ec2ebc0ab0bd8a7b0eb79c8e21e87c18aa3dd282958714
Expected or 240ccc575097784aaae447d29525801d27fb36020f8bfb7280668c37afe4d543
Expected or 53923e5a58e78a5eca8e461e2c6c8a7b01cfeb9e83dfce373de57991dccf20a9
Expected or 147830257fbd3c379b5d9197ca6d62adf23cf3cd7bf447f9b8ba09ea4be2dbda
Expected or d93e3b1ddb18507d79a7d8fb877ab8a5cd49e3284460c43020aad134af1c52e1
Expected or bfbcd5b6f48b9a0b88845b90a5137f482b16fbaea79df8c4b6eeb9cc5dc0da20
Expected or e91057e18eedefeeaca05126493b6f2d1eeb43f45b1654b066b4cb296bc9bbde
Expected or 3cda11c1b4dcf5cdb4b2f8785ca7438a577ffc65ac6d9d235c419ed7d890a297
Expected or 8238b389fc87b64fcfac851ac409e53614a4390077d9a5ffbad1ba99a812d189
Expected or 2247d8906f948c2201d754d7d4aa4556e69de18c11d476c334e4c90fdd03b817
Expected or d7baafc8027735d9525dc7e8275e2201d6ca91ead6b481caf31888615000394e
Expected or 19ec46a2f7991004e5202ecee56c569616b8a7f95686ad7fd0a9ec81cac00269
Got df48777ae9de5e78de8c989c724c9bc7f446b834034fb4c29695576cab79be26
markupsafe==1.1.1 from https://pypi.supersandro.de/packages/MarkupSafe-1.1.1-cp37-cp37m-linux_aarch64.whl#md5=2fc828c720317a7d2aea4487e67d3e0e (from -r https://raw.githubusercontent.com
/jamesjarvis/geo-heatmap/master/requirements.txt (line 37)):
Expected sha256 09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161
Expected or e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7
Expected or 500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183
Expected or b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b
Expected or 98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e
Expected or cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f
Expected or 43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1
Expected or 1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5
Expected or 62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1
Expected or 88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735
Expected or ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21
Expected or 09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235
Expected or 79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b
Expected or c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f
Expected or 7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905
Expected or 6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1
Expected or 9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d
Expected or 24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff
Expected or 00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473
Expected or 717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e
Expected or 535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66
Expected or b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5
Expected or 8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d
Expected or 46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e
Expected or ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6
Expected or b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2
Expected or 9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c
Expected or 29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b
Got dca44b0a131b0710e2efabce718dac4501fb1d4ac88b02fea5874cc4684ae8bd
This also fails on Raspbian cause it uses piwheels by default.
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
ijson==2.5.1 from https://www.piwheels.org/simple/ijson/ijson-2.5.1-py3-none-any.whl#sha256=df48777ae9de5e78de8c989c724c9bc7f446b834034fb4c29695576cab79be26 (from -r https://raw.githubusercontent.com/jamesjarvis/geo-heatmap/master/requirements.txt (line 21)):
Expected sha256 92ee4d4d3d0fd5ad59ec2ebc0ab0bd8a7b0eb79c8e21e87c18aa3dd282958714
Expected or 240ccc575097784aaae447d29525801d27fb36020f8bfb7280668c37afe4d543
Expected or 53923e5a58e78a5eca8e461e2c6c8a7b01cfeb9e83dfce373de57991dccf20a9
Expected or 147830257fbd3c379b5d9197ca6d62adf23cf3cd7bf447f9b8ba09ea4be2dbda
Expected or d93e3b1ddb18507d79a7d8fb877ab8a5cd49e3284460c43020aad134af1c52e1
Expected or bfbcd5b6f48b9a0b88845b90a5137f482b16fbaea79df8c4b6eeb9cc5dc0da20
Expected or e91057e18eedefeeaca05126493b6f2d1eeb43f45b1654b066b4cb296bc9bbde
Expected or 3cda11c1b4dcf5cdb4b2f8785ca7438a577ffc65ac6d9d235c419ed7d890a297
Expected or 8238b389fc87b64fcfac851ac409e53614a4390077d9a5ffbad1ba99a812d189
Expected or 2247d8906f948c2201d754d7d4aa4556e69de18c11d476c334e4c90fdd03b817
Expected or d7baafc8027735d9525dc7e8275e2201d6ca91ead6b481caf31888615000394e
Expected or 19ec46a2f7991004e5202ecee56c569616b8a7f95686ad7fd0a9ec81cac00269
Got df48777ae9de5e78de8c989c724c9bc7f446b834034fb4c29695576cab79be26
There was a problem hiding this comment.
😆 ok I didn't anticipate there would be different uploads of the same version of software, with different hashes...
I can see this is going to continue to be an issue, so I'll remove the hashes from requirements.txt
Thanks for explaining
There was a problem hiding this comment.
Python should hash the source from which the wheel is built and compare that instead of the archive I download.
In response to comments in PR luka1199#42 , adds back a requirements.txt file and removes the "v" limitation in versioning
As per comments in luka1199#42 , add back requirements.txt as exported from poetry (and include info in README how to generate this), and remove the "v" from version tags as per comments.
Due to multiple package repositories being used, hashes do not always match what the target environment requires (see luka1199#42), so this has been removed. For those still wanting a securely locked versioning system, use the poetry files also included, but this may introduce issues with different environment.
|
Sorry for the late reply. Could you bring it up to date? Then I can merge the pull request. You can send me everything necessary for PyPi access by email (luka.steinbach@gmx.de). |
Same as #40 , but rebased to include changes which fix #41
Publishes to PyPi, enables automatic CI/CD on specific releases, locks dependencies and enables virtual environment management with poetry.
In order to ensure that the github actions continue to work, you will need to become a member of the geo-heatmap project on PyPi (contact me for access), and add an API key to the secrets part of this repo.