Hatchbuild retry #379
Hatchbuild retry #379
Conversation
|
Many thanks for all your efforts! 👍
That's fine for the time being. Sometimes I do that too when I did something so horribly wrong that I can't find an escape. 😂 However, proliferating PRs where the same topic is opened and closed again and again should be avoided. After some time, nobody really knows anymore and it gets harder to look through the history of changes if you have a pile of PRs. 😉 The steps for a git rebase (click me)Once you get used to it, it's not really that hard. Before you do it, I would recommend to make a copy/backup of your local directory. Just in case. The simplest case is fairly easy. This is all you need: If you get conflicts after you have rebased, you need a bit more work: If you want a more detailed explanation (with nice images!) I can recommend https://www.geeksforgeeks.org/rebasing-of-branches-in-git/ and for conflict resolution look at https://codeinthehole.com/guides/resolving-conflicts-during-a-git-rebase/
No worries, I can do that for you. However, even Windows supports links. If I'm not mistaken, Git should do most of the things for you.
That sounds a bit strange as there is no ndiff folder, but ndiff is a doctest option.
That's fine, I think you can overwrite the command(s) for Windows. I have seen that Hatch offers some means to do this. I haven't look into the details of the PR yet, so the above reply is just the things that I wanted to share with you. I'll keep you updated! |
|
I wonder if I did not forget the most important thing in my PR.
True, but I only have a Windows-based dev environment set up on my company's computers where the Windows policies forbid users to create symlinks, or hard links. I don't think I will ever bother with setting up Windows for development on my private machines. I'm way behind in schedule to get my Linux machines managed by dotfiles and into shape for doing some development work at home. |
Probably it's not needed anymore. However, that would already anticipate the decision from the discussion about removing Python 3.6, wouldn't it? I have no problem of removing the
Ohh... that's really unfortunate. Does your policy allow to work on WSL or in a virtual machine?
I've tried to look into overwriting commands/scripts and it seems I need a By the way, this line looks suspicious to me: Does the |
|
I've played around a bit and was able to create only one This is the patch: Patch for consolidating test environmentcommit
Author: Tom Schraitle <tomschr@users.noreply.github.com>
Date: Sun Nov 27 21:38:12 2022 +0100
Consolidate test environment
Use _all_ supported Python environments. Add a default matrix.
Run a specific test with "hatch run +py=37 test"
TODO:
* Show errors
* Can the supported Python matrix be written only once?
diff --git a/pyproject.toml b/pyproject.toml
index 4f2099d..5d93f10 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -88,6 +88,10 @@ dependencies = [
"wheel",
]
+[[tool.hatch.envs.default.matrix]]
+python = ["37", "38", "39", "310", "311",]
+
+
[tool.hatch.envs.style]
dependencies = [
"black",
@@ -123,23 +127,13 @@ dependencies = [
]
[tool.hatch.envs.test.scripts]
-python = ["37", ]
+python = ["37", "38", "39", "310", "311", ]
+
[tool.hatch.envs.test.scripts]
-cov = "pytest -vx"
+cov = "pytest"
no-cov = "cov --no-cov"
-[tool.hatch.envs.testall]
-dependencies = [
- "coverage[toml]",
- "pytest-cov",
-]
-[[tool.hatch.envs.testall.matrix]]
-python = ["37", "38", "39", "310", "311", ]
-
-[tool.hatch.envs.testall.scripts]
-cov = "pytest -vx"
-no-cov = "cov --no-cov"
-
-[tool.hatch.envs.testall.scripts]
-cov = "pytest -vx"
-no-cov = "cov --no-cov"
[tool.hatch.envs.tox]
dependencies = [
@@ -149,6 +143,9 @@ dependencies = [
[tool.hatch.envs.tox.scripts]
test = "tox"
+
+## --------------------------------------------------------
+## Other tools
[tool.coverage.run]
source = [
"semver",
@@ -207,12 +204,13 @@ filterwarnings = [
'ignore:Function semver.*:DeprecationWarning',
]
addopts = [
+ "--verbose",
"--no-cov-on-fail",
"--cov=semver",
"--cov-report=term-missing",
"--doctest-glob='*.rst'",
"--doctest-modules",
- "--doctest-report ndiff",
+ "--doctest-report=ndiff",
]
# flake8 does not support configuration in pyproject.tomlAfter the patch, you have only this environments: To run only, let's say, Python 3.7, you run: However, it gives me a return code 1 😢 and it shows me nothing, although pytest was given a @ofek Could you shed some light on our endeavor? I have basically these questions:
Thanks! |
https://hatch.pypa.io/latest/config/environment/advanced/#platform-overrides Non-platform example https://github.com/mkdocs/mkdocs/blob/56b235a8ad43f2300d17f87e6fa4de7a3d764397/pyproject.toml#L114-L127
No https://hatch.pypa.io/latest/config/environment/overview/#inheritance |
|
Thanks @ofek for the great tips! 👍
Thanks. I looked at the documentation. I don't get it. Really sorry, but this part was incomprehensible to me and confused me (that's why I opened the doc issue in your repo). 😉 After I've looked at your example, it seems we're one step further. I did this: [tool.hatch.envs.docs.scripts]
# The default command are on Linux:
build = "make -C docs html"
linkcheck = "make -C docs linkcheck"
serve = "python3 -m webbrowser -t docs/_build/html/index.html"
[[tool.hatch.envs.docs.matrix]]
platform = ["windows", "macos",]
[tool.hatch.envs.docs.overrides]
matrix.platform.scripts = [
{
run = "docs/make.bat -C docs html",
value = "build",
if = ["windows"]
},
{
run = "docs/make.bat -C docs linkcheck",
value = "linkcheck",
if = ["windows"]
},
]However, when I run The line 128 points to the curly open brace. 🤔 That is a minor issue, but still it's a bit confusing.
I was afraid of that. That's really unfortunate. Is it planned to maybe have some sort of "global" variables in the config that can be referenced and used in different parts of the config? I'm sorry, but I don't want to have the same information again in different parts. That only creates inconsistencies. Would it be worth to open a feature request? Many thanks for your help! |
|
Ok, I think, I "fixed" it. It were some TOML problems. [tool.hatch.envs.docs.scripts]
build = "make -C docs html"
linkcheck = "make -C docs linkcheck"
serve = "python3 -m webbrowser -t docs/_build/html/index.html"
_build_win = "docs/make.bat -C docs html"
[tool.hatch.envs.docs.overrides]
"matrix.platform.scripts" = [
{ key="build", value = "build", if = ["linux"] },
{ run="_build_win", value = "build", if = ["windows"] },
{ run="docs/make.bat -C docs linkcheck", value = "linkcheck", if = "windows" },
]Does that make sense? At least it gives me no errors: Output of hatch env showI can run it with: However, if I leave out the |
I don't know these things. Just thinking out loud, I would solve it by having a little utility script around in the repository (excluded from the package build) and there I would program the CLI with the defaults and all the logic. My next step would be to generalise it and make this a utility, since it will be likely needed in other Hatch projects. Or maybe Hatch has already thought of this and it would be possible to write it as a hatch plugin. Maybe it would be possible to write an environment plugin which would dynamically generate a build environment based on some parameters (such environment would have some dynamically generated scripts), and then you could run those scripts. References: |
I had a similar thought; maybe a wrapper around Could you ask on the Hatch discussion forum? Maybe the community could help us.
That would be probably the more general and platform neutral way. However, keep in mind, it's about semver, not hatch. 😉 Hatch is just the means to build and maintain semver. I don't have the time nor the capacity to maintain plugins. Whatever approach we choose, it should be stable, simple, and easy to maintain. |
To ask for a solution in a forum I need to understand the problem. I don't understand why we need to have a matrix of documentation envs and why we occasionally need to run a script which would only do just one piece of the matrix. I'm missing the whole picture here. In your former post I don't get where this Let me lay out my understanding (and the lack thereof) of what we actually want from hatch. Our ultimate goal here is to have a pyproject.toml which could be utilised by hatch to:
I have the feeling I misunderstand here a lot. Maybe you don't want different versions of API-documentations, maybe it's just the sphinx invocation and the whole doc-building script that must be different on Windows than it is on linux and for this we'd need one linux script, and one for script for Windows ( It's like ... I kind of see (or can guess) what you want, but I don't understand why you want to solve it this particular way. From my point of view there are better ways to do this kind of things than to try to do everything manually by invoking hatch scripts. |
First thanks for your great post and really sorry for not be clear enough with my ideas.
It's not in your branch (yet) as you need to allow me to push. I hope I got it right, but you need to allow changes to a pull request branch created from a fork. It's should be only a checkmark, If you are okay with that, I can push my changes for your review.
You raised a lot of good questions. In the following list I try to answer them as good as I can. Hope that make things a bit clearer. Again, many thanks for making this list! ❤️
Yes. If we switch to hatch, the commands should run locally and on our GitHub Action workflow (=CI). Otherwise it becomes inconsistent. Currently, in the CI this is done by tox.
To my understanding, the matrix is needed to provide different commands for Windows and Linux. I may be wrong, I'm not a Hatch expert. This was the part that I was trying to solve in my previous comments.
Oh, you mean dependent on the Python version? No, the API documentation is for all Python versions, there is nothing specific. The build part is done by Sphinx.
I would advise against this. Technically it could be done. However, it doesn't mean we should. In my experience, this opens all sort of problems. Additionally, it would mean we need to add more GitHub Action power. That's why we have only a "check" GH Action job for linting and formatting. It doesn't commit automatically, but will report back to the developer (as you can see in this PR).
Yes, that would be something we could (and should) automate. That would relate to issue #205 and possibly is a task for a different pull request as it's out of scope for this PR (also it needs some thoughts what the workflow should look like).
As described above, I'm hesitant to do automatic commits etc. For my taste, that would be too much automation. From my experiences, changelogs are better done manually.
Probably yes. Also related to issue #205.
Not really. :-) You did already a great job. Thanks for asking and sorry for making your PR harder than it should be. Wasn't my intention.
Well, there are different reasons. 🙂 Some goes mostly into "you can't teach an old dog a lot of new tricks" territory. 😂 At the moment, tox is used to maintain this project. If you run That doesn't mean it's perfect nor it can't be improved or replaced with something better. But if we want to replace this, it should have some benefits. "Modern" is not always equivalent to "better". If we can replicate somehow the tox targets I think it would fit nicely into my workflow. TL;DR: From my point of view, we could use hatch to:
A lot of of these targets overlap with tox (which is intended). Perhaps we can replace tox completely with hatch. I don't know yet if it's possible or desirable. There should be only one tool which does it all. How we really design the targets is probably a matter of taste and how granular we want to have them. Certainly it also depends on what hatch supports. However, I can't do that alone as I need support. A second goal is to adapt the GitHub Action workflow. This would be out of scope for this PR, but should be done soon after we merge your branch. Does that help? |
Retry of PR #377.
I don't know did not find a way how to fix the links
coerce.pyandsemverwithvprefix.py. Maybe because I develop under Windows which does not support symlinks. I had to copy the code from their targets into them, but then I had an import conflict, as pytest tried to import both the "links" and their targets under the same name. So I additionally had to exclude thetestsdir fromtestpaths.Also pytest was complaining that it cannot find the
ndifffolder, so I had to exclude it fromaddoptsas well.And I had to change the matrix from
37to310because I currently only have python 3.10. Then the tests ran! 🥲 . But I undid these changes for this PR. However, I would be happy to solve these errors in the future, as I don't want to do this switcheroo every time I'd contribute some test code.I also had to change the wheel build target (see packages and forced inclusion), lest python would not find the
semverpackage after installing the wheel, or mypy would not find itspy.typesfile, respectively.I also did not test your docs scripts (I have PowerShell, where it's a hassle to run
.batscripts), but I do think that they need to have the invocationdocs/make.batto be conveniently called from the project root. Btw, I commented out thesphinxdependency in .envs.docs, because it comes with either of the other two dependencies.I also was too lazy to find out how to rebase and update the previous PR (maybe another time I'll learn how to do this), so I closed it and here comes a new one.