Add packse scenarios as pip tests #12580
Conversation
| @pytest.fixture(scope="session", autouse=True) | ||
| def start_packse_server() -> Generator[None, None, None]: | ||
| """Starts the packse server before tests run and ensures it's terminated after.""" | ||
| proc = subprocess.Popen( | ||
| ["packse", "serve", "--host", "127.0.0.1", "--port", "3141"] | ||
| ) | ||
| time.sleep(1) | ||
| yield | ||
| proc.terminate() |
There was a problem hiding this comment.
I would recommend using the static index packse generates as mentioned in astral-sh/packse#165 (comment) instead — the serve command is intended for development purposes.
There was a problem hiding this comment.
My concern with that, and maybe it's not justified, is that tests could fail because the scenarios from packse fetch & packse inspect would not align perfectly with the scenarios on the index.
There was a problem hiding this comment.
fetch defaults to pulling the current packse version tag which should always match the corresponding index for that tag. You just need to do packse inspect --no-hash since the index is versioned we don't need to include a hash to uniquely identify scenarios.
There was a problem hiding this comment.
I had a long throught about this and my main concern is that if pip wants to add it's own scenarios, or deviate away from packse scenarios (e.g. because they seemingly don't follow the spec astral-sh/packse#161) then using the astra index won't work.
However, for now, I will attempt to implement using the index, and that problem can be worried about in the future.
|
Some unexpected failures on Mac, but passed on Windows and Linux, probably won't have time to look this weekend but I'm sure I'll be able to figure them out. |
Fixes #12526
Current strategy:
packse fetchwhich git pulls the packse scenariospackse servewhich builds and serves the scenarios on a local indextest_packse_scenarioas a dynamically generated parameter from the information given inpackse inspectI do not have a lot of experience with test frameworks, so I'm not at all claiming this is the best approach, it's just what I got working, happy to significantly change it.
There is still more work to do discussing issues on packse side and identifying if currently failing scenarios are bugs on pip side. Once
EXPECTED_TO_FAILis much smaller I will unmark as draft.