Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use uv for all pip installs #1090

Merged
merged 4 commits into from
May 8, 2024

Conversation

matthewfeickert
Copy link
Contributor

* Use 'uv pip' for all calls to 'pip install' and 'pip uninstall' in
  CI workflows.
   - c.f. https://github.com/astral-sh/uv/
* Fall back to 'python -m pip' for issue with alternative package index.
* Use '--upgrade' as '--upgrade' uses '--resolution highest' by default,
  which is similar to the 'eager' upgrade strategy in pip.
@matthewfeickert
Copy link
Contributor Author

@nsmith- @lgray This is ready for review (assuming all the CI passes).

OS Python pip uv pip
Ubuntu 3.12 2m 17 s 50 s
macOS (Apple silicon) 3.12 2m 0 s 53 s
Windows 3.12 4m 21 s 2m 20 s

These would be faster still if not for astral-sh/uv#3437 requiring a switch to python -m pip at one stage.

@matthewfeickert
Copy link
Contributor Author

Seems this

==================================== ERRORS ====================================
________________________ ERROR at setup of test_listify ________________________

    @pytest.fixture(scope="module")
    def events():
>       return _events()

tests/test_nanoevents_delphes.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_nanoevents_delphes.py:17: in _events
    return factory.events()
src/coffea/nanoevents/factory.py:684: in events
    events = self._mapping(form_mapping=self._schema)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/uproot/_dask.py:296: in dask
    return _get_dak_array_delay_open(
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/uproot/_dask.py:1649: in _get_dak_array_delay_open
    expected_form, form_mapping_info = form_mapping(base_form)
src/coffea/nanoevents/factory.py:133: in __call__
    awkward.forms.form.from_dict(self.schemaclass(lform, self.version).form),
src/coffea/nanoevents/schemas/delphes.py:211: in __init__
    output = self._build_collections(old_style_form)
src/coffea/nanoevents/schemas/delphes.py:276: in _build_collections
    output[name] = zip_forms(content, name, record_name=mixin, offsets=offsets)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

forms = {}, name = 'TrackJet08', record_name = 'Jet'
offsets = {'class': 'NumpyArray', 'form_key': 'TrackJet08_size%2C%21load%2C%21counts2offsets', 'format': 'i', 'itemsize': 8, ...}
bypass = False

    def zip_forms(forms, name, record_name=None, offsets=None, bypass=False):
        if not isinstance(forms, dict):
            raise ValueError("Expected a dictionary")
        if all(form["class"].startswith("ListOffsetArray") for form in forms.values()):
>           first = next(iter(forms.values()))
E           StopIteration

src/coffea/nanoevents/schemas/base.py:28: StopIteration

The above exception was the direct cause of the following exception:

cls = <class '_pytest.runner.CallInfo'>
func = <function call_and_report.<locals>.<lambda> at 0x15e8a53a0>
when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/runner.py:241: in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/unraisableexception.py:85: in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/unraisableexception.py:65: in unraisable_exception_runtest_hook
    yield
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/logging.py:844: in pytest_runtest_setup
    yield from self._runtest_for(item, "setup")
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/logging.py:833: in _runtest_for
    yield
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/capture.py:873: in pytest_runtest_setup
    return (yield)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/threadexception.py:82: in pytest_runtest_setup
    yield from thread_exception_runtest_hook()
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/threadexception.py:63: in thread_exception_runtest_hook
    yield
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/runner.py:159: in pytest_runtest_setup
    item.session._setupstate.setup(item)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/runner.py:515: in setup
    raise exc
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/runner.py:512: in setup
    col.setup()
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/python.py:1630: in setup
    self._request._fillfixtures()
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/fixtures.py:683: in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/fixtures.py:518: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/fixtures.py:603: in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pytest/fixtures.py:1073: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_pytest.config.PytestPluginManager object at 0x105f6bfa0>
hook_name = 'pytest_fixture_setup'
methods = [<HookImpl plugin_name='fixtures', plugin=<module '_pytest.fixtures' from '/Library/Frameworks/Python.framework/Versio...lugin' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytest_asyncio/plugin.py'>>]
kwargs = {'fixturedef': <FixtureDef argname='events' scope='module' baseid='tests/test_nanoevents_delphes.py'>, 'request': <SubRequest 'events' for <Function test_listify>>}
firstresult = True

    def _hookexec(
        self,
        hook_name: str,
        methods: Sequence[HookImpl],
        kwargs: Mapping[str, object],
        firstresult: bool,
    ) -> object | list[object]:
        # called from all hookcaller instances.
        # enable_tracing will set its own wrapping function at self._inner_hookexec
>       return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
E       RuntimeError: generator raised StopIteration

/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pluggy/_manager.py:120: RuntimeError
------------------------------ Captured log setup ------------------------------
DEBUG    fsspec.local:local.py:314 open file: /Users/runner/work/coffea/coffea/tests/samples/delphes.root

is a generic failure at the moment given it is also happening in #1087.

@matthewfeickert
Copy link
Contributor Author

This is being blocked by #1080 (comment).

@lgray lgray enabled auto-merge May 8, 2024 15:10
@lgray lgray merged commit 5e94b87 into CoffeaTeam:master May 8, 2024
12 of 14 checks passed
@matthewfeickert matthewfeickert deleted the ci/use-uv-for-pip-installs branch May 8, 2024 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants