Delay installation of extra requirements after cache hit #805
Comments
PhilippeMoussalli
added a commit
that referenced
this issue
Jan 23, 2024
…ss (#802) PR that updates the cache key estimation to take into account the `ComponentSpec` instead of the `ComponentOp` (operation op has both the component spec and the inner `produces` and `consumes` as well as the `Image` Tested on the example pipeline and caching seems to work fine for lightweight component, only issue is that it will always download the requirements (fondant + extra user dependencies) before skipping the execution of the component. This can become frustrating for iterative development with an increase of dependencies and components. We might be able to mitigate this by providing base images with pre-installed Fondant version and move the installation of the extras after the cache hit estimation. Best to tackle this in a separate PR. Issue ticket [here](#805) The PR also include some small changes to function names to best reflect what they do
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
More info here
The text was updated successfully, but these errors were encountered: