Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get rid of LazyModel #218

Open
juanmc2005 opened this issue Nov 18, 2023 · 0 comments
Open

Get rid of LazyModel #218

juanmc2005 opened this issue Nov 18, 2023 · 0 comments
Labels
API Improvements to the API feature New feature or request
Milestone

Comments

@juanmc2005
Copy link
Owner

Problem

LazyModel makes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reason LazyModel exists is so that we can spawn new processes and share a PipelineConfig instance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.

Idea

To get rid of LazyModel without sacrificing multiprocessing we need to lighten the content of PipelineConfig, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.

This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.

@juanmc2005 juanmc2005 added the bug Something isn't working label Nov 18, 2023
@juanmc2005 juanmc2005 added this to the Version 0.10 milestone Nov 18, 2023
@juanmc2005 juanmc2005 added feature New feature or request API Improvements to the API and removed bug Something isn't working labels May 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Improvements to the API feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant