Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-run models for RAM issues #16

Open
PabloGGaray opened this issue May 31, 2022 · 1 comment
Open

Pre-run models for RAM issues #16

PabloGGaray opened this issue May 31, 2022 · 1 comment

Comments

@PabloGGaray
Copy link
Collaborator

The notebooks friedman.ipynb and friedman_trees_alphas.ipynb need around 21 GB of RAM to run the models in that notebook. Maybe it could be useful to upload to the repository a pre-run model for these notebooks, to have the option of loading the models instead of running them when the user doesn't have enough memory.

@PabloGGaray
Copy link
Collaborator Author

Another option could be to use psutil to check the total RAM (based on this), with something like this:

from psutil import virtual_memory
mem = virtual_memory()
ram = mem.total/(1024.**3)

For friedman.ipynb

if ram > 20.0:
    num_covariables = ["5", "10", "100", "1000"]
elif ram < 20.0:
    num_covariables = ["5", "10", "100"]

And for friedman_trees_alphas.ipynb

if ram > 20.0:
    alphas = [0.1, 0.25, 0.5]
    idatas_at = {
        "10": {"0.1": {}, "0.25": {}, "0.5": {}},
        "20": {"0.1": {}, "0.25": {}, "0.5": {}},
        "50": {"0.1": {}, "0.25": {}, "0.5": {}},
        "100": {"0.1": {}, "0.25": {}, "0.5": {}},
        "200": {"0.1": {}, "0.25": {}, "0.5": {}},
    }
    trees = [10, 20, 50, 100, 200]
elif ram < 20.0:
    alphas = [0.1, 0.25, 0.5]
    idatas_at = {
        "10": {"0.1": {}, "0.25": {}, "0.5": {}},
        "20": {"0.1": {}, "0.25": {}, "0.5": {}},
        "50": {"0.1": {}, "0.25": {}, "0.5": {}},
    }
    trees = [10, 20, 50]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant