Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can the memory footprint be reduced? #8

Open
chi2liu opened this issue Jan 20, 2022 · 1 comment
Open

Can the memory footprint be reduced? #8

chi2liu opened this issue Jan 20, 2022 · 1 comment

Comments

@chi2liu
Copy link

chi2liu commented Jan 20, 2022

With a 14178107*8 vector, a 108GB memory machine is quickly used up. Is there a way to reduce the memory footprint?

The train output:

Input: 14178107 points, dimension 8
scheduler = Parlay-HomeGrown
num-threads = 16
num-cell = 12333095
compute-grid = 5.06638
@wangyiqiu
Copy link
Owner

Hi, I couldn't think of a straight forward way to optimize the memory usage at the moment. I think the high memory usage in your case may be related to a relatively small eps value that you are using. Alternatively, you may also want to try machines with larger memory such as AWS EC2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants