Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit test that confirms performance on large system #473

Open
rsdefever opened this issue Oct 15, 2020 · 5 comments
Open

Add unit test that confirms performance on large system #473

rsdefever opened this issue Oct 15, 2020 · 5 comments
Assignees

Comments

@rsdefever
Copy link
Member

(Somewhat abstract) idea: Add unit test(s) to make sure that new features don't negatively affect the performance of GMSO. I'm not exactly sure what this looks like, but something along the lines of building a large system and ensuring the time doesn't exceed some limit. pytest has timeout capabilities. I could use some brainstorming from others on what the specifics of the test should look like though.

@rsdefever rsdefever changed the title Add unit test that confirms timing on large system Add unit test that confirms performance on large system Oct 15, 2020
@rmatsum836
Copy link
Collaborator

We have something similar on foyer in test_performance.py where we load in a mb.Compound, atom type with foyer, and use pytest.mark.timeout().

@rmatsum836 rmatsum836 self-assigned this Apr 27, 2021
@rmatsum836
Copy link
Collaborator

@rsdefever Here are several ideas for tests. I think we'll eventually want to test atom-typing with GMSO once it's a supported feature.

  • Writers
  • Converter from mBuild
  • Loading in a large FF XML (OPLS perhaps)

Also, just so we're on the same page, what would we consider to be a large system?

@rsdefever
Copy link
Member Author

rsdefever commented Apr 29, 2021

@rmatsum836 I would say at least 100k atoms. If we really wanted to push it, then 500k atoms. But 100k atom systems aren't that "big" anymore.

The main thing is to know when we are degrading performance, so we can decide whether or not the performance degradation is worth whatever new feature we are considering.

@rmatsum836
Copy link
Collaborator

Makes sense. My concern is the wall time required for operations with 100k atoms. Right now these unit tests would be quite long. I wonder if it's possible to only run these tests on PR builds.

I'm not sure how much we've worried about optimization at this point. I think it makes sense to leave this issue open and adjusts the tests once we're happy with performance. Then the tests can ensure that new features aren't degrading performance.

@rsdefever
Copy link
Member Author

I'm fine with keeping the tests smaller for now, and then adding larger once as we get better performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants