Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: poor memory management preventing full test of concat_data #1136

Open
aburrell opened this issue Sep 12, 2023 · 1 comment
Open

BUG: poor memory management preventing full test of concat_data #1136

aburrell opened this issue Sep 12, 2023 · 1 comment
Labels
Milestone

Comments

@aburrell
Copy link
Member

Describe the bug
GitHub Actions cannot run for the operational test, due to poor memory management.

To Reproduce
Steps to reproduce the behavior:

  1. Go to pysat/pysat/tests/classes/cls_instrument_library.py
  2. Change the pytest.skip conditions to allow Python 3.6 to run for the test_load_w_pad test.
  3. Perform the full unit test suite.
  4. See error

Expected behavior
These tests will pass without the full suite locally and the behaviour is correct in iPython. The full test suite should run.

Desktop (please complete the following information):

  • OS: GitHub Actions Ubuntu
  • Version: Python 3.6
  • Other details about your setup that could be relevant

Additional context
The error we are getting is that too much memory is being used. I was able to significantly reduce memory usage in Instrument.load by making changes to __getitem__. It is likely that the use of __slots__ (which requires removing weakref usage), and improving other algorithmic redundancies can solve this problem.

@aburrell
Copy link
Member Author

We should probably establish benchmarking for whatever bits of code we start working on for this to better track improvements (see #811).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants