numba cache file is surprisedly large

[dlee992]

Hi, I met an use-case when enabling cache and using many nested record array and dict.

since CompileResult will also serialize several variables to the .nbc cache file, which makes my cache file pretty big, more than 10MB. And I have dozens of these jitted functions and some have different type signatures, which result in >1GB cache files.

numba/numba/core/compiler.py

Line 202 in 1085604

def _reduce(self):

Not sure if it's expected or we can improve on that somehow.

[dlee992]

Hi, after maintainer triage, any feedback? @kc611 Thanks!

in principle numba cache files shouldn't be this large? I feel it could be related the __str__(a_type_instance) is a naive recursive definition, which results in this issue, given using nested dict, record array, and structref.

[github-actions]

This issue is marked as stale as it has had no activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with any updates and confirm that this issue still needs to be addressed.

[dlee992]

closing this for now, since I lack of concrete test case to show this.