Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mutable defaults, enable bugbear ruff rule #10173

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

xmatthias
Copy link
Member

Summary

Mutable defaults are dangerous - as they can lead to state leaking around.

Quick changelog

  • Enable bugbear ruff rule
  • fix most problems

todo

  • fix DataKitchen.use_strategy_to_populate_indicators() defaults

@robcaulk I'd apreciate your help in fixing use_strategy_to_populate_indicators().
While it seems simple at first (simply remove the default dicts) - it's actually not - as there's several usages without these arguments - and i'm afraid that it's implicitly relying on the cache these (now global) dictionaries provide.

For all calls without the corr_dataframes and base_dataframes arguments these 2 variables become "global state", independent of the DataKitchen instance they're actually used in.
As a consequence, this state could "leak around" across individual DataKitchens (across individual Pairs) - leading to very subtle, almost impossible to debug bugs.

a very simple demonstration of this:

class SomeTest():
    def __init__(self, class_name: str):
        self.class_name = class_name


    def leakTest(self, hello=[]):
        print(f"{self.class_name}: {id(hello)=}")

x = SomeTest("test1")
x.leakTest()
y = SomeTest("test2")
y.leakTest()
x.leakTest()
y.leakTest([])  # This one will cause a different ID
#  Output
test1: id(hello)=130731574154496
test2: id(hello)=130731574154496
test1: id(hello)=130731574154496
test2: id(hello)=130731574164032

Now a simple fix could be to simply make them mandatory, providing empty dictionaries where the arguments are currently missing.
my fear however is that freqAI is (implicitly) relying on this global, shared state - and simply removing their "Optionality" will break things...

@xmatthias xmatthias added the Tech maintenance Technical debt and maintenance - point out issues we should resolve long-term label May 5, 2024
@coveralls
Copy link

Coverage Status

coverage: 94.672% (+0.001%) from 94.671%
when pulling c9d2b85 on fix/mutable_defaults
into 39613c0 on develop.

@robcaulk
Copy link
Member

robcaulk commented May 5, 2024

Thanks @xmatthias , I will try to look at this during the week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Tech maintenance Technical debt and maintenance - point out issues we should resolve long-term
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants