Replies: 1 comment
-
My mental model is "pure composition" but for the reasons you listed, it might lead to the brittlest implementation of Sequoia. That being said I guess it's the one that (in theory at least) would be the easiest to graft a new assumption to add settings. "Pure inheritance" is aligned with our mathematical definition and makes a lot of sense theoretically, but my guess is grafting new assumptions that are not at the bottom of the tree would be a nightmare. that being said, can you get more general than RL? Codewise I guess "Hybrid" makes sense. I think we could view Sequoia v1 as an MVP and decide later if we need a v2 depending on the demand for Sequoia. On the TODOs:
Ok actually maybe main_loop pseudocode is good, as it's less about understanding the backend and more about understanding the settings. love the Pytorch Lightning example. Picture = 1000 words. This solves bullet point 3. |
Beta Was this translation helpful? Give feedback.
-
I have a feeling that Sequoia isn't as clear to new users as it could be.
This issue is meant primarily as a way for me to list out some of the design challenges I've been facing, as well as to share them with you all. I would love to hear other perspectives on this. I might update this thread as new ideas / suggestions pop up.
Goals / Things to address:
New users should get a clear understanding of the "main loop" of any Setting in Sequoia, by just opening the corresponding source file.
It should become evident to users why polymorphism/inheritance makes sense when talking about evaluation "Settings" in ML. It shouldn't feel over-engineered.
It should be more clear what is gained from making a distinction (in code) between the roles of the
Setting
vs theMethod
, as well as where abstractions likeconfigure()
,fit
,get_actions
come from, and why they are useful.Need to decide what kind of inheritance hierarchy (tree) we want to construct long-term:
"pure inheritance":
"pure composition":
(e.g.
PassiveAssumption
+IncrementalAssumption
+TaskAwareAssumption
-->TaskIncrementalSLSetting
).For example this
IncrementalAssumption
might do need to do something slightly differently when creating the environments from each task depending on if the concrete setting is a passive (SL) or active (RL) Setting."Hybrid" (closer to current setup):
IncrementalSetting
(meant to act as theIncrementalAssumption
from above), which doesn't really make sense:ContinualRLSetting
, for instance, should be more 'general' than the IncrementalSetting, since it doesn't have task boundaries, and the environment shifts continuously over time. However, it currently subclasses IncrementalSetting, which is confusing. (Might want to make a new Issue/PR for this)Potential TODOs / things to address:
class_incremental_setting.py
is getting too long!class_incremental_setting.py
continual_rl_setting.py
is getting too long!Let me know what you think!
Beta Was this translation helpful? Give feedback.
All reactions