Mental model and idea behind keystone's tweaks/patches related to mobx observe

[PEZO19]

Hi there!

@xaviergonz, may I ask for your help to clarify the idea/rationale/reasoning/big picture behind keystone's tweaks and patches and its relation to mobx observe?

My understanding is, that keystone tries to provide the (minimal) changes of the living (sub)tree between two states.
For that,

1. partially, it relies on mobx observe
2. partially, it relies @modelAction and tracking patches of the living tree internally InternalPatchRecorder

However the big picture / rule of thumb is not clear to me: which one is preferred (when?) OR necessary and why?

It is not clear to me that:

1. what are "used features/properties" AND the "limitations" of observe
2. what are "used features/properties" AND the "limitations" of internal patch tracking

I have encountered these bits too and it is hard to get my wrap around them:

1. Mobx docs: "don't use observe" Isn't that a problem, that keystone utilizes it? Won't it break?

   ⚠️ Warning: intercept and observe are low level utilities, and should not be needed in practice. Use some form of reaction instead, as observe doesn't respect transactions and doesn't support deep observing of changes. Using these utilities is an anti-pattern. If you intend to get access to the old and new value using observe, use reaction instead. ⚠️_

   It also says "observe doesn't respect transactions and doesn't support deep observing of changes": is that something which keystone attempts to overcome on purpose? If so, what is the difference: what can keystone do what mobx observe does not do (on purpose...)? How is it related to keystone's transaction implementation (UndoManager/flow/async flow)?

2. Why is it a must (from keystone standpoint) to rely on observe?

3. It is mentioned by Michel Weststrate that it might be killed.

I have read the docs, multiple times, but the main goal/rationale/big picture/idea is missing to me. Because of that, I can not really say what are the preferred way for me do a "custom tweaked object/model" if it is necessary for my app or in theory I shall be able to live without custom tweaked objects for my business logic.

One exact question, but is related to the missing mental model:

Let's say I want to have a Set and a filteredSet, but for the filtered: without using @computed/canonical mobx, because the Set is going to get very big (let's say millions of elements). Instead, I want to be able to listen to the changes/JSONPatches and write a "custom" filterChangeHandler which "translates" from "changes" to the target objects API calls. It is not clear to me if I shall try to handle that in mobx world (if so, which keystone abstractions to use, we have both asSet, both ArraySet) and use observe myself OR somehow use (Internal)Patches, but it is not clear to me how I can/ how is it preferred to "intercept" them before emitted globally so I can use the intercepted patches to make downstream changes. This issue is partially behind the rationale of #514.

Can you please help be out what is preferred and what is not preferred/prohibited in keystone related these issues?

@sisp Many times I had the exact questions in my mind as yours in the issue section, that is why I am poking you, I hope you don't mind.