MutationObserver flag to observe mutations in (open) shadow trees to facilitate polyfills of HTML syntax

[LeaVerou]

What problem are you trying to solve?

It is currently exceedingly difficult (and slow) to implement polyfills of certain features that affect HTML syntax across both light and shadow DOM because it requires observing mutations in shadow trees as well, which is currently incredibly difficult, and all solutions have prohibitive performance.

Some examples:

• Any of the custom attributes proposals (example)
• Any new HTML element or attribute

What solutions exist today?

The current options are:

1. 🤷🏽‍♀️ The shrug method: Use a regular document-wide mutation observer and simply accept that the polyfill will not work correctly when shadow trees are involved
2. 🩹 The monkey patch method: Wrap HTMLElement.prototype.attachShadow() with code that calls mutationObserver.observe() on it, like so:

let originalAttachShadow = HTMLElement.prototype.attachShadow;
HTMLElement.prototype.attachShadow = function(...args) {
	let node = originalAttachShadow.call(this, ...args);
	mutationObserver.observe(node, options);
	return node;
}

3. 🤕 The painful method: When the mutation observer detects new elements, check if they have a shadowRoot and call observe() on that too. Loop over all existing elements and attach mutation observers to the shadow roots of those with one.

As you can see, all of these are pretty terrible. They are slow, obtrusive, and wasteful. Can we do better?

How would you solve it?

Option 1

This is the most straightforward, lowest effort option: Simply add a new option to MutationObserverInit to observe (open) shadow trees. Name TBB, some ideas might be shadow, shadowRoots.

The MVP could be a boolean option, which could later be extended to more granularity (e.g. how many levels deep? What types of shadow trees (open, open-stylable etc)).

Option 2

It could be argued that MutationObserver.prototype.observe() is the wrong tool for the job. It was designed to observe mutations under a specific root, and has been since contorted to cater to these types of use cases. Perhaps a different method could be used to signal "observe this everywhere you can", e.g. MutationObserver.prototype.globalObserve().

Anything else?

No response

[WebReflection]

As author of all polyfills from v0 to current state, I fully support this proposal. My workaround is not super slow but it requires me patching the global attachShadow method on native classes which is a practice I usually avoid at all costs and something I've been advocating against for just about forever ... yet polyfill gotta polyfill, so that if this new proposal is the only thing that might need a greedy polyfill around until everyone is onboard, it's definitively worth the effort to me.

On a side note: the closed shadow might still be addressed somehow and I feel like the elephant in the room here is that we don't have a way for Websites authors to decide what is trusted as script that could patch natives, and what isn't ... CSP is not a great answer to this disambiguation, but maybe this discussion should be done in other venues.

[trusktr]

An alternative to this might be to be able to observe the composed tree, because in some cases we might want to know the nodes that will participate in the final output:

I've had to implement composed tree tracking so that webgl-rendered custom elements would render correct results when composed via slots into shadow roots. I'd like to release some of these tools separate libs, maybe it'll help ideate.

Even without custom rendering, it might be beneficial to react to composed nodes, avoiding extra work for trees that are not active. F.e. suppose a client-side route switcher modifies slot attributes to show/hide content, and we want to observe only content in the active tree.

[trusktr]

Over in

• observing the composed tree synchronously webqit/realdom#3

@ox-harris implements options.subtree = 'cross-roots' in the realdom lib for synchronously observing added/removed elements across shadow root boundaries, and gets us a little closer to synchronous observation of the composed tree (f.e. detect composed children, composed parent, etc).

Not only is the usage simpler than with MutationObserver (because MO events fire out of order) and lead to simpler code, but composed tree tracking code will also be as simple when that's ready (ComposedChildrenObserver, ComposedParentObserver, ComposedMutationObserver, etc).