Document Property Introspection for Deserialization

[cowtowncoder]

(note: inspired by #2992)

Currently -- as of 2.15 development starting -- the Number ONE priority for me is to rewrite Property Introspection logic to tackle issues that stem from the problem of "regular" -- getter/setter/field based -- properties, and "creator" properties (arguments to Constructors and/or static Factory methods) -- are not properly linked. This results in cases where some handling does not work the way users would expect: specifically, annotations on Creator properties and non-Creator ("regular") properties are not linked the way they should. Or, in case of Property (re)Naming, some functionality does not work on certain properties (generally, Creator properties).

The fundamental reason for this is that the discovery (introspection) of "regular" and "creator" properties is handled by 2 mostly separate code paths: one that starts with "regular" properties, does some processing, and only then attempts to locate "creator" properties, linking information (like annotations) at the point when some processing has already been done.
Worse, there are two phases to creator discovery:

1. First, early discovery for explicitly annotated (@JsonCreator, or @JsonProperty for parameters) constructors/factory methods
2. Second, later discovery of "implicit" creators (constructors only) based on heuristics

Of these, creators from (2) do not get their properties correctly linked to possible Field / Method based properties.

What needs to happen is that the order is changed a bit -- it should start with Creator (and Creator property) discovery, followed by "regular" property discovery, and mostly deferring handling until both have been properly collected and merged.
With this, a big bunch of open issues would be resolved: not the least ones affecting Java 17 Record types.

What I hope to do, next, is to outline how the processing works in detail: it is not possible to change the processing until understanding current flow and constraints -- some of types used (say, BeanDescription) are existing API types and cannot be changed in arbitrary ways -- whereas others are internal types.
After some documentation effort I may have time to start refactoring things, or maybe others want to try out changes as well.

But first things first: documenting what actually happens as of today (2.14.x).

[JooHyukKim]

Should we revisit the progress of the Property-Introspection rewrite?
It appears to be only blocker for the 3.x release, yet its advancement is not seen.

I'm just concerned about how this delay might impact the wider Jackson projects and usage.
If efforts are already underway that I'm unaware of, that's great.
Otherwise, perhaps we should focus on this task and organize our efforts.

I believe many volunteers, myself included, are willing to contribute while trying best to create little overhead.

cheers

[JooHyukKim]

@JooHyukKim one other idea: I think we could also do a very quick 2.17 release -- there has been tons of fixes, and good clean up.

True, there's been a bunch of fixes.

[JooHyukKim]

If it's possible to do the rewrite in one go (single minor release), I think we can even consider sacrificing skipping the documentation part then document afterwards. Skipping documentation may be an option only because....

• We have descriptive comments around codebase (e.g. // [databind#1234] Since... because...), which sort of served(at least for me) as document
• Code is easy to follow.
• We have received a bunch of tests that failed initially, that we fixed

[JooHyukKim]

Anyway, execution plan would be if we do push 2.17 and work on the rewrite in 2.18 ...

1. Start discussing scope/out-of-scope
   • Estimate how big would be?
   • Can you off-load some works to others?
   • etc...
2. Release 2.17 (only right before we start working on rewrite)
3. Do the rewrite

[cowtowncoder]

To me, Documentation here would be a pre-requisite: writing out how things work now. Meaning that before rewriting, would be necessary to outline how the process of introspection works now, and how it should work after changes.
So I don't mean end-user/developer documentation... but more like writing down the intent. Code alone is hard to follow individually, but more importantly is not shared understanding.

As to scope, I really don't know. It may (I hope!) end up being less humongous task if all goes well.
But it is not easily splittable into concurrent pieces either, I suspect.

Main goal, to me, really is to solve the problem as I originally described: make all properties -- "regular" (field-/method-backed) and "Creator" (constructor/factory method parameter based) be linked, so that annotations work even when Creators are auto-discovered. To do that, the order of introspection logic needs to be changed to start by identifying Creators to use, and only then consider "regular" properties. This should help with Record types, solve most issues.

A secondary goal (or possibly follow-up) would be improving customizability of Creator discovery for modules, in particular Kotlin- and Scala modules.

[JooHyukKim]

Right, documentation does seem neccessary and even help with scoping perspective.
With documentation as a must, question then would be whether to release 2.17 "before" start documentation or "after"

[cowtowncoder]

Started writing:

Deserializer Discovery

which -- I realized -- is more a background document, explaining process surrounding discovery of [Basic]BeanDescription, the actual part to be rewritten. But I think it makes sense to write that as the context, to see how property introspection ties into the overall process of constructing BeanDeserializer.

[cowtowncoder]

Hmmh. Odd, works for me. But full link is:

https://github.com/FasterXML/jackson-databind/wiki/Deserializer-Discovery-2.x

[JooHyukKim]

Seems to occur only in mobile browser, 🤔 thank you tho!

[JooHyukKim]

How's the work going, @cowtowncoder ? Just checking on, if there is anything I can help with 😆

[cowtowncoder]

Thanks for asking. :)

Sigh. Not finding much time for digging deep to see where to start.
If you want, edits to that Wiki page (fixes etc) could be useful.

But otherwise need to figure out steps on refactoring discovery process, to start with Creators (including bit more fine-grained classification for Explicit (annotation-indicated), Canonical (default for Records, probably for Kotlin/Scala too), visible (preferred)).
And then proceeding to find regular properties, link things.

[JooHyukKim]

!!! This is where Property Introspection occurs !!!

How about I try documenting the property introspection part? Probably start at the marker you left.
If I do, I would do it PR style against Jackson repo for better review-ability + comments

[cowtowncoder]

Created #4515 as umbrella issue for actual work (I am feeling over-optimistic about getting to start this, right after releasing 2.17.1 any day now)

[cowtowncoder]

I think documentation side is sufficient; significant work completed for #4515.