Initialization at build time and run time (follow up)

[dmlloyd]

It is time to solve the initialization problem comprehensively.

I have created a PR (#760) to switch standard class initializers to be build time only. This is a pragmatic decision: having two different models is getting to be too difficult to maintain. But it does leave a largely unsolved problem in that we have no good solution for initializing things at run time.

I've re-listened to the virtual meeting we had in May (#523) where we discussed several aspects of initialization. Apart from things we discussed on that call which have since been completed, we identified some requirements:

• Allow the user to express their intent clearly
• Conform to the existing specification whenever possible/practical
• Most if not all of the JDK should be available at build time
• Be able to start up with a premade heap which can be loaded or mapped quickly into memory

And nice-to-haves:

• Develop something which can be proposed to the JVM, JDK, and/or platform specification(s) with some nonzero likelihood of acceptance

The mechanisms that were discussed were:

• Heuristically determine classes that can be initialized at run time
• Field annotations to determine which parts of a class to initialize at run time
• Add a separate run time initializer to each class
• Graal's substitution mechanism
• Some variation on lazy constants or lazy field initializers where the cached constant value(s) are dropped to be re-computed at run time
• Forget build time heap completely

Since then we've made quite a lot of progress of course. But despite floating a lot of ideas, we never answered the basic question of how to handle run time class initialization.

So in order to get something working that we can use ASAP, I'd like to propose an initial solution with multiple parts:

• Define a second type of InitializerElement which applies to run time initialization and call it a run time initializer, and develop a parallel set of initialization rules which apply to run time. I think there are some potentially creative things we can do with these rules but to start with we'd want to keep it as simple as possible; the existing run time initialization support code would likely be largely reusable for this (though in the near future we might want to reevaluate how we are inserting and using class init checks).
• Add new a member flag or flag(s) to signify whether a member is available at build time, run time, or both.
• Re-introduce the "patcher" API and implementation which provides a means to express a run-time aspect for a primary class with the following initial requirements:
  • Allow the run time aspect class to have an initializer; this initializer is added to the class as the class run time initializer
  • Allow static fields which exist on the primary class to be aliased on the run-time aspect class; if these fields are written from the run time initializer, then that field is marked as being re-initialized. The field is present but zeroed in the initial heap and re-initialized using the run time initializer.
  • Allow other aliased static fields to be marked as re-initialized even if no static initializer replaces the value, preventing the corresponding field values from being serialized on the heap.
  • Allow new static fields to be defined which may be initialized solely by the run time aspect class.
  • Allow static (and possibly private instance) methods to be defined in run time aspect classes
    • If the method exists in the primary class, the given method will replace the primary class method at run time
    • Otherwise the method only exists at run time and cannot be called from build time code (an error will be raised)
    • The method will be compiled and thus participates in reachability and initialization analysis
    • Note that bootstraps for invokedynamic must still be computed during build time so that any generated lambda classes are properly constructed and available
  • Run time aspect classes may not be loaded and will be treated as "not found"
  • Provide a file, index, or other means of establishing the association between primary and run-time aspect classes
• Interfaces also have initialization sequences, so these points all apply to interfaces as well

We may also need to consider some or all of the following capabilities in the future:

• Support for patching the build time aspect of a class including generalized replacement or addition of any method
• Support for reinitialization at an instance level (an equivalent to the readObject/readResolve mechanism in serialization)
• Support for pre-serialization hooks (an equivalent to writeObject/writeReplace in serialization)
• Support for on-startup hooks which run in the run time image immediately (could be a method, could be a class runtime initializer marked with some kind of "eager initialization" flag)

@dgrove-oss had a clever take on build time thread creation, wherein we enqueue the created threads and start them as part of a run time initialization sequence. This is in contrast to the GraalVM approach wherein one may generally start threads during build time, with the stipulation that all threads must be exited before the compiler can complete. Deferring thread start to run time might introduce some incompatibilities but overall I think it's a better approach.

Once we have a class initialization solution working, we have to address the next problem: OpenJDK initializes itself using a series of methods on java.lang.System. Since these methods are unaware of the separation between build time and run time, the tasks performed are generally a mix of tasks that can or should be done at build or run time. We can bodge around these issues to an extent - maybe enough to get things up and running - but we should consider creating two new proper initialization sequences for build and run time respectively using the mechanism we define for that purpose.

[dmlloyd]

On a pragmatic note, I think that several run time initialization tasks should be handled using global constructors. Particularly, heap setup and deserialization, GC initialization, and any eager run-time tasks (such as thread starting) should be done from here. This allows us to support usage of program images as a dynamic library, which will surely be a future requirement.

[DanHeidinga]

Thanks for writing this up David. It's an important topic for us to get right.

• Add new a member flag or flag(s) to signify whether a member is available at build time, run time, or both.

Is this intended for both methods and fields? From the writeup, it definitely applies to methods (the new <rtinit> initializer at minimum) but fields are less clear. Fields need to be able to be (re)initialized at runtime but that sounds different than "available". Maybe I'm overthinking the terms?

Moving from the specific to the general:

One of the things this approach does is follow the existing JDK specification for <clinit> by introducing a second <clinit> (`?). This lets us keep the same placement of init checks, though triggering a different method, and is fairly easy to map from one to the other.

The downside is it preserves the "soupy" nature of existing <clinit> which mixes together the initialization actions of multiple fields. What if we used the same patcher mechanism to associate a unique initializer with each static field that needs runtime (re)initialization?

The runtime aspect class can then define multiple methods that each initialize one of the required fields. This lets us start to break the existing soupy clinit into finer grained operations. Hopefully helping explore where / how lazy final static fields could help with this problem.

Not much needs to change from your proposal to make this happen - I think it's mostly in the patcher and in the placement of the runtime initialization operations.

We can have the existing runtime class init model call each of the field initializers on the runtime aspect class so we don't need to add per-field init checks (though it may be an option in the future)

[dgrove-oss]

The <rtinit> model appeals to me as because it "fits" with the existing practices in Java (and thus is relatively unsurprising). Without thinking too deeply, I like making <rtinit> act more like interface initialization: accessing a class member just triggers local <rtinit>.

In terms of making the association between the primary class and its runtime aspect, can we just use the same convention we are using for providing "native" methods. Given a class C, it optionally provides native methods in C$_native and optionally provides a runtime init aspect in C$_rtinit?

[dgrove-oss]

Another thought is that we could further distance ourselves from the disadvantages of class init by treating run time aspect initialization more like interface initialization in that only that class gets initialized/there is no automatic initialization propagation to superclasses.

I like this idea. It may result in more runtime init checks though as we'd have to have them for super-class runtime field access as well, rather than having that be a consequence of initializing the subclass. Actually, the checks may end up being the "same" if we're supporting per-field initialization as each access (not provably following a prior access) would need the check.

The performance aspects of these checks is something we'll have to keep in mind as noping them in the statically compiled code requires writable code segments though the dynamic JVM will have less of an issue with the model.

If we define <rtinit> as only applying to the class (and not rippling up the hierarchy) then it will be easy to elide the check for all the classes that don't have <rtinit> methods. I expect this will mitigate the runtime cost quite a bit vs. the <clinit> checks we are used to seeing.

[DanHeidinga]

Agreed. The cost of checks for <rtinit> is likely lower than the cost of per-field initializers. I still conceptually like the model that separates out a field (or set of fields) and assigns them a different initialization lifecycle. Starting with <rtinit> lets us progress and doesn't close the door on per-field in the future so no objection there.

[dmlloyd]

In terms of making the association between the primary class and its runtime aspect, can we just use the same convention we are using for providing "native" methods. Given a class C, it optionally provides native methods in C$_native and optionally provides a runtime init aspect in C$_rtinit?

Yes, possibly, though the mechanism will probably have to be different (e.g. we'd want to load the class file without defining the type; we'd want the class to not be otherwise loadable; and, it probably has to happen while the parent type is being built rather than at the time of first use like native binding).

If we define <rtinit> as only applying to the class (and not rippling up the hierarchy) then it will be easy to elide the check for all the classes that don't have <rtinit> methods. I expect this will mitigate the runtime cost quite a bit vs. the <clinit> checks we are used to seeing.

Yeah, I think the majority of classes won't have run time initialization at all when all is said and done.

Agreed. The cost of checks for <rtinit> is likely lower than the cost of per-field initializers. I still conceptually like the model that separates out a field (or set of fields) and assigns them a different initialization lifecycle. Starting with <rtinit> lets us progress and doesn't close the door on per-field in the future so no objection there.

One important key differentiator between the two approaches is that with run time init as an aspect of the class, one might expect e.g. new SomeClass() to cause SomeClass to be initialized, whereas with run time init as an aspect of a field or group of fields (even all the fields in the class), one would not be likely to have that expectation (because why have two groups of fields with distinct initializers if they are always initialized together anyway?).

I am however still partial to having the init apply to fields (even if we start off only supporting one group of fields) because it would mean fewer init checks overall.

[dgrove-oss]

Documenting that we ended up implementing an <rtinit> design where a set of static fields from the original class is identified as being runtime initialized. These fields are serialized to the build time heap as containing the type-appropriate zero value and accesses to the fields are guarded with an <rtinit> check. When dynamically accessed for the first time, the <rtinit> method is invoked.

We use the patcher to identify fields for runtime initialization. A patch class is associated with the original class with a @patch or @PatchClass annotation. The patch class is marked with a @RunTimeAspect annotation. All static fields that are added/aliased by the patch class are then associated with the same <rtinit> method (which is the <clinit> defined by the patch class).
The rtinit check is implemented using instances of a Once class that are serialized to the initial heap are referred to directly as ObjectLiterals in the generated code for the static field access. The inline check is relatively efficient (load, compare, branch).

[DanHeidinga]

We may also need to consider some or all of the following capabilities in the future:

• Support for patching the build time aspect of a class including generalized replacement or addition of any method
• Support for reinitialization at an instance level (an equivalent to the readObject/readResolve mechanism in serialization)
• Support for pre-serialization hooks (an equivalent to writeObject/writeReplace in serialization)
• Support for on-startup hooks which run in the run time image immediately (could be a method, could be a class runtime initializer marked with some kind of "eager initialization" flag)

As much as I've wanted to avoid saying this - what we're building is the natural analogue to serialization and I think you're 100% right that we'll need the same kinds of mechanisms to support it as serialization has today.

The more of this work we can do at build time - even if it's just building lists or tables of addresses needing to be patched at startup by the hooks - the better. We may be able to make these hooks efficient by doing the necessary heap walks to identify what needs to be replaced at build time.

[dgrove-oss]

Documenting that we have implemented @SerializeAsZero and @Serialize{Boolean,Integral,FloatingPoint}As annotations to allow simple substitutions of the contents of a field when it is being serialized.

[DanHeidinga]

Looking at the threading aspects:

@dgrove-oss had a clever take on build time thread creation, wherein we enqueue the created threads and start them as part of a run time initialization sequence. This is in contrast to the GraalVM approach wherein one may generally start threads during build time, with the stipulation that all threads must be exited before the compiler can complete. Deferring thread start to run time might introduce some incompatibilities but overall I think it's a better approach.

I think this is a clever idea but I'm not sure it's workable. We may find that as we execute more code at runtime, we'll run into deadlocks or permanent waits due to delaying thread operations. The first <clinit> method we BTI that has parallelStream() in it will block indefinitely if we don't allow creating threads at build time (and unfortunately managing their lifetimes).

As much as possible, we'll want to adapt existing <clinit> to move thread creation to the new runtime init features.

I think we either need to adopt Graal's "threads must be exited" rule or come up with a model to quiesce the threads in way that we can restore (restart?) them at runtime. The CRIU efforts have been looking at a hooks api for this - maybe it's another place where we need to look at serialization read/write hooks?

[dmlloyd]

Looking at the threading aspects:

@dgrove-oss had a clever take on build time thread creation, wherein we enqueue the created threads and start them as part of a run time initialization sequence. This is in contrast to the GraalVM approach wherein one may generally start threads during build time, with the stipulation that all threads must be exited before the compiler can complete. Deferring thread start to run time might introduce some incompatibilities but overall I think it's a better approach.

I think this is a clever idea but I'm not sure it's workable. We may find that as we execute more code at runtime, we'll run into deadlocks or permanent waits due to delaying thread operations. The first <clinit> method we BTI that has parallelStream() in it will block indefinitely if we don't allow creating threads at build time (and unfortunately managing their lifetimes).

I agree to an extent. While delaying thread start is definitely valid in terms of the thread contract, it may be (but is not always) contrary to the user's intention.

We may want to look at a compromise approach where we can identify certain call sites which are allowed to start threads at build time. I believe that in almost all cases, threads started early are intended to exist at run time (I'm thinking about cleaner threads, reference handlers, UI work threads, request-handling thread pools, etc.).

Additionally we might want look at JDK things like parallel streams/fork-join and manage them directly in the interpreter to use the host's fork/join pool directly or one specifically created for the interpreter, and otherwise treat it like any other thing which is not native-image-friendly.

As much as possible, we'll want to adapt existing <clinit> to move thread creation to the new runtime init features.

Sure, that's reasonable, and additionally we may have some patching capabilities we can utilize.

I think we either need to adopt Graal's "threads must be exited" rule or come up with a model to quiesce the threads in way that we can restore (restart?) them at runtime. The CRIU efforts have been looking at a hooks api for this - maybe it's another place where we need to look at serialization read/write hooks?

This seems like a much larger and more complex conversation that might be the subject of its own discussion! 🙂

[DanHeidinga]

We may want to look at a compromise approach where we can identify certain call sites which are allowed to start threads at build time. I believe that in almost all cases, threads started early are intended to exist at run time (I'm thinking about cleaner threads, reference handlers, UI work threads, request-handling thread pools, etc.).

The threads started by the language are definitely a special case and may be easier to deal with than user-initiated thread creation. There's a jdk.internal.misc.InnocuousThread class that may help identify many of these cases.

We may want to look at a compromise approach where we can identify certain call sites which are allowed to start threads at build time. I believe that in almost all cases, threads started early are intended to exist at run time (I'm thinking about cleaner threads, reference handlers, UI work threads, request-handling thread pools, etc.).

This depends on how much of the work can be done at build. Take the example of the JSON library doing all the parsing of a config file at build time and then being able to DCE the library so it's not included in the final image. If that library had spun up a thread for processing (unlikely with a json processor but others libraries? Maybe) and we delayed it's creation till runtime, we'd impede the processing and have a larger image than necessary.

As we enable more and more code to run at buildtime, I think we'll see every possible feature be used and needed at buildtime, including threads. And I can even see a case to be made for a buildtime gui that allowed the user to specify settings that get baked into the final image. It's not the typical usecase when we look at Quarkus but once BTI is available, users will do strange and wonderful things with it.

[dgrove-oss]

I'm pretty sure we do not want to support carry "running" threads from build time init to runtime. Very hard to do on identical platforms and I don't want to think about what this means for cross-compilation scenarios.

I think once we get our <rtinit> story sorted out, we could either continue exploring deferred start semantics or do something maybe less surprising where threads started in <clinit> run as VmThreads in the interpreter and must exit before the ADD phase of compilation completes while threads started in <rtinit> just get started normally at runtime. The various JDK thread pools that we are currently deferring might be dealt with by carefully structuring our own build-time and runtime System.initPhaseN methods to not start threads at build time. This might require more changes to core JDK classes (and to other frameworks) to make them build-time init friendly. The appealing thing about the deferred start strategy is that (for better or worse) it more aggressively reinterprets existing <clinit> methods in a way that is build-time init friendly.

[DanHeidinga]

+1 to not carrying running threads. Though I will mention that with Loom around the corner, we may have to support carrying "virtual threads" from BT to RT.

The appealing thing about the deferred start strategy is that (for better or worse) it more aggressively reinterprets existing <clinit> methods in a way that is build-time init friendly.

This is a reasonable place to start especially if we can special case ie: the forkjoin default thread pool and some of the Cleaner-type threads. My point is that this isn't a place we can finish - our end goal needs to look beyond this.

[dgrove-oss]

Documenting that we are currently taking the approach of using <rtinit> blocks and @ReplaceInit annotations to restructure JDK class initializers so that Thread creation is deferred. The BuildTimeHeap will report an error if a Thread instance is serialized (currently more or less by accident, but eventually in a more intentional fashion: see #1049).

[dgrove-oss]

I've written up the implemented design in a wiki page: https://github.com/qbicc/qbicc/wiki/Class-Initialization-in-qbicc