Better Hot Reload

[agentgt]

The trick of using JMustache for hot reloading has serious limitations.

1. JMustache does not fully implement v1.3 of the mustache spec
2. The reflective model of JMustache does not match the compile model and is generally more loose.
3. Only external templates can be reloaded
4. Model changes cannot be reloaded

The original plan was to create a reflective version of JStachio that implements the v1.3 spec as well as follows the compile time types instead of the instance types. However that only fixes the first two issues.

In a Mustache world especially JStachio world it is extremely common and almost best practice to change the model while you are developing the template.

So ideally we recompile the JStache model class on template OR model change. For things like JSP, Rocker or JTE this is much easier because the annotation processor does not need to run as well as they are basically just a straight translation to Java code. Because of the ambiguous nature of Mustache sections (could be a list, condition, or lambda) it is not a straight translation to Java and the types have to be analyzed.

Furthermore as @dsyer noted in #169 there are already reloading technologies such as Springs reload/devtool, JBoss modules (Jooby), JRebel and possibly Jandex or whatever Quarkus does so one option is to hook into the offerings to say if a mustache external template is modified its model would need to recompile/reload. There are also incremental recompile technologies that need to be dealt with as well such as Eclipse's JDT.

[dsyer]

I did some experiments with inotify. This works (running in the background at the same time as app with devtools):

$ while inotifywait src/main/resources/templates -e close_write; do sleep 1; find src/main/java
 -name \*Model.java -exec touch {} \;; done

For some reason I had to sleep a whole second before touching the Java source - I think that might be a devtools "feature" (it ignores changes that are close together).

[agentgt]

See also #200 for Gradle specific sort of related but not entirely hot reload.

[agentgt]

Like @dsyer I cobbled my own hack for reload as well using fswatch and mvnd.

I have a run script:

https://github.com/agentgt/petclinic/blob/main/petclinic/run.sh

1. Remove all class (.class) files that are JStache models (this could be smarter and actually look for the bytecode). I found touching the .java files not effective as mvn will not recompile them.
2. Run mvnd install (notice maven daemon)
3. java -jar target/output.jar
4. If previous process exits with exit code of 2 we repeat the previous steps

I have added an endpoint that will shutdown and make the exit code 2.

https://github.com/agentgt/petclinic/blob/main/petclinic/watch.sh

The watch script script will hit the shutdown endpoint on change of any file in src/main. That endpoint will shutdown with our special exit code of 2 to differentiate normal shutdowns.

The above works great on new hardware and fast startup stacks. It probably will not work great on slow hardware and giant monoliths.

I thought about trying to setup a maven plugin for the above but @wiverson on reddit let me know they already have a similar plugin.

I think this is generally better than the other two options of:

1. Reflective version of JStachio
2. Recompile models and reload with special classloader

Particularly considering that most folks picking up JStachio are probably new projects and thus probably have faster startup times (as well as modern JDKs are much faster at starting up now).

[dsyer]

Is there any way to do the byte code processing without using APT? APT just never works for me consistently. It sometimes works, but then I get stuck with the IDE in a state that it seems incapable of leaving where it won't or can't compile - maybe it's a JDT issue, so IntelliJ users never see it, but it's a huge problem for the rest of us. It's a massive drain on productivity. I mention it here because if the renderers could be created outside of the compiler it might be easier to control the reload. (Spring Boot went with a non-APT solution for its AOT processor partly if not mainly for this reason.)

Here's a PetClinic with JStachio: https://github.com/dsyer/spring-petclinic-htmx/tree/jstache. It works fine as far as I can tell, but the IDE refuses (usually) to compile the templates. It says there is a non-formattable field, despite the @JStacheFormatterTypes being there and working when you compile on the command line.

[agentgt]

maybe it's a JDT issue, so IntelliJ users never see it, but it's a huge problem for the rest of us.

It is not a JDT issue as I use Eclipse but rather a VSCode Java Extensions using JDT (you use VSCode right?). There are well known issues with annotation processing with VSCode JDT as noted by Micronaut: https://docs.micronaut.io/latest/guide/#vsCodeSetup

Eclipse largely works for me with one giant exception: You must not have JStachio open in the same workspace as project using it otherwise M2E APT will not work correctly and this is a well known issue as well: eclipse-m2e/m2e-core#1550 .

I say the above because that might likely be causing your problems and is tricky because I'm not sure how VSCode handles multiple projects open in workspaces (I assume all project folders open are in the same workspace?).

Byte code processing would be extremely difficult and still would not fix many of the problem (like knowing some random template belongs to a class).

The reflective version would be massively easier but it still suffers from the same problems as byte code.

The major issue with Eclipse for me other than the bug mentioned above is having to press alt-f5 (m2e reload) on projects where a template resource is updated because obviously Eclipse does not know which class(es) need to be recompiled. That could be fixed with an Eclipse plugin I suppose but my hack of just deleting the model classes and having maven incremental rebuild works.

As for other IDE and build systems:

Gradle and IntelliJ have far less issues. Gradle is especially smart as it knows to re run annotation processing for resources if configured correctly. It pains me to admit that as I'm more of a Maven Eclipse guy albeit I use all the IDEs and Gradle from time to time.

As for devtools:

In my Jooby version of petclinic I could not get jooby-run an analog to spring-boot-devtools to reload the annotation processed classes but I believe this is in issue with jooby-run. It uses classloader magic to work.

Ultimately the most reliable solution to reload is to physically restart instead of classloader magic like jooby-run does. I think spring-boot-devtools physically restarts the app now right so it should fair better? I wonder how JRebel fairs.

Regardless I feel your pain @dsyer . Hot reload is always tricky biz.

I will test your version of petclinic today with all the IDEs (Eclipse, IntelliJ, VSCode (both plugins)).

[dsyer]

I think you can fix that sample project by putting the @JStacheFormatterTypes on the main PetClinicApplication. Or maybe it's just in a happy phase and will bite me later. I haven't pushed that change to github yet so would be interested to hear if it works for you.

[dsyer]

UPDATE: If you use Gradle it works even in VSCode (so far for me anyway). I pushed a fixed build.gradle to the sample above. It only worked for me with the Redhat (standard) Java extension. The new one from Microsoft didn't work at all (couldn't compute a classpath even, never mind apply an APT).