A3 - Asset/data compilation without a full server restart - A case for Hot Module Replacement or similar

[stepanjakl]

Hello Apostrophe Team,

Our team is about to switch from A2 to A3. We are very excited! We only just began with learning about the structural changes and new processes in place.

Unfortunately, we already encountered one setback, which we believe affects the Developer Experience (DX).

With A2, we have been able to quickly update HTML (i.e. templates) and CSS files (via PostCSS) and refresh the browser with Browsersync (client-side). This takes only 1-3 seconds at most. When any changes are done to the JS files of any piece, page or widget (mostly configuration index.js files), the whole server has to restart via Nodemon. Restarting the Express server takes a bit longer, but it is fair enough as we understand the project needs to rebuild the data structures (server-side).

A3 needs to restart the Express server in all cases - for changes in any file. This adds extra development/wait time, even when you make a small template (.html) or style (.css) change and want to quickly test it or see the result. Moreover, from our understanding, as the projects grows in complexity and size, the wait times would increase - more information/structures to "crunch" through on every restart (?).

Looking a bit into the code, there is a Webpack module which takes care of the SCSS (only?) compilation. But this, in short, is bit too opinionated, we feel. The easiest and least opinionated approach would be the one from A2, where public files (as well as templates) are immediately available without the server restart.

I am going to give some suggestions (of concepts and ideas) for possible future implementation and how to (maybe) tackle the issue above.

Besides Apostrophe, our team is used to working with Vue.js/Nuxt.js combo ran through Vite (an alternative to Webpack). Vite, Snowpack and similar are no-bundler build tools that have almost instant server start up (this is a crucial aspect for us when selecting a framework to work with) and support the so-called Hot Module Replacement (HMR). HMR refreshes only component/module on the page for which the files changed - without a full page reload. Our understanding is that Apostrophe builds up a full page (or DOM) on the server a.k.a. Server Side Rendering (SSR). The final DOM is sent to the client/browser and rendered there. Why not then have a system which could replace parts of the DOM on the page programatically?

It is a bit of an ask. So, please excuse our bluntness. Apostrophe is a complex package. We understand that aforementioned approaches and/or techniques are not easy to implement.

We would certainly love to contribute somehow in this case. However, like I mentioned before, our understanding of Apostrophe's inner workings is not too deep still. There are certain coding practices that we wouldn't want to break.

We would be happy to get in touch and explain the intended DX/UX further if it is of interest.

Stepan,
Bohemica Team

[boutell]

Hi Stepan,

It's a reasonable question.

Restart times are actually pretty short on typical sites, but still, restarting the whole process with nodemon is clearly overkill if the only things that changed are in ui/src folders.

As you've probably guessed we're currently doing that because it definitely works, it doesn't introduce a lot of breakable parts and dependencies, and the restart times aren't bad unless you add the APOS_DEV=1 environment variable, which is just for when you're actively developing custom apstrophe admin UI and want our UI code to rebuild on changes too.

Also there's the logic involved in handling copying all the ui/src files into place in public/apos-build/modules on each build to satisfy various guarantees Apostrophe makes about how these files behave and keep the webpack loaders simple.

But that doesn't mean there aren't steps that could speed this up with less drastic changes. The most obvious and most likely to be accepted as a PR is to distinguish changes to ui/src, ui/public, etc. from changes that require nodemon, and rebuild just the asset bundle on those, and refresh the page. This would reuse steps and tasks we've already built without blowing up the whole approach so it is probably the quickest path to a performance improvement here.

It could be necessary to use the chokidar module to watch for changes directly in apostrophe for this, and also exclude these folders from the nodemon config in our boilerplates of course.

Or perhaps we could simplify the fix even more by configuring two instances of nodemon, one responsible for running the asset build task when ui/src or ui/public in any module changes, and one responsible for restarting the apostrophe application if other files change. That might not require any code changes to Apostrophe at all, except for a way to recognize the asset bundle has been rebuilt and refresh the page.

As I said though, build time is still pretty good, so our own clients aren't pushing for this so far. Thanks for acknowledging the time and effort involved — we'd welcome contributions and experiments along this line. There is also opportunity to accelerate this feature on our roadmap as part of a commercial partnership. Please reach out to me at tom@apostrophecms.com if you'd like to discuss that.

[stepanjakl]

Hi Tom,

Thank you for your answer.

I got your points, and yes it works absolutely fine as it is. It is just a little disadvantage in comparison to frameworks like Vue.js or bundling tools such as Vite.js. But then, Apostrophe is a (great) CMS - not a framework.

The issue with rebuilding assets in the ui/src, ui/public and other places, is that it concerns only styles, not templates - which are located inside the /modules folder. We really mostly want a quick page reloading for those. JS files make sense to restart the server as they deal with the data structures.

I think I am just pushing an idea for the future (breaking changes) version. It would just be lovely to have the server push the final DOM on any file change (including .html/.js) and refresh (not reload) the page (maybe via a Websocket or something).

So, it's food for thought for now :)

[boutell]

Oh, good point about .html files changing. Yes, there are faster ways to deal with that too. Nunjucks has some features around that but we haven't had the best experience with their thoroughness in the past. We could address it ourselves at some point.

…

On Fri, Feb 11, 2022 at 12:45 PM stepanjakl ***@***.***> wrote: Hi Tom, Thank you for your answer. I got your points, and yes it works absolutely fine as it is. It is just a little disadvantage in comparison to frameworks like Vue.js or bundling tools such as Vite.js. But then, Apostrophe is a (great) CMS - not a framework. The issue with rebuilding assets in the ui/src, ui/public and other places, is that it concerns only styles, not templates - which are located inside the /modules folder. We really mostly want a quick page reloading for those. JS files make sense to restart the server as they deal with the data structures. I think I am just pushing an idea for the future (breaking changes) version. It would just be lovely to have the server push the final DOM on any file change (including .html/.js) and refresh (not reload) the page (maybe via a Websocket or something). So, it's food for thought for now :) — Reply to this email directly, view it on GitHub <#3655 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27I7NOQRBHHGYFT5HRTU2VDKHANCNFSM5OEA6GDQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.*** com>

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[stepanjakl]

https://github.com/readme/featured/server-side-languages-for-front-end

[boutell]

Interesting. Apostrophe's async components and api routes pair well to do similar things. Of course we don't need a special compiler to allow you to write the same language in the browser, since the language was born there 🤪

…

On Wed, Feb 16, 2022 at 8:58 AM stepanjakl ***@***.***> wrote: https://github.com/readme/featured/server-side-languages-for-front-end — Reply to this email directly, view it on GitHub <#3655 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27MIQK2S7NRNTWUPH7TU3OUQZANCNFSM5OEA6GDQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.*** com>

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[stepanjakl]

Definitely, it just indicates where the "world wide web" may be going. The site can be built up partly on the server and partly on the client. Moreover, there are more tools that allow to dynamically change the content through AJAX requests, fetch method or Websockets. Having JavaScript both on the back-end and front-end is one of Apostrophe's beauties - no doubt.