A3 assets

[myovchev]

Are we able to publish module css/js assets as separate files in A3 root public/ folder as it was done in A2 and if yes how?
I have this working HTTP path with A2:
/modules/@corllete/apos-ds-pages/css/bundle.css which was published from location (bundle) node_modules/@corllete/apos-ds/lib/modules/apos-ds-pages/public/css/bundle.css.

I was going to fix the paths (removing lib/) after I see styles and frontend js is back on the page. Could it be part of the problem?

From what I read in the alpha docs, ui/public assets (.js and .css) will be automatically bundled. So I have 2 issues:

1. I can't see it work. There is no trace of module owned JS/CSS in the front-end bundle, so I guess I'm doing it wrong (although reading the asset module code everything looks OK on first glance)
2. I really need separate .js and .css files published to the public folder due to the nature of the project I'm working on

[boutell]

You mention the root public/ folder, but I think you're actually talking about the module's public/ folder. We don't currently directly serve static files from public/ of every module. in 3.x. However, in the 3.0 branch (not the published alpha) we currently incorporate all .css and js files found in the ui/public subdirectory of any activated module into separate CSS and JS bundles. These are incorporated with no preprocessing — no SASS, no LESS, no Babel for JavaScript, etc — because developers tend to have their own preferences for building assets that will be used on the front end. If you need to serve other types of files in a module you could potentially use `apos.express.static` yourself. We haven't committed either way on whether we'll do something like that ourselves.

…

On Mon, Feb 8, 2021 at 11:53 AM Miro Yovchev ***@***.***> wrote: Are we able to publish module css/js assets as separate files in A3 root public/ folder as it was done in A2 and if yes how? I have this working HTTP path with A2: ***@***.***/apos-ds-pages/css/bundle.css which was published from location (bundle) ***@***.***/apos-ds/lib/modules/apos-ds-pages/public/css/bundle.css . I was going to fix the paths (removing lib/) after I see styles and frontend js is back on the page. Could it be part of the problem? From what I read in the alpha docs, ui/public assets (.js and .css) will be automatically bundled. So I have 2 issues: 1. I can't see it work. There is no trace of module owned JS/CSS in the front-end bundle, so I guess I'm doing it wrong (although reading the asset module code everything looks OK on first glance) 2. I really need separate .js and .css files published to the public folder due to the nature of the project I'm working on — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2716>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27IFSTCHSBO6LBXMQ7TS6AJHPANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

This is exactly what the docs says (and exactly what I need) but I fail to understand how it works. Are they exposed directly as express static? Is it required the build core task to be executed in order to publish them?

Let's say we have installed @namespace/product module and inside @namespace/product/ui/public/js/bundle.js
What would be the asset url and what exactly happens?

[boutell]

Any JS file in that folder will get pulled directly into the main .js file that is served to the browser, you do not need express for .css or .js files. This happens in the@apostrophecms/asset:build task.

…

On Thu, Feb 11, 2021 at 11:17 AM Miro Yovchev ***@***.***> wrote: This is exactly what the docs says (and exactly what I need) but I fail to understand how it works. Are they exposed directly as express static? Is it required the build core task to be executed in order to publish them? Let's say we have installed @namespace/product module and inside @namespace/product/ui/public/js/bundle.js What would be the asset url and what exactly happens? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27NC4MOTNGBNE5B5PGTS6P7J3ANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

It's what I thought is happening. In my extremely custom case I need those files separately published, because I need them to NOT interfere with the Apostrophe core at all. I understand it's something which most probably no one would ever need so I'm going to manually copy them to the public folder via internal module build task. I might also investigate how remote asset publish (CDN) works.

Thanks for your time!

[boutell]

Is this about the order of loading? Apostrophe should load them in the order modules are activated in app.js, with the core module assets coming before that. If not we have a bug.

…

On Sat, Feb 13, 2021, 5:38 AM Miro Yovchev ***@***.***> wrote: It's what I thought is happening. In my extremely custom case I need those files separately published, because I need them to NOT interfere with the Apostrophe core at all. I understand it's something which most probably no one would ever need so I'm going to manually copy them to the public folder via internal module build task. I might also investigate how remote asset publish (CDN) works. Thanks for your time! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27IF2VU444KSIKA5LFTS6ZJCZANCNFSM4XJKWT2Q> .

[myovchev]

No, it's about a really custom/rare requirement but the good news the design system bundle A3 prototype is ready to kick :)
I'll publish it to github very soon (next few days) after adding some more documentation and then I'll eagerly need your feedback.
The long story short - half of this module UI needs to be completely separated from the user land and the apostrophe UI context. The full story will be in the bundle docs and I'll open a dedicated discussion here.

Your current asset management looks great. Even adding my own asset management was time of great fun :) I'd only need to discuss a tiny feature request in this direction at some point - before/after core build hook is a magic solution for everyone who needs more magic than you do right now (most probably 3rd party modules). This would allow me to not force users to execute additional asset publish task beefore/after the core asset build one.

And btw I wasn't even able to inject any css/js to the A3 build pipeline, but I really didn't try hard (and it was at a very early stage of the port).

[boutell]

So you're looking for a promise event to be emitted before and after the asset build occurs in the asset module?

…

On Sun, Feb 14, 2021 at 2:23 PM Miro Yovchev ***@***.***> wrote: No, it's about a really custom/rare requirement but the good news the design system bundle A3 prototype is ready to kick :) I'll publish it to github very soon (next few days) after adding some more documentation and then I'll eagerly need your feedback. The long story short - half of this module UI needs to be completely separated from the user land and the apostrophe UI context. The full story will be in the bundle docs and I'll open a dedicated discussion here. Your current asset management looks great. Even adding my own asset management was time of great fun :) I'd only need to discuss a tiny feature request in this direction at some point - before/after core build hook is a magic solution for everyone who needs more magic than you do right now (most probably 3rd party modules). This would allow me to not force users to execute additional asset publish task beefore/after the core asset build one. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27LF6RJO4KBYWLURQVLS7APLJANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

It sounds like the easiest and efficient way. Should be generally useful.

[boutell]

This does sound useful! However, there's an issue. In 2.x our asset build task worked like this, in that every module got to start up before it ran. But this actually caused serious problems because it required 100% of the infrastructure for normal operation, including the database, to be available during an asset build. And since asset builds are often done in continuous integration systems, this has been a problem for cloud deployments of 2.x. Some users have worked around it by supplying a temporary mongodb database for the asset build task, but that is obviously a large requirement to impose.

So for 3.x the asset build task works differently. Only a bare minimum of modules initialize before it runs. No database connection is made.

So for that reason it would be best if you were to achieve your before/after asset build needs in separate tasks that can perhaps be part of a single npm script.

[myovchev]

Adding mongodb to the CI pipe for the sake of boot up sounds like big time cheating to me - definitely needs improvement..
I understand what you are talking about but there is a difference here. My module static build (webpack) is a separate, npm script, independent of the Apostrophe environment and done while it's being published to npm. The @corllete/apos-ds-page-type:publish-asset task is there to integrate the bundle owned (already compiled assets) into Apostrophe and to publish them on CDN provider if the current local app is setup to do so.

It seems this "tiny feature request" needs more thinking than I thought and maybe a dedicated discussion?

[boutell]

If you write your code in an Apostrophe module, it's going to run after other modules start up. That includes core modules like @apostrophecms/db.

We did make an accommodation for @apostrophecms/assets:build to run without the database. That is achieved by placing the assets module earlier in defaults.js and carefully avoiding the need for other modules to completely start up, and using a special flag that causes the task to run as soon as possible and then exit the process.

I understand why you might want an accommodation like that for your own task. But my question then becomes, is your task really getting anything useful from being in an apostrophe module at that point? Or couldn't it just be a script that is run by npm, without an Apostrophe startup occurring at all?

[myovchev]

The source. Everything above could become a standalone script.

Basically two dependencies here, the apos.asset for the business logic to get the releaseId and apos.attachment which connects to uploadfs and passes the configuration of the local app using the module. It doesn't feel right to be a script outside of the apostrophe context. Keep in mind, we are in scenario when there is 3rd party module is being installed and integrated. The module creator doesn't have (and don't want) to have any control or opinion about the local app installation.

BTW I've already hit the brick with just static building - the things fail due to mongodb dependency while I was setting up the GitHub Actions for the project. I've never hit that in A2 due to the fact there was no building for me outside the webpack.

IMHO your asset custom solution should become a standard routine for plugins (tasks without fully booting apostrophe). The scenario would be task which only needs access to module methods (business logic once, no db queries) and options. There shouldn't be any db calls during the boot except the standard A3 checks maybe (e.g. the initializing of empty db). I can't see a "good" module to make requests directly on boot time within your e.g. methods() function. Correct me if I'm wrong. However I can see this as a hard to implement and it needs more thinking about the impact over the core.

A simple and quick solution would be a "webhook like" one. And the most easy one would be:

node app @apostrophe/asset:build \
  --hook=/relative/to/root/path/to/script.js \
  --hook=/another/one.js  \
  --hook=/support/shell/script.sh

It's not perfect, it's semi-automatic (still needs the user to manually add hooks to the npm command) but it's a solid start. Maybe :)

[boutell]

I am not sure I understand what you're saying about hitting the wall with static building in A3. A3 does not invoke MongoDB during the asset build task. Can you clarify? We know this is an issue in A2. That is not going to change for bc reasons but we have removed it for A3.

…

On Wed, Feb 24, 2021 at 9:37 AM Miro Yovchev ***@***.***> wrote: [The source[( https://github.com/corllete/apos-ds/blob/main/packages/apos-ds/modules/%40corllete/apos-ds-page-type/lib/tasks.js#L62). Everything above could become a standalone script. Basically two dependencies here, the apos.asset for the business logic to get the releaseId and apos.attachment which connects to uploadfs and passes the configuration of the local app using the module. It doesn't feel right to be a script outside of the apostrophe context. Keep in mind, we are in scenario when there is 3rd party module is being installed and integrated. The module creator doesn't have (and don't want) to have any control or opinion about the local app installation. BTW I've already hit the brick with just static building - the things fail due to mongodb dependency while I was setting up the GitHub Actions for the project. I've never hit that in A2 due to the fact there was no building for me outside the webpack. IMHO your asset custom solution should become a standard routine for plugins (tasks without fully booting apostrophe). The scenario would be task which only needs access to module methods (business logic once, no db queries) and options. There shouldn't be any db calls during the boot except the standard A3 checks maybe (e.g. the initializing of empty db). I can't see a "good" module to make requests directly on boot time within your e.g. methods() function. Correct me if I'm wrong. However I can see this as a hard to implement and it needs more thinking about the impact over the core. A simple and quick solution would be a "webhook like" one. And the most easy one would be: node app @apostrophe/asset:build \ --hook=/relative/to/root/path/to/script.js \ --hook=/another/one.js \ --hook=/support/shell/script.sh It's not perfect, it's semi-automatic (still needs the user to manually add hooks to the npm command) but it's a solid start. Maybe :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27NUHGKNTOG7MCK2CRDTAUFLNANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

My custom build task did it, core build is doing great without DB access.

[boutell]

OK, right. So the question is what your code gains from being in an apostrophe module and not just being a separate program that is invoked by, let's say, "npm run build" along with the core asset build.

…

On Mon, Mar 1, 2021 at 10:40 AM Miro Yovchev ***@***.***> wrote: My custom build task did it, core build is doing great without DB access. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27OED3QDWMEFAT2IXYDTBOYQHANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

As I explained above - it's about the dependencies described there plus the fact that a 3rd party module needs to expose commands to the end-user (in this case for sure) which doesn't work as a npm or just standalone script (very well).

[boutell]

I know it is frustrating that we don't see eye to eye about this one, especially with the great contributions you've been making. So I read over your code again. But after reading it, I still don't really see why it must be part of an apostrophe task, and can't just be a script that copies files into ui/public of your apostrophe module. And that script could be part of your npm build. So I'm afraid I still don't understand the requirement for this custom apostrophe module to start up during the asset build task.

One important thing though... I think you know this, but since our conversation started we did add support for css in addition to js in ui/public folders. Maybe that helps?

[myovchev]

Not frustrated at all, we are all friends around :)
I'll take the blame of not explaining the problem(s) correctly and it's mainly due to the lack of time and concentration at this moment. I have a solution which is good enough. Let's take our time, bring A3 to the desired place and when the time is right I can start a different discussion with clear and simple description of my idea about the 3rd party modules assets/asset tasks.

And thanks for your time Tom :)

[boutell]

Thanks to you as well!

…

On Thu, Mar 4, 2021 at 10:20 AM Miro Yovchev ***@***.***> wrote: Not frustrated at all, we are all friends around :) I'll take the blame of not explaining the problem(s) correctly and it's mainly due to the lack of time and concentration at this moment. I have a solution which is good enough. Let's take our time, bring A3 to the desired place and when the time is right I can start a different discussion with clear and simple description of my idea about the 3rd party modules assets/asset tasks. And thanks for your time Tom :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2716 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27KPW5BIF3JPURZKSZ3TB6QKFANCNFSM4XJKWT2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his