Nunjucks env

[myovchev]

While doing some and digging around a3, I've discovered a slight change to the templates module. The getEnv signature have been changed from self.getEnv = function(module) to getEnv(req, module).

The latter makes sense for retrieving the nunjucks environment object on request basis, but this change is actually cutting me off from doing nunjucks adjustments on initialization (adding globals, tags, etc). I'm maybe failing to understand the details around the nunjucks integration with apostrophe, but my real question here is - am I able/be able to get my hands on the nunjucks object so I can do adjustments on intialization?

[abea]

It sounds like your issue is that you don't have a request object to pass in since it's on initialization. You might try the self.apos.task.getReq(); to generate one. That req will carry the ability to do anything (admin role), initialization code can do that anyway.

Is that helpful?

[boutell]

I think Miro is trying to change the nunjucks environment before it is used, which is a slightly different issue.

However I don't see how the new situation is different in this regard, because you would have had to do it the same way before, I think... you should be able to do whatever you like to each nunjucks "env" before it's ever used if you use -

// in project level @apostrophecms/template
extendMethods(self, options) {
  return {
    newEnv(_super, req, name, folders) {
      const env = _super(req, name, folders);
      // do something here
      return env;
    }
  };
}

Also note there is a filters option to the templates module that can be used to pass nunjucks filters in with less effort.

(Hmm there may be a minor bug there but not a showstopper probably:

_.each(self.filters, function (filter, name) {
          env.addFilter(name, filter);
        });

        if (self.options.filters) {
          _.each(self.options.filters, function (filter, name) {
            env.addFilter(name, filter);
          });
        }

That certainly seems redundant.)

Also note that any module can inject custom nunjucks tags, too. See "customTags" at the end of "area/index.js" for an example of how this is done.

So there are two cases (filters and tags) where you don't have to do this, but in other cases, the extendMethods pattern should work for you.

[myovchev]

Thanks for your time guys!

@abea It's more close to the scenario described by @boutell.
@boutell Much more clearer now. I'm working in an external module (bundle) context. Overriding on a project level is not an option. However customTags provided by the module should work fine for me (need to look at the implementation though).

I didn't dig too much about "how core is doing it" in a2 (nor a3), I have a slight idea about the core nunjucks implementation. The reasoning here was about the request object which now needs to be explicitly provided which in return makes me feel things (adding custom tags, globals, etc) happen on request time and not statically (boot up). And I can see reasons about that - i18n, permissions etc.
The thing is I'm adding simple "template tools" for front-end developers - simple tags to render responsive layout for easily creating their design system presentation. It doesn't need to be dynamically added, it has simple HTML output. In v2 I was able to just call the template module getEnv(self) inside the construct method of my custom module and do the job - it did sound clean and simple.

I'll play around with the design system module and come back if I'm stuck again ;)

[myovchev]

@boutell @abea I'm coming back with some results and thoughts as promised (long reading alert!).

I think I've wrapped my head around the nunjucks implementation (thanks to the core implementation tips above).
So in general (correct me if I'm wrong) on every request:

1. New nunjucks environment is created for every module having need of render, partial, etc features
2. env created (basically when the template.newEnv() is called) is enhanced with every helper/filter/tag exported by any registered module in the local app (core and 3rd party)
3. env is saved in the request object and referenced by the module name
4. render, partial, etc method of a module is referencing the corresponding env when invoked

Core is nicely adding helpers and tags via exported schema by every module and filters (as far as I found in the core) via the self.template.addFilter() call (I need more information here).

Because of the above and how it's tied to the core (adding area, widget and component tags for example) req object is added to the template.newEnv method signature, respectively template.getEnv.

In any other case the most clean way to directly enhance/get reference the env of every module is to locally (app level) extend the 'template.newEnv` method as pointed out by Tom above. However, I'm not aware of a clean way to do it if you are an module author (as in my particular case).

Let me give you a concrete example. I'm working on Apostrophe design system bundle (containing 2 modules for now) which is prototyped in a2 and now is being migrated to a3. Before I start - please ignore the naming/path conventions, they are still a2 style and will be adapted to a3 conventions when the module is completely ported.

In a2 the -pages module of the bundle is registering nunjucks tags, which aim to help users of the bundle to easily create design system related content - responsive layout (grid), expandable code source, styled sections and hints. I've decided to wrap all of the tags provided by the bundle with nunjucks extension (think of it as a set of custom tags). Simplified implementation is:

// @corllete/apos-ds-pages/index.js
// inside of the module construct, executed on boot only
require('./lib/tempate.js`)(self);

// @corllete/apos-ds-pages/lib/template.js
module.exports = function (self) {
  class DsCommonTags {
    // extension class methods
  }
  // get nunjucks env object
  const env = self.apos.templates.getEnv(self);
  // register
  env.addExtension('DsCommonTags', new DsCommonTags());
}

Here is the only way I found to do it (clean) - NOT in the module, but in the local app (as suggested by Tom). It also does quick fix on missing __ns global helper so that users can try the module with a3 alpha.

// modules/@apostrophecms/template/index.js
// Module author should instruct the users to add this code to their applications
module.exports = {
  extendMethods(self, options) {
    return {
      newEnv(_super, req, name, folders) {
        const env = _super(req, name, folders);

        // Enhance nunjucks env
        require('@corllete/apos-ds/lib/modules/@corllete/apos-ds-pages/lib/template')(self, env);

        return env;
      }
    };
  }
};

Here is a reference to the full template.js implementation for those who are interested.

// @corllete/apos-ds-pages/lib/template.js
module.exports = function (self, env) {
  class DsCommonTags {
    // extension class methods
  }
  // register
  env.addExtension('DsCommonTags', new DsCommonTags());

  // TODO Quick fix till __ns arrives back
  env.addGlobal('__ns', (...args) => {
    if (args.length > 1) {
      return args[1];
    }

    return args[0];
  });

I for sure can rework my nunjucks extension to set of functions used with customTag module schema, but my exercise here is to push the a3 limits (plus I really like the way it hides the implementation details ;) ). Additionally, fixing the __ns helper is crucial in order users/devs/me to test the module at this stage of a3. I haven't found any way to clean-extend the template.newEnv from within my bundle module (which extends the core page-type). I guess brute override of type template.newEnv = myOverrideFuncHere; could work, but it feels like hack and I never tried it.

Sorry for the long post and thank you for still reading it! Here comes the suggestion. I really think allowing module developers to access nunjucks environment directly in a clean way (shouldn't feel as a hack) is extremely important for many reasons and the main one being you don't have to proxy around more than you want - custom tags, helpers, filters are easy with the core and the rest is up to you type of message.

The 'how' is unclear to me because I still miss details of the a3 system and flow. I don't see event (with env context) fired inside of the getEnv/newEnv fit here. I can maybe see additional module export schema type with signature nunjucksEnv(req, env, name) or similar. I'd be happy to discuss it further if you find sense in the suggestion.

[boutell]

In v3 we must create the envs dynamically in order to inject `req`, which is necessary because we allow async components in templates in A3, so we cannot use a global `req` variable during the execution of the template. So we create one env per request per module, rather than just one env per module. I was concerned about the performance impact of this but it turns out it is minimal.

…

On Tue, Jan 26, 2021 at 8:49 AM Miro Yovchev ***@***.***> wrote: Thanks for your time guys! @abea <https://github.com/abea> It's more close to the scenario described by @boutell <https://github.com/boutell>. @boutell <https://github.com/boutell> Much more clearer now. I'm working in an external module (bundle) context. Overriding on a project level is not an option. However customTags provided by the module should work fine for me (need to look at the implementation though). I didn't dig too much about "how core is doing it" in a2 (nor a3), I have a slight idea about the core nunjucks implementation. The reasoning here was about the request object which now needs to be explicitly provided which in return makes me feel things (adding custom tags, globals, etc) happen on request time and not statically (boot up). And I can see reasons about that - i18n, permissions etc. The thing is I'm adding simple "template tools" for front-end developers - simple tags to render responsive layout for easily creating their design system presentation. It doesn't need to be dynamically added, it has simple HTML output. In v2 I was able to just call the template module getEnv(self) inside the construct method of my custom module and do the job - it did sound clean and simple. I'll play around with the design system module and come back if I'm stuck again ;) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2694 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27OTOF22IUVONMBLUY3S33B7LANCNFSM4WMRUB2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

So you've just answered my concerns :)
It's just bunch of functions attached to an object. The performance should be more on the template pre-compile side (cache, I think nunjucks had something there).

[boutell]

Did you see that there is direct support for registering custom nunjucks tags in any module? Check out "area.js" and "component.js" and the way the modules they are in require those in a "customTags" section.

…

On Sun, Feb 7, 2021 at 12:46 PM Miro Yovchev ***@***.***> wrote: @boutell <https://github.com/boutell> @abea <https://github.com/abea> I'm coming back with some results and thoughts as promised (long reading alert!). I think I've wrapped my head around the nunjucks implementation (thanks to the core implementation tips above). So in general (correct me if I'm wrong) on every request: 1. New nunjucks environment is created for every module having need of render, partial, etc features 2. env created (basically when the template.newEnv() is called) is enhanced with every helper/filter/tag exported by any registered module in the local app (core and 3rd party) 3. env is saved in the request object and referenced by the module name 4. render, partial, etc method of a module is referencing the corresponding env when invoked Core is nicely adding helpers and tags via exported schema by every module and filters (as far as I found in the core) via the self.template.addFilter() call (I need more information here). Because of the above and how it's tied to the core (adding area, widget and component tags for example) req object is added to the template.newEnv method signature, respectively template.getEnv. In any other case the most clean way to directly enhance/get reference the env of every module is to locally (app level) extend the 'template.newEnv` method as pointed out by Tom above. However, I'm not aware of a clean way to do it if you are an module author (as in my particular case). Let me give you a concrete example. I'm working on Apostrophe design system bundle (containing 2 modules for now) which is prototyped in a2 and now is being migrated to a3. Before I start - please ignore the naming/path conventions, they are still a2 style and will be adapted to a3 conventions when the module is completely ported. In a2 the -pages module of the bundle is registering nunjucks tags, which aim to help users of the bundle to easily create design system related content - responsive layout (grid), expandable code source, styled sections and hints. I've decided to wrap all of the tags provided by the bundle with nunjucks extension <https://mozilla.github.io/nunjucks/api.html#addextension> (think of it as a set of custom tags). Simplified implementation is: // @corllete/apos-ds-pages/index.js// inside of the module construct, executed on boot onlyrequire('./lib/tempate.js`)(self); // @corllete/apos-ds-pages/lib/template.jsmodule.exports = function (self) { class DsCommonTags { // extension class methods } // get nunjucks env object const env = self.apos.templates.getEnv(self); // register env.addExtension('DsCommonTags', new DsCommonTags());} Here is the only way I found to do it (clean) - NOT in the module, but in the local app (as suggested by Tom). It also does quick fix on missing __ns global helper so that users can try the module with a3 alpha. // ***@***.***/template/index.js// Module author should instruct the users to add this code to their applicationsmodule.exports = { extendMethods(self, options) { return { newEnv(_super, req, name, folders) { const env = _super(req, name, folders); // Enhance nunjucks env ***@***.******@***.***/apos-ds-pages/lib/template')(self, env); return env; } }; }}; Here is a reference to the full template.js implementation <https://gist.github.com/myovchev/2bbfc8166226afe99f94aa1e1d2269e7> for those who are interested. // @corllete/apos-ds-pages/lib/template.jsmodule.exports = function (self, env) { class DsCommonTags { // extension class methods } // register env.addExtension('DsCommonTags', new DsCommonTags()); // TODO Quick fix till __ns arrives back env.addGlobal('__ns', (...args) => { if (args.length > 1) { return args[1]; } return args[0]; }); I for sure can rework my nunjucks extension to set of functions used with customTag module schema, but my exercise here is to push the a3 limits (plus I really like the way it hides the implementation details ;) ). Additionally, fixing the __ns helper is crucial in order users/devs/me to test the module at this stage of a3. I haven't found any way to clean-extend the template.newEnv from within my bundle module (which extends the core page-type). I guess brute override of type template.newEnv = myOverrideFuncHere; could work, but it feels like hack and I never tried it. Sorry for the long post and thank you for still reading it! Here comes the suggestion. I really think allowing module developers to access nunjucks environment directly in a clean way (shouldn't feel as a hack) is extremely important for many reasons and the main one being you don't have to proxy around more than you want - custom tags, helpers, filters are easy with the core and the rest is up to you type of message. The 'how' is unclear to me because I still miss details of the a3 system and flow. I don't see event (with env context) fired inside of the getEnv/newEnv fit here. I can maybe see additional module export schema type with signature nunjucksEnv(req, env, name) or similar. I'd be happy to discuss it further if you find sense in the suggestion. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2694 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27MNZRFKR4AHQ7Q6YITS53GZDANCNFSM4WMRUB2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[myovchev]

Yes, I saw that (they work amazingly well btw), but it doesn't expose env either. I need to register the whole extension class and not single custom tags. I've encapsulated the whole required logic (args parsing/validation, rendering etc) there so reworking it is kind of big effort. I also need to add a global and doing it together with registering the extension makes sense to me. This all is possible via local method rewrite. I've explained everything as detailed as possible in the post.

[boutell]

You can follow my original extendMethods suggestion if you use the improve feature in a bundle. And you know all about bundles (:

So your bundle would contain a modules/template folder, and in that index.js you would write:

module.exports = {
  improve: '@apostrophecms/template',
  extendMethods(self, options) {
    ... extend newEnv here
  }
};

improve means "treat this code like a project-level enhancement of this other module." This is how we do similar things in the workflow module, etc.

I admit it would be nice if a bundle were not necessary to do this cleanly, but I'm not sure there's a better way without introducing an async event into every getEnv call.

[myovchev]

Fascinating! I didn't know anything about the improve feature :)
It's the magic I need exactly.

Thanks Tom
Cheers :)