Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partials/layouts collections discard dot namespaces in view keys #948

Open
mootari opened this issue Jan 25, 2018 · 36 comments
Open

partials/layouts collections discard dot namespaces in view keys #948

mootari opened this issue Jan 25, 2018 · 36 comments

Comments

@mootari
Copy link
Contributor

mootari commented Jan 25, 2018

version

0.24.3

description

  1. Given are two templates that use dots in their filenames to emulate namespaces:
    • list.hbs
    • list.item.hbs
  2. Templates are added to the default partials or layouts collections using a glob string.
  3. When each view reaches the default renameKey callback, view.key is already sanitized to only contain the template basename.
  4. When the renameKey callback gets invoked and view.key still contains ".", any remainder after (and including) the dot will be discarded.
  5. When the view for the template containing the extended name gets added to the collection it silently overwrites the previous view.

Source:

assemble/index.js

Lines 140 to 142 in d385ca8

renameKey: function(fp) {
return path.basename(fp, path.extname(fp));
}

Suggested solution

Warn or error out if adding a view onto a collection replaces an already existing view. Optionally suggest a custom renameKey callback.
This will also help with situations where users have identical basenames or identical filenames in different directories.

@assemblebot
Copy link

@mootari Thanks for the issue! If you're reporting a bug, please be sure to include:

  • The version of assemble you are using.
  • Your assemblefile.js (This can be in a gist)
  • The commandline output. (Screenshot or gist is fine)
  • What you expected to happen instead.

If your issue is related to one of the following, please open an issue there:

  • grunt-assemble Issues with using assemble in grunt or the grunt-assemble library.
  • handlebars-helpers Issues with using handlebars helpers from the handlebars-helpers library.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 25, 2018

Hi @mootari can you show an example of how you're actually setting the templates on the collection? There are existing ways to preserve the keys depending on how you're doing it.

edit: I see that you're using globs, but I' unclear on how dots in the name are being expanded

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

Hi, thanks for the insanely quick response! 😄

I'm adding the templates using the following line:

this.app.partials(pathLib.join(__dirname, 'templates/partials/**/*.hbs'));

(Source: https://github.com/mootari/Documentation/blob/aae6df5755fffa4b4f48caed53dfa7b8acf24270/theme/theme.js#L18)

I noticed the issue after I added two new templates:

  • theme/templates/partials/tocdropdown.hbs
  • theme/templates/partials/tocdropdown.item.hbs

When referencing via {{> tocdropdown}} I would always get the contents of the second template.

@jonschlinkert
Copy link
Member

Strange, I think this might be a different issue than suspected. I'm betting it has to do with how the extname is handled somewhere (in templates or either built-in renameKey function or a custom one). Seems like we encountered this somewhere before. Anything ring a bell @doowb?

@jonschlinkert
Copy link
Member

for discussion, this obviously shouldn't happen just from default node.js path parsing:

const path = require('path');
const stem = name => path.basename(name, path.extname(name));

console.log(stem('theme/templates/partials/tocdropdown.item.hbs')) //=> 'tocdropdown.item'
console.log(stem('theme/templates/partials/tocdropdown.hbs')) //=> 'tocdropdown'

which makes me think we wrote some code that's causing this somewhere

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

Unfortunately I had to overwrite renameKey on the collection itself:

this.app.partials().options.renameKey = p => pathLib.basename(p, '.hbs');

because just passing it to the collection like this:

this.app.partials(pathLib.join(__dirname, 'templates/partials/**/*.hbs'), {
  renameKey: p => pathLib.basename(p, '.hbs')
});

would still ultimately invoke the original callback on the collection.

@jonschlinkert
Copy link
Member

would still ultimately invoke the original callback on the collection.

Hmm, that seems like a bug. I'd love to get some failing unit tests together for an isolated minimum test case. It would be great if you could even do that locally then paste the unit tests back into a comment here, or do a PR, whatever works for you

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

Basic example:

const app = require('assemble')();

app.task('default', () => {
  app.partials('foo', {content: 'foo'});
  app.partials('foo.bar', {
    content: 'bar',
    renameKey: key => key
  });
  const views = app.partials().views;
  if(!views['foo.bar']) {
    // ...
  }
});

module.exports = app;

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

Sorry, forgot the renameKey (I've updated the above example). Although arguably an explicitely set key should not be modified either by default.
Edit: Forgot to hit "Update comment". Updated now.

@jonschlinkert
Copy link
Member

Although arguably an explicitely set key should not be modified either by default.

agreed!

@jonschlinkert
Copy link
Member

Hmm, I'm not able to reproduce the issue. Here is what I did:

// in "foo.js"
const app = require('assemble')();

// create collections
app.create('partials', { viewType: 'partial' });
app.create('pages');

// add views to "partials" collection
app.partial('foo.bar', { content: '"foo.bar"' });
app.partial('foo', { content: '"foo"' });

// add views to "pages" collection
app.page('aaa', { content: 'Partial: {{> foo.bar }}'});
app.page('bbb', { content: 'Partial: {{> foo }}'});
app.page('ccc', { content: 'Partial: {{partial "foo.bar" }}'});
app.page('ddd', { content: 'Partial: {{partial "foo" }}'});

// check the given collection's "views" cache
if (app.partials.views['foo.bar']) {
  console.log('success: foo.bar');
}
if (app.partials.views['foo']) {
  console.log('success: foo');
}

// check the app's "views" cache for the given collection
if (app.views.partials['foo.bar']) {
  console.log('success: foo.bar');
}
if (app.views.partials['foo']) {
  console.log('success: foo');
}

// use a lookup function on the collection
if (app.partials.getView('foo.bar')) {
  console.log('success: foo.bar');
}
if (app.partials.getView('foo')) {
  console.log('success: foo');
}

// use a lookup function on "app"
if (app.find('foo.bar')) {
  console.log('success: foo.bar');
}
if (app.find('foo')) {
  console.log('success: foo');
}

app.toStream('pages')
  .pipe(app.renderFile('hbs'))
  .on('error', console.log)
  .on('data', file => console.log(file.content))
  .on('end', () => console.log('done'))

Then after running $ node foo.js, I see the following output in the terminal:

success: foo.bar
success: foo
success: foo.bar
success: foo
success: foo.bar
success: foo
success: foo.bar
success: foo
Partial: "foo.bar"
Partial: "foo"
Partial: "foo.bar"
Partial: "foo"
done

@jonschlinkert
Copy link
Member

bah, lol. Well I figured out the issue (kind of). I forgot those collections are built-in with assemble, so by re-creating them I "solved" the issue. But that tells us that there is something buggy in one of those built-in collection configs.

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

But that tells us that there is something buggy in one of those built-in collection configs.

Perhaps the default renameKey callbacks defined in Assemble.initViews are unnecessary because they're already implemented via assemble-loader (I didn't check though)?

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

This seems to be where the action happens: https://github.com/jonschlinkert/load-templates/blob/master/index.js#L115-L119

@doowb
Copy link
Member

doowb commented Jan 25, 2018

I just saw these notifications.

From the link that @mootari just posted, I have an idea:
There might be a case when views are being re-loaded onto the collection (this could be something internal). The first time, the .hbs would be removed from the key, then the second time, the .item would be removed from the key. This is just a thought.

@jonschlinkert
Copy link
Member

The first time, the .hbs would be removed from the key, then the second time, the .item would be removed from the key. This is just a thought.

Whatever it is, is being triggered or caused by the configuration settings in assemble. In templates, I'm pretty sure there is a check for an existing view.key before creating a new one, my gut is that it's not what you're saying, but that it's related to how custom .renameKey functions are used.

@jonschlinkert
Copy link
Member

meaning that as @mootari indicated in an earlier comment, the .renameKey isn't working as expected for him:

Unfortunately I had to overwrite renameKey on the collection itself:

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

To clarify, the first call uses the options that got passed with the call to .partials(), the second doesn't.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 25, 2018

Right, to also clarify, I think that although there is loading happening, it's how the .renameKey functions are being cached that is causing the issue. In other words, loading templates multiple times isn't resulting in a bug, considering the following:

  • I was able to reproduce the bug without loading templates from a glob. The templates in my example were loaded one time but still threw an error
  • the bug only happened when a custom .renameKey function was defined (in assemble, not by the user)

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

Here's a partial trace to the second call (renameKey is not set on the item options, defaulting to the collection):

    at Views.<anonymous> (***/node_modules/templates/lib/plugins/item.js:84:16)
    at Views.<anonymous> (***/node_modules/assemble-loader/index.js:79:19)
    at Views.addView (***/node_modules/templates/lib/views.js:133:19)
    at Loader.createView (***/node_modules/load-templates/index.js:124:25)
    at Loader.addView (***/node_modules/load-templates/index.js:145:19)
    at Loader.addViews (***/node_modules/load-templates/index.js:190:12)
    at Loader.globViews (***/node_modules/load-templates/index.js:247:10)

@jonschlinkert
Copy link
Member

while we're on this topic, @mootari and @doowb how do you feel about us changing app.views and collection.views to be instances of Map? I'm doing some refactoring soon and this would obviously be a major bump.

This would also change the interface in important ways. You would no longer be able to directly get views from the object: pages.views.foo won't work, you would need to do pages.get('foo') (a proxy maybe), or pages.views.get('foo'). But there are advantages of the interface as well. Map is intended for this kind of use. We won't need to deal with enumerability of hidden properties on the object, and many other similar nuances...

Thoughts?

@doowb
Copy link
Member

doowb commented Jan 25, 2018

I think changing them to Map is a good idea and I think it can help with some of the other goals mentioned in the templates refactor issue.

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

how do you feel about us changing app.views and collection.views to be instances of Map?

I love maps, but we might need to sugar them a bit. Using Array.from() to convert iterators can get old fast. One thing to keep in mind though is that Map keys can be anything. That can be a blessing or a curse.

or pages.views.get('foo')

Yes, please! Anything that cleans up the exposed interfaces is a welcome change in my book. :)

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 25, 2018

Yes, please! Anything that cleans up the exposed interfaces is a welcome change in my book. :)

agreed!

but we might need to sugar them a bit. Using Array.from() to convert iterators can get old fast.

Also agreed! here is what I've done so far to make them easier to work with for our use case

'use strict';

module.exports = class Hash extends Map {
  get length() {
    return this.size;
  }

  find(fn) {
    for (const [key, value] of this) {
      if (fn(value, key) === true) {
        return value;
      }
    }
  }

  filter(fn) {
    const hash = {};
    for (const [key, value] of this) {
      if (fn(value, key) === true) {
        hash[key] = value;
      }
    }
    return hash;
  }

  map(fn) {
    const hash = {};
    for (const [key, value] of this) {
      const args = fn ? fn(value, key) : [key, value];
      if (!Array.isArray(args)) {
        throw new SyntaxError('expected .map to return an array: [key, value]')
      }
      hash[args[0]] = args[1];
    }
    return hash;
  }

  reduce(fn, init) {
    const hash = {};
    for (const [key, value] of this) {
      fn(hash, value, key, this);
    }
    return hash;
  }
};

edit: in the collection class, we would do:

this.views = new Hash();

@jonschlinkert
Copy link
Member

there might be better ways to do those things, I'm just working on a PoC for the next version. feedback welcome and appreciated

@doowb
Copy link
Member

doowb commented Jan 25, 2018

I like the Hash class. I think the methods like filter, map, reduce should return new instances of Hash instead of an object.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 25, 2018

I think the methods like filter, map, reduce should return new instances of Hash instead of an object.

I would just remove the methods then. IMHO the only time I'd be calling them is when I need an object for rendering or context.

edit: they were originally doing what you're saying, until I actually needed to use them and had to convert them to an object every time.

edit2: fwiw, @mootari and I both just commented on how "Using Array.from() to convert iterators can get old fast." lol. Guess you missed that part.

@doowb
Copy link
Member

doowb commented Jan 25, 2018

Oh, I see. that makes sense.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 25, 2018

here is class I created for lists:

class List extends Set {
  get items() {
    return this.slice();
  }
  get length() {
    return this.size;
  }

  slice(fn) {
    return Array.from(this);
  }

  find(fn) {
    return Array.from(this).find(fn);
  }

  map(fn) {
    return Array.from(this).map(fn);
  }

  reduce(fn, init) {
    return Array.from(this).reduce(fn, init);
  }

  filter(fn) {
    return Array.from(this).filter(fn);
  }

  intersection(set) {
    return this.filter(ele => set.has(ele));
  }

  difference(set) {
    return this.filter(ele => !set.has(ele));
  }

  union(...sets) {
    const list = new List(this);
    for (let set of sets) set.forEach(ele => list.add(ele));
    return Array.from(list);
  }
}

@mootari
Copy link
Contributor Author

mootari commented Jan 25, 2018

if (fn(value, key) === true) {

Is there a reason for the strict comparison?

class List extends Set {

I'm a bit torn on this one. I'm not sure if it's a good idea to mimic part of the Array interface and make people believe they're actually dealing with one. Perhaps it's better to just expose .toArray() and only keep the "non-standard" methods.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 26, 2018

expose .toArray() and only keep the non-standard methods.

I'm not sure what the value of this would be. To convert a set to an array, you just do [...set], or Array.from(set).

I'm a bit torn on this one. I'm not sure if it's a good idea to mimic part of the Array interface and make people believe they're actually dealing with one.

We can easily solve your internal conflict over method name semantics by returning a new Set() as @doowb described. Beyond that, I don't think we need to mislead people. We're all used to seeing methods like .reduce, .filter and .map (all array methods) return objects. Isn't this kind of convenience the key differentiating value for libs like lodash, async, underscore and countless others?

Maybe I'm missing something, but this seems like a non-magical thing that devs would get right away.

  • collection.list is an instance of Set. Convenience methods are exposed for converting the set to an array.
  • collection.views is an instance of Map. Convenience methods are exposed for converting the Map to a plain object.

We can easily do what was mentioned above and return a new Map or Set, then change that language to "convenience methods are exposed for making the Set/Map easier to work with". But then, I'm really not sure what the value of the methods would be, or of even having Map and Set, since everywhere they're used we'll need to convert them to objects or arrays (for starters: streams/pipeline, middleware, rendering, inside helpers, for looping over collections in templates, merging partials onto the context).

The other alternative (which I'm leaning towards) is to just make these private variables and only expose plain objects and arrays so we don't need to worry about confusing devs.

@mootari
Copy link
Contributor Author

mootari commented Jan 26, 2018

We can easily solve your internal conflict over method name semantics [...] Maybe I'm missing something, but this seems like a non-magical thing that devs would get right away.

No need for snark, I'm just trying to give an outside perspective as someone who spent some time digging and stepping through the sources to discover what can or can't be done. Assemble users appear to cover a wide range of proficiency, and not everyone has their debugger set up and ready to go. "If it quacks like a duck ..." etc.

The other alternative (which I'm leaning towards) is to just make these private variables and only expose plain objects and arrays so we don't need to worry about confusing devs.

Sounds good to me.

Btw, I'm really not trying to bikeshed here but to provide some value with my feedback. If you think I'm being anal let me know and I'll try to dial it back.

@jonschlinkert
Copy link
Member

jonschlinkert commented Jan 26, 2018

No need for snark, I

I was just joking! sorry humor comes across poorly here. I have a dry sense of humor

edit: the first part was joking, this next part I was not being snarky or funny, just stating my view

[...] Maybe I'm missing something, but this seems like a non-magical thing that devs would get right away.

I was speaking to your comment: "I'm not sure if it's a good idea to mimic part of the Array interface and make people believe they're actually dealing with one.", which sounds pretty clear that you were saying we would be doing "magic" when you say "making people believe".

I'm really not trying to bikeshed here but to provide some value with my feedback. If you think I'm being anal let me know and I'll try to dial it back.

Not at all, it's good feeback. I hate the bikeshedding meme since it causes exactly the kind of second-guessing that you're indicating. Please feel free to keep giving us your feedback, it's valuable

@mootari
Copy link
Contributor Author

mootari commented Jan 26, 2018

which sounds pretty clear that you were saying we would be doing "magic" when you say "making people believe".

Nah, more like "emulating". I count myself among the group of people that see stuff like .map(), .filter(), .length and immediately think "ah, it's an array, so I can also use .every(), .some(), .concat() and access indices". And then they can't. Might just be a case of RTFM, but it might also hurt first experiences because you have to stay aware of which of those methods are actually provided.

@doowb
Copy link
Member

doowb commented Jan 26, 2018

I meant to reply earlier, but I was away from my computer...

After @jonschlinkert's comment about returning an object from the Hash methods, I realized that these classes are meant to be classes used in the templates library and to help with building context and rendering collections and lists of information in templates (usually handlebars in the case of assemble). In those situations, working with arrays and objects is easier than working with Maps and Sets, which is why I think it's valuable to take advantage of Map and Set by inheriting from those classes to create our own classes. Then by adding additional methods to our classes, we're able to make them easier to work with to build context and use in templates and template helpers.

Might just be a case of RTFM

This is how I see it... if we document the api for List then it's just a class that has familiar methods. The methods might be similar to Array methods but I feel like this is similar to implementing a Queue or other data structure (the push and pop methods on a FIFO queue are going to act differently than the methods on a LIFO queue) that exposes convenience methods for interacting with the stored data.

We've tried to make classes for Collection and List and said, "Let's just use helpers to interact with the methods on them." which started making the templates look like a mess, or ended up forcing us to make helpers that repeated a lot of the same code just to get an object or an array. For this reason, I vote on the classes that inherit from Map and Set but have methods that can easily be used to build objects and arrays that can be added to the context for rendering templates. This will let users use regular helpers that would normally be used with objects and arrays.

I hope all of this makes sense 😀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants