feat(source-contentful) multi-language support

[helloiamlukas]

This pull request adds proper multi-language support for the Contentful source.

Before

I previously opened a pull request to define search parameters in the plugin configuration (#1331). This was mainly used to add the locale search parameter, in order to retrieve localized nodes.

{
   use: "@gridsome/source-contentful",
   options: {
      space: process.env.CTF_SPACE_ID,
      accessToken: process.env.CTF_ACCESS_TOKEN,
      host: "cdn.contentful.com",
      environment: "master",
      typeName: "Contentful",
      parameters: {
         locale: "*"
      }
   }
}

Unfortunately the structure of the returned nodes is not so straight forward to work with. A query within a template would look like this:

query News($id: ID!) {
  news: contentfulNews(id: $id) {
    title {
       en, de
    }
  }
}

If you have a lot of fields, the need for the localized subfields adds up and your query will end up in a huge repetitive mess. In addition to that, you can't define localized template paths, as node.locale is always undefined.

Now

This pull request fixes the problems mentioned above (and is also a proper solution for #748). You can now define your locales in the source directly:

{
   use: '@gridsome/source-contentful',
   options: {
     space: 'YOUR_SPACE', // required
     accessToken: 'YOUR_ACCESS_TOKEN', // required
     host: 'cdn.contentful.com',
     environment: 'master',
     typeName: 'Contentful',
     locales: ['en', 'de']
   }
}

This will import your nodes once per defined locale. Due to this, it is possible to define localized template paths in the Gridsome configuration.

module.exports = {
  templates: {
    ContentfulNews: [
      {
        path: (node) =>  `/${node.locale}/${node.locale === 'de' ? 'nachrichten' : 'news'}/${node.slug}`
      }
    ]
  }
}

The query within a template is the same, as it would be in a single language application. In addition to that, the locale field returns the current locale.

query News($path: String!) {
   news: contentfulNews(path: $path) {
      title,
      locale
   }
}

[kasperstorgaard]

Great feature, and good docs, works like a charm for me alongside gridsome-plugin-i18n

I have a small issue with multi language entries that are only valid for one locale, eg. their url is empty in some locales.
This results in this warning:
ContentfulRecipe > Failed to add node: Duplicate key for property path: /recipes/undefined/

with this template setup

// gridsome.config.js
templates: {
  ContentfulRecipe: [
    {
      path: (node) =>  `/recipes/${node.url}`
    }
  ]
}

I don't really see a way around it though, as @gridsome/source-contentful will always gather all entries first, then let you filter after.

The end result is actually what we want, eg. the html file never gets created if it doesn't have a url, but it seems sort of flaky to rely on the collection failing to do this filtering.

[helloiamlukas]

@kasperstorgaard Yes, multi language entries that are only valid for one locale can't be supported at the moment.

What you can do in your special use case though, is to import the entries with the parameters option

{
   use: "@gridsome/source-contentful",
   options: {
      space: process.env.CTF_SPACE_ID,
      accessToken: process.env.CTF_ACCESS_TOKEN,
      host: "cdn.contentful.com",
      environment: "master",
      typeName: "Contentful",
      parameters: {
         locale: "*"
      }
   }
}

and then create the pages in your gridsome.server.js manually.

module.exports = function (api) {
  api.loadSource(({ getCollection, createPage }) => {
    const collectionName = "ContentfulRecipe";
    const locales = ["en", "de"];
    const entries = getCollection(collectionName);

    entries.data().forEach(
      (entry) =>
        entry.url !== undefined &&
        locales.forEach((locale) =>
          createPage({
            path: `/recipes/${entry.url}`,
            component: `./src/templates/${collectionName}.vue`,
            context: {
              id: entry.id,
              locale,
            },
            route: {
              meta: {
                locale,
              },
            },
          })
        )
    );
  });
};

[louisnovick]

Thank you for working on this! I've been experimenting with it lately, and one issue I seem to be running into is that linked entries are all returning null or coming back as empty arrays. The references seem to exist in the playground but when using ... on LinkedEntryName { ... } within a query the expected results are not returned. Any idea what is causing this? Happy to help anyway I can.

[helloiamlukas]

Thanks @louisnovick for the feedback! You are right, this also doesn't work for me. I already see what's the problem, I will work on a solution and keep you updated.

[helloiamlukas]

@louisnovick This should work now!

The previous version included some major bugs. It looks like I mixed up the method parameters of addNode and createReference, but somehow it was still working 🥴 In addition to that, I didn't take the references into consideration at all (as you pointed out with your comment). I fixed both of it now!

[louisnovick]

@helloiamlukas Thanks for the quick update! It seems to be working as expected now 🙂 I'll keep playing with it and let you know if anything else comes up.

[jakehandwork]

@helloiamlukas Thanks a lot for the work you've done on this. I was able to get it working... sort of. I've also installed the gridsome-plugin-i18n.

Updates

Update 1/15/21
I went back and looked at your original message on this PR about the Before and Now. I adjusted the ContentfulNews in the template to this: ContentfulNews: [{ path: (node) => `/${node.locale}/news/${node.slug}` . This seemed like it worked at first, but then example.com/news/article doesn't work, only if it has the locale. So I adjusted to /${node.locale !== 'en' ? 'es/' : ''}news/${node.slug} so that it would be example.com/news/article and example.com/es/news/article. This resulted in the same Duplicate key for property path: /news/article/, just that this time it errored and did not finish starting the server. So I reverted back to /${node.locale}/news/${node.slug} and there is good and bad to this. The good being that it solves problems 1 (adding $locale variable to page-query) and 2 (failed to add node error). It does not fail on build and the site works fine afterwards leaving me with problem 3 (default URL loads both locales even though $context.locale is set to 'en') and a 4th problem (the default URL 404's on templates - basically example.com/news/article does not work)

Along with this update, I've been reading more about the page-query documentation to ensure that I understood them correctly and the $context.locale should be passed to the query and it seems that it should, so I'm not sure why it doesn't get passed. I've also been reading about the gridsome-plugin-i18n and it seems to give examples of the exact functionality that I'm looking for. So I'm trying to figure out why I'm hitting so many speedbumps with it.

Update 1/15/21 Pt 2
I used the Route Translation section from the i18n plugin documentation to create my own routes.js that looks like:

en: [
    {
      path: '/',
      component: './src/pages/Index.vue'
    },
    {
      path: '/en/',
      component: './src/pages/Index.vue'
    },
    ...
],
es: [
    {
      path: '/es/',
      component: './src/pages/Index.vue'
    }
    ...
]

I then added these lines to my config file:

enablePathGeneration: false,
routes: require('./routes.js')

And it works great! The default url returns just english, /en/ returns just english and /es/ returns just spanish. Now I'm just stuck with the newest problem where templates don't work on the default url. Working on that now.

Goal

My goal is to set up the site for English and Spanish where:

example.com returns English
example.com/en/ returns English
example.com/es/ returns Spanish

Current Config

My current config for gridsome-plugin-i18n is this (with locales = ['en', 'es']:

options: {
    locales,
    fallbackLocale: locales[0],
    defaultLocale: locales[0],
    enablePathRewrite: true,
    rewriteDefaultLanguage: false
}

I set rewriteDefaultLanguage to false since I don't want /en/ to be the default for English, just to keep the URL cleaner.

In my page-queries I format them like this:

query News ($locale: String!) {
   slides: allContentfulNews (filter: {locale: {eq: $locale}}, order: ASC) {
    edges {
      node {
        id
        title
      }
    }
  }
}

My understanding is that the idea is for the $context.locale set by the i18n plugin will be passed into the page query to filter by locale.

Problems

Here is where my issues come in. There are 3:

1. Variable not working in page-query on gridsome build
On gridsome build, I get this error: Error: Variable "$locale" of required type "String!" was not provided. referencing the locale variable I passed into the page-query. This doesn't show up anywhere on gridsome develop and seems to work just fine.

2. Duplicate key for property path
Also on gridsome build, I get several warnings: ContentfulNews > Failed to add node: Duplicate key for property path: /news/article-name/. These are all coming from the ContentfulNews type which is the only type I have set up as a /template so far. That config is just this: ContentfulNews: '/news/:slug' and the $slug variable is also passed to the page-query.

3. No locale in URL loads both locales
Locally, when I go to example.com without the locale in the url, $context.locale defaults to en as it should. But the queries don't query only the en locale. They pull in from both locales and I end up with double everything. This would like be the same case on the production site if I could get it to build. However, everything works as it should when using /en/ and /es/ in the url. What's most strange about this is that if I go and edit the page-query in anyway (e.g. add a field or remove a field), it straightens up and just loads the en locale, but only on that component and only while the current server is running. Once I restart it, it returns to having duplicates.

Attempted solutions

1. Variable not working in page-query on gridsome build
I've attempted removing the $locale filter from the query of the component that errored on gridsome build to rule out an issue with that specific component. This resulted in it just an error on the next component being built. So on gridsome build the $locale variable is not being passed at all. Doing some research on this, I didn't find much. I'm not quite sure which variables the page-query gets access to or how they get set. Still looking into that

2. Duplicate key for property path
Still working on this one. My edit above shows what I'm working with so far. I've attempted the server config recommended with no success. But I'll keep going at it

3. No locale in URL loads both locales
I'm also working on this one still. Ultimately I think I need to dive more into the behind the scenes of Gridsome, Contentful and the gridsome/vue-i18n libraries as I'm not super familiar with them.

Conclusion

If anyone has any thoughts or similar issues, let me know! I will update this as I figure things out. Thanks!

[helloiamlukas]

1. Variable not working in page-query on gridsome build
On gridsome build, I get this error: Error: Variable "$locale" of required type "String!" was not provided. referencing the locale variable I passed into the page-query. This doesn't show up anywhere on gridsome develop and seems to work just fine.

This happens because Gridsome will not just create a /en/index.html and /de/index.html file, it will also always create a file without the language path being added. In this case, it will create a /index.html file. In the query of this file $locale is not set – and this is the reason why you are getting this error. To solve this, you have to add a default parameter to the query (e.g. $locale: String! = "en").

query News ($locale: String! = "en") {
   slides: allContentfulNews (filter: {locale: {eq: $locale}}, order: ASC) {
    edges {
      node {
        id
        title
      }
    }
  }
}

I forgot about this default parameter in the docs, I will edit it ☺️

[helloiamlukas]

@jakehandwork Regarding your other problems: The Gridsome i18n Plugin unfortunately has some bug where it will always fall back to the default locale in your Templates (not in Pages though).

I made a pull-request to fix this problem. Please make sure to use this version of the plugin by adding it to your package.json

{
  "dependencies": {
     ...
    "gridsome-plugin-i18n": "https://github.com/helloiamlukas/gridsome-plugin-i18n",
     ...
  }
}

Let me know if this works out for you. Otherwise you can also provide a test repo, so I can have a closer look into it.

[jakehandwork]

Hey @helloiamlukas. Thanks for all the feedback and for updating the PR. I've added the default $locale... should've thought of that myself, though I'm not really familiar with GraphQL yet.

Anyways, it looks like your two responses here have resolved all my problems. I updated the $locale variable to have a default and I'm using your forked i18n plugin.

This left me with one final issue. For templated routes, I was only able to get either / to work or /en/ but not both. This was my configuration in gridsome.config.js:

templates: {
    ContentfulNews: [{
      path: node => `/${node.locale === locales[0] ? '' : `${node.locale}/`}news/${node.slug}`,
    }],
    ContentfulEvent: [{
      path: node => `/${node.locale === locales[0] ? '' : `${node.locale}/`}schedule/${node.slug}`,
    }],
  },

Unable to figure out how to add the english locale twice with different base routes, I decided to look into the Pages API using gridsome.server.js and removed the templates object from gridsome.config.js. Specifically a configuration similar to the comment above about entries only valid with one locale:

module.exports = function (api) {
  api.loadSource(({ getCollection, createPage }) => {
    const collectionName = "ContentfulRecipe";
    const locales = ["en", "de"];
    const entries = getCollection(collectionName);

    entries.data().forEach(
      (entry) =>
        entry.url !== undefined &&
        locales.forEach((locale) =>
          createPage({
            path: `/recipes/${entry.url}`,
            component: `./src/templates/${collectionName}.vue`,
            context: {
              id: entry.id,
              locale,
            },
            route: {
              meta: {
                locale,
              },
            },
          })
        )
    );
  });
};

I tried the same code, adjusting it to my situation, and ran into a few issues. To start, I slowly found out that entry.url is a custom field and not some default Contentful property. I changed that to entry.slug since I called it slug instead of url on my content type. It also appears that since createPage is not a part of the Data Store API, it is undefined when you pass it as a parameter to the method api.loadSource. To get around this, I'd have to use the Data Store API and Pages API, but one after the other. Here is my final solution:

module.exports = function (api) {
  const locales = ['en', 'en', 'es']; // duplicate the default locale to get `/` and `/en/`
  const templates = [
    {collection: 'ContentfulNews', path: '/news/'},
    {collection: 'ContentfulEvent', path: '/schedule/'}
  ] // makes it easy to add more templates in the future

  // loop through each template type, load the data, then create the pages for each entry
  templates.forEach(template => {
    let templateEntries = [] // store the entries outside of the Date Store API to access them in the Pages API

    // load the data from contentful
    api.loadSource(({ getCollection }) => {
      const entries = getCollection(template.collection)
      entries.data().forEach(entry => { templateEntries.push(entry) })
    })

    // create the 3 locale routes for each page
    api.createPages(({ createPage }) => {
      templateEntries.forEach(entry => {
        if (!entry.slug) return
        locales.forEach((locale, i) => {
          const localeRoute = i === 0 ? '' : `/${locale}` // simplify creating the locale route for each locale
          createPage({
            path: `${localeRoute}${template.path}${entry.slug}`,
            component: `./src/templates/${template.collection}.vue`,
            context: {
              id: entry.id,
              locale,
              slug: entry.slug 
            },
            route: {
              meta: {
                locale
              }
            }
          })
        })
      })
    })
  })
};

This ends up solving my problem. Now all routes are properly working on pages and templates. Thanks for all your help @helloiamlukas. Let me know if you have any questions or if it seems I misunderstood something.

[dglovernfa]

Hi @helloiamlukas - question I'm hoping you might be able to help with. The fix seems to work well for getting entries and their data, but it seems to not work at all for getting the correct data from assets. Are you able to provide some guidance on that? Working on a multilingual site and need to have alt tags for images in the correct language, but right now all assets return English content no matter what, despite the assets themselves having fields for every language on my localization settings.

[dglovernfa]

Hi again @helloiamlukas. I've actually resolved the issue by using much the same logic as you're already doing on the entries themselves. I'm hosting a version of your PR (down the rabbit hole we go!) for my own use, and you can reference it here if you want to add my changes to your own:

https://github.com/DouglasKGlover/gridsome-src-contentful-custom

[felixdenoix]

Hi there @helloiamlukas !
Thanks for the smart update and the thourough documentation,
Would there be any chance that you could find the time and rebase your pr so that @hjvedvik could merge the PR ?
Many thanks for the framework !