[bug] ☂️ umbrella issue for schema customization issues

[freiksenet]

This is a meta issue for all the issues with 2.2.0, that were introduced by the schema refactoring.

What?

See the blog post for the details about why we did the refactoring and what's it all about.

See the release blog post for release notes and final updates.

How?

Install latest version of Gatsby and try running your site. Hopefully it will all just work. If you want, you could also try the two new APIs (createTypes and createResolvers).

yarn add gatsby

Changelog

gatsby@2.5.0

• Published

gatsby@2.5.0-rc.1

• Moved to explicit directives instead of addResolver

gatsby@2.4.0-alpha.2

• Master back merge, upgrade graphql-compose

gatsby@2.4.0-alpha.1

• See feat(schema): Inference controls and improvements #13028

gatsby@2.2.0

• 🍾

gatsby@2.2.0-rc.2

• Fixed a regression when empty strings mixed with dates cause stuff to not be intrepreted as dates

gatsby@2.2.0-rc.1

• merge latest master
• docs update

gatsby@2.2.0-alpha.6

• merge latest master
• better SDL parse error messages

gatsby@2.2.0-alpha.5

• fix regression in connection naming in case types are lowercase named

gatsby@2.2.0-alpha.4

• Fixed custom scalars not having any filters
• Added errors when trying to override Node interface or generated filter/sort types.

gatsby@2.2.0-alpha.3

• Added a new convenience API modelled after graphql-compose. See using-type-definitions example.

exports.sourceNodes = ({ actions, schema }) => {
  const { createTypes } = actions
  createTypes([
    schema.buildObjectType({
      name: `CommentJson`,
      fields: {
        text: `String!`,
        blog: {
          type: `BlogJson`,
          resolve(parent, args, context) {
            return context.nodeModel.getNodeById({
              id: parent.author,
              type: `BlogJson`,
            })
          },
        },
        author: {
          type: `AuthorJson`,
          resolve(parent, args, context) {
            return context.nodeModel.getNodeById({
              id: parent.author,
              type: `AuthorJson`,
            })
          },
        },
      },
      interfaces: [`Node`],
    }),
  ])
}

gatsby@2.2.0-alpha.2

• Fixed regression with Node id field not being a String, like in current master.
• Upgraded to graphql-compose@5.11.0
• FilterInput types are now not prefixed by output type, reducing proliferation of types

gatsby@2.2.0-alpha.1

• Fixed issue with pagination over null query results
• @dontInfer(noDefaultResolvers: false) actually works
• in createResolvers resolvers info.originalResolver is available even if there is no resolver on original field

gatsby@2.2.0-alpha.0

• Redone alpha version with proper release version
• Fix Node types without actual nodes not being added

gatsby@2.1.20-alpha.0

• Initial alpha

[rexxars]

The filters for ID fields now expect an ID as input where they previously wanted a string. This breaks certain queries, for instance:

export const query = graphql`
  query BlogPostTemplateQuery($id: String!) {
    post: sanityPost(id: { eq: $id }) {
      id
      title
    }
  }
`

Will report:

error GraphQL Error Variable "$id" of type "String!" used in position expecting type "ID".

While it may be more correct to update the query to reflect this, it is a breaking change, so I thought I would report it.

[freiksenet]

@rexxars Nice find! I assume we used to convert ID filters to String. I'll restore old behaviour.

[freiksenet]

Released new version.

[NicoleEtLui]

Trying to query the schema in the resolver:

exports.createResolvers = ({ createResolvers, schema }) => {
  createResolvers({
    MenuJson: {
      someResolver: {
        type: `String!`,
        async resolve(source, args, context, info) {
          const foo = await graphql(schema, `
            {
              allPageJson {
                nodes {
                  id
                }
              }
            }
          `, {})

          console.log(foo)

          return 'WIP'
        },
      },
    },
  })
}

TypeError: Cannot read property 'nodeModel' of undefined
         at /private/tmp/test-gatsby/node_modules/gatsby/dist/schema/resolvers.js:22:15
         at /private/tmp/test-gatsby/node_modules/gatsby/dist/schema/resolvers.js:49:44
         at Generator.next (<anonymous>)
[...]

[stefanprobst]

@NicoleEtLui For queries in the field resolver, use the methods provided on context.nodeModel, i.e. you can use context.nodeModel.getAllNodes({ type: 'PageJson' }) or for more sophisticated queries you can use context.nodeModel.runQuery. There are some basic examples here.

If you need to access to the schema, note that the schema argument is just an intermediate representation - in the resolver you have access to the final built schema on info.schema.

[freiksenet]

Published gatsby@2.2.0-alpha.3

[NicoleEtLui]

@stefanprobst Thanks for the quick answer and all the great work you did !

[LoicMahieu]

Hi!
Any suggestion to handle relationships between nodes ?

For example, files are stored in JSON (without id field, assume the file name as id):
data/comments/some-uuid.json = { "message": "Hello", "postId": "some-post" }
data/posts/some-post.json = { "content": "post" }

Using the source-filesystem and transformer-json plugins, that makes the nodes have an unpredictable ID since the transformer use createNodeId(). It makes difficult to find post from comment.

[NicoleEtLui]

Hi !
Trying to start gatsby project with any of these in gatsby-config.js:

• gatsby-transformer-sharp
• gatsby-plugin-sharp
• gatsby-plugin-manifest

will throw:

error Plugin gatsby-transformer-sharp returned an error


  Error: Cannot find module 'gatsby/dist/utils/cpu-core-count'

  - loader.js:581 Function.Module._resolveFilename
    internal/modules/cjs/loader.js:581:15

  - loader.js:507 Function.Module._load
    internal/modules/cjs/loader.js:507:25

  - loader.js:637 Module.require
    internal/modules/cjs/loader.js:637:17

  - v8-compile-cache.js:159 require
    [keemotion-corporate]/[v8-compile-cache]/v8-compile-cache.js:159:20
[...]

[pieh]

@NicoleEtLui Please update gatsby-plugin-manifest and gatsby-plugin-sharp, this was fixed in those packages with #12332

[freiksenet]

Published gatsby@2.2.0-alpha.4

[freiksenet]

@LoicMahieu you can manually provide ids for the nodes, then you can do relationships by specifying fieldName___NODE fields.

[freiksenet]

Published gatsby@2.2.0-alpha.5

[skinandbones]

Q: Is this use case suitable for createResolvers?

I am using a remote CMS via gatsby-source-graphql which includes references to several remote files. I am currently pulling in those files with createRemoteFileNode. However, the page queries get awkward quickly because it's easy to get into situations where I need the result of the cms query (on the gatsby-source-graphql data source) to figure out which files I will need from gatsby-source-filesystem.

Ideally, I'd like to add/link/join (?) these remote file nodes into the cms nodes from gatsby-source-graphql. Is this a situation that createResolvers can help with?

[stefanprobst]

@skinandbones If I am understanding correctly, the short answer is "maybe but probably not quite yet".

We do support extending field configs on types added from a third-party schema, so it is possible to add a field resolver to a type from your CMS schema with createResolvers, and use createRemoteFileNode in the resolver. For example:
https://github.com/stefanprobst/gatsby/blob/5bbfee29b5ec38f13a3070b13de4877aaddd6483/examples/using-gatsby-source-graphql/gatsby-node.js#L56-L71

The problem is that createRemoteFileNode will trigger onCreateNode API calls, and in the field resolver we have currently no way of knowing when those subsequent API calls have finished and it is safe for the resolver to return. (One approach to solve this could be #12202.) So depending on what exactly you intend to do with the remote files, this might or might not yet work.

[stefanprobst]

@LoicMahieu Do you have an example project you can link to?

[skinandbones]

@stefanprobst Yes, you understand correctly and the async issue makes sense. The example you linked is pretty similar to what I was expecting to do so I can give that a shot and see what happens. My plan is to run these file nodes through gatsby-transformer-sharp in the page query.

Is another viable approach to use createRemoteFileNode via the sourceNodes API (as usual) and then link those nodes into the 3rd party schema using the new API? Up to this point, I haven't been able to get into the 3rd party schema to do this.

[stefanprobst]

@skinandbones sorry, i should have clarified: the part in the example i linked to is currently not yet working, exactly because of the issue that when the field resolver returns, the File node will have been created, but the ImageSharp node (which is created in the triggered onCreateNode API call) not yet.

As for the second approach, I'd be interested in your findings -- it should be possible to query for the added remote File nodes in the resolver with context.nodeModel.getAllNodes({ type: 'File' }) or with something like context.nodeModel.runQuery({ type: 'File', query: { filter: { name: { regex: "/^remote/" } } } })

[LoicMahieu]

@stefanprobst
Here is a example:

1. https://github.com/LoicMahieu/test-gatsby-refactor-schema
   Here we get able to link comments to post by a complex lookup on parent File.

2. https://github.com/LoicMahieu/test-gatsby-refactor-schema/tree/custom-transformer-json
   Here we get able to link them by using a custom JSON transformer where we could transform the object and also change the id.
   This method works but: if post is deleted and reference still exists in comments, gatsby will fail.
   It could be fixed by not use the ___NODE way but the new createResolvers: demo

[baobabKoodaa]

Hey, I just read this blog post about schema customization and the birthday example kinda seemed strange to me. In the example you're creating a custom resolver, which tries to resolve a date field into a Date, and falls back to a bogus value (01-01-1970) if the input date is not a proper Date. If you ever actually had a date in a real system, you would never want to replace incorrect inputs with bogus data. You would want to flag them as unknown/incorrect with something like null values. The fact that null wasn't used in the example made me wonder: is there some kind of limitation in Gatsby/GraphQL wrt. null values?

[stefanprobst]

@baobabKoodaa

is there some kind of limitation in Gatsby/GraphQL wrt. null values?

No. In GraphQL, you can explicitly set if a field should be nullable or not. More info here.

[skinandbones]

@stefanprobst I got this working and it works with ImageSharp. Very very cool and a game changer for working with a 3rd party schema 🎉 🎉

As for the second approach, I'd be interested in your findings -- it should be possible to query for the added remote File nodes in the resolver with context.nodeModel.getAllNodes({ type: 'File' }) or with something like context.nodeModel.runQuery({ type: 'File', query: { filter: { name: { regex: "/^remote/" } } } })

Here's what I did ...

exports.sourceNodes = async ({ actions, store, cache, createNodeId }) => {
  ... do createRemoteFileNode stuff ...
}

exports.createResolvers = ({ createResolvers, schema }) => {
  createResolvers({
    CMS_Thing: {
      thumbFile: {
        type: 'File!',
        async resolve(source, args, context, info) {
          const data = await context.nodeModel.runQuery({
            type: 'File',
            query: { filter: { fields: { ThingThumb: { eq: 'true' }, thingId: { eq: source.id } } } }
          })
          return data[0];
        }
      }
    }
  });
}

(This depends on me creating the file nodes with some fields, obviously. )

The optimal path (for my use case) will be to be able to use createRemoteFileNode in createResolvers so hopefully we can figure that out.

[stefanprobst]

@skinandbones Very cool! Btw, you can use firstOnly: true in runQuery to only get the first result.

[laradevitt]

@stefanprobst - Thanks for your reply! Yes, it does, but the point of the plugin is that you should not have to use relative paths:

A gatsby plugin to change file paths in your markdown files to Gatsby-friendly paths when using Netlify CMS to edit them.

I've created a pull request with a fix that makes the returned relative path platform agnostic.

I still don't know why it broke with 2.2.0. 🤷‍♀

[stefanprobst]

@laradevitt ah, sorry, should have checked the readme.

I still don't know why it broke with 2.2.0.

This is very probably a regression in Gatsby, as with 2.2.0 type inference changed a bit -- but normalizing paths in gatsby-plugin-netlify-cms-paths definitely seems more correct 👍

[ryanwiemer]

@stefanprobst

I just created a new ticket related to schema customization. If it is better for me to close that and put it in this umbrella thread just let me know.

#16099

Thanks!

[NickyMeuleman]

Description

Arguments in a graphQL query don't work when using schema customization.

After watching the learn with Jason stream on advanced GraphQL(broken up in multiple parts due to technical difficulties) I incorporated a lot of what was talked about into my theme.
However using graphQL arguments (like filter and sort) that rely on values from the customized schema doesn't work.

Steps to reproduce

run this branch of my theme locally.
Fire up /___graphql and query allBlogPosts.
Now try to query allBlogPosts again with a sort by descending date.
RESULT: no change in order (ASC, or DESC)

Example 2

Query for all posts and show their titles.

query MyQuery {
  allBlogPost {
    nodes {
      title
    }
  }
}

RESULT: works

now query for all posts that are have a tag with slug "lorem-ipsum"

query MyQuery {
  allBlogPost(filter: {tags: {elemMatch: {slug: {eq: "lorem-ipsum"}}}}) {
    nodes {
      title
    }
  }
}

RESULT: empty nodes array.

temporary workaround, possible reason why it doesn't work?

Try to query a single BlogPost (picks first one by default if no arguments are given).
Now try to query blogPost(slug: {eq: "herp-derpsum"}).
This works, I think because I added the slug to the MdxBlogPost node.
https://github.com/NickyMeuleman/gatsby-theme-nicky-blog/blob/d29c966e639f4733caf9ee43e9f5755df42db71d/theme/gatsby-node.js#L209-L210

It seems like the graphql arguments use data from the MdxBlogPost node rather than the eventual result after it runs its resolvers. Is my suspicion close?

Environment

System:
OS: Linux 4.19 Ubuntu 18.04.2 LTS (Bionic Beaver)
CPU: (4) x64 Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Shell: 4.4.19 - /bin/bash
Binaries:
Node: 12.4.0 - /tmp/yarn--1565124765846-0.45931526383359134/node
Yarn: 1.16.0 - /tmp/yarn--1565124765846-0.45931526383359134/yarn
npm: 6.9.0 - ~/.nvm/versions/node/v12.4.0/bin/npm
Languages:
Python: 2.7.15+ - /usr/bin/python

Additional notes

Relevant parts of code are in gatsby-node.js in the theme directory.
specifically the createSchemaCustomization and onCreateNode lifecycles.
I tried to heavily comment to show my though process.

[stefanprobst]

@NickyMeuleman sorry haven't had time yet to look closely but it seems you are missing a link extension on the BlogPost interface.

You define your MdxBlogPost.tags field to link by name:

tags: {
  type: "[Tag]",
  extensions: {
    link: { by: "name" },
  },
},

in the typedefs for the BlogPost interface you have:

  interface BlogPost @nodeInterface {
    tags: [Tag]
  }

Does it work with

  interface BlogPost @nodeInterface {
    tags: [Tag] @link(by: "name")
  }

[stefanprobst]

Related: #16466

[NickyMeuleman]

After adding your suggestion, querying allBlogPost with a filter based on a tag did work! 🎉 Thanks!

n the future, will I be able to remove the @link from the interface, since Tags might be linked differently per type that implements the BlogPost interface? (same question for Author, filtering there doesn't work, even with the @link moved up to the interface level.

[stefanprobst]

@NickyMeuleman
thanks for experimenting!
after looking at this i think it's something we need to fix in core: both issues have to with the fact that we use the interfaces's resolver when manually preparing nodes so we can run a filter on them. instead, we should run the resolver of the type we get from the interface's resolveType, which is also what happens when graphql processes the selection set.

[NickyMeuleman]

After #17284 got merged I tried removing the logic in my theme to work around this.

Used Gatsby version: 2.15.14
It doesn't appear to be working.
https://github.com/NickyMeuleman/gatsby-theme-nicky-blog/blob/bbc782332e6938daaa2fca1b25d6df7e78f19c6c/theme/gatsby-node.js#L278-L282
When you comment out the lines linked above, filtering in GraphQL on these fields is no longer possible.

[Everspace]

LekoArts suggested I ask this question more publicly than in the discord, since it might be a good topic for documentation.

How do I describe Many to Many relationships with createSchemaCustomization?

The use cases I have are the following:

1. I have a Product and a Store. Stores can hold any number of products, and products are non exclusive to any one store. What is the best way to express these two types of nodes?
2. I am trying to make a page for a game system. I have expressed each Ability as a Node. Abilities have any number of prerequisites, and any number of dependant skills. Preferably I would like to link to those pages (prerequisites and dependants) in the page for the skill itself. How do I model these relationships in graphQL, and furthermore, in Gatsby's api for that?

[Wolfsun]

@Everspace Try this for 1. add to gatsby-node.js. I've not tried this but I am doing something similar linking from a custom type (using the node interface) to an image, and it works. It does rely on the link extension so you will need to use IDs assigned by Gatsby. I guess you could map whatever IDs you are using in your datasource to Gatsby IDs during createPages?

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = [
    schema.buildObjectType({
      name: 'Store',
      fields: {
        products: {
          type: "[Product]",
          extensions: {
            link: {},
          },
        },
      },
      interfaces: ['Node'],
    }),
    schema.buildObjectType({
      name: 'Product',
      fields: {
        stores: {
          type: "[Store]",
          extensions: {
            link: {},
          },
        },
      },
      interfaces: ['Node'],
    }),
  ]
  createTypes(typeDefs)
}

[freiksenet]

@NickyMeuleman Do you mean you don't get results or that you don't see fields in input object?

[NickyMeuleman]

When I comment out the date key in that link, sorting by date no longer has any effect.

query MyQuery {
  allBlogPost(sort: {fields: date, order: DESC}) {
    nodes {
      date(fromNow: true)
    }
  }
}

has the same (non-empty) result as the same query with the order set to ASC.
Was hoping that wouldn't be necessary since there is a custom date resolver

[ryanwiemer]

Are there any examples out there for how to handle schema customization for an optional image coming from Contentful?

[sidharthachatterjee]

Been a while since Schema Customisation has been released and things are stable. I think it's time to close this issue 🙂

Incredible work @freiksenet and @stefanoverna ❤️

Folks, if you're having trouble related to Schema Customisation, let's open independent issues. Thanks!