No way to get requested fields of the object inside resolve

[xslim]

Let's say I have a Database with User entity with a lot of fields:

var Sequelize = require('sequelize');
var sequelize = new Sequelize('sqlite://database.db');

var User = sequelize.define('User', {
  name: Sequelize.STRING,
  email: Sequelize.STRING,
  otherField: Sequelize.STRING,
});

And I have a GraphQL:

var queryType = new GraphQLObjectType({
  name: 'Query',
  fields: () => ({
    user: {
      type: userType,
      args: { id: { type: GraphQLID } },
      resolve: (_, {id}) => User.findById(id)
    },
    users: {
      type: new GraphQLList(userType),
      resolve: (source, args) => {
        User.all(); // How to get the requested fields?
        //User.all({ attributes: args._requestedFields });
      }
    },
  })
});

And I do a GraphQL query to get name of all users

 curl -s -X POST http://localhost:8080/api/graph -d '{ users { name } }' | jq '.'

I don't want to fetch all the fields from the database. I want to know what fields to fetch...

Is there any way to get a list of fields that were requested in resolve method?

[devknoll]

Here's one idea... The 4th argument to resolve is the AST for the field. In your simple example, you could get the fields with something like:

resolve: (source, args, root, ast) => {
  var args = ast.selectionSet.selections.map(selection => selection.name.value);
  User.all({ attributes: args });
}

That would work, as long as you didn't have any fragments/spreads/etc. For more advanced cases, you could probably use the AST visitor utils. There's also a curious // TODO: provide all fieldASTs, not just the first field for that param too...

I believe that I heard that Facebook optimizes things like this internally, so hopefully Lee will show up and drop some wisdom 😄

[xslim]

Thanks for thd tip! I'll give it a try!

I believe getting requested fields in resolve will be needed not only for me)

[leebyron]

@devknoll is on the money with this. This is an area that is not yet complete, hence the TODO in the source. There is a more complex case that we don't yet support, and I plan to investigate ways of doing so.

Here's a contrived, but valid query: we want the names of our friends and the birthdates of our friends.

{
  me {
    ...friendNames
    ...friendBirthdays
  }
}

fragment friendNames on User {
  friends { name }
}

fragment friendBirthdays on User {
  friends { birthdate }
}

GraphQL's execution engine will notice that these fragments overlap, and will fetch friends one time, and will fetch { name, birthdate } afterwards. However we're not yet sharing this information wholly to the resolve function.

[devknoll]

I wonder if the problem could be generalized by adding some metadata to fields to provide hints.

For a SQL backend, maybe you'd say if there's an access to first_name, include that field in the query. name would prefetch both first_name and last_name. friends would generate another query, using the hints beneath it.

Then at the start, you would recursively go through the tree of requested fields, generate all the queries that you'll need, execute them in parallel, and then turn the results into your business objects. And finally, run resolve on the root object (-- no idea how this would work with fragments).

I would love to know if you have a more elegant solution at Facebook, @leebyron 😄

[leebyron]

@devknoll we have never tackled this problem of writing SQL queries from GraphQL queries at Facebook - we never write SQL queries directly in the product anyhow, so this has never come up, so it's fresh research for those trying to do this.

What you described of keeping a mapping between GraphQL fields and SQL fields is what @schrockn was considering doing as he's been toying with building this in the past. Given a GraphQL query you should be able to determine which fields need to be queried from which tables based on those mappings and assemble the right query.

[devknoll]

@leebyron Ah, I meant the more general problem of avoiding calling out to the database multiple times. For some reason I thought this was already optimized. Thanks for the detailed response though!

[leebyron]

One thing we have figured out at Facebook is a debounced query dispatcher with memoized caching.

When we issue a query from our application logic tier, we actually wait for the frame of execution to unwind before dispatching that query to a service tier. This way if any other queries are issued during the same frame of execution, all can be dispatched in a single go.

If later in the same process we issue a query for the same object, we can quickly fulfill that query by returning an from the in-memory cache.

Between these two optimizations, we have seen really efficient query dispatch and fulfillment between our app layer and service layer without requiring any change in how people (or more recently, GraphQL) actually perform the queries.

Does that help answer?

[devknoll]

Absolutely, thank you.

[leebyron]

Implementing this in JS is really easy as well. Something along the lines of (warning: coding directly in this editor):

export function sendQuery(str) {
  return new Promise((resolve, reject) => {
    if (queue.length === 0) {
      process.nextTick(dispatchQueue);
    }
    queue.push([str, resolve, reject]);
  });
}

var queue = [];

function dispatchQueue() {
  var toDispatch = queue;
  queue = [];
  yourBatchQueryMethodHere(toDispatch.map(([str]) => str)).then(results => {
    results.forEach((result, index) => {
      var [,resolve] = toDispatch[index];
      resolve(result);
    });
  });
}

[gyzerok]

@leebyron What you mean by yourBatchQueryMethodHere? Can you share an example of such method?

I'm currently want to batch GraphQL client to my server.

[mnylen]

@gyzerok, I implemented this in our api aggregation layer. See https://gist.github.com/mnylen/e944cd4e3255f4421a0b for example. Hopefully it helps.

[gyzerok]

@mnylen Thank you!

[leebyron]

@mnylen that's a great and terse example! @gyzerok the idea is that function is going to be very different depending on what backend storage system you're using :)

[gyzerok]

@leebyron The point is I do not want batching for backend, but for frontend. The reason is I'm currently trying to make tool for Redux to achieve Relay-like functionality. All because I'm highly interested in Relay and already bored to wait its public release. Here is a dirty proof of concept.

[leebyron]

Ah, then I misunderstood. GraphQL is the tool we use for batching requests from the frontend. Relay helps us build single GraphQL queries to represent all the data we need at any one point.

The example I explained above and @mnylen and @devknoll were interested in was how to debounce round trips to a backing storage from GraphQL type implementations.

[leebyron]

Closing this now since v0.3.0 will include access to all fieldASTs as well as fragments which makes this one step easier. Specifically is is now possible.

[mnpenner]

Sorry to jump in on a closed ticket, but I'm running 0.4.2 now. Is there a newer example of how we would get a list of all the requested fields?

[fson]

@mnpenner The third argument to resolve is a GraphQLResolveInfo object now and it includes the fieldASTs property. So you can do

resolve(source, args, { fieldASTs }) {
  // access fieldASTs
}

[Nexi]

Sorry to write on a closed ticket too, but @leebyron I can't understand how exactly this debounced query dispatcher aggregates the queries on the next processor tick.

if I have the following graphQL:

post(id: 3000) {
  author {
    name
  }
}

In the current graphql implementation, first "post" needs to be resolved and then, after the post's data is available (promise has been resolved), executor goes inside "author" type end resolves it. In other words, aggregation layer doesn't know about "author", before post has been resolved. In this case, how aggregation layer can combine "post" and "author" query in one query, when "post" needs to be fetched first in order to aggregation layer figures out we need "author" too.

Do I miss something here?

[leebyron]

@Nexi The query debouncing technique only works if your backends supports a batch fetching (e.g. SQL select from myTable where id in ...), and only when there is inherent parallelism in your query. In your example, there is no inherent parallelism because as you rightly point out, you must first request the "post" before you can request the "author".

[Nexi]

@leebyron Got it! Thank you for your quick response :)

[leebyron]

I should mention that https://github.com/facebook/dataloader was released today which expands upon my example code above. In fact, this issue was the original inspiration for writing this new repo!

[bigblind]

I have a possibly stupid question, because I'm new to graphql, but it seems that any query with fragments and other bells and whistles could be rewritten to a query with just fields and subfields, so

{
  me {
    ...friendNames
    ...friendBirthdays
  }
}

fragment friendNames on User {
  friends { name }
}

fragment friendBirthdays on User {
  friends { birthdate }
}

could be rewritten as
{
me {
friends {
name
birthdate
}
}
}

It looks like the executor needs to think about queries in this way anyway, so could it modify the ast to look like this before passing it to resolve functions? I can see no reason why a resolve function should care about whether some field is in a fragment anyway.

[clintwood]

@bigblind ATM I use fragments to represent the fields from a specific backend store (i.e. fragment fields <=> store collection fields). That way when I compose my GraphQL query I know which store collections/tables need to be queried to compose the final set of fields for final output. Of course this is not mandatory, but useful...

[jakepusateri]

@clintwood I made a library that handles fields, fragments, inline fragments, skip and include directives that may solve your problem: graphql-list-fields

[benjie]

For anyone else who lands here; fieldASTs was renamed to fieldNodes in v0.8.0

[Mikhus]

I've landed here too late, as far as already did another one lib for dealing with query fields, so, maybe someone will find it useful, it supports array of fields extraction and fields object maps taking into account query fragmentation and skip and include directives as well. So here it is: https://github.com/Mikhus/graphql-fields-list
Hope it would save someone couple of hours or days of work....
And it also have TypeScript feature included, for those who need it...

[dandv]

@jakepusateri and @Mikhus: thanks for the packages. Would be helpful to explain how they differ from prior art, namely @robrichard's graphql-fields.

[benjie]

If you need more depth than just the next layer, there’s also https://www.npmjs.com/package/graphql-parse-resolve-info which is what we use for PostGraphile.

[Mikhus]

@jakepusateri and @Mikhus: thanks for the packages. Would be helpful to explain how they differ from prior art, namely @robrichard's graphql-fields.

What I was missing in original graphql-fields is fields transformation. For example, GraphQL API contains "id" field, but I need to query "_id" field in MongoDB, with the lib I did you can do it as configuration and do not need extra-code. So my lib differs in a way that it provides not only data extraction from GraphQLResolveInfo object, but also has fields name transformation API.

Also the difference is that fields map functionality returns tree which has false value on it's leafs which simplifies recursive traversal in some cases...

And one more thing - it gives an ability to fetch only a sub-tree of fields by a given path...

As well one more thing is that it provides ability to skip some paths from a resulting fields map tree to be returned, using wildcard patterns as well.

At least, all those were differences on the moment of creation of the library...

[oleksandr-andrushchenko]

Checker variant, not ideal but maybe useful

  /**
   * Example operation:
   * query PaginatedCategories($size: Int) {
   *   paginatedCategories(size: $size) {
   *     data {
   *       id
   *     }
   *     meta {
   *       nextCursor
   *       prevCursor
   *     }
   *   }
   * }
   *
   * Example use:
   * checkSelectedField(info, 'PaginatedCategories.paginatedCategories.meta.nextCursor') => TRUE
   * checkSelectedField(info, 'paginatedCategories.meta') => TRUE
   * checkSelectedField(info, 'paginatedCategories.any') => FALSE
   * checkSelectedField(info, 'any') => FALSE
   *
   * @param {GraphQLResolveInfo} info
   * @param {string} path
   * @returns {boolean}
   */
  public checkSelectedField(info: GraphQLResolveInfo, path: string): boolean {
    for (const _path of this.iteratePaths(info.operation.selectionSet.selections)) {
      if ([ ...[ info.operation.name.value ], ..._path ].join('.').includes(path)) {
        return true
      }
    }

    return false
  }

  private* iteratePaths(selections: ReadonlyArray<SelectionNode>): Generator<any, [], any> {
    if (selections.length == 0) return yield []

    for (const selection of selections) {
      if (selection['selectionSet'] && selection['selectionSet']['selections']) {
        for (const path of this.iteratePaths(selection['selectionSet']['selections'])) {
          yield [ selection['name']['value'], ...path ]
        }
      } else {
        yield [ selection['name']['value'] ]
      }
    }
  }

https://github.com/oleksandr-andrushchenko/ExamMeApi/blob/main/src/services/graphql/SelectedFieldGraphqlChecker.ts

[benjie]

As an update and alternative to my graphql-parse-resolve-info library shared above, Grafast now exists as a completely independent execution engine to address this need. It eschews resolvers in favour of "plan resolvers" which enable deep optimizations by first planning the entire request, then optimizing the plan, and finally executing this optimized plan. I talked about it at GraphQLConf 2023 in case you prefer videos.