How to fetch associations #382

MarcGodard · 2016-08-07T22:06:51Z

So I have developed an app with the feathers framework. I have a hook attached to a database service that I want to access data from another service. Is this possible, and easy? I want that service to be able to access several neDB files. How do I do this?

'use strict';

module.exports = function(options) {
  return function(hook) {

    // access other db files here and get data

    hook.data = {
        putDataHere: newData
    };
  };
};

The text was updated successfully, but these errors were encountered:

daffl · 2016-08-07T22:33:38Z

Yes. Have a look at the example how to fetch related items. Basically, in your hook you can access another service via hook.app.service('servicename') and call the method you need. For the hook to wait you can return a Promise (that resolves with the hook object) which conveniently is what all the service methods already do:

module.exports = function(options) {
  return function(hook) {
    return hook.app.service('otherservice').find({
      query: { name: 'David' }
    }).then(page => {
      const davids = page.data;

      hook.data.davids = davids;
      // Or you can replace all the data with
      hook.data = { davids };

      // IMPORTANT: always return the `hook` object in the end
      return hook;
    });
  };
};

MarcGodard · 2016-08-08T01:07:00Z

@daffl Thank you very much. I figured it would be this easy, but couldn't find it in the documentation, I must have missed it.

MarcGodard · 2016-08-08T17:01:28Z

Ok so this is a find call that I have this before hook (just if you need that info)

I am only getting 25 but there are hundreds. If I remove the limit, I only get 5 items.

What is going on and how do I get all?

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      query: { dataSet: dataSet, $limit:2000 }
    }).then(page => {

      // lots of magic here

      //console.log(page.data);

      hook.result = page.data;

      return hook;
    });
  };
};

daffl · 2016-08-08T17:32:43Z

It depends on what option you set for paginate.max in the configuration. It's a safeguard so that someone can't just request millions of records remotely. In all newer adapters you can turn off pagination to get all record like this:

return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet }
    }).then(data => {
      // data is an array
    });

MarcGodard · 2016-08-08T18:07:22Z

When I do that I get 0 records. When I remove the pagination: false I get 25.

daffl · 2016-08-08T18:15:17Z

Does npm ls feathers-nedb show v2.4.1? If not you may have to update your package.json accordingly.

MarcGodard · 2016-08-08T18:16:22Z

└── feathers-nedb@2.4.1

daffl · 2016-08-08T19:00:46Z

I just verified that paginate: false works with the following example:

const feathers = require('feathers');
const rest = require('feathers-rest');
const socketio = require('feathers-socketio');
const hooks = require('feathers-hooks');
const nedb = require('feathers-nedb');
const bodyParser = require('body-parser');
const handler = require('feathers-errors/handler');
const NeDB = require('nedb');

// A Feathers app is the same as an Express app
const app = feathers();
const db = new NeDB({
  filename: './data/messages.db',
  autoload: true
});

// Parse HTTP JSON bodies
app.use(bodyParser.json());
// Parse URL-encoded params
app.use(bodyParser.urlencoded({ extended: true }));
// Register hooks module
app.configure(hooks());
// Add REST API support
app.configure(rest());
// Configure Socket.io real-time APIs
app.configure(socketio());
// Register our memory "users" service
app.use('/todos', nedb({
  Model: db,
  paginate: {
    default: 20,
    max: 50
  }
}));
// Register a nicer error handler than the default Express one
app.use(handler());

const promises = [];

for(let i = 0; i < 700; i++) {
  promises.push(app.service('todos').create({ text: `Item #${i}`}));
}

Promise.all(promises).then(function() {
  app.service('todos').find({
    paginate: false,
    query: {}
  }).then(data => console.log(data));
});

// Start the server
app.listen(3333);

I am getting 700 items logged to the console. Did you accidentally add paginate: false into the query instead of the main parameters?

MarcGodard · 2016-08-08T22:44:04Z

This is what I did:

'use strict';

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet, $limit:2000 }
    }).then(page => {

      // lots of magic here
      console.log(page.data);

      hook.result = page.data;

      return hook;
    });
  };
};

Still get 0 results and 25 with pagination variable not there and 5 results without limit.

However, I got it working with this:

'use strict';

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet }
    }).then(page => {

      // lots of magic here
      console.log(page);

      hook.result = page;

      return hook;
    });
  };
};

As you see the results are no longer in the page.data but is just in the page variable.

Lesson learned :)

MarcGodard · 2016-08-08T22:44:45Z

Oh and thanks @daffl for your help.

MarcGodard · 2016-08-08T23:03:40Z

On the client side it doesn't work. (the paginate: false, variable).

daffl · 2016-08-08T23:22:37Z

Only query is passed between the client and the server. I'd try and avoid letting a client request all entires (if your database has a million records it will kill both, the server and the client). If there is no way around it you have to map a special query parameter (e.g. $paginate) from params.query into params in a hook on the server:

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.$paginate) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});

MarcGodard · 2016-08-08T23:28:56Z

I understand the warning of not allowing pagination on the client side, however, the tool I am making is just for me to run locally. Thanks again for everything.

arkokoley · 2016-08-28T14:05:49Z

@daffl Hi, I'm using the code you have posted to retrieve posts a user has.

module.exports = function(options) {
  return function(hook) {
    // hook.getPosts = true;
    const id = hook.result.id;
    console.log("id");

    return hook.app.service('posts').find({
      paginate: false,
      query: { postedBy: id }
    }).then(function(posts){
      console.log("posts");
      // Set the posts on the user property
      hook.result.posts = posts.data;
      // Always return the hook object or `undefined`
      return hook;
    });
  };
};

But this causes the server to go in an infinite loop and the node app ultimately crashes.

I am using MySQL as my DB connected by sequelize.

What am I doing wrong?

Update

I had setup a similar hook for posts to populate the user field based on the postedBy field. So it seems, the user hook would trigger the post hook which in turn would trigger the original user hook and the loop continues resulting in an infinite loop and memory overflow.

Any ideas how to populate only a shallow related item, i.e, the hook would pull only the related items on only the first level.

fortunes-technology · 2017-03-21T15:14:12Z

@daffl Your idea is awesome, though the code didn't work.
I had to change hook.paginate to hook.params.paginate to make it work.
And I made a little change so you can send whatever you want there.
so you can $paginate : {value: false} to disable pagination or
$paginate: { value: {default: 100, max: 2000}} to override pagination

app.service('data').before({ find(hook) { if(hook.params.query.$paginate) { hook.params.paginate = hook.params.query.$paginate.value; delete hook.params.query.$paginate; } } });

One more thing to consider is, when pagination disabled, the data is not in res.data but res itself.

daffl · 2017-03-21T15:58:25Z

@fortunes-technology You are right. I updated my code snippet. This should work:

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.$paginate) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});

nathandial · 2017-09-12T21:15:10Z

I think if $paginate is set to false (which is the point of this hook, to disable it... ), the if is going to be false, too. Maybe

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.hasOwnProperty('$paginate')) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});

maksnester · 2017-09-19T20:33:05Z

Hi there. Could someone tell me if feathers services have findOne functionality or how to implement it? Thanks!
cc @daffl

daffl · 2017-09-20T00:05:14Z

A findOne is just a find with a $limit of 1:

app.service('myservice').find({ query: { name: 'test', $limit: 1 } })
 .then(page => page.data[0])
 .then(result => console.log('Got', result);

maksnester · 2017-09-20T08:09:36Z

@daffl I asked because mongoose has findOne and I suggested that it doesn't search over all rest collection if something already found. So, I thought about findOne as something smarter then search with limit=1... That's why I asked.

A common scenario is we need to access another service from inside the hook, I propose to revive a legacy documentation found on this issue feathersjs/feathers#382

roelvan · 2018-01-18T14:29:13Z

I usually do this for a quick findOne:

const employee = (await context.app.service('employees').find({ query: { userId: user.id }, paginate: false }))[0];

fridays · 2018-01-21T19:06:31Z

I just made a plugin for a .findOne() method: feathers-findone

joakimstrandell · 2018-07-09T09:16:41Z

I couldn't find any example on how to add the $paginate workaround for the client. So I created a before hook that runs from the app.hooks.js file with some modifications:

module.exports = function () {
  return context => {
    if (context.params.query && hook.params.query.hasOwnProperty('$paginate')) {
      context.params.paginate = context.params.query.$paginate === 'false' || context.params.query.$paginate === false;
      delete context.params.query.$paginate;
    }
  };
};

Currently it issues the following warning: (node:6686) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.

dtwilliams · 2018-11-20T21:23:56Z

How would you go about writing a unit or integration test for this?

daffl · 2018-11-20T21:26:08Z

The same as writing any other test for Feathers.

dtwilliams · 2018-11-20T21:39:35Z

so if i have the following in my hook

const records = getItems(context);
records.authors = await context.app.service('users') .find({ query: { _id: { $in: records.authorIds } } }, context.params);

how do i "mock" the context.app.service in my unit test?

do i need to add it to my contextBefore object?

contextBefore = { type: 'before', params: { provider: 'socketio', user: { _id: 1 } }, data: { _id: 1, }, };

i'm using the unit-test auto generated from the feathers-plus generator.
https://generator.feathers-plus.com/get-started/#unit-test-for-a-hook

or should i be using an integration test instead?

eddyystop · 2018-11-25T19:48:33Z

I have gone the mocking route some time ago. I think its a very bad idea. You will eventually have to mock more and more of Feathers. There is no guarentee your mocks function as feathers do and you can get false negatives, as I did.

Instanciating a Feathers app is very fast, so use Feathers instead of mocks. That's how the appraoch the cli+ test take.

I wrote an article that refers to this. https://medium.com/feathers-plus/automatic-tests-with-feathers-plus-cli-4844721a29cf

dtwilliams · 2018-11-26T18:56:43Z

thanks eddyy, i've seen that, great article, really helped.

i was struggling on how to add the app to my context object but working it out eventually, i think!

    const app = feathers();

    contextBefore = {
      type: 'before',
      params: { provider: 'socketio', user: { _id: 1 } },
      data: {
        _id: 1,
      },
      app,
    };

Then i had to amend the test so it used async.

it('patch test', async () => {
    contextBefore.app.use('users', {
      async find() {
        return { data: [ { expectedResultObj1 }, { expectedResultObj2 } ] };
      }
    });
    contextBefore.data = {
      _id: 1,
    };
    assert(true);

    await hookToTest()(contextBefore);

    assert.deepEqual(contextBefore.data, {
      _id: 1,
      expectedResultObject1,
      expectedResultObject2,
    });
  });

Is that the right way to do it or is there a better way?

eddyystop · 2018-11-26T20:39:46Z

Looks good

phantomlinux · 2019-01-10T06:16:22Z

@fortunes-technology You are right. I updated my code snippet. This should work:

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.$paginate) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});

can use query $limit : null to bypass too

svendeckers · 2021-08-11T07:56:11Z

What strikes me, is that this line:
hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
will set hook.params.paginate to TRUE if hook.params.query.$paginate is FALSE and thus essentially reversing the result.

MarcGodard closed this as completed Aug 8, 2016

MarcGodard reopened this Aug 8, 2016

MarcGodard closed this as completed Aug 8, 2016

tuxrace mentioned this issue Oct 11, 2017

Proposal documentation to bring back the access another service from inside a hook. feathersjs/docs#866

Closed

daffl changed the title ~~Noob question I am sure~~ Fetching associations Sep 18, 2018

daffl changed the title ~~Fetching associations~~ How to fetch associations Sep 18, 2018

ionphractal mentioned this issue Oct 13, 2018

Fix client-side duplicate comments bug on submit and load more Human-Connection/WebApp#322

Merged

hanzlamateen mentioned this issue Sep 14, 2023

Changes for project-setting service to use schema, knex & feathers 5 EtherealEngine/etherealengine#8746

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to fetch associations #382

How to fetch associations #382

MarcGodard commented Aug 7, 2016

daffl commented Aug 7, 2016 •

edited

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016 •

edited

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016 •

edited

MarcGodard commented Aug 8, 2016

arkokoley commented Aug 28, 2016 •

edited

fortunes-technology commented Mar 21, 2017 •

edited

daffl commented Mar 21, 2017

nathandial commented Sep 12, 2017

maksnester commented Sep 19, 2017

daffl commented Sep 20, 2017

maksnester commented Sep 20, 2017

roelvan commented Jan 18, 2018

fridays commented Jan 21, 2018

joakimstrandell commented Jul 9, 2018 •

edited

dtwilliams commented Nov 20, 2018

daffl commented Nov 20, 2018 •

edited

dtwilliams commented Nov 20, 2018 •

edited

eddyystop commented Nov 25, 2018

dtwilliams commented Nov 26, 2018 •

edited

eddyystop commented Nov 26, 2018

phantomlinux commented Jan 10, 2019

svendeckers commented Aug 11, 2021

How to fetch associations #382

How to fetch associations #382

Comments

MarcGodard commented Aug 7, 2016

daffl commented Aug 7, 2016 • edited

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016 • edited

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

MarcGodard commented Aug 8, 2016

daffl commented Aug 8, 2016 • edited

MarcGodard commented Aug 8, 2016

arkokoley commented Aug 28, 2016 • edited

Update

fortunes-technology commented Mar 21, 2017 • edited

daffl commented Mar 21, 2017

nathandial commented Sep 12, 2017

maksnester commented Sep 19, 2017

daffl commented Sep 20, 2017

maksnester commented Sep 20, 2017

roelvan commented Jan 18, 2018

fridays commented Jan 21, 2018

joakimstrandell commented Jul 9, 2018 • edited

dtwilliams commented Nov 20, 2018

daffl commented Nov 20, 2018 • edited

dtwilliams commented Nov 20, 2018 • edited

eddyystop commented Nov 25, 2018

dtwilliams commented Nov 26, 2018 • edited

eddyystop commented Nov 26, 2018

phantomlinux commented Jan 10, 2019

svendeckers commented Aug 11, 2021

daffl commented Aug 7, 2016 •

edited

daffl commented Aug 8, 2016 •

edited

daffl commented Aug 8, 2016 •

edited

arkokoley commented Aug 28, 2016 •

edited

fortunes-technology commented Mar 21, 2017 •

edited

joakimstrandell commented Jul 9, 2018 •

edited

daffl commented Nov 20, 2018 •

edited

dtwilliams commented Nov 20, 2018 •

edited

dtwilliams commented Nov 26, 2018 •

edited