Seletive Models, delete documents and slow #1
Comments
|
Hi Gabriel, nice to hear that you like the idea and want to use it :)
The main problem is that for each document has to be loaded the corresponding node (and the other way round) from neo4j / mongodb. So this design construct doubles the requests on each record. If you have now enabled the populate documents for relationships (it's enabled by default and is too convenient to turn it off ^^) for each relationship will be loaded all related documents. Currently each document is queried separately... that is going to be changed to "query all documents of one collection at once" in one of the next releases. Beside that I see currently not much room for performance improvement. But maybe some benchmark tests would help to find bottlenecks and make different operations more comparable. Cheers |
|
I added benchmark tests that shows that mongraph is much slower than mongoose: But as you can see the neo4j nodes are mostly the bottleneck. Maybe an option to make deleting corresponding nodes as well and/or deleting node asynchronously would "solve" the »waiting for neo4j to finish the deletion job« problem... |
|
Hi Philipp @pstaender , First of all, thanks for your answer and benchmark! About the first question Q -> Do you have some gotchas to use mongraph to use selective mongoose models? About the 3 Question, You can think about use this guy to comunicate with neo4j, https://github.com/scttnlsn/mubsub i think that way u can add ou delete node asynchronously. what you think? sorry about comment in a close issue :-/ |
|
no problem - i closed too soon and you problem is correct (I understood it the wrong way ^^). the problem is, once a schema is compiled in mongoose, you can't recompile it, so the remove hook is not replaceable. but according to your first suggestion to make graphability optional could be a good way to handle with that for each model. would you say it would be a good way to define that in your schema, s.th. like: schema = {
name: String,
...
// enables by default all extensions (schema + middleware/hooks)
graphability: true,
// or specific ones
graphability: {
schema: true,
middleware: [ 'preRemove', 'preSave' ]
}
}what do you think? |
|
Here is what I found about problems with overwriting middleware: Automattic/mongoose#1251 schema = new mongoose.Schema({ name: String })
schema.set('graphability', false) #disable schema extension and middleware
# or set specific options via json
schema.set('graphability', { middleware: { preSave: true, preRemove: false } }) # disables preRemove hook
…See more on the tests @: it 'expect that schema extensions and hooks can be optional', (done) ->
…Channels and listeners sound interesting... bu not sure that it makes it more comfortable / easier to handle… |
|
Hi @pstaender , That's a great solution, i will try to implement in my app, stay this issue open i will back with the feedback. Off course we can implement the asinc option :-D, i made something like that, but using ES and a plugin river in here take a look but the mongraph is in coffescript so it can be more harder for me :-/ but not impossible, lets go! :-D |
|
Hi @pstaender , I think i m doing something whrong here, can u help me? I load my models that way var mongoose = require('mongoose'),
fs = require('fs'),
di = require('injector'),
mongraph = require('mongraph'),
neo4j = require('neo4j'); ;
// Load Models
module.exports = function (app) {
mongraph.init({
neo4j: new neo4j.GraphDatabase(app.settings.config.graphdb),
mongoose: mongoose
});
// Connect db
mongoose.connect(app.settings.config.db);
// Bootstrap models
fs.readdirSync(__dirname).forEach(function (file) {
if (file !== __filename.replace(__dirname + '/', '') && (file !== 'plugins')) {
di.process(require('./'+file));
console.log('load model: ', file);
}
});
function handleModelError(err) {
console.log('err: %s', err);
}
//Register Mongoose Modules on di
for(var k in mongoose.models) {
//Add Plugin to ElasticSearch
var model = mongoose.models[k];
model.river();
model.on('error', handleModelError);
di.register(k, model);
}
};and there is an model i dont wanna create relation on neo4j so i do this: // Tag schema
var mongoose = require('mongoose'),
mongoosastic = require('mongoosastic');
module.exports = function(app) {
var Schema = mongoose.Schema;
// Schema
var TagSchema = new Schema({
name: {type : String, 'default' : '', trim : true, required: true},
});
//Disabled mongraph
TagSchema.set('graphability', false);
// Plugins
TagSchema.plugin(mongoosastic, app.settings.config.mongoosastic);
mongoose.model('Tag', TagSchema);
};but when i run my tests, on neo4j db the Node tag was created :-/ what i do wrong? |
|
mmh, can't see a mistake... are you using the latest version? Have you tried it without the plugin? Is the node attached to the document? In the tests it's done the following way: var Guitar, guitar, schema;
schema = new mongoose.Schema({
name: String
});
schema.set('graphability', false);
Guitar = mongoose.model("Guitar", schema);
guitar = new Guitar({
name: 'Fender'
});
guitar.save(function(err, doc) {
expect(err).to.be(null);
expect(doc._node_id).to.be(void 0);
return doc.getNode(function(err, node) {
expect(err).not.to.be(null);
return expect(node).to.be(null);
});
});Cheers Philipp |
|
AHHH!!!! Dammed, its a version, sorry about that.... i still have problems with remove documents, |
|
Hi @pstaender when i set schema.set('graphability', true); i had this error: |
|
@gabrielmancini , it's seems that there is middleware is set as an undefined but it should be set as {} by default # set default values, both hooks
schemaOptions.graphability.middleware = {} if schemaOptions.graphability.middleware and typeof schemaOptions.graphability.middleware isnt 'object'
schemaOptions.graphability.middleware.preRemove ?= true
schemaOptions.graphability.middleware.preSave ?= true Please try to set it as true in the options ( s.th. like { graphibility = true } ) |
Hi Philipp,
Before at all, congrats for your great job on this project, its awesome, be sides coffescripe 3:D.
Your idea makes all sens, join together the powerful mongo document and neo4j graph relations.
-Do you have some gotchas to use mongraph to use selective mongoose models?
-When i delete a document on mongo the node is delete to?
-I feel a little slow down in my tests ween i start mongraph, i need to tuning something?
thanks for coding this
cheers
ps: sorry about my english :-/
The text was updated successfully, but these errors were encountered: