setNext used within a pre-save hook works but generates an error

[rernens]

@ramiel

hi Fabrizio

Pursuing previous exchange on scoped counter, I have the requirement in my application to manage when a counter is incremented when a new document is added.

I have seen that the setNext method is made just for this and looking at your code, I see that it invokes the same function than what you normally do in the pre-save hook that you attach to the schema in order to increment the counter for a new document.

In my case, each new document should not get a specific counter incremented.

In fact, due to legal reasons, employees documents in my application can't be updated in place. In order to keep the previous value of the document, the current document is marked as archived, a new document is created as a clone of the one just archived and changes are applied to this new document.

As employees must have a unique sequence number, the counter should not be incremented in this case. So if a value already exists in the sequence attribute the sequence number should not be set with a new value, leaving all employees documents for a specific employee with the same sequence counter value but a different updateSequence counter incremented at each update.

Here below is what I have implemented :

import { Document, Schema, Model, model} from "mongoose";

import * as autoincrement from "mongoose-sequence";
import * as mongoose from "mongoose";import { EmailSchema } from "./email.schema";

import { PhoneSchema } from "./phone.schema";
import { AddressSchema } from "./address.schema";
import { ProviderSchema } from "./provider.schema";
import { TraineeSchema } from "./trainee.schema";
import { ForeignerSchema } from "./foreigner.schema";

import { IEmployee } from "../interfaces/employee";

export interface IEmployeeDocument extends IEmployee, Document {
    setNext( id: string, callback: any ):void;
}

export interface IEmployeeModel extends Model<IEmployeeDocument> {
    counterReset( id: string, reference: any, callback: any ): void;
}

export const EmployeeSchema: Schema = new Schema({
    sequence: Number,
    customer: String,
    establishment: String,
    nir: String,
    quality: String,
    firstName: String,
    lastName: String,
    birthCountry: String,
    birthCity: String,
    birthDate: Date,
    startDate: Date,
    exitDate: Date,
    contract: String,
    duration: String,
    ratio: Number,
    category: String,
    replacing: String,
    address: AddressSchema,
    emails: [EmailSchema],
    phones: [PhoneSchema],
    idcc: String,
    citizenship: String,
    position: String,
    firingAllowance: Date,
    provider: ProviderSchema,
    foreigner: ForeignerSchema,
    trainee: TraineeSchema,
    isTrainee: Boolean,
    archived: Date,
    updateSequence: Number
}, { timestamps: true });


EmployeeSchema.pre<IEmployeeDocument>('save',  true,function (next, done ) {

        next();
        if (this.sequence) {
            return done();
        }
        this.setNext('employees_counter', (err) => {
            if ( err ) {
                console.log('Error while incrementing employees_counter', err);
            }
        });
        done();
    }
);

const AutoIncrement = autoincrement(mongoose);

EmployeeSchema.plugin( AutoIncrement, { id: 'employees_counter', inc_field: 'sequence', reference_fields: ['customer', 'establishment', 'isTrainee'] , disable_hooks: true });
EmployeeSchema.plugin( AutoIncrement, { id: 'employees_update_counter', inc_field: 'updateSequence', reference_fields: ['customer', 'establishment', 'nir'] } );

export const Employee: IEmployeeModel = model<IEmployeeDocument, IEmployeeModel>("Employee", EmployeeSchema);

Counters are well incremented as expected.

A new employees_counter value is set in the sequence attribute when a new employee is created as well as 1 in the updateSequence.

When an employee is udpated, a new document is created, the sequence remains the same and the updateSequence receive an incremented value.

So far so good.

But the rest api console displays the following message when a new employee is created :

Error while incrementing employees_counter { ParallelSaveError: Can't save() the same doc multiple times in parallel. Document: 5c6526f884611085d073b031
    at ParallelSaveError.MongooseError [as constructor] (/Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose/lib/error/mongooseError.js:13:11)
    at new ParallelSaveError (/Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose/lib/error/parallelSave.js:18:17)
    at model.Model.save (/Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose/lib/model.js:440:20)
    at model.<anonymous> (/Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose-sequence/lib/sequence.js:224:18)
    at /Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/async/dist/async.js:4617:26
    at /Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose-sequence/lib/sequence.js:269:24
    at /Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose/lib/model.js:4670:16
    at process.nextTick (/Users/rer/Documents/Private/Angular Applications/mongodb-monCSE-restful-api/node_modules/mongoose/lib/query.js:2682:28)
    at process._tickCallback (internal/process/next_tick.js:61:11)
  message:
   'Can\'t save() the same doc multiple times in parallel. Document: 5c6526f884611085d073b031',
  name: 'ParallelSaveError' }

Could it be that there might be a conflict between the automatic update of the updateSequence counter and the manual update of the sequence counter.

Am I doing something wrong ?

[rernens]

@ramiel

correction !

Counters are incremented but sequence counter is not set properly in all employees documents.

Missing in some, those for which the error shows up in the log.

further correction :

not linked to the fact that there is two counters.

Same happens with just employees_counter / sequence.

[rernens]

@ramiel

hi Fabrizio

Looking further into this and getting deeper in pre-save hook documentation, I found out that the issue comes from the save you do in the setNext method.

A very little change would then allow to use the setNext method in a pre-save hook as I wish to :

this._schema.method('setNext', function(id, save, callback) {
    var sequence = sequenceArchive.getSequence(id);
/*
 * for compatibility reason check if the second parameter if of type function meaning the new
 * save args is not provided  as per current way of invoking setNext
 */
    if (save instanceof Function ) {
      withSave = true ;
      callback = save;
   } else {
     withSave = save;
  }


    if (_.isNull(sequence)) {
      return callback(new Error('Trying to increment a wrong sequence using the id ' + id));
    }
        // sequence = sequence.sequence;

    sequence._setNextCounter(this, function(err, seq) {
      if (err) return callback(err);
      this[sequence._options.inc_field] = seq;
/*
 * conditionnaly save the document base on args, unchanged default behavior
*/
      if ( withSave ) {
         this.save(callback);
     } else
       callback();
    }
    }.bind(this));
  });

I hope you will agree that this is a minor change which provide a major benefit : allowing to set counters on demand in pre-save hook.

[ramiel]

Hello @rernens.
I have no time in these days to check this. I'll be back the next week. If there some code change you feel is necessary, you can make a pull request, it's very welcome :)

[rernens]

@ramiel

hi Fabrizio

yes I will make the pull request and submit the code change, straight away.

[rernens]

@ramiel

Hi Fabrizio,

Have you had a chance to review my PR ?

I works fine for me but I had like to get it committed to the base code in order to be able to update my other dependencies.

Thanks in advance

[btknorr]

Any update on this? We also need the ability to use setNext in a pre save hook. I personally think setNext should not call save by default. Thanks!

[ramiel]

There was a PR about this but it didn't pass the tests and now it may be outdated. Have a look at the PR to see if you can use the code from @rernens directly.