Is it possible define the schema before import the data ?

[theseeker16]

Hi, I was wondering if is it possible to define the schema to data (not a typeORM model ), it's because in my schema I defined some attributes unique and generate some indexes, so when I insert the data it skipped all these rules.

[pkosiec]

Hi @theseeker16, you can achieve that with custom app using mongo-seeding JS/TS library, by making a separate DB connection etc. There is no way to do it with CLI or Docker Image.

There is a room for improvement though - e.g. we could reuse DB connection to do our stuff or something like that.

[theseeker16]

@pkosiec I'm using mongo-seeding in a custom project(TS) without CLI or Docker, I already define some schemas, DB connection, etc.. is ready, but how can I define the schema into my data?

[pkosiec]

@theseeker16 Unfortunately, you cannot do that with Mongo Seeding itself. But, of course, you can make a separate connection to the DB on your own and define a schema - before the data import with Mongo Seeding. That's the only option for now.

[theseeker16]

@pkosiec I see, it could be great if you can set to which scheme to reference the data, so I can reutilize the schema that I made it. Thanks for the help @pkosiec 👍

[pkosiec]

Hmm, do you have any specific ideas, how the API could look like?

[theseeker16]

Sorry, I don't have any ideas right now, this is my second time using mongo-seeding, actually, I'm using it for some automatized test, because I need to verify the information can't be duplicated.

[pkosiec]

Hi again @theseeker16,
I want to revisit this topic - Just to clarify: are we talking about JSON schema validation from MongoDB 3.2 or do you mean something else? https://docs.mongodb.com/manual/core/schema-validation/

You also mentioned creating indexes, which means the following functionality, right? https://docs.mongodb.com/manual/indexes/#create-an-index

I'm still thinking about end-user API, and I conclude that it has to be done with optional file inside collection directory with some reserved name (like .mongo-seeding.config.json), where you could define this kind of data.

[theseeker16]

Hello again @pkosiec

This is an example of what indexes and schema I'm talking about:

import * as mongoose from 'mongoose';
import * as mongoosePaginate from 'mongoose-paginate';

export const UserSchema = new mongoose.Schema({
  email: {
    type: String,
    required: true,
    unique: true,
    index: true,
    dropDups: true,
  }
});

UserSchema.plugin(mongoosePaginate);

My thought at that moment, it's was because when you insert the data as a JSON object without any Model Type (in the example, you defined a TypeORM, but I thought in reutilized my model), so, mongo-seeding opens a DB connection, the model created using mongo-seeding did not contain any indexes or properties like this:

When I made some tests with mongo-seeding to push the data automatically and then tested it, I could not test, for example, the email could be unique.

sorry for my English it could not be good.

[pkosiec]

Ah, okay, so it's all about Mongoose models support.

Currently, Mongo Seeding doesn't support Mongoose models import as it would require custom logic for them (which means it would need a dependency to mongoose itself).
Unfortunately, It would be a huge change and I can't promise I will implement it 😞. Sorry for that! I already described the downsizes of using mongoose here: https://github.com/pkosiec/mongo-seeding#problem-2-no-data-model-validation

Did you try other solutions prepared specifically for Mongoose, for example https://github.com/seanemmer/mongoose-seed?

[theseeker16]

@pkosiec don't worry about it. I'll check the other solution just in case if I need it. Thank you

[pkosiec]

Alright, thank you. I will close the issue for now, but feel free to comment it in case you have an idea, how I could adjust Mongo Seeding to support "raw" MongoDB import, as well as Mongoose one.

In future I want to rewrite Mongo Seeding (planned for 4.0). I will keep in mind this case and I will try figure out generic, pluggable solution.