Ability to add object-arrays to schema

[grabbou]

It'd be useful to have the ability to force specific values and keys on all properties inside your object.

In my use-case, I have provider property attached to my main schema that contains multiple providers in it (Facebook, google+ and other passport goodies). The problem is that I don't want to use arrays as dot noted access matters here.

Here's the current snippet of code:

new Schema({
    provider: {
        facebook: {
            accessToken: String,
            refreshToken: String,
            id: String,
            displayName: String
        },
        instagram: {
            accessToken: String,
            refreshToken: String,
            id: String,
            displayName: String
        },
        // and so on....
   }
});

It'd be super handy to do something like:

new Schema({
    provider: {
        [validation-rule-for-key]: {
            accessToken: String,
            refreshToken: String,
            id: String,
            displayName: String
        }
   }
});

Kind of similar to #681, but that issue is only speaking about validation for keys, not about ability to have multiple properties. Validation of the key is actually not that important for me actually, can be anything. Should be faster to develop then.

What you think?

[aedanexplosive]

I agree with this as it would save a lot of code duplication when repeating the same structure for different keys.

[FelikZ]

Is there any workaround on this now?

[vkarpov15]

Not quite, and no real ETA for one unfortunately. Right now the best way to do this would be an array and the $elemMatch operator.

[rickdoesdev]

Not sure how to upvote this as a "yes please" feature, but, 'yes please'. I'm wanting to store custom form fields in one document and their values in another, and being able to look up values[field._id.toString()] is much faster than having to loop a values array and look for the value with matching fieldId property. Currently having to do values : Schema.Types.Mixed and just pushing new properties onto the values object for each value. Would love to be able to define something like

values : { 
  [key] : { ... schema for value validation ...  }
}

instead. Similar to OP's request, but for speed of look up of dynamic content in the DB, rather than to avoid copy pasting schema blocks, which I -can't- do in current situation

[vkarpov15]

@rcuddy if you're worried about performance, multi-key index and $elemMatch would work about as well performance-wise. Otherwise, you can just wait for #2689, which will fix this.

[paglias]

@vkarpov15 hi, I need something similar to what @grabbou requested (object-arrays) and I'm probably going to try adding it as a plugin but I want to make sure it's doable before starting (of course unless you planned to add this in the near future :) ):

1. Adding keys not defined in the schema to an instance is not supported:

   NOTE: Any key/val set on the instance that does not exist in your schema is always ignored, regardless of schema option.

   The key/val is simply ignored when saving or there's some other blocker that might prevent my plugin from working? Like it's not even added to the non-saved instance

2. I didn't read the code enough but I suppose each field in the schema is assigned a setter to intercept modifications to it without requiring to call markModified() every time. Is the setter avalaible publicly so that I can reuse it or is strictly internal?

3. Any suggestion that I might find useful?

Thank!

[vkarpov15]

In theory you could use type: Mixed and have your plugin handle validation in a pre('validate') hook.

Re: point 2, Document.prototype.set() should be what you're looking for.

Re: point 3, don't really know what the difficulties are off the top of my head. I haven't done any work on this yet though. Let me know how your attempts at a plugin go :)

[paglias]

I think I'm going to try this approach: slightly modify the
EmbeddedDocument type to work with an object of documents and setting the
modified paths on the parent document. And then add two methods ($add and
$remove) that will be used to add and remove documents from the object and
convert them to the proper type instance as there's no way to intercept new
and removed properties on an object.
Il 15/apr/2015 21:12, "Valeri Karpov" notifications@github.com ha scritto:

In theory you could use type: Mixed and have your plugin handle
validation in a pre('validate') hook.

Re: point 2, Document.prototype.set() should be what you're looking for.

Re: point 3, don't really know what the difficulties are off the top of my
head. I haven't done any work on this yet though. Let me know how your
attempts at a plugin go :)

—
Reply to this email directly or view it on GitHub
#2376 (comment)
.

[vkarpov15]

A custom type is an interesting avenue that I had not really considered, good idea :)

[paglias]

My idea was using Mixed with a custom validator but I'm wondering of
the performance implications of validating (or just checking if
modified and then validate) 1-2k sub documents stored in the object
arrays

2015-04-16 15:36 GMT+02:00, Valeri Karpov notifications@github.com:

A custom type is an interesting avenue that I had not really considered,
good idea :)

---

Reply to this email directly or view it on GitHub:
#2376 (comment)

Matteo Pagliazzi - paglias.net

[vkarpov15]

If you have a thousand subdocs you should really watch out for the 16MB doc size limit - just because it isn't an array doesn't mean that it's safe to have a document size that grows without bound :)

[andreafdaf]

Hi @vkarpov15, sorry for the tag but this is such an old issue that it has fallen out of view :)

We have an application where users can create custom fields and then attach them to existing documents, right now we are using Schema.Types.Mixed and it does the job but we miss the ability to then use mongoose stuff on the child documents, what we would like to accomplish is (simplified version):

const CustomFieldSchema = new Schema({
  name: { type: String },
  options: { ... },
})

const OtherSchema = new Schema({
  ...
  customFields: {
    [Schema.Keys.String]: { // that's just a supposition
      customField: { type: ObjectId, ref: 'CustomField' },
      value: { type: String },
    },
  },
})

We tried the array way but it gets really messy when using OtherSchema on aggregation pipelines with $project, or other querying nuances (we let users build their own tables in the frontend and then we automatically create a pipeline to fetch the data on the server, with lots of $lookup and $project in them), so the following way doesn't work out:

const CustomFieldSchema = new Schema({
  name: { type: String },
  options: { ... },
})

const OtherSchema = new Schema({
  ...
  customFields: [{
    customField: { type: ObjectId, ref: 'CustomField' },
    value: { type: String },
  }]
})

The first example is what we would like to have, but since we miss this "generic key" feature we end up with:

const CustomFieldSchema = new Schema({
  name: { type: String },
  options: { ... },
})

const OtherSchema = new Schema({
  ...
  customFields: { type: Mixed },
})

It works, but having mongoose magic on those subdocuments would be awesome!

As always, thanks for your time

[vkarpov15]

@andreafdaf I think you're looking for maps, here's some more docs on maps.

[andreafdaf]

Oh my god that's amazing, thank you so much! I guess I need to read release notes more often, that's a great feature that I never noticed

But now there is another question, how can I populate a map since the keys can be anything?
With an array like:

const OtherSchema = new Schema({
  ...
  customFields: [{
    customField: { type: ObjectId, ref: 'CustomField' },
    value: { type: String },
  }]
})

We can simply:

OtherModel.find(query).populate('customFields.customField')

But with:

const OtherToCustomFieldSchema = new Schema({
  customField: { type: ObjectId, ref: 'CustomField' },
  value: { type: String },
}, { _id: false })

const OtherSchema = new Schema({
  ...
  customFields: { type: Map: of: OtherToCustomFieldSchema },
})

How do we:

OtherModel.find(query).populate('customFields.(???).customField')

[vkarpov15]

OtherModel.find(query).populate('customFields.$*.customField') . We need to add that to the docs.