Constraint enforcements on JOINed references should be optional

[onhate]

Description

I would like to bring a discussion on the topic of how JOINS work now.

As of the current implementation Typesense is enforcing constraints validations, for example, if you index a ticket that has a field agentId that references an agents collection Typesense will reject the indexing of tickets whose agentId value does not exists on the agents collection.

I understand the benefits of this approach, but thinking as Typesense not being the source of true and it in most cases being a source that synced from other database it adds complexity to the syncing of data (specially batch re-indexes) because now I am forced to first index all the agents and then only being able to index the tickets.

Also, it will not work if you already use NoSql databases that don't have constraint validations and allow you to have things like ticket.assigee = 'group:XXX' that does not really refers to an agent...

I'll raise a issue on the repository to continue the discussion, but in my opinion there would be an option to allow "loose constraints" on references.

[kishorenc]

We could add an optional property that can be enabled for the reference field, e.g. async_reference: true. When indexing documents of any collection we'll check if it's being referenced and whether the field is marked as async_reference and the add the references appropriately.

[onhate]

and internally it would work as we discussed on slack? creating a placeholder for missing references that later will be updated when the record is inserted in the referred collection? If so, the filed name async_references is very use case specific, a name like loose or no_constraint would allow better understanding of the feature, wdyt?

[kishorenc]

Yes, correct. We will have more details when we get down to the implementation details.