Export a set of documents with given ids

[hshenMcc]

We have a db with 2 million documents in prod. I need to export 14 of them to a testing environment. I noticed an option "Use Transform script". But really have no idea how to construct the script. Help please. Thanks.

[ayende]

The easiest way to do that is to run a query:

from @all_docs where id() in ('employees/1-A', 'orders/1-A')

Then export as CSV and import as CSV

If you want to do that using import / export (which is important if you want to include attachments, time series, etc). You can filter during the export process (or the import one, for that matter).

Here is the relevant script:

const docIds = ["employees/1-A", "orders/1-A"];

if(docIds.indexOf(id(this)) == -1)
    throw 'skip';

Note that this still needs to scan through all the documents in the database. That is why if you just need to push the raw document data, you can do that more easily through CSV.

[hshenMcc]

Thanks. This is very helpful. I need the attachments to be exported. I have to use "Transform script". Do you have a document to specify how to construct such a script in general? Next time, I may need to export a set of documents based on a range of document created date. Thanks again.

[ayende]

That is just JavaScript, and you can throw 'skip' to skip documents.
If you need created date, you can access this["@metadata"]["@last-modified"] or a CreatedDate property

[hshenMcc]

I am wondering other than "skip", is there a "include". In my case, it seems not that right to throw exception 2 million times just to get 14 documents.

[ayende]

No, you are going to scan through them anyway. The throwing doesn't matter

[hshenMcc]

Ok. Thanks.

[hshenMcc]

May I ask what's the syntax to filter on a property? For example, document has a property 'UserName'. How to filter on UserName matching 'ABC' and 'DEF'.

Thanks.

[ayende]

const matches = ["ABC", "DEF"];

if(matches.indexOf(this.UserName) == -1)
    throw 'skip';

[hshenMcc]

Thanks. Tried it in prod. It takes about 7 minutes to dump 35 documents out of 2.7 million. Does this have any impact on other normal operations?

We have a static index on UserName. I think it's more efficient to query on the index and then do dump. Is that possible?

[ayende]

It scans through all the data, that is expensive, yes.
If you can do that using a query, that is better. Note that it would just do the JSON if you are using the studio / CSV method.
Easier to just write some code that would copy those directly if it happens oftern.