esbulk 0.4.7

• 1.4.X compat fixes

esbulk 0.4.6

• try to handle case, where creating an already existing index results in a HTTP 400 (e.g. elasticsearch 1.4.3)

esbulk 0.4.5

• fixes #10

esbulk 0.4.4

• API adjustements and fixes: #9

esbulk 0.4.3

• add support for nested ids and concatenated ids via #8, thanks @sakshambathla

esbulk 0.4.2

• add X-Pack, HTTP basic auth support, with curl syntax:

$ esbulk -u username:password -index hi file.ldj

esbulk 0.4.1

Previously it was possible to silently lose documents, if a bulk operation failed. Starting with this release, the bulk operation error is checked and the indexing is aborted, in case the operation failed.

There are two ways to deal with such an error:

• reduce the number of workers (using the defaults should be fine)
• increase thread_pool.bulk.queue_size in your elasticsearch settings

This severe error occured in scenarios, where esbulk would exhaust the thread pool quickly (e.g. high number of workers and small elasticsearch thread pool queue size). Using the defaults (e.g. number of workers equals number of cpus) would result in correct results in most cases, which is probably why this bug could linger so long.

esbulk 0.4.0

• attempted fix for #5

esbulk 0.3.9

Allow integer ids in document, when using -id, see also #3.

Update 2016-09-20: Replaced outdated binaries with binaries built from 201490f.

esbulk 0.3.8

Allow reuse of document ids for elasticsearch via -id flag.

If you want to reuse IDs from your documents in elasticsearch, you
can specify the ID field via -id flag:

$ cat file.json
{"x": "doc-1", "db": "mysql"}
{"x": "doc-2", "db": "mongo"}

Here, we would like to reuse the ID from field x.

$ esbulk -id x -index throwaway -verbose file.json
...

$ curl -s http://localhost:9200/throwaway/_search | jq
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "throwaway",
        "_type": "default",
        "_id": "doc-2",
        "_score": 1,
        "_source": {
          "x": "doc-2",
          "db": "mongo"
        }
      },
      {
        "_index": "throwaway",
        "_type": "default",
        "_id": "doc-1",
        "_score": 1,
        "_source": {
          "x": "doc-1",
          "db": "mysql"
        }
      }
    ]
  }
}