Skip to content
/ fakerdb Public

Generate an unlimited stream of JSON schema instances using json-schema-faker, faker, chance and insert the data into a supported database, e.g.: nedb, mongodb, postgres, mssql

License

Notifications You must be signed in to change notification settings

sius/fakerdb

Repository files navigation

Build Statuscodecovdependencies StatusdevDependencies StatusGitHub

fakerdb

Generate an unlimited stream of JSON schema instances using json-schema-faker, faker, chance and insert the data into a supported database, e.g.: nedb, mongodb, postgres, mssql.

Getting Started

npm install

# run docker container
docker-compose up -d

npm test

# stop docker container
docker-compose down

Test Results and Performance

Test Environment

  • Intel® Core™ i7-3520M CPU @ 2.90GHz × 4
  • 7.5 GiB
  • Linux 64 bit

JSON schema

// person.json
{
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "faker": "name.findName"
    }
  },
  "required": [
    "name"
  ]
}

Insert 10^5 Records

insert 10.000 generated person records

Insert 10^6 Records

insert 100.000 generated person records

Insert 10^7 Records

insert 1.000.000 generated person records

API Usage

Install

npm i fakerdb

Stream to STDOUT

// examples/faker-stdout.js

var { replay, jsf } = require('fakerdb')
, faker = require('faker')
, jsonSchemaFaker = require('json-schema-faker')
, path = require('path');

jsonSchemaFaker.extend('faker', () => faker);

const SCHEMA_OBJ = {
  type: 'object',
  properties: {
    name: {
      type: 'string',
      faker: 'name.findName'
    }
  },
  required: [
    'name'
  ]
};

replay(SCHEMA_OBJ, { replay: 2 })
  .pipe(jsf(jsonSchemaFaker))
  .pipe(process.stdout);

const SCHEMA_FILE = path.join(__dirname, './schema/person.json');
replay(SCHEMA_FILE, { replay: 2 })
    .pipe(jsf(jsonSchemaFaker))
    .pipe(process.stdout);

Stream to file with nedb

// examples/faker-nedb.js

let { generate }  = require('fakerdb')
  , Datasource = require('nedb')
  , db = new Datasource({ filename: 'faker.db', autoload: true })
  , path = require('path');

const REPLAY = 1000;
const SCHEMA  = path.join(__dirname, './schema/person.json');
const OPTS = { replay: REPLAY, insert: { blockSize: 1000 } };

generate(db, SCHEMA, OPTS);

Stream to database

fakerdb CLI

Use

npm i -g fakerdb
fakerdb --help

Custom Faker Example

Create Standard Normal Distributed Samples

Use

cd examples/faker
fakerdb g -r 200000 -i person.json -f custom-jsf.js -b 1000 -o people.db 

References

About

Generate an unlimited stream of JSON schema instances using json-schema-faker, faker, chance and insert the data into a supported database, e.g.: nedb, mongodb, postgres, mssql

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published