Skip to content

Latest commit

 

History

History
166 lines (142 loc) · 5.08 KB

readme.md

File metadata and controls

166 lines (142 loc) · 5.08 KB
Raw

Benchmark 0chain smart-contract endpoints.

Runs testing.Benchmark on each 0chain endpoint. The blockchain database used in these tests is constructed from the parameters in the benchmark.yaml. file. Smartcontracts do not (or should not) access tha chain so a populated MPT database is enough to give a realistic benchmark.

To run

DOCKER

  1. run init
./docker.local/bin/init.setup.sh
  1. build base image
./docker.local/bin/build.base.sh
  1. build docker image
./docker.local/bin/build.benchmark.sh
  1. change dir to benchmarks
cd docker.local/benchmarks
  1. run tests
../bin/start.benchmarks.sh

Script can be run with different options:

  • load
  • tests
  • config
  • verbose
  • omit

BARE METAL

go build -tags bn256 && ./main benchmark | column -t -s,

It can take a long time to generate a MPT for the simulation. To help with this it is possible to save a MPT for use later, set the options.save_path key in benchmark.yaml.

options:
  save_path: ./saved_data

You can now reuse this database using the load option in the command line

go build -tags bn256 && ./main benchmark --load saved_data  | column -t -s,

To run only a subset of the test suits

Note: when run from docker list of tests should be passed as space delimited list, not comma delimited, it caused with bug in viper spf13/viper#380

go build -tags bn256
./main benchmark benchmark --tests "miner, storage" | column -t -s,

To only print out the comma delimited data without any trace outputs, use the --verbose=false flag

go build -tags bn256
./main benchmark  --verbose=false | column -t -s,

To filter out test from the benchmark use the -ommit option, and enter them in a comma delimited list.

go build -tags bn256
./main benchmark --omit "storage_rest.allocation, storage_rest.allocations" | column -t -s,

To use the event database you need a to set up a local postgreSQL database. Login in parameters are read from the benchmark yaml, dbs.events section.

  • MacOS
    1. brew install postgres
    2. initdb /usr/local/var/postgres
    3. pg_ctl -D /usr/local/var/postgres start
    4. /usr/local/opt/postgres/bin/createuser -s postgres

Create zchain_user

CREATE ROLE zchain_user WITH
LOGIN
NOSUPERUSER
NOCREATEDB
NOCREATEROLE
INHERIT
NOREPLICATION
CONNECTION LIMIT -1
PASSWORD 'zchian';

Create events_ds database

CREATE DATABASE events_db
WITH
OWNER = zchain_user
ENCODING = 'UTF8'
CONNECTION LIMIT = -1;

Add connectivity details to the config

dbs:
  events:
    enabled: true
    name: events_db
    user: zchain_user
    password: zchian
    host: localhost
    port: 5432
    max_idle_conns: 100
    max_open_conns: 200
    conn_max_lifetime: 20s

Set enabled to false if you have not setup a postgreSQL database. Some of the Rest Api endpoint will not work without an event database.

You can also set all these options in the benchmark.yaml. file. The command line options will take precedence over those in the .yaml file.

The benchmark results are unlikely to be false positives but could be false negatives, if benchmark parameters are such that a particularly long running block of code is accidentally skipped.

The output results are coloured, red > 50ms, purple >10ms, yellow >1ms otherwise green. To turn off, set colour=false in benchmark.yaml. or use --verbose=false.

For best results try to choose parameters so that benchmark timings are below a second.

Event database benchmark

To run a event database benchmark set the option.event_database_benchmarks: true in the config file. This will test the events saved in the file indicated in the config file, option.event_database_event_file.

The event_database_event_file is a JSON file that contains a map of event obejcts. The map key is used as a label for displaying the results.

The use can make their own event_database_event_file file or use the one generated output from the smart-contract benchmarks. To do this set the two JSON files to be the same.

The option.smart_contract_event_file option will save the events generated by the smart-contract to a JSON file with the correct format to be used for the event database benchmark.

Example config file settings to link the smart-contract benchmark events to the event database benchmarks.

options:
  event_database_benchmarks: true
  smart_contract_event_file: "edb_in.json"
  event_database_event_file: "edb_in.json"