Read scalability

Introduction

WiredTiger is architected to scale linearly on multi-threaded workloads. This test demonstrates that scaling for a read only workload. It also compares the results to another popular key/value store - LevelDB.

Results

The test executes 10 million reads in each of the threads. So the 32 thread version is doing 32 times as much work as the single threaded version.

Here are those results in a table:

Thread count	WiredTiger us/op	LevelDB us/op
1	3.44	7.19
2	3.41	7.63
4	2.97	7.85
8	3.39	9.71
12	3.31	19.05
16	3.31	27.31
20	3.55	35.63
24	3.73	45.44
28	3.94	56.97
32	4.25	69.72

The same results as operations per second rather than latency.

Thread count	WiredTiger ops/second	LevelDB ops/second
1	290,697	139,682
2	586,510	262,123
4	1,346,801	509,554
8	2,359,882	823,892
12	3,625,377	629,921
16	4,833,836	585,865
20	5,633,802	561,324
24	6,434,316	528,169
28	7,106,598	491,486
32	7,529,411	458,978

Setup

Benchmark configuration

The benchmark inserts 10 million items into a WiredTiger table, with 16 byte keys and 100 byte values creating about 800 MB of data.

The read phase of the benchmark uses a 10GB cache, has a variable number of threads doing WT_CURSOR::search operations for random values over the key range. During the read phase each thread completes 10 million operations.

Machine information

Run on an AWS c3.8xlarge instance type, which has:

• 60 GB RAM
• 2 x 320 GB SSD drives configured with raid 0 and xfs
• 4 physical Xeon E5-2680 CPUs - 32 vCPUs

Code setup

Benchmark and LevelDB source code at revision: https://github.com/wiredtiger/leveldb_wt/tree/ffd56f7eb55d750bd64ef8e77e1b3f8c1d1996cc

With compression disabled for the WiredTiger version.

WiredTiger source code at revision: https://github.com/wiredtiger/wiredtiger/commit/562892d2c5ed3c2e4679e1650411224324059f58

Configuration and Compilation

WiredTiger

./configure --enable-snappy && make

LevelDB and Benchmark Code

LD_LIBRARY_PATH=~/work/wiredtiger/build_posix/.libs:~/work/wiredtiger/build_posix/ext/compressors/snappy/.libs
env LDFLAGS="-L~/work/wiredtiger/build_posix/.libs" CXXFLAGS="-I~/work/wiredtiger/build_posix" make all db_bench_wiredtiger

Load command

./db_bench_wiredtiger --use_lsm=0 --cache_size=10000000000 --benchmarks=fillseq --num=10000000 --value_size=100 --db=results --use_existing_db=0 --threads=1

Read phase command

for i in 1 2 4 8 12 16 20 24 28 32 ; do
./db_bench_wiredtiger --use_lsm=0 --cache_size=10000000000 --benchmarks=readrandom --num=10000000 --reads=10000000 --value_size=100 --db=results --use_existing_db=1 --threads=$i
done

To run the equivalent with LevelDB, use the same commands, but run ./db_bench rather than ./db_bench_wiredtiger and remove the --use_lsm=0 option.