Skip to content

YCSB Mapkeeper benchmark

Jump to bottom
agorrod edited this page Jan 7, 2013 · 3 revisions

Introduction

The goal of this benchmark is to compare WiredTiger, MySQL and LevelDB for disk I/O bound, multiple-reader/multiple-writer workload, based on the original YCSB mapkeeper benchmark.

Configurations

The YCSB mapkeeper benchmark code used can be cloned from github:

https://github.com/wiredtiger/mapkeeper

This page has information on the process for setting up other dependencies required to run the YCSB mapkeeper benchmark:

https://github.com/m1ch1/mapkeeper/wiki/Getting-Started

Hardware

  • Amazon Web Services (AWS) High I/O instance:
  • 60.5 GiB of memory
  • 35 EC2 Compute Units (16 virtual cores*)
  • 2 SSD-based volumes each with 1024 GB of instance storage
  • 64-bit platform
  • I/O Performance: Very High (10 Gigabit Ethernet)
  • API name: hi1.4xlarge

Operating System

  • Amazon Linux AMI x86_64 EBS (3.2.30-49.59.amzn1.x86_64)

File System

  • XFS

WiredTiger

  • Compression: Snappy compression
  • LSM configuration: "internal_page_max=128K,leaf_page_max=128k,lsm_chunk_size=200MB,lsm_bloom_config=(leaf_page_max=10M),block_compressor=snappy"
  • Database configuration: "transactional,cache_size=2GB,sync=false,session_max=120"
  • Solid line: 1.3.8 WiredTiger release

MySQL

  • MySQL Server 5.5.28 Percona
  • Compression: Enabled via InnoDB RAW_FORMAT=compressed
  • innodb_buffer_pool_size = 2G
  • innodb_flush_log_at_trx_commit = 0

NOTE: The MySQL configuration differs from that of the original benchmark. We enabled compression and disabled DIRECT_IO (enabled the disk buffer cache). The changes were made based on feedback we received so the different data stores are configured as similarly as possible.

LevelDB

  • Release 1.7 (svn revision r69)
  • Compression: Snappy compression
  • options.write_buffer_size = 500 * 1048576; // 500MB write buffer
  • options.block_cache = leveldb::NewLRUCache(1500 * 1048576); // 1.5GB cache
  • options.sync = false;
  • Blind inserts and updates configured
  • Options to mapkeeper_leveldb: -w 500 -b 1500 -i -u

YCSB Workload

  • Use the same box as MySQL/LevelDB
  • Record size: 4KB
  • Number of records: 10 million
  • Number of client threads: 100
  • Read/Write ratio: 80/20

Results

The raw data and scripts used to process it are here.

Sequential Insert Performance

WiredTiger sustains a much higher throughput and lower latency than either LevelDB or MySQL. LevelDB maintains a higher overall throughput than MySQL, though there are long periods where no inserts at all are being processed in LevelDB.

[(https://raw.github.com/wiki/wiredtiger/wiredtiger/attachments/mapkeeper_ycsb_insert_latency.pdf)

80% Read / 20% Update Performance

WiredTiger performs significantly better than LevelDB and MySQL in this benchmark. It maintains much lower and more consistent average and 99th percentile latency even when heavily loaded.

\

Comments?

Send me email at wiredtiger-users@googlegroups.com or post your comments in the wiredtiger-users group if you have any questions/suggestions.

Clone this wiki locally