No matter what kind of DB you take, it will almost always be a set of key-value pairs collections. Every table in your SQL DB, every index, every document DB and Graph DB relies on a fast underlying Key-Value Store (KVS) Engine. Those are generally implemented as persistent tree datastructures, such as B-Trees, B+-Trees or LSM Trees. Having faster KVS is an essential prerequisite to building fast DBMS and is of insane value itself.

To compare those, we used the Yahoo Cloud Serving Benchmark (YCSB), which has beed the industry standard since 2010. It randomly generates data on the fly and compares DBs in a broad variety of mixed read/write/scan workloads.

Reproduce Our Results

Setup

Device

  • CPU:
    • Model: 2x AMD Epyc 7302.
    • Cores: 32 (64 threads @ 3 Ghz).
  • RAM Space: 256 GB.
  • Disk Space: 2 TB.

Working with 100 MB data

100MB: Initialization

Name CPUs Avg MEM Size on Disk Throughput (ops/s)
WiredTiger 312.4% 106 MB 131 MB 31'306
RocksDB 71.2% 53 MB 118 MB 50'358
LevelDB 105.8% 54 MB 118 MB 58'153
LMDB 89.8% 100 MB 215 MB 32'396
UnumDB 303.4% 57 MB 116 MB 81'877

100MB: Read/Update (50/50)

Name CPUs Avg MEM Size on Disk Throughput (ops/s)
WiredTiger 237.9% 131 MB 243 MB 55'588
RocksDB 85.5% 67 MB 177 MB 45'379
LevelDB 116.4% 122 MB 177 MB 53'973
LMDB 81.8% 182 MB 215 MB 64'162
UnumDB 254.5% 12 MB 120 MB 169'512

Working with 1 GB data

1GB: Initialization

Name CPUs Avg MEM Size on Disk Throughput (ops/s)
WiredTiger 408.4% 192 MB 1.32 GB 35'338
RocksDB 141.2% 267 MB 1.18 GB 66'444
LevelDB 167.8% 216 MB 1.40 GB 55'298
LMDB 99.8% 989 MB 2.06 GB 40'408
UnumDB 480.4% 126 MB 1.16 GB 132'744

1GB: Read/Update (50/50)

Name CPUs Avg MEM Size on Disk Throughput (ops/s)
WiredTiger 295.9% 223 MB 2.44 GB 46'670
RocksDB 135.5% 239 MB 1.55 GB 52'538
LevelDB 182.4% 135 MB 1.59 GB 55'688
LMDB 96.8% 1.99 GB 2.07 GB 77'656
UnumDB 469.5% 70 MB 1.19 GB 225'632
  • Tweet
  • Subscribe to receive similar articles 📨
  • Reach Us to try the fastest data processing software 🔥