Leveldb v/s berkeley db

I had a requirement to index a good chunk of data (around 250M key value pairs) and wanted to try out both berkeleydb and leveldb. Here are some metrics when running on an amazon m1.xlarge machine. 

  1.  Time to build the index / db:
  • Berkeley DB: 1hr 52 minutes
  • Level DB: 33 minutes

    2.  Time to lookup 6719350 keys from the db (random lookup) on the second run 

  • Berkeley DB (cold): 25m 2s
  • Berkeley DB (hot): 1m 41s
  • Level DB (cold): 3m 16s
  • Level DB (hot) : 2m 18s

    3. DB Size

  • Berkeley DB: 9.5G
  • Level DB: 3.5G
Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Leveldb v/s berkeley db

  1. Danielle says:

    Nice post. Would love to add your review of Oracle Berkeley DB to IT Central Station. Other Berkeley DB users have enjoyed using it as well: https://goo.gl/RPtwu3.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s