Alternative to BerkeleyDB?

Question

I'm looking for a dbm-like library that I can use in place of Berkeley DB, which I'm currently using. My main reason for switching is the licensing fees for BDB are pretty high (free for open source apps, but my employer does not want to open source this particular app for various reasons).

I've looked briefly at qdbm but it doesn't look like it will fill my needs -- lots of keys (several million) and large data items (> 1-5 megabytes). Before I continue my search I figured I'd ask because it seems there are tons of dbm-like libraries out there.

Answer 1

C/C++

• http://sphia.org/ a Key/Value Store, Written in C and BSD-Licensed.
• https://code.google.com/p/leveldb/ Written in C++, also BSD-Licensed
• http://luxio.sourceforge.net/ Written in C++, LGPL, faster than Berkeley DB
• http://symas.com/mdb/ Written in C, License is OpenLDAP Public License (BSD-Like)
• https://github.com/davidar/sdbm Written in C, Public Domain, Reimplementation of "ndbm"
• http://repetae.net/computer/vsdb/ Written in C, License unknown, Experimental
• https://github.com/stevedekorte/skipdb Written in C, BSD-Licensed, Key-value Store using Skip-List indexes
• http://unqlite.org/ written in C/C++, Embedded NoSQL Database Engine, BSD-Licensed, Similar to SQLite.
• http://hamsterdb.com/ Written in C/C++, Embedded NoSQL Database Engine, Apache License 2.0.

Java

• https://github.com/dain/leveldb A Java port of LevelDB, Apache License 2.0
• https://github.com/jingwei/krati A former LinkedIn Project, Apache License 2.0
• http://www.mapdb.org/ An Embedded Java Database, Apache License 2.0, supports Key-Value Storage, very efficient Object serialisation
• https://github.com/OpenHFT/Chronicle-Map, LGPL v3, embedded pure Java key-value store, faster than Berkeley DB
• https://github.com/jetbrains/xodus, Apache License 2.0, a Java transactional schema-less embedded database, faster than Berkeley DB
• https://github.com/lmdbjava/lmdbjava, JNI binding for LMDB, Apache License 2.0

Answer 2

You could try JDBM. It is a free (Apache 2) key-value store with disk persistence. Simple API and high performance

Answer 3

SQLite is public domain, meaning you can use it for any purpose whatsoever, and is widely used and supported.

Answer 4

db4o is pretty cheap and fast but it can only be used with java or .net

Answer 5

Firebird is your best friend.

Answer 6

You could look at Tokyo Cabinet. Its the successor to qdbm/gdbm, and if you decide to scale has a nice network front-end available.

Edit:

Another variant is Kyoto Cabinet; developed by the same person, but supposedly easier to use.

Answer 7

Postgres or HSQLDB and possible even H2 database

Answer 8

You can get much improved performance out of any dbm (even qdbm) and improved parallelism with a simple level of indirection: Simply take your keys and hash them, and use data_dir/H(key)/ as the database to store those keys. Limit the hash output to a small value (say) 255 for best results.

This approach has a number of benefits, easily summarized:

• Conceptually simple
• Easy to implement and test
• Doesn't lock the entire database for updates
• Can support much larger databases
• Easy to replace the DBM component

The hash probably doesn't even need to be cryptographically secure; just mostly uniform. DJB's cdb hash works well in most cases.

Answer 9

If you are on Windows then you can use the builtin esent database engine. http://blogs.msdn.com/windowssdk/archive/2008/10/23/esent-extensible-storage-engine-api-in-the-windows-sdk.aspx