We have a database with following parameters:
- 30k records, 7mb in size
- 20 inserts/second
- 1000 updates/second
- 1000 range selects/second, by secondary index, approx 10 rows each
- needs at least one secondary index
- needs some mechanism to expire keys if they are not updated for 75 secs (can be done via programmatic garbage collector but will require additional 'last_update' index and will add some load)
- consistency is not required
- durability is not required
- db should be stored in memory
For now we use Redis, but it does not have secondary index and it's keys index:foo:*
is too slow. Membase also does not have secondary index (as far as I know). MongoDB and MySQL memory engine have table-level locks. What engine will fit our use case?
I think Redis does indeed support secondary indexes, but they are not implicit: you have to maintain them explicitly. A sorted set can be used for a range scan, but it is less convenient than what you could have in MongoDB or a relational database.
If you are looking for a Redis-like server with better support for range scans, I would suggest to have a look at the discontinued AlchemyDB project, or the Aerospike 3 (commercial) product (which includes part of the features of AlchemyDB).
http://code.google.com/p/alchemydatabase/
http://www.aerospike.com/