Redis 2.0.3
I need to store a huge list of items in Redis. Each item is a short string (less than 256 characters).
I need to do two operations on the list:
Add many (several thousands to a million) identical items. (Several times a day)
Remove one random item from the list. It is not necessary to have "fair" random. Any "good enough" approach will do. (Up to several hundred times a second)
I do not have enough RAM to store all items in a list one by one.
I think that I need to store items in batches, name and a counter. (There will be up to several thousands distinct items, more like several hundreds.)
But I'm not sure how to organize this effectively.
Any hints?
Well, since nobody is up to help me, here is a "dumb" solution, in pseudocode.
Get random element:
Insert new elements
Please do suggest a better solution if it exists, I'm still groking Redis, and may miss something obvious.
I suspect that
LOCK
/UNLOCK
ininsert_items()
may be superfluous and can be replaced withMULTI
/EXEC
, but I think that it is needed forLOCK
/UNLOCK
inmaybe_get_next_item()
to work properly (which I do not know how to replace withMULTI
/EXEC
)...