I am implementing microservice on Tarantool vinyl database and need to make basic architecture decision - should I use one big space for all the data, or use many spaces with same tuple format.
The space will be used as some kind of state storage of many instances of application. This space will be base for calculations, so it will take intensive read/write load.
I can store all the tuples for all app instances in one big table, or I can separate data into many spaces per one app instance - each one space named for app instance ID. And this separation will be more convinient from the code point of view.
So the question is what cons (if any) could I get with this separation? I mean will Tarantool spend more time or recources to "switch" between spaces or something like that? Or maybe it works more efficient with caches when only one space is used?
The whole base size is about 1TB The number of app instances 10000-30000 The number of records in one app instance space vary from 1000 to 2000000
Thanks!
In short - you need sharding. There is no reason to split one space into multiple because one single Tarantool instance won't be able to handle that many requests per second.
What you need is vshard and (probably) cartridge to make a large number of instances easier to manage.
In your case, the best way will be to calculate the sharding key from the app ID.