What is the best way to upload large data set to tarantool

267 Views Asked by At

I have defined a simple table in Tarantool and I'm trying to fulfil it with 1 millions of record. I'm using a c# code with progaudi.tarantool library. I'm using two approaches:

box.ExecuteSql(/*My insert statement*/) and space.Insert<Item>(item)

Both are extremlly slow. Using box.stat() in the terminal I can see that I'm able to insert only 63 requests per second.

I haven't noticed any approach where I can do the bulk insert. I tried to send a multiple sql statements but it's failing. Is there any approach that I can use to upload data quickly or investigate why only 63 insert requests per second are done?

Edit 1: I found the following information when executing box.slab.info()

- items_size: 32203920
   items_used_ratio: 97.42%   !!
quota_size: 268435456   
quota_used_ratio: 18.75%   
arena_used_ratio: 82.1%   
items_used: 31373320   
quota_used: 50331648   
arena_size: 50331648   
arena_used: 41302024 ...

Those settings were defined:

 box.cfg{memtx_max_tuple_size=100*1048576}
 box.cfg{memtx_memory=3*268435456}
0

There are 0 best solutions below