I am importing data from Kafka server using VoltDB Kafka Importer, my current setup is in AWS and all the 9 nodes has following config
8 vCPUs
200 GB HDD
8 GB RAM
The import rate is 10,000 recs per second.
My problem is the cluster enters into in Read-only mode after importing 42 million records, even if other 7 nodes are using only 20-30% of memory.
My table and stored procedure are partitioned. I enabled auto snapshot as well with 1 Hr frequency.
I am expecting 144 million records
What changes I should do to my configuration and how I can move data from memory to disk.
Please help.