I am looking for suggestions on my RocksDB configuration. Our use case is to load 100GB of key-value pairs into rocksdb and at run time only serve the key-value pairs in the db. Key is 32 bytes and value is 1.6 KB in size.
What we have right now is we used hadoop to generate a 100GB sst file using SstFileWriter api and save it off in S3. Each new server that comes up ingests the file using: db.ingestExternalFile(..). We use a i3.large machine (15.25 GiB | 2 vCPUs | 475 GiB NVMe SSD). P95 and avg response from rocksdb given the current configuration:
- BlockSize = 2KB
- FormatVersion = 4
- Read-Write=100% read at runtime
is ~1ms but P99 and PMAX are pretty bad. We are looking for some way to reduce the PMAX response times which is ~10x P95.
Thanks.
Can you load all of the db in memory using tmpfs ... basically just copy the data to RAM and see if that helps .... also it may make sense to compact the sst files in a separate job rather than ingesting at startup ... this may need a change in your machine config to be geared more towards RAM and less towards SSD storage