Too large batch size or timeout while waiting for server response while exporting data into cassandra

419 Views Asked by At

I'm trying to import data from CSV files to cassandra using cassandra-loader: https://github.com/brianmhess/cassandra-loader

Cassandra-version: 3.1

Export data from cassandra into CSV files takes ~ 20 minutes which is ok. Import data using this command: cassandra-loader -f "path to files" -host "host" -schema "scema" -numThreads 40 takes more than 7 hours. If i try to increase -batchSize over than ~1.5k and -rate over than 50k than i get an error.

Error inserting: Batch too large
com.datastax.driver.core.exceptions.InvalidQueryException: Batch too large

Sometimes i get error like "Server responce timeout"

Timed out waiting for server response
com.datastax.driver.core.exceptions.OperationTimedOutException: [] Timed out waiting for server response
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:772)

How can i fix it and speed it up?

Or maybe there is another way to import fast data from CSV into cassandra

0

There are 0 best solutions below