We are getting some weird load distribution on our cassandra (3.7) cluster. The cluster has 13 nodes, with 16 cores & 32G ram, and it is running behind Apache Usergrid (forked). When we put (artificial) stress on the application, same 3 nodes get almost all the CPU load and the rest of the nodes operate normally. We could not create the same scenario using cassandra-stress tool, with that all the load is distributed evenly. There is nothing different on the logs of these nodes either. gc logs also seem to be similar (running G1 gc for high heap memory). The queries originate from Usergrid, which we do not have much control. I need guidance for finding the problem. (So far nothing helpful from the internet.) Where should I look to diagnose the problem?
Uneven CPU load on Cassandra cluster
538 Views Asked by Eren Yilmaz At
1
There are 1 best solutions below
Related Questions in CASSANDRA
- how to create a chess board with Queen in the central position and all its moves in assembler code
- Passing arguments to ENTRYPOINT causes the container to start and run indefinitely
- Apache Cassandra Node Driver Connection
- Simulate Cassandra DB timeout
- How to update Cassandra Lucene index with a new column? rebuild or update index?
- Cassandra JDBC connection string for logstash
- Cassandra OversizedMessageException
- dsbulk unload is failing after ran couple of hours with OOM issue
- Cassandra: "Model keyspace not set" and "Connection name doesn't exist in the registry" Errors
- Unable to cqlsh to a cassandra docker container remotely
- Forward pagination with object mapper in java asyn
- Allow filter in cassandra query
- How to fix bytes unrepaired in cassandra
- Can't install Cassandra using RPM packages for RHEL 9
- Why can't get a connection to Cassandra running on Docker from a Spring Boot instace using spring-boot-starter-data-cassandra on first boot?
Related Questions in USERGRID
- Updated Maven dependencies and still getting warnings
- Resolving dependencies in Maven fails
- Issue with ng-map (googlemaps) and usergrid backend
- swift JSON calls, closures using usergrid IOS SDK and multiple calls
- Is usergrid recommended for apps with real time functionality?
- Usergrid system requirements?
- Usergrid Upgrade from 2.1.0 to 2.1.1
- Uneven CPU load on Cassandra cluster
- Usergrid does not run. Internal Server Error
- Usergrid node.js usergrid.init() error
- usergrid 2.1 ql not working with old records. portal also not giving the old records, but records count is available
- Usergrid is failed to start. usergrid.cluster_name property must be set
- Extremely high CPU load on ES and Cassandra
- Apigee BaaS expired token
- Using Apigee or usergrid REST API, how can I query entities by a list of UUIDs?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
It looks like uneven distribution of data/keys/requests between nodes. Check that your partition key distributes data evenly, and you are not requesting same key.