Cassandra nodetool status down and generation time very old

456 Views Asked by At

Our cassandra cluster went down. It was not maintained properly. The nodetool status from each node is different. So, when I tried the gossip info, the generation timestamp if very old.

/192.168.0.35
  generation:1539742113
  heartbeat:8705953
  DC:datacenter1
  STATUS:NORMAL,-1117919639473599473
  SEVERITY:0.0
  HOST_ID:4bc33d49-6f71-4e33-9790-f9ae46180613
  LOAD:1.097982388E9
  NET_VERSION:8
  RPC_ADDRESS:192.168.0.35
  RACK:rack1
  RELEASE_VERSION:2.1.6
  SCHEMA:04c4cc4f-7598-3e69-9455-8d8896b0624d
/192.168.0.36
  generation:1504926520
  heartbeat:2147483647
  DC:datacenter1
  STATUS:shutdown,true
  SEVERITY:0.0
  HOST_ID:0708fc6f-0423-46fc-8f56-8e7512a83de4
  LOAD:1.2352913099E10
  NET_VERSION:8
  RPC_ADDRESS:192.168.0.36
  RACK:rack1
  RELEASE_VERSION:2.1.6
  SCHEMA:04c4cc4f-7598-3e69-9455-8d8896b0624d
/192.168.0.34
  generation:1541989374
  heartbeat:1890169
  DC:datacenter1
  STATUS:NORMAL,-100967197570522715
  SEVERITY:0.20000000298023224
  HOST_ID:f3e8f987-055d-42f9-b9aa-0516d6890235
  LOAD:4.243156213E9
  NET_VERSION:8
  RPC_ADDRESS:192.168.0.34
  RACK:rack1
  RELEASE_VERSION:2.1.6
  SCHEMA:04c4cc4f-7598-3e69-9455-8d8896b0624d
/192.168.0.33
  generation:1540772736
  heartbeat:5580091
  DC:datacenter1
  STATUS:NORMAL,-1055061873163068507
  SEVERITY:0.20080320537090302
  HOST_ID:c827ef32-f076-456f-a0ad-2f3289ef3e5b
  LOAD:3.123707903E9
  NET_VERSION:8
  RPC_ADDRESS:192.168.0.33
  RACK:rack1
  RELEASE_VERSION:2.1.6
  SCHEMA:04c4cc4f-7598-3e69-9455-8d8896b0624d

What should I do to make nodes gossip more? Thank you

1

There are 1 best solutions below

0
On

In this case, you should restart the node check the status. because when the node get started it establish gossip again on all the nodes.