Node wont finish recovering, with error ERROR: Waiting for node to catch up with cluster state before starting replication management

Help
1

Hi, we have a 6 node cluster with 4 leaf nodes. We had memsql.service crash on the master node. We tried to restart the clusters using the Startup Sequence and Process in a Cluster · SingleStore Documentation.

But after restarting the nodes the leaf nodes are erroring out with error
120122787280 2022-02-20 12:00:44.347 ERROR: Thread 114999: Fn: Waiting for node to catch up with cluster state before starting replication management, due to which the node is still in recovering state. Trying to find how to resolve this ?

2

Hi dixiabh-1! :wave: Thanks for submitting your first question. Can you tell me if you are on the managed or self-hosted and what version you are using?

3

Hi Maria,

We are using the self-hosted.
Memsql 7.1.3 version
Memsql-studio 1.9.6
memsql-toolbox 1.5.5

thanks,
Abhi