Leav Node struch as Recovering

Off-Topic
1

Hi, One of the leaf node in my cluster got Detached.When i attached it, Its showing status as RECOVERING for long time. Now the status is still DETACHED.
when i try to attach it again to cluster its throwing an Error like
ERROR 1811 (HY000): Unable to ATTACH leaf ‘10.48.80.139’:3307. database rtctracker_0 on ‘10.48.80.139’:3307 is currently recovering (ReplayLSN S13583:47264415, CommitLSN S13583:47264415). Try running ATTACH LEAF again when the leaf finishes recovering. Use SHOW DATABASES EXTENDED on the leaf to examine its progress.

Capture11
Capture11759×151 36.9 KB

It is recovering for a long time and Finally getting down, Along with it its’s HA pair node is also getting down

2

If the node is not able to recover the most common issue is related to the memory configuration. The node may not have enough memory to replay data back into memory ((maximum_memory set too low or some other OS configuratoin issue which is blocking memsql from allocating memory). The tracelog (memsqlbin/tracelogs/memsql.log) for 10.48.80.139:3307 should have more detailed information.

1 Like