Problem creating node

Off-Topic
1

Hey all,

I am trying to create a leaf node on a cluster. When I try I receive the following error:

dkoutsou@euler01:~$ memsql-admin create-node --host euler07 --port 3336 --password <password>
The target user dkoutsou on host euler07 does not have the privileges to perform this action.
Please enter your password to proceed with sudo.  (For details, see https://docs.memsql.com/toolbox-redir/sudo-prompt).
sudo password for dkoutsou@euler07:
Toolbox is about to perform the following actions on host euler07:
  · Run 'memsqlctl create-node --port 3336 --password ●●●●●●'

Would you like to continue? [y/N]: y
error running memsqlctl: error running command: `"/usr/bin/ssh" "-oBatchMode=yes" "-q" "-tt" "-oControlPath=/tmp/memsql-toolbox714268994/1.socket" "euler07" "'sudo'" "'-S'" "'-k'" "'--'" "'/bin/sh'" "'-c'" "''\\''printf'\\'' '\\''7ab024ec-0269-4adf-a82e-87dd8bdedc02'\\'' && '\\''exec'\\'' '\\''memsqlctl'\\'' '\\''--json'\\'' '\\''--yes'\\'' '\\''create-node'\\'' '\\''--port'\\'' '\\''3336'\\'' '\\''--secure-password'\\'' '\\''7srdngQmjoMz0ppi9tCxjcXwxGd/UJuakRJYY6P0mZFhWw=='\\'''"`: exit status 1
stderr: Latest errors from MemSQL tracelog:

: Failed to connect to MemSQL: process exited: exit status 255

I have tried different ports but the error still remains. When I see my cluster configuration I see the following:

+------------+---------+---------+------+---------------+--------------+---------+----------------+--------------------+--------------+
| MemSQL ID  |  Role   |  Host   | Port | Process State | Connectable? | Version | Recovery State | Availability Group | Bind Address |
+------------+---------+---------+------+---------------+--------------+---------+----------------+--------------------+--------------+
| B7E3D4EFCC | Master  | euler01 | 3306 | Running       | True         | 7.0.12  | Online         |                    | 0.0.0.0      |
| D1D0EA00E9 | Leaf    | euler02 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| 7AABE3C4E4 | Leaf    | euler03 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| 2DF06A8615 | Leaf    | euler04 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| C323ABB21F | Leaf    | euler05 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| 4DF3819D4D | Leaf    | euler06 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| 5432B65B0B | Leaf    | euler08 | 3306 | Running       | True         | 7.0.12  | Online         | 1                  | 0.0.0.0      |
| F9113841D0 | Unknown | euler07 | 3336 | Stopped       | False        | 7.0.12  | Unknown        |                    | 0.0.0.0      |
+------------+---------+---------+------+---------------+--------------+---------+----------------+--------------------+--------------+

Any ideas on how to fix this? I suspect this issue is due to some broken metadata.

Thanks a lot

2

Hi dkoutsou,

Can you take a look at the tracelogs/memsql.log file under the MemSQL data directories? That may give you some more clues on the actual issue. What kind of license are you using? Hope you are not running into a license limitation.

Thanks,

3

Hi rnarayanan,

I managed to fix the issue. The problem was the extremely low space in the hard drive. Checking the log didn’t help, I found the issue by accident and when I emptied some space then the node was created successfully. It would be valuable to report such incidents in the command line in some future version.

Thanks,
Dimitris

4

Thanks for your update Dimitris. I have notified our engineers to look into a better error messaging for the same.

Glad you are up and running now.

Ramesh