Upgrading Cluster fails operation not permitted

Hi,

I’m having an issue when running a cluster upgrade with the command ‘./sdb-deploy upgrade --online’.

It throws the following error (on all nodes):

Failed to stop Leaf node on 10.0.0.120 (1/1) upgrade failed.
The cluster might be in an unhealthy state. The last error was: error running memsqlctl:

Error running command:
/usr/bin/ssh -oBatchMode=yes -q -tt -oControlPath=/tmp/singlestoredb-toolbox750145730/2.socket 10.0.0.120 /home/ec2-user/memsql/singlestoredb-server-8.1.31-d5850b922e/memsqlctl --json --yes --config /home/ec2-user/memsql/memsqlctl.hcl stop-node --memsql-id 2DA17590E2FD5B1969C54EF40EA8F308101B7E2B: exit status 1:
stdout: failed to signal process (real uid=0, effective uid=0, saved uid=0, fs uid=0), we are uid=1000(ec2-user) gid=1000(ec2-user): operation not permitted
stderr: (empty)
Online upgrade failed, but we will try to proceed with an offline upgrade

The offline upgrade fails with the same error.

running the upgrade with sudo priviliges doesn’t work either:

Toolbox does not support running with sudo. Please add ‘user = “root”’ to /root/.config/singlestoredb-toolbox/toolbox.hcl to run Toolbox commands as the root user. See Toolbox · SingleStore Documentation

Can someone point me in the right direct?

Kind regards

Is it on-prem setup? Also check sudoers setup on linux os. May be you can share cat /etc/sudoers file output to help us guide you further.

Also, let us know your SingleStore version and toolbox version.