Intermittent ProcessHandshakeResponsePacket

Hi -

I am currently evaluating Singlestore again in preparation for an upcoming project and am having near constant connection issues.

Connecting from DataGrip was successful for the first couple days the cluster was live, but then DataGrip begins to ask for credentials and will not connect when the credentials are provided - despite the connection being successful with the same user and credentials in the past. The server logs indicates ERROR: ProcessHandshakeResponsePacket() failed. Sending back 1045: Access denied for user 'admin'@'localhost' (using password: YES)

Connecting from Python using SQL Alchemy, mysqlclient, PyMySQL all have similar behaviors when executed using a Windows machine - Mac and Linux result in the connection being refused most of the time. On the client side, the error message indicates Access Denied due to password. Server side, the logs indicates ERROR: ProcessHandshakeResponsePacket() failed. Sending back 1045: Access denied for user 'admin'@'localhost' (using password: YES)

When connecting from Superset (which uses SQL Alchemy), I am able to occasionally have a successful connection by repeatedly attempting, quite literally spam clicking connect. But, the server logs are flooded with the same error as above.

I understand there are issues with connecting from some libraries due to requiring setting the authorization plugin for MySQL to “mysql_native_password”, but that does not explain intermittent connectivity leading to near zero connectivity.

This pattern is repeated no matter which user I attempt. I am creating users by issuing commands similar to this one: CREATE USER user_name@'%' identified by 'password123';
with permissions being granted like this: GRANT SELECT, INSERT, UPDATE, CREATE ON schema_name.* to user_name@'%';

My deployment is modelled off of the deployment documentation for Kubernetes found here, with slight changes to operate as a Helm Chart to pass along values from a values.yaml file.
For the cluster operator, I am using memsql/operator:1.2.5-83e8133a, for nodes I am using memsql/node:centos-7.1.20-f5ac027e02.

I’m a bit lost at this point as to what the issue could be. Any help or suggestions would be greatly appreciated. Thank you for your time thus far.