Hi,
I am running SingleStore 7.1.11. I have a primary cluster where I defined a Kafka pipeline that uses a Python transform to perform data enrichment on the incoming records before they are loaded into a table.
In testing replication from the primary cluster to a DR secondary cluster (e.g. REPLICATE DATABASE db_name FROM root:‘root_password’@primary-MA:3306;) the replication of data appears to be fine, and the pipeline definitions are also in the replicated cluster. But the transform files that are normally under the ${MEMSQL_BASE}/data/transforms directory are not there on the replica cluster. So on the replica, after doing “STOP REPLICATING db_name” the pipelines are running but not working to pull in data. There are leaf errors in the information_schema.PIPELINES_ERRORS table along the lines of ERROR_CODE = 1965 and ERROR_MESSAGE like "Leaf got file with bad checksum: Got da39a3ee5e6b4b0d3255bfef95601890afd80709, expected 0f9857a0a25095135d97c63ba9df389af078a351"
I searched but could not find any documentation on how to successfully fail over to the replica cluster and have the pipelines working. Do the pipelines need to be recreated on the replica cluster after the fail over?
Thanks.
Shaun