Hi,
I was comparing the MemSQL versions 7.0.15 and 6.8.3 and I found that reference data is getting asynchronously replicated to all leaves and child aggregators in version 7.0.15. However, on version 6.8.3, it is getting synchronously replicated.
In the screenshot, I ran show databases extended on a leaf and it says async slave. On version 6.8.3, it says sync slave.
I am not sure about the change in default behavior as in case their is latency during asynchronous replication of reference data and leaves have to perform join operation to local copy of reference data then leaves will not be seeing up to date copy of data which will result in incorrect data display.
Does anyone have an idea about the reason of this change? Also, How do we change it back to synchronous in version 7.0.15?
That difference is entirely an internal difference. The semantics, however, have not changed. Reference databases now use the new mechanism for asynchronous replication to replicate transaction logs, but have a higher level mechanism that at the end of every write waits for that write to be visible in all online nodes. This exactly matches the behavior prior to 7.0, so your workload should not see any differences.
Replication works in a completely new way starting in 7.0. Synchronous replication is a lot higher performance, but also has slightly different semantics. In particular, synchronous replication now relates only to replication of the transaction logs, which does not imply writes are visible on replicas by the time they commit.
This is fine for partitions, as those work as hot spares, and at the moment are never read until after they become masters (at which point we guarantee everything is visible), but it doesn’t work for ref dbs where we want writes to be visible on every node after they’re done committing.
That, coupled with some more details of how the new replication works, made us take a different approach for reference database replication, where the lower layers are asynchronous, but higher layers guarantee that writes are visible everywhere by the time they finish, achieving behavior parity with prior versions.
Although from your explanation it does not look like we need to change it back to synchronous replication for reference data, but just out of curiosity, is there a way to change it back to synchronous using some variables etc.?
No, there isn’t a mechanism to change the replication mode to sync replication. Sync replication in 7.0 is resigned in a manner that no longer suites reference table replication (sync replication that is roughly the same performance as async replication was one of the bigger improvements in 7.0).
So quick question on SYNC REPLICATION in 7.0 and how it will affect writes on a given leaf node, if the sister node (with the corresponding slave partition) is unresponsive. As memsql will now promote the slave on the primary node (whose sister node is down) from slave -> master, would that mean the SYNC replication does not undergo a “infinite wait” scenario (vs async, whereby replication does not hinder writes).
In simple terms : I would not want the SYNC nature in 7.0 to make writes wait infinitely…
Obviously, when we go to HA-3 for example, the chances of the above go away drastically.
At the lowest level, sync replication does do an infinite wait, but at higher levels, memsql reacts to writes blocked due to replication by doing a slave demotion from sync to async. This also happens on failovers. Once that demotion is done, the writes are unblocked. The constraint that SYNC replication introduces in memsql 7.0 is that you can’t failover if your slave was not synchronously replicating, so that demotion temporarily reduces a partition’s redundancy to 1, until the slave becomes available and synchronous again, or a new slave is setup.
very well explained. yes. this seems to be the right option to tackle node non-availability in a sync replication mode.
Hoping this will all make it to the documentation, as these are few of the questions which we constantly get from end-users.
