Sync portion of larger MemSQL to smaller read only MemSQL instance

Off-Topic
1

I would like to create a setup that involves a “main” database that contains the full set of data for my application.

I would like to be able to create read only replicas of portions of the data on “satellite” databases.

something like “select from insert into” - for the master to satellite transfer.

Has anyone done anything like this?

-john

2

I’d suggest doing this with application software. A few approaches to consider are:

  1. do SELECT INTO OUTFILE from the “main” DB and load the results to the “satellite” DB
  2. have the app that creates the data insert/update the data in “main” and “satellite”
  3. in 7.0, use SELECT INTO KAFKA queue on “main” and use a pipeline to load the data to the “satellite”

Item 3 is not documented yet but will be within a few weeks.

3

Wow select into KAFKA sounds awesome.

Thanks for the advice.

-john

4

is there any possibility of something like an “exporting” pipeline that will continuously stream data out via KAFKA?

5

We’re considering options for change data capture (CDC) out of MemSQL. This is an option we have thought of. No concrete plans or timelines yet.

6

Hanson,

thanks again!

-john