Oracle GoldenGate, an industry-leading data replication tool, has released support for SingleStoreDB and Singlestore Helios as a target database. This blog explores the details of this new integration.
Oracle GoldenGate, an industry-leading data replication tool, has just released support for SingleStoreDB as a target database. This exciting development allows SingleStoreDB users to easily integrate their data with other systems in real time, without the need for complicated and time consuming data migration processes.
With this new connector, users can enjoy the benefits of Oracle GoldenGate’s powerful data replication capabilities while leveraging SingleStoreDB’s high-performance, distributed SQL database.
In this blog post, we’ll explore the details of the new integration and what it means to users looking to improve their data integration workflows.
What Is Oracle GoldenGate?
Oracle GoldenGate is a real-time data replication platform that enables businesses to capture, transform and move transactional data across heterogeneous systems. Thousands of global banks, retailers, telecoms, healthcare companies, etc. run their operational data platforms on the foundation of GoldenGate, which can detect data events and route them across networks at very low latencies. GoldenGate’s CDC capabilities are also used for detecting and transmitting data events, including database DML and DDL.
Key features of Oracle GoldenGate include:
- Real-time data movement, minimizing latency
- Only committed transactions are moved, enabling consistency and improving performance
- Supports a wide range of heterogeneous databases running on a variety of operating systems. Data can be replicated from Oracle to SingleStoreDB
- Simple architecture and easy configuration
- High performance with minimal overhead on the underlying infrastructure
What Is SingleStore?
SingleStore is a real-time, distributed SQL database that handles both analytical (OLAP) and transactional (OLTP) workloads in the same table type. SingleStoreDB provides fast ingestion and high query speed for complex OLTP and OLAP queries. It provides a robust, scalable solution that is levels above what other legacy, single-node databases can do. In addition to Singlestore Helios, there is also a managed service that can be deployed on AWS, GCP or Microsoft Azure.
How to Connect Oracle GoldenGate to SingleStoreDB
Step 1: Download & install GoldenGate
Download base install OGGMA for MySQL 21.3 from the following link:
https://www.oracle.com/in/middleware/technologies/goldengate-downloads.html
Perform the GoldenGate installation per the following screenshots:
You can choose the location of the software based on your filesystem availability.
Using ./oggca, create a new GoldenGate deployment.
Step 2: Download & apply GoldenGate patch for SingleStoreDB
For SingleStoreDB to be the target database, we need to download and apply a GoldenGate Microservices patch named : Patch 35271106: Oracle GoldenGate 21.10.0.0.1 Microservices for MySQL-compaIble DBs
Use the following link to download the patch, and refer to the attached file screenshots on how to apply it:
Step 3: Create & configure the GoldenGate processes
3.1. Create connections to the source and target databases.
Connections are created from the Administration Service 〉Configuration 〉Database 〉Credentials page, supplying the Database Server, Port and User ID/Password information for the connections. Click Submit when completed.
MySQL source database credentials:
SingleStoreDB credentials:
3.2. Create a checkpoint table for SingleStoreDB
Once the Credential Alias is created, login to the database and create a checkpoint table to be used by the Replicat.
Click the plus sign (➕) and enter your chosen name for a checkpoint table to be created. The schema listed must already exist in the database, and the user listed in the Credential Alias must have permissions to create tables in the SingleStoreDB database. Once entered, click the Submit button.
Note: Skip this step if you intend to create a Parallel Replicat. The checkpoint table for a Parallel Replicat will be created atomically when the Replicat starts.
3.3. Create a new extract
Now, create a source GoldenGate Extract whose trails will be consumed by a SingleStoreDB Replicat. In this example, a source MySQL Extract was created in the same deployment that will be used for the SingleStoreDB target — writing to a trail sequence called ‘em’, so no distribution path is needed.
Any database supported by Oracle GoldenGate as a source supports a SingleStoreDB Replicat.
From the Administration Service 〉Overview page, click the ➕ sign in the Extracts table to add a new Extract.
In this example, we’re creating a Change Data Capture (CDC) Extract that will pick up changed DML operations from the MySQL database, starting at this point in time.
3.4 Create a new Replicat
From the Administration Service’s Overview page, create a new Replicat for the target SingleStoreDB database.
Choose from the available types of Replicats and click Next.
Enter the required information: Process Name (eight characters or less), Credential Alias and Trail Name, the two-character name that is created by an Extract running in the same deployment — or by a distribution path if the trail is sent from another deployment. Click Next.
3.5. Edit the Replicat parameter file
Edit the parameter settings to include the desired source schema tables that will be mapped to the SingleStoreDB target schema tables:
When completed, click Create and Run, verifying the new Replicat is up and running.
3.6. CDC replication check
After performing new DML operations to the source MySQL table, you can check the stats for both the MySQL Extract and the SingleStoreDB Replicat by selecting Details from the Action drop-down menu for each process.
Now, navigate to the Statistics tab within the process Overview window.
As both the processes are running (Extract & Replicat), any data flowing from the MySQL database will be replicated real time into SingleStoreDB.
So, what are you waiting for? Get started with SingleStoreDB and integrate your workload to SingleStoreDB from your existing data sources using Oracle GoldenGate.
Appendix:
