Spin Up a SingleStore Cluster on Docker Desktop in 10 Minutes

In this post we’ll quickly build a single-instance SingleStore cluster running on Docker Desktop, on a laptop computer, for free. You’ll need a machine with at least 8GB RAM and four CPUs. This is ideal for quickly provisioning a system to understand the capabilities of the SQL engine. Everything we build today will be running on your machine, and with the magic of Docker containers, we’ll not need to install or configure much of SingleStore to get it running.

The steps here are: get a free SingleStore license; install Docker Desktop; create a Docker Compose file; start the SingleStore cluster through Docker; and browse to SingleStore Studio. You’ll have a bare-bones SingleStore cluster running on your laptop machine in no time.

Why Docker, and Why a Single Docker Container?

Docker is a great way to run software in a protected sandbox and easy-to-manage environment, with less overhead than a virtual machine (VM) – and much less than a dedicated server. You can use it to spin up applications, systems, and virtual hardware to try out software or to quickly spin up a database to support local application development. We’ll use Docker here to provision and spin up a free SingleStore cluster, and just as easily, destroy it when we’re done.

Using Docker makes it much easier to run a small SingleStore cluster without interfering with other software running on the machine. Pre-installed in the cluster-in-a-box container image is an aggregator node, a leaf node, and SingleStore Studio, which is a browser-based SQL editor and database maintenance tool, all running in one place, all pre-configured to work together. The minimal hardware footprint wouldn’t be nearly enough for production workloads, but it allows us to quickly spin up a cluster, connect it to our project, and try things out.

We could use a Virtual Machine (VM), but Docker containers are lighter-weight than a virtual machine (VM). Like virtual machines, Docker provides a sandbox between processes. But unlike VMs, containers virtualize the operating system instead of the hardware, and Docker’s configuration-as-code mindset ensures we can quickly provision a complete virtual system from a small text file stored in Git.

With the single-container SingleStore cluster described here, you can craft the simplest of tables all the way up to running a complex app, a dashboard, a machine learning model, streaming ingest from Kafka or Spark, or anything else you can think of against SingleStore. You’ll quickly understand the methodology and features of SingleStore, and can plan accordingly.

The SingleStore cluster-in-a-box container has minimum hardware specs disabled, but you’ll still want a machine with at least 8 GB RAM and four CPUs. With specs well below SingleStore’s limits, you’ll see poor performance, so this system is definitely not the right setup for a proof of concept (PoC). But you can use this setup to experience the full features of SingleStore, and understand how it applies to your business problems.

Once you know how SingleStore works, you can take these experiments and use what you learn to help you achieve your service-level agreements (SLAs) on distributed clusters that meets SingleStore’s minimum requirements and your high-availability needs. That’s when you can really open up the throttle, learn how your data performs on SingleStore, and dial in system performance for your production workloads.

Sign Up for SingleStore

To get a free license for SingleStore, register a singlestore.com/cloud-trial/ and click the link in the confirmation email. Then go to the SingleStore customer portal and login. Click “Licenses” and you’ll see your license for running SingleStore for free. This license never expires, and is good for clusters up to four machines and up to 128GB of combined RAM. This is definitely not the license you’ll want for a production cluster, but it’s great for these “kick the tires” scenarios. Note this license key. We’ll need to copy/paste it into place next.

Install Docker Desktop

The first step in getting our SingleStore cluster running in Docker Desktop is to get Docker Desktop installed. If you already have a recent version of Docker Desktop, you need only ensure you’re in Linux containers mode.

Docker’s install requirements are quite specific, though most modern mid-range systems will do. Docker Desktop for Windows runs a Linux VM in Hyper-V, and Hyper-V requires Windows 10 Pro or Enterprise. Docker Desktop for Mac runs a Linux VM in xhyve, and requires a 2010 or newer model with macOS 10.13 or better.

To install Docker Desktop, go to Docker Hub and choose the Docker Desktop version for your operating system. The download will require you to create a free account. Run the downloaded installer and accept all the defaults.

Note for Windows users: If you are doing a fresh install, ensure you choose “Linux Containers” mode. If you installed Docker previously, ensure you’re running in Linux containers mode. Right-click on the Docker whale in the system tray (bottom-right by the clock), and choose “Switch to Linux Containers”. If it says “Switch to Windows Containers”, you’re already in the right place – that is, in Linux Containers mode.

Note: Adding more RAM: Though not required, SingleStore will definitely behave better when Docker Desktop has more capacity. Click on the Docker whale, choose “Settings…” on Windows or “Preferences…” on Mac, click on the “Advanced” tab. If your machine has more than 8 GB RAM, set this to 8192. If your machine has 8 GB RAM or less, set it as high as you can. Then change the CPU count from 2 to 4.

Create a Docker Compose File

A docker-compose.yaml file gives Docker Desktop instructions to spin up one or more containers together. It’s a great way to capture all the docker pull, docker build, and docker run details. This file doesn’t replace Dockerfile but rather makes it much easier to use them.

We’ll use the memsql/cluster-in-a-box image built by SingleStore and available on Docker Hub. Pre-installed in this image is the SingleStore database engine and SingleStore Studio. The minimum system requirements are disabled in this “cluster-in-a-box” configuration.

Create an empty directory and create a file named docker-compose.yaml inside. Open this file in your favorite code editor and paste in this content:

A yaml file is a text file that’s great for capturing our architecture setup. As with Python source code, white space is significant. Yaml uses two spaces, not tabs. Double-check the yaml file to ensure each section is indented with exactly two spaces. If you have more or fewer spaces, or if you’re using tabs, you’ll get an error on startup.

Here are the sections in the docker-compose.yaml file:

1. We’re using the Docker Compose syntax Version 2.
2. The services array lists all the containers that should start up together. We’ve only defined a single container here, named memsql, which uses the memsql/cluster-in-a-box image built by SingleStore.
3. In the Ports section, we identify inbound traffic that’ll route into the container. Open port 3306 for the database engine and port 8080 for SingleStore Studio. If either of these ports are in use on your machine, change only the left port. For example, to connect from outside Docker to the database on port 3307 use 3307:3306.
4. The first environment variable, START_AFTER_INIT, exists for legacy reasons. Without this environment variable, the container will spin up, initialize the SingleStore cluster, and immediately stop. This is great for debugging, but not the behavior we want here.
5. The second environment variable, LICENSE_KEY, is the placeholder for the license we got from the customer portal. Don’t copy the license key into place here — you’ll accidentally leak secrets into source control. Instead, this syntax notes that we’ll reference an environment variable set in the terminal.

In time, we could easily add our application, business intelligence (BI) dashboard, and other resources to this file. If you have an existing docker-compose.yaml file, you can copy that content into place here too. Save the file, and we’re ready to launch Docker.

Starting the SingleStore Cluster

Open a new terminal window in the same directory as the docker-compose.yaml file. This could be Powershell, a command prompt, or a regular terminal.

First, we’ll set the license key as an environment variable. Copy the license from the customer portal, in the Licenses tab, and create an environment variable in the terminal:

Command prompt:\ set LICENSE_KEY=paste_license_key_here\ Powershell:\ $env:LICENSE_KEY = 'paste_license_key_here'\ Mac/Linux/Git Bash:\ export LICENSE_KEY=paste_license_key_here

Paste your actual license key in place of paste_license_key_here. It’s really long and probably ends with ==.

Next, type this in the shell:

This tells Docker to pull or build all the images, start up all the containers in our docker-compose.yaml file, and stream the console output from each container to our terminal. I find it fascinating to watch each application spew their innards here.

If you get an error starting the cluster, double-check that the license key is correct and that Docker is running. If you get an image pull failure, ensure your network connection is working as expected. To retry, type Cntrl-C in the terminal, then type:

docker-compose down\ docker-compose up

Congratulations! We’ve launched a SingleStore cluster. Let’s dive in and start using it.

Start SingleStore Studio

Now that SingleStore is running in Docker, let’s dive in and start using it. Open the browser to http://localhost:8080 to launch SingleStore Studio. Click on the local cluster, enter username of root, use the password you used above, and login.

On this main dashboard screen, we can see the health of the cluster. Note that this is a two-node cluster. Clicking on Nodes on the left, we see one node is a leaf node, one is an aggregator node, and they’re both running in the same container. In production, we’d want more machines running together to support production-level workloads and to provide high availability.

Click on the SQL Editor page, and we see the query window. In the query window, type each command, select the line, then push the execute button on the top-right:

For more details on SingleStore Studio, check out the docs or watch the SingleStore Studio tour video.

Where Can We Go From Here?

SingleStore is now ready for all the “kick the tires” tasks we need. You can:

• Hook up your analytics dashboard to SingleStore.
• Add your application to the docker-compose.yaml file and connect it to SingleStore using any MySQL connector.
• Create an ingest pipeline from Kafka, S3, Spark, or other data source.

Being able to run these tasks from such a simple setup is a real time-saver. However, don’t expect the same robustness or performance as you would have with a full install on a full hardware configuration.

Cleanup

We’ve finished our experiment for today. To stop the database, push Ctrl-C in the terminal, and type:

This will stop all the containers in the docker-compose.yaml file. If you’re done experimenting with SingleStore, delete the image by running docker image rm memsql/cluster-in-a-box in the terminal. Or better yet, leave this image in place to quickly start up your next experiment. Run docker system prune to remove any stored data or dangling containers left in Docker Desktop and free up this space on your hard disk.

Conclusion

With the SingleStore cluster-in-a-box container and Docker Desktop, we quickly provisioned a “kick the tires” SingleStore cluster. Aside from Docker itself, there was nothing to install, and thus cleanup is a breeze. We saw how easy it is to spin up a cluster, connect to it with SingleStore Studio, and start being productive. Now get a copy of SingleStore for free and go build great things!