SingleStore Now: Making Real-Time AI a Reality

At SingleStore, we focus on helping customers transact and analyze different types of data in real time.

Along that journey, we’ve helped hundreds of customers deliver real-time applications and analytics. Today, we’re expanding our mission to AI by helping customers also contextualize their data in real time. At SingleStore Now: The Real-Time AI Conference, taking place today at the Chase Center in San Francisco, we’re announcing the first wave of exciting product innovations that we believe open up the frontier of real-time AI. 

The announcements fall under three main categories:

1. Faster and better search, including for vectors

2. Layer of intelligence on top of real-time data

3. Exciting partnerships and integrations with the wider technology ecosystem

SingleStore Scope hybrid search: Advanced vector indexes + full-text search

SingleStore Scope indexed vector search

As with books, indexes in databases help quickly find relevant information sifting through an astronomically large corpus of data. While SingleStoreDB has supported vector functions and vector search for several years, new generative AI applications require looking up top matches from among tens of millions (or billions) of vector embeddings in a split second.

This scale and the unstructured nature of data used for AI makes efficient indexing extremely important — from both a cost and performance perspective. 

Today, in addition to our existing exact k-Nearest Neighbor (kNN) vector search that several customers use for higher accuracy, we’re thrilled to announce support for multiple indexing techniques to enable Approximate Nearest Neighbor (ANN) search. Our preliminary tests indicate orders of magnitude faster vector search performance.

Unique approach to adding vector indexes

SingleStore Scope takes a different approach to vector indexing compared to other vector-only or general-purpose databases:

• State-of-the-art algorithms for vector indexing. SingleStore Scope will provide Inverted File (IVF) with Product Quantization (PQ) to bring significantly lower index build times, strong compression ratio and smaller memory footprint with very good performance for vector similarity search. But we are not stopping here, and will also be adding the Hierarchical Navigable Small World (HNSW) approach to support high dimensionality. The overall result is a compact vector index and search retrieval performance that is orders of magnitude faster. Our architecture also allows us to add new indexing strategies as modular components so we can incorporate the best available approaches as this space evolves.

• Hybrid full-text and vector search. We believe that real-time AI requires not just a vector based semantic search but a hybrid search approach that combines semantic with exact keyword/lexical search. We are bringing significant improvements to our already stellar keyword match by taking advantage of the latest libraries and advances in this area. Vector embeddings can be ingested at high throughput into memory and then committed to disk. The new vector data is immediately searchable rather than requiring to wait for the index to finish building in the background.
  
  Combined with improved vector indexes, SingleStoreDB now has a differentiated hybrid search that makes Retrieval Augmented Generation (RAG) based use cases into real-time ones.

• Fast, full-featured database. Developers can employ  familiar SQL or MongoDB^® queries (with SingleStore Kai™) rather than learning multiple new query languages of pure vector databases,  offering very limited functionality (Compared with SQL that offers filters, aggregations and joins). Vector databases also don’t offer enterprise-grade security or resiliency. Vector-capable relational databases do use SQL but don’t support vector search in complicated query shapes.

In addition to SingleStore Scope vector search, we are also introducing a new data type specifically built for vectors that enhances the developer experience, making it easier to write and understand code related to vectors.

Layer of intelligence over real-time data

We believe there will be an AI data plane that sits between an ensemble of LLMs and corporate data. With SingleStore, one can think about this as a layer of intelligence on top of real-time data. Enterprise use cases for generative AI require this  ensemble of LLMs  to generate responses custom to your organization,  based on fresh data and available only within enterprise firewalls. To accelerate the development of these applications we are announcing a slew of new services including SingleStore Aura compute service, SingleStore Aura job service and SingleStore Radian integration service.

SingleStore Aura compute service (limited preview)

SingleStore Aura brings together a set of services that enable users to deploy specialized compute resources (CPUs and GPUs) for AI, ML or ETL workloads right next to where the data resides in their database. This empowers developers to transform and pass live data to their LLMs and ML models without the need to move data outside of secure SingleStoreDB environments — or to provision and maintain dedicated compute resources.

SingleStore Aura run-time environments are highly configurable and can be scaled in an isolated manner, while ensuring security and governance throughout the data lifecycle. While data lakehouses can do this today, they are geared towards batch processing. SingleStoreDB, on the other hand, now enables processing and curating massive amounts of data in real time.

Aura allows you to:

• Scale compute (CPUs or GPUs) on demand for AI workloads, data prep or hosting and fine tuning private LLMs
• Bring your own ML models, third party software, libraries and Python UDFs, running them in proximity with your database so your applications have full context from enterprise data
• Parallelize data processing to improve database performance

SingleStore Aura job service (private preview)

Another important workflow Aura enables is the ability to schedule SQL and Python jobs from within SingleStore Notebooks:

• Users can create and schedule jobs to run at specified times or frequencies
• With memory-optimized CPUs and GPUs, users can execute complex data preparation and ML flow pipelines to feed AI applications
• Users can also schedule SQL jobs to automate day-to-day tasks like calling Stored Procedures as part of the workflow

SingleStore Notebooks (generally available)

Starting today, SingleStore Notebooks is GA. Notebooks are web-based Jupyter notebooks that allow developers to create, explore, visualize and collaborate on data analysis and workflows using SQL or Python code. Developers can also generate code with the help of SQrL, the SingleStore AI chatbot that is embedded within Notebooks. Notebook users will now be able to use more powerful compute infrastructure using SingleStore Aura.

Related to Notebooks, we are announcing SingleStore Spaces, a collection of pre-built Notebooks for key use cases like building generative AI applications, recommendation engines, doing semantic search and analyzing real-time data. With SingleStore Spaces, developers and data scientists alike can quickly browse through and get started using and building LLMs and other real-time data apps — without writing a single line of code.

Notebooks now also offer access to a file system for users to upload CSVs and store their files for processing within Notebooks. This feature is called Stage, and is available today.

SingleStore Radian data integration service

SingleStore Radian is a set of services designed to integrate data from heterogeneous sources into SingleStoreDB seamlessly and at no additional cost. SingleStore Radian currently supports CDC-in from MongoDB® (public preview), and CDC-in from MySQL and Iceberg data format (both in private preview). SingleStore Radian can also extract data out from SingleStoreDB to other destinations — currently in JSON format and in the coming months in Iceberg format.

In the future, SingleStore Radian will include an SDK for letting customers and partners build their own customer integrations from a variety of data sources and data formats.

Connected ecosystem

To help developers create applications more quickly, we are expanding our collaboration with hyperscalers like AWS, Google and IBM, as well as enlarging our ecosystem of open-source and commercial technology partners.

• With Google, we are announcing a new extension for Google Vertex AI. This enables customers to take advantage of the rich features of Vertex AI like ensemble LLMs, fine-tuning of commercial and open source models and taking advantage of hybrid search of data including vectors, all in the same place.

• With AWS, we are announcing a one-click deployment feature that helps developers deploy LLM apps on AWS leveraging Amazon SageMaker.
• With IBM we are announcing two integrations for watsonx. An AI and data platform, watsonx is designed to scale and accelerate the impact of AI with trusted data across your business. Our integrations with watsonx include:
  • watson.ai brings together new generative AI capabilities, combining foundation models and traditional machine learning into a powerful studio spanning the AI lifecycle
  • watsonx.data, an open, hybrid and governed data store optimized for all data and AI workloads built on a data lakehouse architecture

• With the open-source community and technology partners, we are very excited about our growing collaboration and connectors for the following products:

  • FlowiseAI, a no-code platform to build custom LLM applications with LangChain
  • LangChain, a framework to simplify the creation of applications using LLMs
  • LlamaIndex, a data framework for LLM applications to ingest, structure and access private or domain-specific data
  • OpenAI, access up to date information using ChatGPT and SingleStoreDB
  • Twilio Segment (coming soon), a Customer Data Platform (CDP) for collecting and assimilating all customer touchpoint data
  • Unstructured, a library designed to help preprocess unstructured text documents for use in downstream ML tasks
  • Vercel, a React platform for building and deploying full-stack applications

In addition to these services across search, intelligence layer and ecosystem, we are also announcing a new free tier, and a new SDK for building web apps.

SingleStore free tier

While we already offer a cloud trial with $600 of credits, developers will now have access to an ‘always-free’ tier. The free tier will provide a no-cost sandbox environment to host your data and test any workloads and use cases on SingleStore Helios (SingleStore’s cloud managed service).

SingleStore Elegance

We are also releasing a new SDK called SingleStore Elegance. Written using the React framework, Elegance can be downloaded and used to build full-stack real-time and AI applications that are pre-configured to work with SingleStoreDB. Users can connect to their SingleStoreDB database and start taking advantage of SQL, JSON and vector data by choosing to start from a host of pre-built React components.

The path forward

We believe our announcements today are a major leap forward in making the vision of real-time AI a reality for customers with vast amounts of data by enabling fast vector search and other AI capabilities directly into our modern data platform, accessible from SQL and our API for MongoDB® applications.

As AI proliferates, LLMs and other multi-structured foundational models will need to respond to requests in milliseconds and, in turn, need their data planes to have real-time capabilities for processing and analyzing data in diverse formats.

To execute on real-time AI, enterprises need to continuously vectorize data streams as they are ingested and utilize those for AI applications. Consequently, organizations will increasingly move toward a zero ETL philosophy to minimize data movement, complexities and latencies to power their AI apps.

In case you missed attending the SingleStore Now in-person, keep an eye out on our social media for on-demand sessions.

In the meantime if you’re interested in building your own generative AI applications, you can try SingleStoreDB free today. We encourage customers to reach out to us to be included in the limited preview for Aura.