July 25, 2024 in Product

Accelerate Your MongoDB® Applications With SingleStore Kai™ $vectorSearch Capabilities

Spandana Rachamadugu

Product Manager

We are thrilled to announce that SingleStore Kai™, a MongoDB® wire-compatible API, now offers integrated vector search capabilities.

Accelerate Your MongoDB® Applications With SingleStore Kai™ $vectorSearch Capabilities

This enhancement enables MongoDB developers to connect their applications to SingleStore and achieve 100x faster analytics, empowering gen AI use cases without any code modifications.

The demand for intelligent AI applications is surging, driven by advancements in vector data utilization across various domains including content summarization, semantic search and image recognition. These applications often require complex data manipulations that traditional databases struggle to handle efficiently.

With the latest update, Kai extends its powerful capabilities to include vector search, allowing MongoDB/NoSQL developers to store and query vector data alongside their operational data. This enables the creation of sophisticated features within applications, combining vector search with full-text search and operational analytics — all powered by a single database without the need to add additional complexity in your data ecosystem.

Enhanced indexing options

SingleStore Kai provides multiple indexing options beyond the basic HSNW offered by MongoDB. With SingleStore, developers can choose from a variety of indexes like IVF_PQFS, which offers lower memory usage and faster index build times, ensuring quicker real-time searches.

Recent benchmarks show that SingleStore's vector index outperforms competitors including Milvus and pgvector in terms of performance, recall and index build times.

How to use vector search through SingleStore Kai

1. Create a collection

Define a collection with a specific column for embedding fields, which optimizes storage and enhances query performance. In the following, example emb is the field name that stores the vector embeddings of five dimensions.

create_collection("profiles",
  columns=[{ 'id': "emb", 'type': "VECTOR(5) NOT NULL" }],
);

2. Insert documents with embeddings

Insert documents along with their embeddings. These embeddings can be generated using foundational models from OpenAI or other sources:

db.profiles.insertMany([
  {
    name: "Sophia Hart",
    bio: "Sophia Hart is a data scientist with expertise in machine learning 
and big data technologies.",
    emd: [0.45, 0.22, 0.33, 0.91, 0.85]
  },
  { 
    name: "Liam Smith",
    bio: "Liam Smith is a software engineer specializing in scalable cloud 
applications.",
    emd: [0.58, 0.77, 0.69, 0.43, 0.65]
  },
  {
    name: "Isabella Johnson",
    bio: "Isabella Johnson is an expert in cybersecurity and cryptographic 
techniques.",
    emd: [0.71, 0.59, 0.12, 0.88, 0.34]
  }
]);

3. Create a vector index

Set up an index for efficient vector searches. In this example I am using AUTO vector type that currently defaults to IVF_PQFS

client[DB].command({
    'createIndexes': 'profiles',
    'indexes': [{
        'key': {'emb': 'vector'},
        'name': 'vector_index',
        'kaiIndexOptions': {"index_type":"AUTO", "metric_type": 
"EUCLIDEAN_DISTANCE", "dimensions": 5}
    }],
});

4. Perform vector search

Query the index using MongoDB's familiar syntax

{
    '$vectorSearch': {
        "index": "vector_index", 
        "path": "emb",
        "queryVector": query_vector,
        "limit": 3,
    }
}

Enterprise capabilities and integrations

SingleStore is not just about high performance; it's a comprehensive solution suited for enterprises. It supports workload isolation, fast ingest of vector embeddings and hybrid searches that combine vector, text and operational data for enriched analytics and insights.

As MongoDB wire protocol compatible, SingleStore Kai supports different MongoDB integrations within the gen AI landscape like LangChain, LlamaIndex and more.