June 2024: Unfreeze Your Data Lakehouse to Power Intelligent Applications

Today, we’re excited to share four new platform capabilities:

1. Bi-directional integration for Apache Iceberg
2. Build intelligent applications with faster vector search + enhanced full-text search
3. Scale applications, not complexity with Autoscaling
4. Deploy SingleStore Helios in your own VPC

Let’s dive in.

Bi-directional integration for Apache Iceberg

The biggest lakehouse vendors have embraced Apache Iceberg as the de facto standard, which means it’s finally possible to write data with one system and read the same data with another. To quote Ali Ghodsi, "May the best engine win."

Not since the separation of storage and compute has there been a bigger revolution in data management. Companies today are building the intelligent apps of the future — these apps require a data platform that supports sub-second analytics, real-time ingestion, vector search, full-text search and more. And, organizations want to use the data in their lakehouse to power these intelligent apps.

Now, with SingleStore’s bi-directional, low-latency operational engine for Iceberg, you can. What’s more, you don’t need any ETL tools or expensive queries on the source system to do it. This opens up exciting new use cases including real-time fraud detection, real-time RAG, customer experience personalization and more.

Now in public preview, SingleStore’s zero ETL integration from Apache Iceberg helps you power rich, interactive applications or point-in-time analytics in combination with Snowflake or any open data lakehouse. We support automatic schema inference (with support for schema evolution coming soon); and, snapshot loads and incremental updates will be supported using Amazon Glue and Snowflake Rest catalog. In the coming months, you’ll be able to query external Iceberg tables at similar performance as native tables in SingleStore.

Additionally, SingleStore already supports exporting data to lakehouses with support for Glue catalog, and support for others coming soon. With this bi-directional sync, SingleStore will serve as the speedy layer on your lakehouse, delivering sub-second analytics for all structured and unstructured data.

To try our Iceberg integration, start free on SingleStore. You can also learn more about our data integration services here.

Build intelligent applications with faster vector search + enhanced full-text search

SingleStore pioneered hybrid search, allowing you to combine vector and full-text search in a SQL-accessible form factor. Now we’re advancing both these dimensions of hybrid search with faster vector search and enhanced full-text search.

Faster vector search. Vector search using HNSW Flat will soon be 40% faster in release 8.7, compared to release 8.5. Our IVF Flat index is already extremely fast, and proven to be between 47 to 100x faster than pgvector. And, our vector index build time is 2-3x shorter than Milvus and pgVector. We will continue to add state-of-the-art vector search libraries to maintain our industry-leading vector search performance.

Enhanced full-text search. While SingleStore has long supported full-text search, we’ve recently invested heavily here and now run on Java Lucene, the same engine used by Solr and ElasticSearch. With release 8.7, you get (in general availability) improved relevance scoring with BM25 (Best Match 25), keyword boosting, fuzzy matching and proximity-based search. We’re also unveiling search over both text and JSON columns. International language support is planned for later this year.

Our enhancements to vector search and full-text search make using SingleStore a huge time and money saver if you consider the alternative of stitching together multiple databases including Solr, Milvus and MySQL. Even more, SingleStore can scale to handle any data set and workload size. Check out more on our hybrid search capabilities — and of course, you can start free to try this anytime.

Scale applications, not complexity with Autoscaling

Building your applications on free, open-source databases can prove costly in the long run. When your business takes off, the accompanying user growth can lead to slow app performance and potential revenue loss. Bottom line? Scaling applications on single-node relational databases is no fun.

While offloading analytics to a data warehouse is an option, this adds ETL delays — not to mention licensing costs and management overheads. We want you to be able to scale your applications, not your complexity. SingleStore already offers a distributed architecture, separation of compute and storage and isolated compute workspaces. And today, we’re making it easy for you to scale applications quickly and dynamically.

Our new Autoscaling and fast scaling capabilities allow you to ensure application performance by scaling compute resources up or down in minutes to adjust to unpredictable workloads. We also help you avoid billing surprises with user-defined thresholds for CPU and memory. Autoscaling is available now in public preview — try it on SingleStore Standard/Enterprise Edition, and get $600 in free credits.

Deploy SingleStore Helios in your own VPC

You can already deploy SingleStore anywhere you’d like, on-prem or cloud. Customers choose Self-Managed for full control of database resources and Helios® cloud service to offload database management overhead to SingleStore. Some of you want to blend both these options for the best of all worlds — the ease of Helios while retaining data in your own cloud tenancy. Now, SingleStore Helios BYOC makes this possible: You maintain control of your data while enjoying the manageability benefits of Helios.

Helios BYOC is currently in private preview on AWS. Sign up to get early access, and learn more here.

Thanks for reading. We’d love for you to have full access to the details including demos and exciting fireside chats. Watch here!

We’re only in mid-2024 and there’s so much more in the works today that will make building intelligent applications easy. Stay tuned!