We are proud to announce that we have just released SingleStore 8.9! Major improvements in this release include:
Full-text Search Enhancements
SingleStore 8.9 introduces custom analyzers, enabling high-quality search results for documents in all major languages of the world, not just English. This includes support for language-aware word breaking, stemming, and stop word removal. All standard Java Lucene tokenizers and filters are supported. In addition, special analyzers are available for common document types such as email and web pages so that markup is properly ignored.
We also now support global BM25 scoring, which provides high-quality, uniform scoring that’s not sensitive to how your data is partitioned.
SQL Improvements
Writeable views are now supported, allowing improved compatibility with Oracle and MySQL and easier application migration from those systems. This allows you to, for example, create a new physical table (with a new name) every day for an important data set, but reference the table for both query and update via a single view name that stays the same over time. This can simplify data preparation and application development.
LATERAL JOIN is new since we shipped 8.7. It’s very popular in other database products for allowing you to join a table on the left to a subquery on the right, with correlated conditions.
Multi-column IN-list expressions in the WHERE clause of a query are now supported (e.g. “(t.x,t.y) IN ((1,2), (3,4))”).
Iceberg
You can now set up a pipeline from an Iceberg table to a SingleStore table and as data changes in the source table, the pipeline will bring the changes into the target table. Both inserts and upserts can be moved (not deletes at the present time).
Snowflake, REST, JDBC, Hive, and Polaris catalogs are now supported using PIPELINES from Iceberg.
JSON & Kai Improvements
Many developers use JSON documents that contain denormalized data, such as keeping a collection of Orders, which have order details plus an array of Lineitems inside them. The 1-many relationship is contained within a single object. SingleStore allows you to run aggregate queries that group by order and lineitem information, and filter on order and lineitem information, in this example, up to 20x faster than before – at least an order of magnitude faster than any object database we know of for large aggregates that touch a large percentage of rows (objects). These improvements benefit SQL query processing and Mongo-compatible processing via Kai as well.
Performance can be improved by many times when inserting or loading data into tables with many persisted computed columns derived from a JSON column.
The JSON_MERGE_PATCH() function has been added as well for easy merging of JSON documents.
Vector Search Enhancements
We now support building vector ANN indexes on a NULLable VECTOR-type column. This makes life easier for developers who may have a text column and a vector column they will fill in later after they go get a vector embedding for the text asynchronously. To speed up vector searches, they can put an ANN index (like HNSW_FLAT) on the column, which can be NULLable and will contain NULL values until the vectors are fetched.
Vector index memory usage tracking is improved.
General Improvements and Fixes
8.9 also has dozens of other functional and performance improvements and bug fixes. See the release notes for a full list.
8.9 will be rolled out on SingleStore Helios gradually over the next several weeks. Customers may opt in to 8.9 for dev clusters beginning approximately 20 Nov 2024.
Best regards,
Eric Hanson
Director of Product Management
SingleStore