Data today moves in extremely large volumes at light speed all while requiring high levels of protection on the content. Traditional database and data warehouse technologies are unable to handle these workloads without the need for expensive hardware appliances. The NoSQL and Hadoop-based systems are very complex and unable to support real-time analysis. A new way to process fast-moving, structured content is needed to achieve this level of scale and security. Why not a highly secure and extreme scalable SQL-based approach?
Imagine the volume of telemetry data that is required for processing a swarm of hundreds of mini-drones. This data would be rich with geospatial, video, and metadata content. What if we needed to use this real-time data as input to a predictive analytics behavior model to determine the patterns of life in a specific geographic location? Maybe this content is collected from an area of interest surrounding a world event, such as the Olympics. Real-time dashboard applications and who has access to those dashboards need to be locked down to your most trusted analysts. The content needs to be highly available and remain confidential. This hypothetical use case requires a technology fabric that can ingest, transform, model, store, analyze, and secure data in terabyte and petabyte scale in real time.
Welcome to SingleStore. SingleStore is a purpose-built data management system for high volume workloads and provides a fast and secure platform for mission-critical analytics. The platform provides a data pipeline framework that allows real-time ingest, transform, model, and persistent storage of streaming content. A direct connection to a distributed message queue system such as Kafka can be made. Data in flight can be transformed with a distributed processing engine such as Apache Spark to run your machine learning algorithms and store directly in the engine. The database is then locked down to secure activity, content, and real-time access to your most trusted administrators, developers, and analysts.
The key security tenants behind SingleStore are:
- Separation of duties to protect against rogue administrators and controlled access
- A 360° view of all database activity through full audit logging
- Database vault functionality thru Strict mode
- User authentication and encryption for data in transit and on disk
- Enforced password protection
- Highly available data thru level 2 redundancy and cluster replication
With SingleStore, high performance at large scale has never been more achievable. Query speeds range in hundreds of millions to over a billion data points per second per core on four core commodity hardware. With this capability for example, Thorn can compare facial images in real time using linear algebra operations on vectors to match image fingerprints using a basic SQL query to SingleStore. The architecture behind SingleStore is based on large scale parallelism, data compression, in-memory processing, vectorization of queries, compiled queries, and query optimization. Given these architecture principles, SingleStore is able to run in a secure configuration minimizing impact on performance. The guiding principles behind SingleStore Enterprise Security are based upon confidentiality, integrity, and availability.
In conclusion, SingleStore, a real-time data warehouse, is a platform for building your highly secure systems requiring speed-of-thought analytics on streaming data into your organization. No longer does your organization need to sacrifice security over performance.
To learn more about how SingleStore is keeping real-time analytics secure, come chat with us next week at the Department of Defense Intelligence Information System Worldwide Conference at booth 634.
