Infiswift uses AI to help meet real-world challenges, largely through Internet of Things (IoT) deployments. They optimize the operation of physical devices using connectivity and data. The company’s innovative platform makes it easy to connect and manage any number of endpoints at scale, with security, and to build solutions that open new services or improve existing ones. Infiswift empowers its customers to be data-driven in industries such as renewable energy, agriculture, and manufacturing. Infiswift has chosen SingleStore as the real-time insights engine of its platform to deliver fast, reliable analytics, and to power constantly updated machine learning (ML) models. We’ve updated this blog post with fresh insights from our recent interview with Infiswift CTO Jay Srinivasan.
The Impact of Device Insights
The Infiswift platform reliably and securely ingests data from any endpoint, combining scalability and simplicity, a combination which most IoT environments struggle to deliver. The platform delivers real-time insights across streams of data as they enter the system. Operators use the data to improve equipment reliability and optimize costs.
The platform uses the MQTT protocol at the core of a sophisticated publish-subscribe mechanism, including multi-level, hierarchical security protection, down to the level of an individual data point. A key ingredient to the analytics system is the need for an adaptable database platform that can manage fast ingestion of intermittent data. Infiswift has found the platform they need: SingleStore.
Real-Time and Historical Data for IoT Systems
IoT systems access millions of devices that generate large amounts of streaming data. For some equipment, a single event may prove critical to understanding and responding to the health of the machine in real time, increasing the importance of accurate, reliable data. While real-time data remains important, storing and analyzing the historical data also unveils opportunities for new optimizations and operating techniques. The combination of both real-time and historical data provides the most complete view for the IoT platform.
Database Requirements for IoT
Business demands put extra pressure on the Infiswift platform to process data quickly, including device authentication and state. Equipment messages come at intervals of seconds or subseconds, driving the requirement for a high-throughput, memory-optimized database. The engineering team needed a database to rapidly ingest changing data, while also persisting to disk for long-term analysis. Because of the large data volumes, the database needs to optimize disk resources by leveraging columnstore data compression.
“We need the data to be processed extremely rapidly, basically in real time. We chose SingleStore because we needed a high-throughput database that can handle our data volumes and velocity with the reliability our customers require.” – Jay Srinivasan
SingleStore Advantages
The Infiswift team needed a database platform that could handle the scale of ingesting and analyzing millions of events in real time. After several technology evaluations, the team chose SingleStore.
“It’s a really well-thought-out product; that’s why I love it.”
Rowstore and Columnstore for Optimal Performance
The unique combination of an in-memory rowstore engine with a memory and disk-based columnstore made for efficient processing and a simplified architecture.
“We wanted to store lots of sensor data, so obviously that will be disk-based storage. But at the same time, we wanted the platform to be fast for real-time data distribution, which means in-memory.”
JSON Support
SingleStore was an early leader in making JSON data a first-class data type within a relational database, making it fast and easy to run SQL commands against semi-structured JSON data. JSON data is fundamental to IoT, so this capability is critical to Infiswift.
Scalable Transactions
The application requires transactional consistency to deliver highly accurate updates for constantly changing sensor events. Infiswift could not get what it needed from NoSQL-based solutions; the speed and compatibility of SQL were vital to them.
“We need a platform that can support these high-energy, high-frequency transactions… we take only 24ms. to complete the round trip.”
Relational SQL
The Infiswift engineering team has an investment in ANSI SQL, along with MySQL, for app development. SingleStore ensures existing technology commitments could remain intact. In particular, SingleStore is compatible with MySQL wire protocol, making it easy to use SingleStore as a drop-in replacement.
“At the end of my time at Google, we were moving to Google Spanner, which is actually a SQL-based interface.”
Compression
The large amount of data collected required sophisticated compression to maximize server resources. SingleStore provides compression of roughly 7x for the application.
AI/ML Integration
AI is a key part of Infiswift’s value proposition, and SingleStore provides fast, simultaneous access to both new, streaming data and historical data, as needed for AI. In particular, Infiswift needs to update machine learning models, running within a Spark framework, in real time – and SingleStore readily supports Spark integration.
“For our machine learning models… in the training phase, our workload runs directly against the same SingleStore cluster that also provides us analytics support and 7x compression for columnstore data.”
Licensing and Deployment Flexibility
SingleStore offers a free edition that allows innovative, fast-moving startups like Infiswift to build and scale. Infiswift is able to flexibly move from SingleStore on-premises, SingleStore in the cloud, and MySQL deployments for small accounts.
“SingleStore has the ability to be completely hostable on-premises for us. It’s not just a matter of moving across clouds; it’s one of our most important requirements, that we also be able to host completely on-prem. SingleStore is hostable anywhere we want. ”
Conclusion
Infiswift was looking to meet four key requirements, not expecting to find a single product that could offer all of them: 1. an in-memory rowstore, 2. on-disk columnstore with compression, 3. SQL support, and 4. availability both on-premises and in the cloud. Not only has SingleStore met all these requirements, with excellent performance; it has also provided two additional capabilities that proved to be important: support for JSON, which is used my nearly all modern IoT devices; and MySQL wire protocol compatibility, which makes SingleStore a drop-in replacement for MySQL, without changing application code.
Building IoT applications involves a balance of technology to enable the scale, security, and performance for operational requirements. The Infiswift platform eliminates complexity and delivers on the promise of a high-performance IoT system that can efficiently enable data-driven operations. Now, with SingleStore, each application built on the platform has the reliability and performance that customers require to drive transformational analytics, machine learning, and AI for their operations.
To learn more about SingleStore for IoT applications, try SingleStore for free or contact SingleStore.
