Analyst firm 451 Research has come up with new research that sees a bright future for HOAP – hybrid operational and analytical processing. The report is titled 451 Perspective: A HOAP-ful future for hybrid operational and analytical processing.
This new type of processing has received several different names, from different analyst firms and consultancies:
- HTAP (Gartner)
- Translytical processing (Forrester)
- HOAP (451 Research)
- Operational analytics (common amongst other analyst firms)
By any of these names, this new style of data processing – which unifies transactions and operational analytics, including many data warehousing-type functions – is widely believed to have a bright future. And SingleStore is right in the middle of it.
What HOAP Replaces
In a previous report, 451 Research identified HOAP as an emerging category. Now, they see HOAP experiencing broad adoption. HOAP seeks to unify two formerly separate data processing categories, and to largely eliminate the need for a third:
- Online transaction processing (OLTP). Transaction processing systems have various needs for reliability, with requirements which begin at strong – “nearly always works” – to absolute -“must work every time, across time zones and disparate data centers, safeguarding against serious financial and reputational consequences for data loss or significant downtime.”
- Online analytics processing (OLAP). Analytics processing systems, which include data warehousing systems, data marts, and data shoeshine stands (just kidding), typically work on copies of existing data. They must be fast, reliable, scalable to multiple apps and users, and affordable, with SQL support.
- Extract, transform, and load (ETL). Systems which transform and format data from ingest or OLTP systems to OLAP systems have become a separate category of their own. Using an ETL system reduces the requirements for the OLTP and OLAP systems that an ETL product connects to, but adding a third system to the mix increase cost and complexity, as well as ensuring significant end-to-end latency.
One of the advantages that 451 Research cites for OLTP systems, and their use of rowstore tables, is the ability to handle complex queries with joins. And they also cite architectural advantages to the separation of transactions, given that these are well suited to rowstore tables, and analytics, which usually benefit from the use of columnstore tables.
SingleStore, however, fuzzes over these distinctions – and, with Universal Storage, is on track to nearly eliminate them. SingleStore not only supports both table types in a single database; it supports joins, and other operations, for rowstore tables, columnstore tables, and across table types.
A diagram from the previous 451 Research HOAP report shows HOAP revenues
as making up roughly 20% of all database revenues by 2021
The Benefits of HOAP
451 Research describes HOAP databases as “taking a bite out of (formerly) pure OLTP workloads,” with more of this expected in the future. There are many benefits to HOAP, largely deriving from its ability to handle both transactional and analytics processing – without the need for ETL. Benefits include:
- Fewer systems to maintain. By eliminating ETL, and taking on many OLTP and OLAP workloads, HOAP systems reduce operational and maintenance complexity within customer organizations.
- Eliminating latencies. Latency due to ETL is eliminated, and the latencies adhering to OLTP and OLAP are reduced to a single overall latency, which may be measured in milliseconds – vs. many hours. or even several days, for the OLTP-ETL-OLAP combination.
- Real-time decision making. When properly exposed to apps, business intelligence (BI) programs, and ad hoc SQL queries, a HOAP database can take the same data which is used for transactions and also make it available for analytics, in near real time.
Spreading HOAP within Organizations
When 451 Research surveyed organizations about the type of workloads they run, HOAP showed up fifth among seven types of workloads, a bit ahead of data science workloads. And 451 sees HOAP revenue growing by more than a third in five years, rising from nearly $60B in 2018 to nearly $80B in 2022.
451 Research also sees HOAP nearly doubling its share of workloads, to more than a quarter of all workloads, in the same five-year period. Why the growing use of HOAP? Reasons include:
- Better new products and services
- Lower costs
- Higher product sales
- Competitive advantages
451 Research describes HOAP as playing an important role in companies becoming data-driven.
Different Kinds of HOAP
According to the report, there are different ways to deliver a HOAP system. 451 Research largely recasts the database wars of recent years through the lens of HOAP. So the HOAP market is made up of both distributed, relational systems, such as SingleStore, and NoSQL systems – which we here at SingleStore believe are overrated and overused.
Cloud vendors also offer HOAP systems. They use their operational control in the cloud to hide system complexity. This allows them to meet customer needs while trading off resources, including human intervention in database operations, behind the scenes. At the same time, a cloud database that’s exclusive to a cloud provider is not, ironically, cloud-native. These systems typically run on just one cloud vendor and can’t be run on-premises, even using cloud-native tools such as Kubernetes.
SingleStore, as fully cloud-native software, runs anyplace you can run Linux – on-premises, or in any cloud. SingleStore offers a Kubernetes Operator for easily managed, cloud-native operation in any environment. And Singlestore Helios is a database-as-a-service (DBaaS) offering, managed by SingleStore using Kubernetes.
All of these capabilities fit 451’s desiderata, which include database-as-a-service across hybrid IT systems and vendors that offer a consistent experience across a broad portfolio of options.
Bringing HOAP to Life in Your Organization
To get a copy of this report, contact 451 Research. And, you can try it yourself: take advantage of a time-limited free trial of Singlestore Helios, or run SingleStoreDB Self-Managed for free. Download SingleStoreDB Self-Managed or connect to Singlestore Helios for free today. Or, if you want to know more before you proceed, contact SingleStore.
