Sometimes we become so accustomed to certain things, we stop asking questions and accept the status quo. That can last a long time.
Such is the case with batch processing in the enterprise. And it was made abundantly clear in a survey titled “Enterprise Big Data, Business Intelligence, and Analytics Trends” by the Enterprise Strategy Group, that there remains an ongoing complacency with the time it takes to complete batch processing. This is in stark contrast to the recognition that real-time processing still mandates measurement in microseconds.
The survey involved 375 IT and business professionals familiar with their organization’s current database, business intelligence, and analytics solutions. Respondents are primarily large mid-market organizations (500 to 999 employees) and enterprise organizations (1,000 employees or more) in North America. Industry verticals including financial, business services, manufacturing and retail.
A Dichotomy of Opinions
When surveying users about real-time operations, 45% of respondents indicated data is updated in seconds to support real-time databases, with 29% citing milliseconds, and 12% citing microseconds. That is 86% of users working with second or sub-second updates.
Figure 1: Frequency of Updating a Data Set for Teal-Time Analytics. Source ESG
However, the picture on the batch processing side is at the opposite end of the spectrum, with most respondents indicating they update daily.
Figure 2: Frequency of Updating a Data Set for Batch Analytics Source ESG
Why Such Discrepancy?
For too long, the database world has been split into silos, where individual datastores each have a portion of data. In order to get systems to work together, companies rely on time consuming ETL (extract, transform, and load) to move data into place for batch processing. Ultimately, that leads to lengthy update cycles.
Real-time Transactions and Analytics in a Single Database
Today, customers have a choice. And one of those is to close the batch gap by adopting databases that can handle real-time transactions and analytics concurrently. In doing so, customers can eliminate many of the delays with ETL to achieve complete visibility to their data, even as it continues to change. As industry expert Curt Monash recently noted,
Internet interaction applications increasingly require data freshness to the last click…
Curt Monash
“Up to the last click,” is now possible with modern approaches that merge transactions and analytics, eliminating the delays of ETL and help close the batch gap. These are core tenets of SingleStore and why we built our real-time database for transactions and analytics.
To learn more visit www.singlestore.com give it a try for free, visit www.singlestore.com/cloud-trial/.
