The phrase “real-time,” like love, means different things to different people.
At its most basic, the term implies near simultaneity. However, the amount of time that constitutes the “real-time window” differs across industries, professions, and even organizations. Definitions vary and the term is so often (ab)used by marketers and analysts, that some dismiss “real-time” as a meaningless buzzword.
However, there is an important distinction between “real-time” and “what we have now but faster.” A real-time system is not just faster, but fast enough to cross a performance threshold such that your business can reproducibly gain net new value.
This abstract definition is likely too general to assuage the real-time absolutists. However, there is no way to select a single number value definition of “real-time” that works for all use cases. Rather, it’s better to talk about “real-time” as a heuristic and allow stakeholders to establish conventions tailored to their own idiosyncratic business problems.
Instead of claiming real-time means X seconds, this article will describe two classes of real-time applications and their performance requirements.
Machines Acting in Real-Time
One class of real-time applications is where machines programmatically make data-driven decisions. The ability to automate data-driven decisions is especially valuable for applications where the volumes of data or demanding service level agreements (SLAs) make it impossible for the decision to hinge on human input.
Example: Real-Time Bidding
Take the example of digital advertising, where real-time user targeting and real-time bidding on web traffic have revolutionized the industry. Selecting a display ad or choosing whether to buy traffic based on the viewer’s demographic and browsing information can boost click-through and conversion rates. Clearly, the process of choosing ads and whether to buy traffic must be done programmatically – the volume of traffic on a busy site is too large and the decisions must be made too quickly for it to be done by humans.
For this application, “real-time” means roughly “before the web page loads in the browser window.” This brief lag period is essentially free computation time while the viewer waits a fraction of a second for the page to load.
This definition of real-time may not be numerically absolute, but it is well-defined. While businesses implementing real-time advertising platforms will often impose particular SLAs (“this database call must return in x milliseconds”), these time values are just heuristics representing an acceptable execution time. In practice, there may not be a hard and fast cutoff time beyond which it “doesn’t work.” The business may determine that clicks tail off at some rate as page load time lengthens, and shrinking average load time causes an increase in clickthrough rate.
Example load times and clickthrough rate
| Time to load (s) | Clickthrough rate |
|---|---|
| .2 | 3 |
| .4 | 2 |
| .6 | 1 |
| .8 | .5 |
| 1.0 | .1 |
This real-time window is not a discreet interval that guarantees uniform outcomes—rather, it’s defined probabilistically. Every time a user views a web page with a display ad, we know they will click on an ad with some probability (i.e. clickthrough rate). If the page or display ad loads slowly, the viewer is more likely to overlook the ad, or navigate to a different page entirely, decreasing the average clickthrough rate. If the page and ad load quickly, the viewer is more likely to click on the ad, increasing the average clickthrough rate.
While this definition of real-time allows a range of response times, in practice the range tails off quickly. For instance, the clickthrough rate at 2 seconds load time is likely near 0%. This is what I mean when I say a real-time application is one that is “fast enough” to capture some untapped value. The “real-time” approach of dynamically choosing display ads or bidding on traffic based on user profile information is fundamentally different from the legacy approach of statically serving ads regardless of viewer profile. However, real-time digital advertising is only worth implementing if it can be done fast enough to lift intended user behavior.
There are many applications for machines programmatically making decisions in real time, not just digital advertising. Applications include fraud detection, geo-fencing, and load-balancing a datacenter or CDN, to name a few.
Humans Acting on Real-Time Data
The other class of real-time applications is where humans respond to events and make data-driven decisions in real time.
Despite strides in artificial intelligence and predictive analytics, many business problems still require a human touch. Often, solutions require synthesizing information about a complex system or responding to anomalous events. While these problems require the critical thinking of a human, they are still data-driven. Providing better information sooner lets humans reach a solution faster.
Example: Data Center Management
A good example of this type of problem is managing complex systems like data centers. Some of the management can be automated, but ultimately humans need to respond to unexpected failure scenarios. For online retailers and service providers, uptime directly correlates with revenue.
With or without a real-time monitoring system in place, data center administrators can access live server data through remote access and systems monitoring utilities. But standard systems monitoring tools can only provide so much information. The challenge lies in collecting, summarizing, and understanding the flood of machine-generated data flowing from hundreds or thousands of severs. Doing this in real-time has some demanding requirements:
- Granular logging of network traffic, memory usage, and other important system metric
- Interactive query access to both recent and historical log and performance data so administrators can spot anomalies and act on them
- The ability to generate statistics reporting on recent machine data without tying up the database and blocking new data from being written
The third requirement is arguably the hardest, and the one on which the definition of real-time hinges. It entails processing and recording all machine data (an operational or OLTP workload) and aggregating the data into useful performance statistics (a reporting or OLAP workload). The reporting queries must execute quickly without blocking the inflow of new data.
Once again, there is no hard and fast rule for what constitutes a real-time window. It could be a second or a few seconds. Rather, the distinguishing feature of a real-time monitoring system is the ability to converge live data with historical data, and to interactively analyze and report on them together. The technical challenge is not simply collecting data, but how quickly can you extract actionable information from it.
There are many applications for real-time monitoring beyond data center administration. It can be applied to understand and optimize complex dynamic systems such as an airline or shipping network. It can also be used for financial applications like position tracking and risk management.
Moving to Real-Time Data Pipelines
While the specific numerical values associated with “real-time” may vary between organizations, many enterprises are deploying similar data processing architectures to power data-driven applications. In particular, enterprises are replacing legacy architectures, that separate operational data processing from analytical data processing, with real-time data pipelines that can ingest, serve, and query data simultaneously. SingleStore forms the core of many such pipelines, often used in conjunction with Apache Kafka and Spark Streaming, for distributed, fault-tolerant, and high throughput data processing.
