impact.com Accelerates The Partnership Economy with SingleStore

In performance-based (or simply performance) advertising, the advertiser pays only when there are measurable results, normally classified as conversions. This is by contrast with brand marketing and advertising, which focuses on building awareness in the marketplace. As if performance advertising were not already challenging enough by definition, it  has also faced external challenges, such as Apple blocking third-party cookies, leaving marketers struggling with campaign tracking and attribution. Performance advertising companies needed new ways to analyze performance and manage partnership programs that evolved beyond traditional affiliate marketing, and impact.com stepped into the void.

impact.com enables companies to tap into the massive potential of The Partnership Economy. It offers the world’s leading partnership management platform for performance advertising that supports brands, publishers, and agencies in building relationships with publishers and consumers, and provides visibility across the entire consumer journey. It offers streaming analytics for reporting and insights on digital marketing, affiliate success, fraud detection and compliance. Its customers pay not for impressions or clicks but for outcomes (conversions) such as sales, leads, app installs, and signups.

impact.com has 1,000 employees across 11 global locations serving more than 1,000 customers including 1-800-FLOWERS, Adidas, Canva, HSBC, HubSpot, Lenovo, Levi Strauss & Co., McAfee, Microsoft, Ticketmaster, Uber, and Walmart.

Challenges/Goals 

impact.com must be able to process 20 million events per hour for 1,000 concurrent users who need to run many reports throughout the day and expect real-time data, low latency, and an interactive experience. impact.com was experiencing rapid growth in datasets across multiple databases and ran into problems with ingestion speed at the concurrency it needed. The existing system used Cloudera Impala and Kudu and was unable to achieve sub-second reporting.

“As much as we tried to optimize the data structures in our reports, with Cloudera we could never get reporting to be sub-second. It was anywhere from two to 10 seconds, which was not the experience we wanted to deliver to our customers,” said Mauricio Aristizabal, Principal Data Architect, impact.com. “And I’m not just talking about exploratory, ad hoc business intelligence. I’m also talking about standard reports, dashboards, and widgets. We needed reporting to be a lot faster than we were able to deliver at the time.”

Mauricio Aristizabal, Principal Data Architect, impact.com

impact.com was also dealing with a poorly defined static cost structure and a lack of account team support and engagement with Cloudera.

“Historically for us with Cloudera, it has been difficult because until recently, it required us to use Kerberos [a network authentication protocol]. We had to set up Kerberos tickets and everything else for any client out there. In some cases, that wasn’t possible because some clients simply did not support it,” said Aristizabal.

All of these factors led to negative business consequences. impact.com was missing performance service level agreements (SLAs), delivering poor end-customer experience — and paying more in the bargain. It was dealing with increasing costs associated with supporting and scaling the database infrastructure, and unforeseen budgetary increases along with little pricing flexibility.

Technology Requirements 

Aristizabal and the team started evaluating other solutions to meet its growing data-intensive platform needs, including:

• Low latency analytics to meet user expectations and speed up the time-to-insight
• Support for a variety of business intelligence (BI) reports, dashboards, and other visualizations, APIs and by-key lookups
• Distributed (scale-out) model with push-button online cluster resize to continuously optimize cost while meeting SLAs
• High concurrency capable of supporting 1,000 users and full multi-tenant functionality 
• Ultra-fast performance for processing 20 million events per hour.
• Primary keys to support idempotency — the abiilty to apply operations multiple times without changing the result beyond the initial application — and fast lookups from the same dataset that is also serving ad hoc queries
• Support for updating data at scale
• Separation of storage and compute, and among tenants (workspaces)
• Low management overhead: no manual compactions, stats gathering, or rebalancing
• Easy client connectivity

Why SingleStore

“We looked at four players: BigQuery, Cloudera CDP (Impala/Kudu), SingleStore, and Snowflake. Several of the competitors have good features, but they all had a pitfall or two that held us back. Whereas SingleStore really ticked all the boxes,” explained Aristizabal. 

impact.com had added SingleStore to its evaluation process due to the company’s original name.

“Frankly, it was the name at the time, MemSQL, that compelled us to reach out to the SingleStore team. We were looking for an in-memory data store because we thought that’s what would enable the sub-second response we required,” said Aristizabal.

SingleStore is the modern database for data-intensive applications. impact.com was particularly drawn to these capabilities: 

• Sub-second reporting thanks to a blazing fast, infinitely scalable database engine and data structures
• Separation of storage and compute with SingleStore Workspaces to avoid maxing out memory and budget, creating an infinitely scalable system
• Easy connectivity with a simple username and password, and with user-friendliness reminiscent of MySQL
• The ability to have transactions, analytics, and multiple data types in one modern database, minimizing data movement and delivering ultra-fast speed
• Fast key lookups and multiple indexes

Solution 

impact.com deployed SingleStore Helios on Google Cloud. SingleStore checks all the boxes with sub-second reporting, low-latency analytics, high concurrency, and more. SingleStore provides impact.com with real-time analytics on its multi-tenant digital marketing, affiliate success, and fraud platform. When the migration is complete, SingleStore will be Impact.com’s data warehouse for 100% of its data and reporting. In short: All Data. One Platform™ — and Impact.com will be managing six petabytes of data with SingleStore.

For the initial deployment, impact.com moved 20-30% of its data from Cloudera into SingleStore (acting as a data mart), which generated 80% of the reporting requests. It used a resource-heavy deployment model for extra performance and peak season capacity.

“With SingleStore, we can find a specific record among billions via fast by-key lookups. Our team can do analytic queries on any column because it’s columnar, and they can also do sub-second super-fast reporting on a couple of columns that have indexes,” explained Aristizabal.

Outcomes

impact.com’s partnership management platform now handles affiliate and cash back programs, as well as influencer and B2B use cases, with features that include: 

• Fraud detection to verify that real humans are taking actions, not bots
• Compliance checking to ensure partners are using company brand names appropriately
• Tracking and attribution that identifies the click that drove the conversion and other partners that contributed to the customer journey
• The ability to handle all fund transfers and support for more than 60 currencies, allowing users to fund in their currency, from one bank account, to pay partners globally in their local currencies

“It's a pleasure working with SingleStore. It has a great sales organization and delivers exceptional service. It's not just the fact that we have well planned and structured quarterly business reviews, but I have peace of mind knowing my SingleStore team is a Slack message or or phone call away if we run into any trouble,” said Aristizabal.

Low-Latency Analytics Delivers Stellar Customer Experiences

With SingleStore Helios running on Google Cloud. impact.com is able to achieve its sub-second latency analytics goal. Many concurrent users can quickly access many types of reports, allowing them to make better business decisions faster with real-time analytics.

“In the end, we learned that sub-second latency wasn’t really just about having an in-memory database, but also about having SingleStore’s robust engine and data structures,” said Aristizabal.

Eliminating Database Sprawl Reduces Management Overhead

“We can have one set of data, one table, and one data structure that can satisfy many use cases with SingleStore. It enables us to do all kinds of things with one piece of data without having to duplicate it, use separate pipelines, or worry about data discrepancies,” explained Aristizabal.

Partner Automation Leader Moving Forward with a Stable, Future-Proof Data Partner

“The platform has been very stable. We’re using the managed service, so we don’t really have to do much management on our end. The few times we’ve had some fleeting hiccups, SingleStore has been right on it. Working with SingleStore the last two years gave us the confidence to go all in on the platform. That was a big decision, and it has paid off. We’re looking forward to continuing this successful partnership for many years to come.”

“In the near future, SingleStore will become our main data warehouse that will handle 100% of data and reporting requests for all of our use cases. We’re looking forward to that migration,” said Aristizabal.

impact.com is also looking forward to using SingleStore WorkSpaces, which separates compute and storage for management and workload isolation. “Workspaces is very exciting to us because the separation of storage and compute makes it possible to serve even more tenants while ensuring their workloads are isolated and don't affect each other, and allows us to fine tune resources and cost (via workspace size) to match their specific requirements and SLAs,” explained Aristizabal. “In Cloudera, for example, there are different database pools for a little bit of isolation between different workloads, but it’s not fine-grained.” With SingleStore, impact.com will be able to create individual workspaces for tenants so that one workload does not impact the other. If an API gets out of control, it won’t affect other components, such as reporting.

“With SingleStore WorkSpaces, we will have a main read-write workspace where we do our extract, transfer, load (ETL) and maintain the data," said Aristizabal. "Then we will have individual workspaces that are read-only for all our tenants, so they’re using their own compute and not affecting everyone else."

---

Watch this Fireside Chat as impact.com’s Mauricio Aristizabal talks about how when Cloudera’s Impala and Kudu could not keep up with the speed business, SingleStore delivered.

SingleStore is helping companies compete and win across every vertical. Learn more →