DirectlyApply Saves $100K/Year while Helping Job Seekers Find Opportunities 400X Faster with SingleStore on AWS

DirectlyApply, an angel-funded and bootstrapped company based in London, United Kingdom,  is a job discovery platform and vertical search engine that connects job seekers with employers across North America. The full-time team of eight developed a search engine that provides job seekers with job listings to which they can directly apply.

The platform fetches thousands of vacancies from job sites, portals, and career websites, then intelligently filters the results to provide a curated list of up-to-date opportunities at real employers that are actively hiring.

Challenges/Goals

“We have always had, and will always have, only one business goal: to improve job search and discovery for our users,” explained Dylan Buckley, Co-Founder, DirectlyApply.

The team at DirectlyApply believes nothing should stand between the job seeker and their next opportunity. The platform focuses on helping job seekers conduct smarter searches without receiving anonymous job adverts, fake posts, or irrelevant sponsored results. DirectlyApply removes the barriers that have traditionally existed between job seekers and real jobs.

“By solely focusing on chasing this goal, we are, in turn, pushed to constantly improve and innovate the platform. As we are able to deliver quality candidates to our employers’ vacancies, we increase our own revenue and profitability, and ultimately are able to grow our market share,” said Buckley.

The company has prioritized profitability from day one, allowing its small team to achieve financial results on par with much larger competitors.

“We are predominantly remunerated by our clients on a cost per application (CPA) basis, although in some instances we are paid on a cost per click (CPC) basis, which is the way most of our competitors are paid,” said Buckley.

The DirectlyApply platform was originally built on MongoDB with an ElasticSearch layer on top to try to resolve performance issues that had crept in over time. However, as the platform scaled, MongoDB quickly fell behind from both business and technical perspectives. Querying became unnecessarily cumbersome because MongoDB’s Document database model necessitated creating hundreds of millions to billions of documents. As an additional frustration, the cost of running two replica database systems that were performing suboptimally added up fast in monthly hosting costs.

“We didn't want to impact user experience further or risk getting to a stage where we would be limping along because migration is typically painful. We therefore knew we needed to find a solution and quickly,” said Buckley.

Platform search time was increasing, which was beginning to cause some user frustration. Furthermore, DirectlyApply had to scale back the initial database queries and do a lot of client-side processing, which was also beginning to impact the user experience.

“We were concerned that if this continued, our ability to provide great job results was at serious risk,” said Buckley.

Technology Requirements

Querying was completely unusable on MongoDB as the company grew, and ElasticSearch was struggling to deliver the speed the platform needed for the best possible user experience. 

“We knew we had outgrown NoSQL, and MongoDB was chosen prior to realizing how quickly DirectlyApply would scale,” explained Buckley. “MongoDB was great for helping provide an initial proof of concept, but our needs soon outran it."

The team started looking for a fully managed SQL solution built for high-volume data ingress. Speed was the most important factor DirectlyApply was optimizing for.

Why SingleStore

During DirectlyApply’s search for a SQL-compatible database capable of delivering the speed and scale it needed, the team discovered Jack Ellis, Co-Founder, Fathom Analytics, a SingleStore customer who has detailed his experiences with this technology in a series of blog posts linked at the bottom of this Fathom Analytics customer impact story page.

DirectlyApply saw that it faced similar challenges to Fathom Analytics and began exploring whether SingleStore would solve its problems, too. SingleStore delivers real-time, unified, distributed SQL. With a unified data engine for transactional, operational, and analytical workloads, SingleStore powers real-time data, applications, and analytics.Its support for multi-cloud deployments was also a compelling factor in DirectlyApply’s decision. SingleStore runs on the Big 3 clouds and SingleStore has hundreds of customers including 100+ Fortune 500, Forbes Global 2000, and Inc. 5000 brands.

“We started using SingleStore as a test on an internal analytics project in late 2021, and began exploring how we could expand it across the platform,” said Buckley.

It wasn’t long before DirectlyApply decided to migrate from MongoDB and Elasticsearch to SingleStore Helios and the team completed the process in November 2022.

Solution

“We have transitioned not only analytics but also our catalog, customer 360, and personalization systems to SingleStore Helios. It just made sense to run it all on one platform, and that platform is SingleStore,” Buckley explained. 

SingleStore Helios powers 80% of the DirectlyApply platform following the migration process. It coexists with the remaining 20%, which consists of a few small systems and MongoDB clusters that have not yet been moved over, such as backend portals.

“It was very simple to get started with SingleStore Helios and implement it. Now, our whole team is able to build fast, as SingleStore presented a lower barrier to entry than ElasticSearch. And our engineers commented on the quality of the documentation,” said Buckley.

Everything DirectlyApply customers use is entirely on SingleStore. The main platform is a Next.JS application, with services hosted on Vercel, Heroku, and Amazon Web Services (AWS). The backend systems are built in GoLang and Python, and are hosted on a DigitalOcean Kubernetes cluster.

Outcomes

With SingleStore, DirectlyApply was able to achieve goals and build features many years ahead of its expected schedule.

400X Query Performance for Vastly Improved User Experience 

ElasticSearch queries took 2600 milliseconds to show results. SingleStore Helios reduced that to six milliseconds, resulting in a 400X improvement. DirectlyApply users are now able to search quickly for jobs and get as many listings in front of them as possible. Additionally, thanks to SingleStore, DirectApply now delivers a far better user experience.

Cost Savings of $100,000 per Year

DirectlyApply has always been lean and profitable, and was able to reduce its database costs substantially by adopting SingleStore Helios for 80% of its platform. The company no longer had to maintain two replica database systems.

Radically Faster Product Development Due to Familiar SQL 

The entire DirectlyApply team is able to build quickly, as SingleStore Helios’s SQL support reduces the skills barrier compared to ElasticSearch.

Improved Revenue Through Better Platform Performance

“For every 10 milliseconds you can shave off search and load times on your site, you gain readership, page performance improvements, and revenue. If your site takes more than 1.5 seconds to load or show results, people are likely to bounce,” explained Buckley. 

“Through solely focusing on chasing  our one goal, we are able to deliver quality candidates to our employers’ vacancies, which has allowed us to increase our own revenue and profitability. Ultimately we are able to  grow our market share.”

Expanded the Platform’s Customer Engagement Capabilities

“We’re incredibly successful and we’ve only scratched the surface, attaining only 1% of where we can go,” said Buckley.

One of DirectlyApply’s major areas of focus for 2023 is growing the customer reengagement functionality of the platform through processing, storing, and indexing large volumes of customer data to provide even greater quality insights for job seekers on their job search. SingleStore Helios allows DirectlyApply to do this considerably faster and with more accuracy than first anticipated.

Many of these features have been on its roadmap since inception. “We’ve been dreaming about doing this for four years,” concluded Buckley. “Now with the help of SingleStore, we can accelerate helping job seekers find new jobs.”

---

SingleStore is helping companies compete and win across every vertical. Learn more →