Cybercrime damages in 2021 are expected to reach \$6 trillion, with organizations of all sizes and industries exploring ways to protect themselves better. Potential security vulnerabilities come in many forms, from cross site scripting to improper privilege management. Over 17,000 new security vulnerabilities were discovered in 2020 alone. IT security teams are responsible for securing infrastructure that’s continually increasing in complexity, so it’s challenging to respond quickly to newly discovered exploits or confirm that a known vulnerability is addressed. It doesn’t take long for an organization to end up with a substantial vulnerability backlog. On average, over six months, organizations end up with a backlog exceeding 57,000 and fail to mitigate 28 percent of these vulnerabilities. A pre-emptive, holistic, and scalable vulnerability management approach is needed to keep systems safe.
Addressing a Critical Vulnerability Management Process Gap
The founders of Nucleus Security, Stephen Carter, CEO, and Scott Kuffer, COO, had decades of experience providing vulnerability management services to United States government agencies. They used a variety of vulnerability scanning tools for this purpose. Whenever they found a vulnerability, they had to manually enter a remediation request into a ticketing system, such as Jira or ServiceNow. The gap between the identification of vulnerabilities and the creation of tickets created an efficiency bottleneck. All of the vulnerability reports needed to be normalized across a range of formats used by different tools and prioritized by the user. The response then needs to be automated for greater speed, manageability, and scalability.
Carter and Kuffer created the Nucleus vulnerability management platform to make it faster and easier to provide services to their clients. Their original intent was to have a custom tool for their own operation, but they quickly discovered a major need for this type of platform.
The users most interested in Nucleus Security were:
- Large enterprises and government agencies needing to manage security vulnerabilities at scale.
- Singlestore Helios Security Providers (MSSPs) looking for a platform that supports their vulnerability management processes with multiple clients, offers client access, and provides a way to expand it with their proprietary functionality.
Handling Unified Vulnerability Management at Scale
Since Nucleus was created by and for vulnerability management experts, the platform solved many problems that stood in the way of discovering and remediating vulnerabilities before they became exploits.
Smart automation and workflow optimization eliminated many tedious and time-consuming parts of vulnerability management through:
- Centralizing vulnerability aggregation, normalization, and prioritization in a single platform.
- Delivering the right information to the right people in the right format by connecting vulnerability, ticketing, alerting, and issue tracking tools together.
- Increasing vulnerability management speed and capacity from end-to-end.
- Reducing costs for vulnerability management.
- Improving the visibility of vulnerability management data and remediation efforts.
How Nucleus Works
The Nucleus application was delivered as a software-as-a-service (SaaS) offering and was designed with a traditional three-tier architecture. “There’s a web application, which is customer-facing, and a job queue that processes data. Then there’s a database on the backend, serving both,” says Kuffer.
- Users log in to the customer-facing Nucleus web application.
- They set up integrations with the tools they’re using from a list of more than 70 supported applications.
- The users create rules to ingest data and trigger alerts, reports, and tickets.
- They can access a centralized vulnerability management dashboard, perform searches, and analyze their scan results.
- A separate backend job queue ingests and processes the data on the user-specific schedule from the selected data sources’ APIs.
- A database powers the frontend and backend operations.
The product licensing is based on the number of assets that an organization is working with. In Nucleus, an asset is defined as an item that the user scans. Each asset can have multiple vulnerabilities.
The Challenges of Pivoting to a Commercially Viable Vulnerability Management Platform
The decision to launch a separate company, Nucleus Security, had not originally been the founders’ plan. Kuffer explains, “We actually just went to a conference called Hacker Halted in Atlanta, Georgia, and we had Nucleus as basically a line item in a data sheet for our old company. We just got basically hammered with leads at that point.”
“We were not prepared at all for any leads whatsoever, and so none of us had any sales experience, we didn’t have a company, we had nothing. We didn’t really have much of anything, other than a product that didn’t scale for all of these needs.”
Nucleus had many leads coming in that were bigger names and companies, which helped to expedite the decision to fork it off into a separate business at the end of 2018. However, the founders needed a way to scale the platform for these Fortune 500 companies, and they needed it fast.
Finding the Right Database Solution
When Nucleus Security began architecting the Nucleus platform, they explored several database options, including document-oriented databases, graph databases, and traditional relational databases.
Carter says, “All of the options had their strengths and weaknesses, and we felt that several database technologies could work well for the initial proof of concept. However, it quickly became apparent that a high-performance relational database system was a hard requirement to support our data model and the features we were planning to bring to market.”
They started out developing the solution they would have needed when they were working directly with government agencies. These clients are highly focused on compliance and run scans weekly or monthly, based on the regulatory requirements. The database solutions that Nucleus tried often took a long time to process vulnerabilities, but it wasn’t a big issue at a weekly or monthly cadence.
Struggles with MariaDB
The Nucleus prototype used MariaDB, which is on many federal government-approved software lists. “MariaDB comes bundled with the government-approved operating system we were using, which is Red Hat Linux,” explains Carter. “For the prototype, this worked just fine. But when we started to onboard larger customers, we hit some ceilings performance-wise, and some pretty major performance issues.”
“As a workaround, we were trying to precalculate a bunch of stuff, to show it in the UI. But if you’re scanning every hour, and it takes an hour and a half to do the calculations, then you get a huge backlog,” says Carter.
The team spent a lot of time tuning queries to squeeze out every bit of performance that they could to keep up with the capacity needed for their beta customers. However, even with the clustering and load-balanced configurations available, it was clear that a different database solution would be needed for Nucleus to support very large enterprises, which have hundreds of thousands of devices and tens of millions of vulnerabilities.
Commercial clients scan for vulnerabilities multiple times daily or weekly. They wanted to see the results of a scan in minutes, not hours or longer. Over time, the backlog built up and hit a ceiling where the Nucleus application couldn’t process jobs fast enough. Batch processing worked with federal agencies, but enterprises demand real-time vulnerability management.
“It was the database that was the bottleneck all along,” says Kuffer. “We looked at some NoSQL solutions. We looked at Percona for clustering, but we would have had to rewrite a lot of our code – and all of our queries.” Nucleus Security also investigated other SQL solutions based on PostgreSQL core, such as Greenplum.
The relational database for a commercially viable version of Nucleus needed:
- Real-time processing
- Support for 10s of millions of vulnerabilities and 100s of thousands of devices
- High scalability
The Benefits of SingleStoreDB for Vulnerability Management Platforms
Nucleus Security started looking into alternatives to MariaDB that were better suited for its vulnerability management platform. They found Percona first, but the initial tests indicated that it wouldn’t help with their use case. The scaling focused more on being a high-availability cluster. While they could add to the cluster and use load balancing schemes with different nodes, it was an extremely manual process. It also required a minimum of three servers.
SingleStore came up during Nucleus Security’s search for a database and impressed them from the start. “SingleStore was a great option, because SingleStore is not only relational; it also supports the MySQL wire protocol, which of course is inherent in MariaDB,” explains Carter. “It was almost a drop-in replacement for MariaDB, and it’s way less complex, and also much easier to maintain than the Percona cluster solution that we were looking at.”
SingleStore is The Single Database for All Data-Intensive Applications powering modern applications and analytical systems with a cloud-native, scalable architecture for maximum ingest and query performance at the highest concurrency. It delivered many powerful capabilities, such as:
- Optimization to run anywhere from bare metal to the hybrid cloud with commodity hardware, including multi-cloud
- Simple to deploy
- Better performance than legacy RDBMS and NoSQL databases for high-velocity big data workloads
- Memory-optimized
- Low total cost of ownership by integrating with existing systems
- Ingest millions of events per second with ACID transactions while simultaneously analyzing billions of rows of data in relational SQL, JSON, geospatial, and full-text search formats
- Data sharding
- Latency-free analytics
- Seamless horizontal and vertical scaling
- Workload management
- Compressed on-disk tables
Switching to SingleStoreDB from MariaDB in Nucleus
Nucleus Security started with the free tier of SingleStoreDB Self-Managed for the proof of concept early in 2019. The founders wanted to determine how hard it would be to migrate the application to the new database. They imported 80-100 gigs of data from MariaDB that included several tables with several 100 million rows to test their slowest queries. It only took an afternoon to migrate the development database to SingleStoreDB Self-Managed and get Nucleus working without any architectural changes.
Carter says, “It was dead simple to get set up. Whereas, I’ve got experience getting Oracle database clusters set up, and those things can be nightmares. And our experience with SingleStore was very good. We do not spend a lot of time maintaining it, troubleshooting it.”
Following the successful proof of concept, the Nucleus application moved to a three-node cluster along with several single server instances for customers in Australia, the United States, and the European Union. It took 3-4 weeks to get Nucleus tested, deployed, and entirely in production on SingleStoreDB Self-Managed. Nucleus got its first keystone client, the Australian Post Office, in March 2019, shortly after the migration. They had 2,000 code repositories and required approximately 50,000 scans per day.
Kuffer says, “They paid us a lot of money upfront on a multi-year deal, and plus they had the brand name and it allowed us to transition that into a lot of later success. We definitely wouldn’t have been able to support the scale that they had without SingleStore.”
No Architectural Changes
The Nucleus app brings in data directly from the APIs of vulnerability scanning tools used by customers, and interacts directly with their job scheduling systems, such as Jira or ServiceNow, directly. There’s no need, at this time, to use Kafka or other streaming technologies.
Nucleus did not need to make any architectural changes to their application to move to SingleStore; it has served as a drop-in replacement for MariaDB. Since MySQL wire compatibility is shared by both, making the move was easy. By replacing MariaDB with SingleStore, Nucleus customers can now support MSSPs with full scalability.
Reference Architecture: Nucleus replacing MariaDB with SingleStore
The Nucleus team did not need to make any architectural changes to move to SingleStore, and they describe using SingleStore as “dead simple.” Stephen said, “It was dead simple to get set up. Whereas, I’ve got experience getting Oracle database clusters set up, and those things can be nightmares. And our experience with SingleStore was very good. We do not spend a lot of time maintaining it, troubleshooting it. Anyone that’s using MariaDB, we would recommend it to. No question.”
Why Nucleus Security Deployed SingleStoreDB Self-Managed on AWS
Nucleus Security opted to use SingleStoreDB Self-Managed on AWS rather than Singlestore Helios for several reasons:
- Enterprise clients have strict security and compliance requirements for their sensitive data.
- Some Nucleus clients require their data to be hosted on servers within their country or region.
- Enterprises may be uncomfortable with smaller third-party providers.
What SingleStoreDB Self-Managed Helped Nucleus Security Accomplish
By leveraging SingleStoreDB Self-Managed for its vulnerability management platform, Nucleus Security has:
- Improved query speeds from 2 – 20x faster without any schema or query changes.
- Increased customer size capacity by 5 – 10x on the same hardware through SingleStore’s data compression, which reduced 90 gigs down to 5 gigs.
- Enabled quick and easy horizontal and vertical database scaling to support large enterprise clients.
- Achieved a 50X improvement on asset scan ingestions, going from 5,000 assets in 3 hours to 100,000 assets in 45 minutes.
- Kept AWS costs at one-third of what Percona would have required.
Carter explains: “Several vendors have tried to build vulnerability management products that scale, but nothing in the market actually did. Then we found SingleStore, which allows us to handle customers 10 times the size, and to be ready for future growth as well. For years, large enterprises and managed service security providers have been looking for a vulnerability management solution that can scale; many had tried to build such a platform themselves. Now we have it, thanks to SingleStore, and we are growing rapidly. ”
Learn more from this youtube video: Launch Codes for SaaS Success – The Know Show
What’s Next for Nucleus Security?
As a company with a quickly expanding customer base, Nucleus Security is just getting started. The team is constantly working to keep up with increasing volumes of data generated by enterprise vulnerability scanning tools and continually expanding its growing list of 70+ integrations with scanners and external tools. Nucleus Security will continue to work closely with SingleStore and looks forward to the new features of SingleStoreDB Self-Managed to power the growth of its platform.
Get started with SingleStoreDB Self-Managed for free, or contact us to learn more about our distributed SQL database built for operational analytics.
