Epigen Technology depends heavily on SingleStore as part of its core toolkit. “Without SingleStore, I can’t do what I do,” says Terry Rice, founder of Epigen.
Visit SingleStore at AWS re:Invent booth #1905 in Las Vegas, November 27-30.
Epigen Technology uses software to innovate in many fast-changing areas of technology, including cybersecurity, analytics, machine learning, artificial intelligence (AI), and the emerging area of cognitive computing. Now, Epigen has developed a framework for facial recognition in the cloud that combines AWS S3, SingleStore, machine learning, AI, and visualization.
Terry Rice, founder of Epigen, is a noted IT consultant and is trusted by senior figures in business and government to deliver solutions to challenging problems, including national security threats. He’s also a fifth-generation Marine. Terry graduated early from the University of Oklahoma with dual degrees in Information Systems and Statistics. After finishing college, he enlisted as a Marine, following a family tradition going back more than a century. He served for six years.
Terry then moved into roles as an IT consultant and software architect for companies, such as Capgemini, Lockheed Martin, Network Solutions, and Verizon. He’s also worked for government, serving as a solutions architect for the Department of Homeland Security. In his current role, Terry is helping to lead the development of innovative systems in several areas. He speaks at conferences on blockchain, big data, machine learning, and AI. And he uses SingleStore as a core part of Epigen’s toolkit.
Recognition Faces Challenges
Facial recognition is one of the most challenging topics in AI. Facial recognition for isolated static images is improving – as experienced by anyone who uses Facebook, which can often suggest names for most of the faces in a photograph of family or friends.
The Facebook example shows some of the characteristics that make facial recognition easier:
- A well-lit, high-resolution, static image.
- A small list of candidate names for each face (the user’s Facebook friends).
- Subjects not trying to disguise themselves.
- Low consequences for failure – in the Facebook case, a non-identification or misidentification is an annoyance, or even humorous, rather than a serious problem.
However, Epigen’s clients in areas such as the military, law enforcement, and air transportation need facial recognition that works under increasingly challenging circumstances, including working at scale and using video frames, not just static images.
Currently, facial recognition is being used at some airports to verify the identity of passengers on a per-flight basis. The comparison is a four-step process:
- For a specific flight, passport images for all ticket holders are vectorized and stored.
- People who appear at the gate to board a flight are photographed live.
- Live passenger photos are compared to the stored passport photos.
- If a live passenger photo doesn’t match any of the stored passport photos, the passenger is asked to verify their identity as a match against their passport.
After years of testing, this system is now in use at more than a dozen airports and at land ports of entry to the United States. In one recent case, it prevented a passenger who was using a passport that was valid, but not their own, from boarding a flight at Dulles Airport in Washington DC.
The current approach requires that each live photo be compared to as many as nearly a thousand passport photos (given that an Airbus A380, the largest passenger airplane, can carry more than 800 passengers). This is only the beginning. Current, and potential future, users of facial recognition systems need more capability.
Epigen is working on a series of improvements:
- Higher accuracy – for the current, one in a thousand use case. The system used to check passengers against the passport photo database is only about 85 percent accurate. There is a short-term goal to improve accuracy to 97 percent, then further improvement will be required from there.
- Compare against other databases – for use cases of one to thousands and one to millions. Today, each live photo is only compared to the passport photos of one specific flight’s passengers. In the future, the system should also compare live photos to databases from Immigration and Customs Enforcement, the FBI, other law enforcement agencies, motor vehicle departments, and others.
- Use full-motion video – lower-resolution, moving images. Video cameras are everywhere, even more so in some countries than others. Automatically spotting wanted people on the move is an important goal for law enforcement. One high-priority use case, hard to deter by other means, is recognizing missing and kidnapped children. AI-based recognition from video could augment existing systems, such as the AMBER Alert system used in the US.
One target application for video is to shorten lines at airports. If people can be identified and matched against passport photos as they move through the terminal, they can be, in essence, pre-approved. The small number of people who raise concern can be contacted by officials before reaching a gate. This pre-checking can avoid the need for people to queue at entry points, potentially eliminating the long lines at Customs, for instance, when entering a country.
Throwing Resources – and SingleStore – at the Problem
How can facial recognition be made to work better, and to scale to handle more comparisons, at higher speeds, and to handle video?
Epigen is developing a scalable architecture for facial recognition. Requirements are strict. For instance, “We need less than 10ms response time, round trip,” according to Rice.
SingleStore is a great fit for this kind of challenge. The Pipelines feature combines rapid data ingest and data processing. “I like SingleStore because it’s lightweight and runs on bare metal,” Terry says. “I can make the technology disappear.”
Epigen is incorporating GPUs, rather than CPUs, for vectorization of facial images from video and comparison of those images to a database of candidates for matching, such as a law enforcement watchlist. GPUs are an excellent tool for solving many problems that require the rapid execution of complex mathematical algorithms – a class that includes many challenging problems in machine learning and AI, including this one. (Or, as recently described in Forbes: for AI, GPUs are the new CPUs.)
Just as GPUs are designed to be used alongside CPUs, with the GPU serving as an engine for graphical calculations, GPUs can also be used for the mathematical calculations in machine learning and AI – whether those calculations are specifically graphical or not. The problem faced by many solutions that pair CPUs and GPUs together is getting data into the GPUs fast enough. “The GPUs are typically starved for data,” says Terry. So SingleStore is used to partially process data, then rapidly transfer that data to GPUs for further processing.
Epigen’s facial recognition platform takes full advantage of several SingleStore capabilities:
- Ingest – rapidly bringing in photographic data from Amazon S3 and, in the future, from other sources.
- CPU-based processing – SingleStore’s ability to handle the “transform” part of an ETL operation within a pipeline is crucial to rapid processing, as is SingleStore’s Pipeline to Stored Procedures capability when more complex processing is required.
- GPU-based processing – rapid transfer of processed data into the GPUs, which is crucial to performance of the entire system.
Data is initially read into an Amazon S3 database in the cloud. Epigen then uses a SingleStore Pipeline to extract the data from S3 at top speed, then rapidly transform and load it into SingleStore. From there, the data goes to the GPU array.
The advantage of SingleStore is that it’s scalable at every point in the process. Ingest is crucial when one photo is being compared to thousands or millions. But so is initial processing in SingleStore and total throughput, especially the ability to flow data smoothly into the GPUs. “GPUs are often only used at about 10 percent of capacity,” says Terry. “With SingleStore, I’m already up into the 40-50 percent range,” and he’s working to bring the GPUs to full utilization.
In this kind of implementation, Terry has come to expect roughly a 4-5x increase in performance. “I was running Python scripts against S3, and now I don’t have to write or run those scripts and performance is optimized,” says Terry. “I was then reading the data into MongoDB, which worked poorly,” he continues. “MongoDB doesn’t have the ability to roll back the system.” (Because SingleStore is transaction-based, transactions can be rolled back with no lost of data. NoSQL solutions such as MongoDB lack this capability.) “You need to be able to do that with visual recognition.”
“We want throughput and productivity. And people are scared of the price tag that typically comes with that,” offers Terry. With SingleStore, “I can use 100 percent of system capacity and a smaller number of machines,” he said. “Direct costs might drop from around $2 million a year on a different stack to about $150,000.” And, with SingleStore, “I can have fewer people running that infrastructure.”
Technology as Teamwork
Epigen often pairs SingleStore with a visualization tool named Graphistry, which is frequently used in developing and using AI applications. Graphistry helps expose relationships among different kinds of data. Like so many other analytics and visualization tools, the Graphistry API has built-in functionality for working with SQL data, making it a good match for SingleStore.
Graphistry promises “the analytical power of GPUswithout having to think about GPUs."
Developing technology of this sort is a collaborative venture. When Epigen provides a system, the client may put in as much work customizing and optimizing it for their use case as Epigen did to develop the system in the first place. “I set the reference architecture,” explains Terry. “But, in order for the technology to work, the client has to be able to use the tools I provide them to improve the system further. SingleStore is an important part of that set of tools.”
Next Steps for Epigen with SingleStore
Facial recognition is only one of the areas in which SingleStore is helping Epigen offer advanced technology for customers. “Some of the problems we are investigating for future work include replacing a data warehouse implementation so as to improve fraud detection; preventing opioid abuse; geospatial applications; and Oracle database consolidation, combining many disparate database instances into a single data platform on SingleStore. Also, most of what we learn from facial recognition, we can apply to audio and voice detection as well.”
With this initial facial recognition application, Epigen is creating a reference architecture for cognitive computing. This design goes beyond today’s machine learning and AI tools to make decisions and offer support for decision-making by humans. For instance, a narrowly targeted AI tool might be able to show that someone boarding an airplane is not the same person as shown on their passport photo. With cognitive computing, the live photo would then be matched against law enforcement databases, with possible matches identified and each possible match assigned a probability. A tool such as Graphistry would then be used to present a complete picture of the information uncovered to security personnel for them to act on.
Epigen is creatively using the latest technology – SingleStore as the database, GPUs for vector processing, Graphistry in visualization, and machine learning and AI tools – to tackle some of the toughest problems facing business, the military, and government. Its prospects for success are promising.
