In the wake of the Equifax breach and the global WannaCry ransomware outbreak earlier this year, tensions around cybersecurity are running high.
According to a recent study conducted by the Ponemon Institute and sponsored by Barkly, seven out of 10 organizations are reporting their security risk has significantly increased during the past 12 months. Only 54 percent believe the attacks they’re facing can realistically be stopped, but they are feeling pressure to do more to keep their data safe. While many harbor significant doubts that any solution can help them keep up with the rapidly evolving nature of today’s threats, all are hoping for a breakthrough.
The combination of “breakthrough” and “artificial intelligence,” or “machine learning,” is showing up more and more frequently in research papers and the press. Advanced systems are even learning how to train themselves. Given the remarkable volume and dynamism in security data, it seems obvious that security vendors are now integrating these advances in new approaches to tackle the seemingly intractable problem of blocking attacks.
Automated machine learning “factories” can process extremely large datasets on a regular basis, making them the right infrastructure to analyze the huge volume of malware produced every day. These outputs can be used to train models to identify similarities across malware samples, balancing that understanding with equally frequent and comprehensive training on evolving good software. Taken together, protection software can learn to distinguish malicious software from good software. Automating the training process can even help models keep pace as malware continuously and rapidly evolves—something traditional security solutions have been overwhelmed with for some time.
As with any new technology, there are implementation challenges as well as limits to the level of accuracy these models are currently able to achieve.
—Challenge #1: Models need to be trained on the right data. To accurately differentiate between malware and “goodware,” a model’s datasets need to consist of a diverse range of both. Otherwise, imbalances in sample types can produce biases. For example, when models are over-trained on malicious software, it causes them to be prone to false positives, classifying legitimate programs as malware and requiring additional effort to create and maintain substantial whitelists or exceptions. This is especially true when organizations deploy custom-built applications that are unlikely to be represented in a generic training set of well-known software.
—Challenge #2: Models need to be trained and refreshed regularly. One of the major benefits of applying machine learning to security is the predictive value of the resulting models. In security, though, new techniques and new malware appear constantly, produced by attackers at a rapid, unrelenting pace. As a result, as time passes, machine learning models deteriorate. Unlike other uses cases machine learning has been applied to—like natural language processing (NLP), machine-aided diagnostics in healthcare, or beating players in Go—the relevance of security sample data decreases over time. Malware from the past—Code Red, Nimda, or the Morris Worm, for example—have little relevance to today’s Ursnif, Locky, or WannaCry attacks. As a result, the training of security models needs to be sensitized to the newest and most prevalent attack techniques, while older, less relevant (and sometimes obsolete) attacks must be deprioritized. When this is done, machine learning models can resist deterioration, so long as they are also consistently updated with components of the end-user protection platform.
It’s worth noting the adoption of machine learning in cybersecurity is still in its early days. Some security vendor implementations of machine learning lack refinement and currently serve as coarse-grained filters that operate with a clear over-sensitivity to malware over goodware.
They note that this creates additional workload for IT administrators in the form of whitelist and blacklist management between software updates. In part, the issue stems from vendors creating models that are optimized to provide wider coverage (blocking more malware) while sacrificing accuracy (making sure malware is the only thing getting blocked). As a result, false positives are becoming the accepted downside of stronger protection, even though they are understood to be a significant barrier to widespread adoption of machine learning-enabled security products.
As more security vendors try to harness machine learning, however, advances may soon make the tradeoff of coverage over accuracy unnecessary.
It begins with the realization that organizations are more unique than we usually think in terms of their software environments. The right approach to sample collection and training needs to recognize that while malware and attacks are constantly evolving, it is equally true that patches, updates, and new applications are constantly changing the content of goodware. The resulting models are responsive, both to the changing threats and to the distinctive collection of software in use at specific organizations. This is what we refer to as “responsive machine learning.”
Using this approach, vendors develop tailored models for each organization, providing them with responsive protection that is more accurate and less disruptive because it is up to date and has been developed specifically for their systems.
The adoption of machine learning in the security industry is still in its infancy, and this more responsive approach to developing protection models represents just one exciting step forward. As research and development continues, we can anticipate more breakthroughs that may finally tip the scales against attackers.