MITRE ATT&CK® Evaluation results: Malwarebytes’ efficiency, delivered simply, earns high marks
Credit to Author: Malwarebytes Labs| Date: Fri, 01 Apr 2022 03:31:37 +0000
Cybersecurity can be complex work, as security teams need to regularly decipher and prioritize alerts, protect against daily threats, and possibly implement product configuration changes, all while staying abreast of the latest intelligence on new and evolving threats. For organizations that lack fully staffed, internal security teams, or sometimes even one security hire, cybersecurity is then a function of simplicity and time: Any good cybersecurity product must be effective and intuitive out of the box.
With the results of this year’s MITRE ATT&CK® Evaluation, announced today, Malwarebytes’ standout performance proves that our Endpoint Detection and Response (EDR) solution is just that type of product.
In the two categories of Protection and No Delays or Configuration Changes, we achieved a 100 percent success rate, meaning that our business product stopped every tested cyberthreat before it reached a machine, and we did it “out of the box,” meaning that we did not have to change any configuration settings to achieve total protection. For the many businesses that lack a SOC, this simple, intuitive protection is vital to their daily operations, as it allows those businesses to focus on any threats that do manage to get through.
We also detected 83 out of 90 steps that were included in the MITRE ATT&CK Evaluation, and of the corresponding 83 alerts for those steps, 82 were of the highest quality, providing actionable insight that could help stop an attack as it happens. Combined with our 100 percent protection score and our user-friendly simplicity, Malwarebytes can give even small businesses everything they need to get back to doing what they do best—their jobs.
In this post, we will focus less on how Malwarebytes performed in the MITRE ATT&CK evaluation, and more on what the test means and how it can be best interpreted by businesses of all sizes.
Why MITRE ATT&CK matters to you
Choosing a cybersecurity product can be just as difficult as understanding its advertising. Vendors constantly vie for customers, promising new capabilities baked into varying, three-letter acronyms coupled with the “latest,” “greatest,” “best-in-class” features. Adding to this complexity is the fact that, seemingly every day, a new cyberthreat emerges that can derail any business, small or large, and it’s difficult to know if any product under consideration will be effective against evolving attacker techniques.
Fortunately for customers, there are multiple third-party tests that can provide better insight into how cybersecurity products would perform under real-world testing. Interpreting those results, though, and actually applying them to the real world can be challenging. For the many businesses that still see cybersecurity as a solution that can hopefully be implemented directly “out of the box,” how do the various comparison graphs and tables of stats reveal which solution will deliver on that promise?
We’re going to help explain the 2022 MITRE Engenuity ATT&CK Evaluation results with the same approach we take to cybersecurity overall: We want to make it simple and actionable for you.
The MITRE ATT&CK Evaluation third-party test involves the work of cybersecurity researchers testing individual cybersecurity vendors’ products against documented attack methods. This year the testing was modeled after real-world threat actors Wizard Spider and Sandworm. MITRE Engenuity’s researchers record how a product performs across separate activities: Did a product catch every “substep” of an attack? Did it provide a quality alert that provided robust information to the end-user? During Protection evaluation, did it prevent an attack that could have distracted an end-user from the primary attack? And did a vendor go into the product and change any settings to improve results after missing a step?
To understand the results this year, we are going to explain three evaluation categories that provide the best judgment opportunity for businesses choosing a cybersecurity vendor. Those categories are Visibility, Analytic Coverage, and Protection. We will also explain the importance of configuration changes.
Visibility
Cyberattacks do not happen in a vacuum. Instead, when attackers pull off something like a ransomware attack, they are often planning and implementing the attack over a period of weeks or days, leaving behind clues that they have breached a network and started to spread laterally before delivering a final payload. And while spending weeks on an attack may seem long to some, it’s important to remember that these very same attacks used to take months. Threat actors are working faster and more efficiently than ever before, often equipped with many automated technologies, which is why these types of third-party tests are so important: Organizations have to more quickly recognize warning signs and respond to them to avoid falling victim to an attack. As always, it is better to stop an attack at the start of its chain rather than trying to recover after it has struck, especially when an attack results in encrypted data, like the attack used in this year’s MITRE evaluation.
Without good visibility, your cybersecurity team is lost. With no prior warnings sent to your team, every cybersecurity event will be an emergency, noticed likely by an employee with seemingly isolated computer issues. In actuality, that one employee’s problems could be replicated across your business, indicative of an attack that began months ago and which you and your teammates are only seeing the results of today.
Strong cybersecurity products provide visibility into those attacks as they happen, warning users about suspicious activity along the way and helping provide information so that an attack can be stopped before it escalates. In fact, in this year’s MITRE ATT&CK testing, one of the attacks started in the simplest way—a user accidentally opening a malicious file.
Others steps in the attack included access using unsecured credentials, lateral movement through Remote Desktop Protocol, and multiple instances of Ingress Tool Transfer, which could point to threat actors bringing their own files or tools into your now-compromised network. The sequence of attacks recorded in the evaluation show a clear concept of “attack flow,” from an attack’s earliest stages, including the execution of a malicious file, to its later progression through other tactics like credential access, discovery, defense evasion, and lateral movement, until, at the completion of the attack, a victim’s data is encrypted.
The MITRE ATT&CK Evaluation’s 90 steps show a clear intent of attack, and a good cybersecurity product will catch these types of activities and warn your security team about them when they happen. In the testing, the number of steps detected provided the product’s “Visibility” score, because the more steps a security team is warned about, the clearer a picture they have to stop an attack as it happens.
Malwarebytes detected 83 out of 90 steps, or 92 percent of all steps.
Alert quality
As we explained above, visibility is a crucial component for any cybersecurity product, as a collection of warnings can provide some patterns for the human users on the other end to interpret. But warnings themselves are not always enough. Often, security teams need to know more about a specific warning to understand whether it is an issue or not, and how, specifically, they should respond, depending on what a warning tells them. After all, a non-informative warning can be as ineffective as a non-existent one.
This is where alert quality comes into play.
Not every alert is equal. Some provide far more detailed information that can be acted upon by security teams, while other alerts only notify a security team of a problem. In the MITRE ATT&CK evaluation results, alerts are given three tiers of specificity, from least to most specific—General, Tactic, and Technique.
Techniques are the types of alerts that empower security teams to solve problems faster. Going beyond a basic description of what happened, like whether a PowerShell script was executed on a machine, a Technique alert will explain the surrounding context. That can include what threat actors are trying to accomplish with the script—like persisting in an environment even after a system reboot—and how threat actors could achieve that—by changing a registry value to leverage a Winlogon key to execute arbitrary binaries upon logon or logoff.
These quality alerts will help not just small- to medium-sized businesses equipped with equally small IT teams, it will also help the Managed Service Providers who often support these businesses. For small IT teams, quality alerts will help prioritize what problems need to be addressed, while MSPs can also gain better intelligence for the SOCs. In our MITRE results this year, Malwarebytes delivered 82 Technique alerts out of the 83 alerts delivered in total, meaning that nearly every piece of information that we offered to security teams was of the highest quality.
Protection
Nearly every important component of the MITRE ATT&CK Evaluation results fits into the broader umbrella of actionable intelligence. Quality alerts help all teams—from small to large to MDR to MSP—prioritize the cybersecurity issues that will affect them most, while raw telemetry across the landscape of the attack gives more evidence of what is happening and when.
But the value of both components could diminish under an onslaught of cyberevents that bog down any response. What helps in these situations is protection—preventing attacks before they happen. Without protection, even a wealth of high-quality alerts will only stretch your IT team to its limit, unable to meaningfully prioritize, forced to make every alert a priority.
Here, Malwarebytes stands particularly proud, having achieved a 100 percent success rate in the Protection category for Windows. We must note that for this latest MITRE ATT&CK round, we did not submit our solution for Linux testing—a process that, while entirely optional, still impacted vendors’ Visibility scores. Malwarebytes recognizes the importance of cybersecurity for every operating system within any organization, and our EDR solution for Linux will be available on April 5.
A note on configuration changes
Designing a third-party cybersecurity test isn’t easy, as any meaningful evaluation must consider how a product is used by real organizations, from the smallest nonprofit to the largest enterprise. But what’s good for one subset of organizations isn’t necessarily good for the other.
One of the allowances MITRE Engenuity gives their participants is the ability to change configuration settings in a security product once an evaluation has already begun. This reflects the real-world application of larger enterprise companies that have security professionals who prefer a robust interface of settings and configurations that they, personally, can adapt to their own business environment.
There is a flip side to this, though, in that there are countless customers who do not have the time to configure their security product’s settings. There are just as many customers who do not have the internal or external resources for such a project. For these customers, endpoint security is still a product that needs to function as a “set it and forget it” tool. Importantly, these customers may actually lose some value if they try to implement the same types of configuration changes that MITRE Engenuity allows, as these changes will likely produce a greater quantity of alerts, leaving these customers to spend more time deciphering the importance of these alerts and how to respond. This adversely affects the visibility and alert quality components as customers spend time sifting through a potentially significant number of additional, low-quality alerts in order to determine priority actions. A productivity loss no organization—big or small—is willing to accept. Malwarebytes completed the MITRE ATT&CK Evaluation completely without delays or configuration changes, so our results reflect the out-of-box efficacy customers expect.
We consistently want to provide cybersecurity in a simple, accessible, and effective way, and that means developing a product that responds to cyberthreats even without any configuration changes. This isn’t easy, as every business environment is different, but it’s worth the trouble if it means our users are safer than they were without us.
Making use of MITRE
Third-party tests should empower your team to choose the right cybersecurity product for their own business goals. No matter your organization’s size or complexity, though, a few criteria must be met: Your solution must catch the warning signs of an attack in progress, it must provide the highest-quality alerts possible, and it must prevent as many attacks as possible, so as to not overwhelm your security team and your employees.
The MITRE ATT&CK Evaluation thankfully evaluates many cybersecurity tools on these exact metrics. We hope that, with our breakdown on what matters when reading the results, you and your business are able to thrive, away from cyberthreats.
If you want to learn more about how Malwarebytes performed in the MITRE ATT&CK Evaluation, you can contact us here.
The post MITRE ATT&CK® Evaluation results: Malwarebytes’ efficiency, delivered simply, earns high marks appeared first on Malwarebytes Labs.