What Are MITRE Evaluations?

August 14, 2023

Last Updated: November 22, 2023

Share on:

What Is MITRE Engenuity?

MITRE Engenuity is a not-for-profit foundation designed to create and apply innovative solutions to critical infrastructure problems. MITRE Engenuity works hand in hand with the government, academia, and private companies to develop effective cybersecurity measures. Its primary focus is to enhance security across all sectors, with a particular emphasis on cybersecurity.

MITRE is perhaps best known for its ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) framework. This globally-accessible knowledge base is used by defenders to understand and classify an adversary’s behavior and actions. ATT&CK provides a common language and methodology to describe these behaviors, facilitating communication, collaboration, and coordination among cybersecurity professionals.

This is part of our series of articles about endpoint protection.

What Are MITRE Engenuity ATT&CK Evaluations?

MITRE Engenuity ATT&CK Evaluations are a series of assessments that test the effectiveness of cybersecurity products against real-world threats. These evaluations are designed to give businesses and organizations a clear and unbiased understanding of how well cybersecurity products can protect their systems and data.

Vendor Neutrality

The evaluations are not influenced by the interests of any particular vendor. Instead, they provide an unbiased assessment of the security product’s capabilities. This neutrality ensures that the evaluations are fair, accurate, and reliable, allowing organizations to make informed decisions about their cybersecurity measures.

Real-World Testing

MITRE Engenuity ATT&CK Evaluations are based on real-world testing. They simulate actual cyber attacks to assess how well a product can defend against them. This approach ensures that the evaluations reflect the reality of the threat landscape. It also allows for the identification of any potential weaknesses or vulnerabilities in the product, which can then be addressed to improve its effectiveness.

Detailed Insights

Through the evaluations, organizations receive comprehensive insights into the performance of cybersecurity products. These insights include detailed information about how the product responded to the simulated attack, the tactics and techniques used by the attackers, and how well the product was able to detect and mitigate these threats. They can help organizations to strengthen their cybersecurity infrastructure and prepare for future threats.

Comparison Between Security Solutions

The evaluations provide a standardized framework for assessing the effectiveness of different products, allowing organizations to compare them on a like-for-like basis. This comparison can assist organizations in choosing the most suitable security solution for their specific needs.

The MITRE Engenuity Evaluation Process

The evaluation process developed by MITRE Engenuity serves as a benchmark for identifying the most effective security products available in the market. It is a rigorous and in-depth procedure that assesses the performance of various security solutions against real-world cyber threats. Let’s break down this process into its five main stages:

1. Selection of Security Products

The first step in the MITRE Engenuity evaluation process begins with the selection of security products. MITRE Engenuity invites vendors from across the globe to participate in their evaluations. The selection process is open and transparent, ensuring that a wide variety of security products, ranging from established names to emerging players, are considered.

The selection isn’t based on popularity or market share but rather on the potential effectiveness of the product against cyber threats. To this end, MITRE Engenuity maintains a rich and diverse pool of security products to respond to the ever-evolving landscape of cyber threats.

2. Design of Test Scenarios

After selecting security products, the next step is to design test scenarios. These scenarios are meticulously crafted to simulate real-world cyber-attacks. This ensures that the evaluation isn’t merely a theoretical exercise but is instead grounded in practical, real-life situations that organizations might face.

These test scenarios are based on the tactics, techniques, and procedures (TTPs) used by cybercriminals. They are designed to mirror the strategies adopted by threat actors in the wild. This allows the evaluation process to gauge the robustness of various security products.

3. Execution of Attacks and Monitoring

In this phase, the selected security products are subjected to the test scenarios. Each product is exposed to a range of cyber threats, and its response is carefully monitored.

This phase is critical for understanding how security products respond to different types of attacks. The performance of these products under various attack scenarios provides useful insights into their effectiveness. It helps assess the product’s detection capability, response speed, and overall resilience against cyber threats.

4. Data Collection and Analysis

During the attack execution and monitoring stage, a vast amount of data is generated. This data is carefully collected and then subjected to in-depth analysis. This analysis includes assessing how accurately the security products identified the threats, how quickly they responded, and how effectively they mitigated the impact of the attack.

This stage also takes into account any false positives generated by the security products. The data analysis phase forms the basis for the final scoring and ranking of the security products.

5. Scoring and Ranking Methodology

Based on the data collected and analyzed in the previous stage, each security product is scored. This scoring takes into account factors such as the product’s detection abilities, response time, mitigation strategies, and the number of false positives.

Once the scoring is completed, the security products are ranked. This ranking helps organizations identify the security products that are most effective against the prevalent cyber threats. The results of the MITRE Engenuity evaluations are made publicly available, contributing to the collective knowledge of the cybersecurity industry.

Learn more about Cynet in the MITRE 2023 Evaluations

MITRE Engenuity Enterprise Evaluation 2023: Turla

In 2023, MITRE will perform its Enterprise Evaluation of cybersecurity vendors based on tactics, techniques, and procedures (TTPs) used by Turla, a Russian threat group that has carried out successful attacks in 45 countries. Below is a partial view of the Turla TTPs that will be used in the evaluation.

Below are the participants in the 2023 evaluation. Our very own Cynet 360 platform will be participating for the third year in a row.

Cynet 360 Performs Strongly in 2022 MITRE Evaluation Results

Cynet emerged as a top performer in the 2022 MITRE ATT&CK Evaluation, achieving impressive results that placed it ahead of many other vendors in multiple crucial sectors.

Key achievements:

Full Visibility and Detection: Cynet demonstrated a comprehensive detection capability, achieving 100% visibility and detection across each of the 19 MITRE ATT&CK steps evaluated.
Exceptional Prevention Rate: The cybersecurity solution was adept at halting threats in their tracks, boasting a 100% prevention rate across the 9 tests MITRE conducted.
High Rankings in Speed and Prevention: Cynet’s effectiveness in halting attacks was evident in its rankings. The product stood as the third-best vendor in terms of the number of prevented attacks and speed of prevention.
Outstanding Detection Coverage: When it came to detecting threats, Cynet secured its position as the third-best vendor by achieving a detection coverage of 98.2% across the 109 substeps that were part of the MITRE ATT&CK® Evaluation.

Given the diverse threat landscape, cybersecurity solutions need to be agile, robust, and comprehensive. Cynet’s performance in the 2022 MITRE ATT&CK Evaluation is an affirmation of its capabilities and its commitment to providing advanced detection solutions for businesses and organizations.