MITRE ATT&CK Evaluations Of Cyber Security Products

The threat group commonly known as APT3 are primarily interested in stealing documents and other intellectual property from aerospace, defense, construction, telecommunication, and other high tech industries. Their attack pattern can be broken down into three phases as shown below. After making the initial compromise, the attackers instantiate a… Read More

The threat group commonly known as APT3 are primarily interested in stealing documents and other intellectual property from aerospace, defense, construction, telecommunication, and other high tech industries. Their attack pattern can be broken down into three phases as shown below.

After making the initial compromise, the attackers instantiate a command and control setup so that they can orchestrate the rest of their operation. The next phase is to discover all aspects of their target users, environments, and systems. They hunt for users who have elevated privileges, get access to their credentials, look for network shares they can then access, and move laterally to systems where they might find documents and other files of interest. This phase can extend for a considerable amount of time, especially if they remain undetected. The final phase is collecting the documents and then compressing, encrypting, and temporarily storing them on a staging server from which they are exfiltrated.

The Mitre team analyzed threat intel reports on the APT3 group and came up with a series of detailed steps that closely emulate the adversary. They defined a test range in Azure consisting of one Windows domain controller, a file server, and three test machines. Each of the security vendors got to install their software on these systems, and their detection capabilities were then evaluated at each step of the process. You can find a step-by-step account for each of these at the Mitre ATT&CK Eval site

In all, there were 56 different Mitre ATT&CK Techniques that were used by the APT3 emulation. The figure on the left lists the techniques that were used five or more times during the course of the attack. The most heavily used TTPs were T1059 and T1086, which are Command-Line Interface and Windows PowerShell. This is not surprising, as most attackers these days prefer “Living off the Land” (LotL) techniques, wherein trusted system tools are repurposed to discover resources and launch attacks. In addition to being present in every system, they have the added benefit of being hard to discover when they are used nefariously.

Some other notable, high-roller techniques are T1007, T1012, T1016, T1057, and T1069, which are System Service Discovery, Query Registry, Network, Process, and Permission Groups Discovery. Again this highlights a common attacker modus operandi: learn as much about the system resources, the environment, and user profiles as you can before launching the next phase of the attack. T1087 is Account Discovery, which is typically used to learn more details about the users on the system so that attackers can then elevate privileges to run as a particular user who has access to the resources they desire. T1083 is used to enumerate files and directories to search for and discover materials of interest. The last one of the oft-used techniques is T1106, where the adversary uses a system API to launch malicious scripts or binaries.

While most of the Security controls in the Mitre evaluation detected all of the above techniques, it is important to highlight that the Mitre environment was not subject to the normal high volume of background noise that is prevalent in most enterprises. It may be unrealistic to expect your security analyst to be watching for many of these detections, and an adversary can easily go unnoticed if their activities are a very small part of the many different events and alerts that your SoC is being inundated with. Thus it is important to run an emulated attack in the real environment during normal business operations to validate that your security controls are working as desired.

The Mitre ATT&CKCK evaluations also uncovered several techniques that were not detected by most of the vendor offerings. T1010, T1057, and T1063 refer to the discovery of Application Windows, Processes, and Security Software deployed on the system. T1083 discovers and searches through files, and T1081 looks for

The credentials that may be embedded in them. Security software is constantly being updated to detect more and more of these techniques, and I am sure it is only a matter of time before these potential gaps are addressed as well.

However, the real question is not whether the techniques are discoverable through a security vendor’s offering, but rather evaluating if the security analyst in the customer’s environment is able to detect these activities in a timely manner. The sooner they are able to detect a threat, the less time the attacker has to discover resources and inflict damage. The only way to find out in the real world is to institute a system where you have a continuous validation of your security posture to ferret out the chinks in the armor that are likely to be exploited by an attacker. The AttackIQ platform provides the capability to define and run this program in a production environment.