Ensuring your threat detection rules work as intended and provide sufficient coverage for major threats is a critical component of a security program. Red Canary’s Atomic Red Team—an open source library of detection tests that help teams validate the effectiveness of their security measures—has historically been the tool of choice for detection testing. But while the methods in Atomic Red Team are tried and true for traditional Windows and Linux hosts, evaluating detection coverage for containers and cloud environments can be more difficult and often requires extensive manual work.
Datadog’s Workload Security Evaluator simplifies the process of running Atomic Red Team detection tests for container environments. It provides an automated workflow that stands up a container-based testing platform where you can run these tests in a consistent, standardized manner. In this post, we’ll cover:
- Key considerations and challenges for validating runtime detection coverage in container environments
- How to use Workload Security Evaluator to stand up a container-based detection testing platform for Atomic Red Team tests
Considerations and challenges for validating runtime detection coverage in container environments
Detection coverage is typically discussed in the context of MITRE ATT&CK, a framework of tactics and techniques commonly used by real-world adversaries. Tactics refer to the goals an adversary is trying to accomplish in an attack, such as persistence (i.e., maintaining a foothold in an environment they’ve already breached) or exfiltration (i.e., stealing data). Techniques are the specific methods an attacker may use to accomplish a goal. MITRE organizes these tactics and techniques into a matrix so defenders can easily visualize detections, example incidents, mitigations, and red team capabilities for each attack type.
Atomic Red Team builds on MITRE ATT&CK, providing simple, focused tests that are mapped to ATT&CK techniques. This enables organizations to approach detection testing in a standardized manner to ensure consistent results.
With any testing framework, however, accurate results require a representative environment—i.e., one that is as similar as possible to your production environment—where you can simulate attacks specific to production workloads. In order to trigger the detections you are testing, the expected files, network endpoints, and other elements of the environment must be present.
This comes with the downside of additional setup—particularly for cloud and container environments. Setting up test containers, installing an agent, and integrating the required tools can make the process of evaluating the effectiveness of your detections quite time-consuming. And while time constraints sometimes make it not possible to deploy a product in a staging account for security testing, these tests at least need to be executed from a container hosted in the cloud.
Use Workload Security Evaluator to stand up a container-based detection testing platform for Atomic Red Team tests
Workload Security Evaluator is designed to speed up the time-consuming process of standing up a container-based environment for detection testing. It allows anyone, regardless of experience, to test the detections in Datadog Cloud Security Management Threats using Atomic Red Team’s atomics. The Evaluator provides a ready-made container environment that is easy to spin up, saving you time and manual work.
Workload Security Evaluator also helps you identify and execute the relevant Atomic Red Team tests—which, at the time of this writing, number 1,139, covering 224 ATT&CK techniques. MITRE ATT&CK’s matrixes are geared toward different platforms like Windows, Linux, and cloud environments, but container-based production workloads intersect a few of these matrixes. We analyzed the techniques and determined which are and are not relevant for container environments, so it’s easier for you to run the corresponding Atomic Red Team tests.
Workload Security Evaluator runs on Docker. To get started, we recommend launching a compute instance in your preferred cloud provider, which will provide the most accurate results. After accessing the instance, install Docker, then clone the Workload Security Evaluator repository.
git clone https://github.com/DataDog/workload-security-evaluator.gitWorkload Security Evaluator is made of two containers. The test environment is a plain Ubuntu image with PowerShell and the Invoke-AtomicRedTeam execution engine installed. All activity will be completed in the test container and monitored by the Datadog Agent, which runs in a separate container. The only input is a Datadog API key, which can be found here.
export DD_API_KEY="<api-key>"
docker compose build
docker compose up -dNow, you can enter the container and freely execute atomics or perform other attack emulation activities for security testing. PowerShell is used to execute the Invoke-AtomicTest module.
docker exec -it atomicredteam /usr/bin/pwshThe -ShowDetails argument displays the name, description, and commands used by the atomic. The atomic T1053.003-2 adds a script to directories such as /etc/cron.daily.
Invoke-AtomicTest T1053.003-2 -ShowDetailsThe -GetPrereqs argument will attempt to install the required packages and download any required files, such as a mock kernel module.
Invoke-AtomicTest T1053.003-2 -GetPrereqsExecuting the atomic ID with no arguments will execute the atomic, which will begin emulating malicious activity.
Invoke-AtomicTest T1053.003-2After the emulated attack is complete, the signal appears in Datadog. Signals from Workload Security Evaluator are tagged with env:emulation to differentiate them from real security threats.
Finally, the -Cleanup argument reverts any changes made by the atomic.
Invoke-AtomicTest T1053.003-2 -CleanupTest detections more efficiently with Workload Security Evaluator
Ensuring your detections are working as intended is critical for your security program’s success, but for container-based infrastructures, setting up a testing environment can be complex and time-intensive. Datadog’s Workload Security Evaluator provides an automated way to stand up a representative container environment, so you can quickly and efficiently start running Atomic Red Team tests for your detections in Datadog Cloud Security Management Threats.
Visit our GitHub page for Workload Security Evaluator to get started. If you’re new to Datadog, sign up for a 14-day free trial.
