How to improve the security of your Infrastructure-as-Code deployments

Infrastructure-as-Code (IaC) usage has grown significantly, but adopting IaC doesn't automatically mean that your infrastructure is secure. For AWS environments, Datadog's recent DevSecOps report shows that 80% of organizations use at least one IaC tool, but at least 38% still manually provision infrastructure in production. These manual deployments—known as ClickOps—can create inconsistencies in how IaC is implemented across your environment and introduce real security risks, such as misconfigured defaults, hardcoded secrets, and configuration drift.

This post looks at the following goals for identifying and closing the most common security gaps in IaC:

• Inventory your infrastructure with tags
• Define IaC and Policy-as-Code (PaC) rules
• Shift your infrastructure policies left
• Enable continuous monitoring on your infrastructure

Regardless of the maturity of your IaC setup, these goals can help strengthen your security posture and reduce risks from misconfigurations and drift.

Goals for securing IaC deployments

When we think about how to implement security-focused IaC, it's important to look beyond adopting specific tools. Many of the same risks that exist with manually provisioned resources can still occur in IaC. For example, deploying a new cloud function with a default service account could introduce excessive permissions in production—regardless of whether it was deployed manually via ClickOps or automatically using IaC. In complex environments, such as those that use a multi-repository approach to manage Terraform modules across teams or business domains, these risks can quickly multiply.

Instead of listing specific recommendations, we'll take a step back to look at goals for addressing the common challenges with securing IaC.

Goal 1: Inventory your infrastructure with tags

Without a clear view of what infrastructure exists in your environment and how it's configured, security policies can't be enforced. Tags—a foundational part of classifying infrastructure—are often missing or inconsistent, especially in environments where ClickOps is common. This lack of standardization makes it difficult to track critical resources, maintain compliance, or detect misconfigurations.

Developing an established tagging system can significantly improve visibility into security and provisioning gaps within your infrastructure. Consider a scenario in which an organization in a highly regulated industry, such as financial services, needs to comply with PCI-DSS requirements. The organization audits its existing infrastructure for critical systems that need to be compliant with PCI-DSS regulations, such as payment services and associated databases. In order to easily identify these systems and configure them appropriately, the organization decides to create a structured tagging system for them.

The following state file includes example security tags for an Amazon S3 bucket, based on AWS's tagging recommendations for data security:

1
terraform {...}
2

3
provider "aws" {
4
  default_tags {
5
    tags = {
6
      Environment = "production"
7
    }
8
  }
9
}
10

11
resource "aws_s3_bucket" "payment" {
12
  bucket = "payment-bucket"
13
  tags = {
14
    Team           = "payment-compliance"
15
    Service        = "cc-payments"
16
    Compliance     = "PCI-DSS"
17
    Purpose        = "sensitive-data-storage"
18
    Classification = "restricted"
19
  }
20
}

This snippet applies a default environment tag for all resources and categorizes the payment-bucket S3 bucket as a resource that needs to maintain compliance with PCI-DSS standards.

You can also create tagging policies, such as those supported by Datadog Resource Catalog. Standardizing and enforcing tags early creates a foundation for infrastructure governance, providing teams with the necessary context to address the common security gaps introduced by ClickOps, such as untrackable resources, overly permissive roles, or publicly exposed resources.

Goal 2: Define IaC and PaC rules

Even when teams can see what's deployed, they may lack the guidance needed to improve infrastructure security. Without uniform rules or consistent enforcement for deployments, teams can inadvertently introduce security risks in production environments. Datadog's recent AWS security report highlights this risk with a finding that 36% of organizations using Amazon S3 have at least one publicly exposed S3 bucket.

PaC frameworks, such as Open Policy Agent (OPA) and Terraform Sentinel, strengthen IaC by systematically defining and enforcing the security and compliance guidelines for provisioned infrastructure. These policies ensure that your environment is supported by immutable infrastructure, which you can trust will not change until a new policy is required.

For example, you can create an OPA policy that flags any Amazon EC2 instances that do not use IMDSv2. While newly created instances use IMDSv2 by default, it's important to ensure that legacy instances—and other manually provisioned resources—follow the same security best practices. This step is especially necessary in environments where infrastructure may not be easily replaced.

Beyond policies for individual resources, there are a few recommendations that will strengthen IaC security as you scale:

• Track all IaC changes via a version control system for your implementation of Terraform, such as organizing code in either centralized, team-based, or decoupled repositories
• Store Terraform state files remotely, such as on Amazon S3 or the Google Cloud Storage state backend
• Reduce permissions sprawl by deploying least privilege IAM policies via Terraform's IAM Policy resource
• Store and limit access to secrets with platforms such as Vault and AWS Secrets Manager

After you have defined policies in place for both Terraform and your infrastructure, the next step is to ensure that they are efficiently applied across your environment.

Goal 3: Shift your infrastructure policies left

Security checks often detect issues after a resource has already been deployed to production. Even a short-lived misconfiguration can cascade into significant drift or introduce a security risk, like exposing sensitive data. Embedding policies into your CI/CD pipelines shifts critical security checks to the left and ensures that they are enforced early and consistently. This step significantly minimizes the risk of deploying misconfigured or vulnerable resources in production.

A common stage for embedding IaC and PaC policies is during a pull request, where teams can scan new Terraform-defined infrastructure for any configurations that do not follow best practices. Consider an example where a team accidentally provisions an EC2 instance with IMDSv1 enabled, a legacy configuration that can expose metadata via an SSRF attack. Datadog IaC Security flags the instance during a PR review, as seen in the following screenshot:

Close dialog

With this information, the team can enable the recommended IMDSv2 configuration before deploying the EC2 instance to production.

Goal 4: Enable continuous monitoring on your infrastructure

Even with defined policies, impromptu changes can still slip through. During incidents, teams may apply security patches or hotfixes in production—or forget to revert a temporary configuration change. This drift between policies and deployed infrastructure can quickly create a variety of issues. For example, Terraform might revert a ClickOps change, such as manually increasing database capacity to address storage exhaustion, creating a recurring incident.

By enabling continuous monitoring, you can identify when policies fail or infrastructure changes unexpectedly. You can apply this step per resource and for your Terraform installation, giving you end-to-end visibility into your infrastructure.

Per resource

Configuring policies that enforce logging for provisioned infrastructure, such as access logs for S3 buckets, will help you detect any malicious activity or unauthorized changes. The following snippet enables logging on our example payment-bucket S3 bucket:

1
terraform {...}
2

3
provider "aws" {...}
4

5
resource "aws_s3_bucket" "payment" {
6
  bucket = "payment-bucket"
7
  acl = "private"
8
  tags = {...}
9

10
  target_bucket = aws_s3_bucket.log_bucket.id
11
  target_prefix = "log/"
12

13
}
14

15
resource "aws_s3_bucket" "log_bucket" {
16
  bucket = "payment-bucket-logs"
17
  acl    = "log-delivery-write"
18
}

With this simple example, the payment-bucket S3 bucket will log server access logs to the payment-bucket-logs S3 bucket. You can then forward the logs for monitoring.

Terraform installation

Continuous monitoring also requires visibility into your Terraform installation. You can enable audit logging for the HCP platform itself and monitor activity with Datadog Cloud SIEM's Terraform Content Pack. This step enables you to review all audit events, such as when certain policies fail or are canceled, in a built-in Terraform dashboard.

Close dialog

To take it a step further, you can build custom detection rules on Terraform audit logs to surface key security events, including:

• Run task changes, such as newly created or destroyed tasks
• Policy checks, such as when certain policy sets—defined by PaC frameworks like Terraform Sentinel or Open Policy Agent (OPA)—consistently fail or are canceled
• Team and workspace changes, such as newly added teams or updated team permissions

With these insights, you have the ability to instantly detect security risks caused by ClickOps and continually refine your IaC policies.

Prioritize security-focused IaC policies with Datadog

Integrating security into IaC is difficult to do well. It requires visibility into your entire infrastructure, understanding where security gaps exist, and consistently enforcing rules in a way that doesn't disrupt your teams' work. Without appropriate guardrails, even mature teams can find themselves patching resources in production. The goals discussed in this post provide a starting point for tackling the security challenges associated with deploying infrastructure and limiting security risks introduced by ClickOps.

Datadog helps teams act on these goals by providing visibility into IaC configurations, surfacing misconfigurations before they are deployed, and detecting configuration drift. Datadog IaC Security is now available in preview—sign up to get started. You can also check out more of Datadog's security offerings. If you don't already have a Datadog account, you can sign up for a free 14-day trial.