Compromised secrets, such as leaked API and SSH keys, credentials, and session tokens, are the leading cause of cloud security incidents. While attackers can directly compromise secrets through methods like phishing, they can also gain control by finding and taking advantage of simple misconfigurations in your environment. The most common cause of data breaches in the cloud, for example, are long-lived credentials, such as AWS IAM user access keys and keys for GCP service accounts and Entra ID applications.
To minimize these risks, it’s recommended to implement processes that prevent you from storing secrets in the first place. Steps like creating auto-expiration policies and using centralized identity management tools can help you efficiently manage authentication and authorization workflows in your environment, without the reliance on secrets for individual accounts and applications.
Even with these measures, you still need adequate visibility into existing secrets so you can replace them with more secure options and limit costly issues. In this post, we’ll look at a few examples of how you can improve secrets management by efficiently tracking the lifecycle of your existing credentials, keys, and more. These steps include monitoring when secrets need to be revoked or have expired (or should expire), and detecting anomalies and their usage.
Get visibility into which secrets need to be revoked
Cloud environments—and their authentication and authorization requirements—are in a constant state of flux. Organizations routinely create cloud accounts for new employees, workloads spin up and tear down on demand to process traffic efficiently, and additional virtual machines and storage are created to support a growing user base. Each new user, workload, and service will require some level of access to associated resources to complete their tasks.
With this continuous change, it’s important to find and remove any compromised or unnecessary secrets in order to restrict access to sensitive resources and data. Searching for individual secrets is often a fruitless pursuit, but there are a few areas you can start with.
First, misconfigurations in your code, services, and resources can be easily overlooked depending on the complexity of your environment and how often you deploy new features. For example, though it’s considered a well-known anti-pattern, hardcoding or passing secrets as environment variables can still be a common occurrence and requires immediate action if found in your environment.
The ability to surface hardcoded secrets in your environment, such as the following example of a hardcoded password, enables you to resolve the vulnerability without needing to manually sift through your environment.
Other areas to monitor include your IAM and account configurations, which is a critical part of managing your overall security posture. Visualizations like the following security report can help you quickly surface which keys are problematic and where they are configured.
For example, it can help you identify when root accounts have access keys, which are examples of long-lived credentials. A root account in AWS has complete access to the environment, which would enable an attacker to move to any resource if they gained control of the account via a compromised access key. Monitoring these areas will help you find and remove stale, vulnerable secrets in your environment and limit the ways an attacker takes advantage of them.
Visibility into cloud misconfiguration is also critical for revoking secrets that were used as part of an attack, such as the following signal for a compromised AWS IAM access key.
Some other examples of activity related to misused secrets include:
- Amazon SNS enumeration in multiple regions using a long-term access key
- Anomalous number of secrets retrieved from AWS Secrets Manager
- Unfamiliar IAM user retrieved secret from AWS Secrets Manager
- User enumerated AWS Secrets Manager
Know when secrets have expired (or should expire)
As previously mentioned, adding auto-expiration policies to your secrets can help minimize the risk of exposure and is a recommended best practice. An important step in this workflow is understanding which secrets have these policies configured and when they are set to expire. For example, passwords for IAM users should not be active for more than 90 days. Any other credentials for AWS IAM users should also be deactivated or removed if not used for 45 days, as seen in the following signal.
These rules also apply to platforms like Azure Key Vault, where keys are required to have an expiration date. Measures like these can help you find and regularly remove existing secrets as they expire, so you can phase them out via recommended steps like replacing dedicated IAM users with an identity provider platform.
Track anomalies in secrets or secrets policy activity
As you continue to find and phase out compromised or unnecessary secrets, it can still be difficult—and often fruitless—to individually track changes to secrets or their policies in dynamic cloud environments. As an alternative, it can be helpful to be aware of potentially vulnerable areas or areas of interest for an attacker. For example, understanding which AWS API calls in your environment return secrets can help you create the appropriate access policies for AWS resources and monitor them for any potentially unnecessary changes.
Focusing on anomalies in these day-to-day workflows can also help minimize the risk of wasting time investigating false positives. For example, changing an IAM policy on its own may not be a sign of an attack—but if the associated user doesn’t typically update policies, or begins changing other configurations, then this activity could be malicious. Having visibility into the chain of events after a policy change, like what’s captured in the following signal, can help you determine whether or not the behavior is expected or is part of an attack path.
Some other activities worth monitoring for anomalous behavior include:
- AWS access key created by a previously unseen identity
- AWS KMS key deleted or scheduled for deletion
- Google Cloud Service Account key created
- Google Compute Engine instance metadata SSH key added or modified
- Attempt to add SSH key to Google Compute Engine project metadata by a previously unseen user
Phase out vulnerable secrets with Datadog
Stale secrets—such as, cloud credentials, access and SSH keys, and session tokens—can give attackers unrestricted access to your environment and its data if managed improperly. Best practices like using temporary credentials minimizes this risk, but it’s also important to keep track of existing secrets in case issues present themselves.
Datadog provides you with this critical visibility, so you can gradually phase out vulnerable secrets for secure alternatives. For example, Datadog Cloud SIEM includes built-in detection rules that allow you to monitor patterns in your cloud activity and audit logs to get a better understanding of the why behind that activity. Datadog Cloud Security Management will automatically notify you of misconfigurations or secrets that do not have the appropriate expiration policies. And Datadog Sensitive Data Scanner automatically monitors your environment for secrets and credentials, so you can surface those that were accidentally hardcoded in application code, logs, or other telemetry.
Check out our documentation to learn more about Datadog’s security capabilities. If you don’t already have a Datadog account, you can sign up for a free 14-day trial.
