As a DevOps engineer or platform leader for an expanding, highly distributed infrastructure, you may be responsible for overseeing thousands of on-premise and cloud resources from multiple providers—governed under dozens of accounts by a complex nest of role-based access control (RBAC) rules. To query all these resources for purposes such as compliance audits and access management, you may be required to write custom scripts and painstakingly sift through data across disparate tools. To efficiently govern all these resources, you need to access ownership, configuration, and security posture data about all of them in one place.
By providing a unified inventory of all your resources and mapping their interrelationships, the Datadog Resource Catalog helps you proactively govern your resources more effectively. With the Resource Catalog Inventory view, you can sort, filter, and drill into all your resources to view key metadata, ownership info, and service dependencies. And with the Resource Catalog’s Governance view, you can define policies that help close attribution gaps and accomplish broad platform engineering projects within your organization. You can request Preview access to the Governance view here.
In this post, we’ll explore how you can use the Resource Catalog in a variety of contexts, including:
- Understanding and fixing tagging to close attribution gaps
- Drive awareness and compliance for infrastructure best practices
- Maintaining your security posture by finding and resolving misconfigurations and vulnerabilities
- Optimizing resource usage to reduce the costs of your most expensive resources
Understand and fix tagging to close attribution gaps
The Resource Catalog’s Inventory tab enables you to surface insights about all the resources in your environment. You can search for resources by name, filter them by any shared attributes (such as region, environment, account, or cloud platform), and group them using out-of-the-box categorization options.
You can group resources not only by specific types—such as Amazon EC2, Amazon S3, or Azure Blob Storage—but also by any user-defined tag as well as resource attributes such as account and region. This way, even if your resources are distributed across multi-cloud or hybrid environments, you can still group them according to their architectural function and understand their service and team attribution at this level. For example, the following screenshot shows resources filtered to the eastus region and grouped by team and reveals that a large number of resources in this region are missing team attribution.
You’ll want to add team and service tags to these resources so that they can provide helpful context for incident investigations and initiatives like upgrade planning and cost optimization. This way, you’ll be able to spot orphaned resources, identify the services impacted by misconfigurations, and know who to contact to help address these issues.
Continuing the example, let’s say you want to investigate these untagged resources. You can click into the “Resources without team” group to get a full list of those resources. The following screenshot shows the resulting list grouped by resource type, revealing that most of the untagged resources are Azure security groups.
When you find that key tags are missing on important resources, you can define a tagging policy in the Resource Catalog Governance view. Implementing a tagging policy can be helpful for large, longer-term tag coverage improvement efforts that make a significant impact on attribution health.
In this example, you might resolve the lack of team attribution in the VM instances by creating a tagging policy mandating that all VM instances must have a team tag. Once the policy is created, you can use the Governance view to see a list of noncompliant resources, the number of resources that break the policy, and a compliance score. This helps you quickly audit teams’ compliance with the policy and identify untagged resources to remedy.
Drive awareness and compliance for infrastructure best practices
The Resource Catalog’s Governance view offers custom policies to help you track compliance of your cloud resources against your organization’s best practices. For example, let’s say your company is experiencing higher peak-time traffic and wants to migrate all x86-based EC2 instances to Arm to reduce costs. To define the custom policy, you would select the “aws_ec2_instance” resource type, the “architecture” attribute on the resource, and the desired “arm64” value.
Each policy has a detailed, dedicated page where you can see the list of noncompliant resources, number of resources that break the policy, and a compliance score. For the safest migration, you’ll want to do this region by region. The following screenshot shows a query that groups these noncompliant resources by region so you can more easily plan each step of the migration.
Discover resource-level misconfigurations and threats to jumpstart security investigations
If you’re a Datadog CSM customer, you can also access the Resource Catalog’s Security tab to understand your attack surface and receive actionable insights about security misconfigurations and active threats. Just like with the Inventory tab, you can filter and group your resources in the Security tab to quickly find active issues.
In addition to the Security tab’s list of resources and their misconfigurations and threats, the Resource Catalog also offers a map that can help you assess the security posture of your resources at a glance. For example, let’s say you want to find misconfigurations in your Google Cloud compute instances. You can filter to those resources and fill the nodes according to the severity of misconfigurations. The following screenshot shows the resulting map, which surfaces a number of high-severity misconfigurations in your buckets across your three biggest regions. You can click on any node in the graph to view that resource’s side panel and investigate the misconfigurations on it further.
In addition to using the map view for high-level visibility into your attack surface, the Resource Catalog provides a list of your resources and their misconfigurations and threats. The following example shows security information for an Appgate service grouped by resource type. You can see that some Google Cloud compute instances in the us-central1 region have external IP addresses that could expose the service to attacks over the internet.
The Security tab lets you not only search for and identify misconfigurations and threats but also kick off remediation. When you click to view the details of this misconfigured resource, you can see a description of the issue and access a runbook that provides steps you can use to remove the public IP addresses using the Google Cloud console or CLI. The side panel also includes a button you can use to pivot directly to your cloud provider console, so you can quickly implement this guidance.
You can also leverage Datadog Workflow Automation to trigger an automated process that resolves the issue, or create a new Jira ticket to loop in more collaborators.
Optimize cloud spend with visibility into resource cost
If you’re a Datadog CCM customer, you can also access the Resource Catalog’s Cost tab, which provides visibility into resource costs, including total cost per resource, change in cost over time, and actionable recommendations to reduce cost.
The Cost tab also supports sorting by total cost per resource and change in cost over time, making it easy to identify the most expensive resources. By additionally leveraging the Resource Catalog’s out-of-the-box resource categorization, you are able to identify the most expensive hosts across all cloud providers, helping compute admins easily get started with cost optimization. You can request Preview access to the Cost tab here.
Govern your cloud from an encyclopedic view
The Datadog Resource Catalog provides a powerful way to proactively govern your infrastructure, drill into your resources to improve attribution, stay ahead of misconfigurations and security risks, and discover cost insights to control your spend. The Resource Catalog is now generally available for all Datadog customers, and you can sign up for the Governance view in Preview here. Any host running the Datadog Agent will automatically appear in your Resource Catalog. For full visibility into your entire infrastructure in the Resource Catalog, you will need to add more resources to Datadog by enabling resource collection for your cloud account, project, or subscription. If you’re brand new to Datadog, sign up for a free trial to get started.
