Import Backstage YAML files into Datadog to manage all your services in one place

The Datadog Service Catalog centralizes your organization’s knowledge about the ownership, reliability, performance, costs, and security of your services. If you’re also using Backstage to keep track of your services, you can leverage our support for Backstage YAML to easily consolidate and maintain all your service information in the Service Catalog.

By automatically importing your Backstage YAML, Datadog lets you manage all your services in a single platform, so you can view service data side-by-side with observability data from your applications and infrastructure and eliminate knowledge gaps. Datadog also keeps your Service Catalog complete and up to date, even as you create and modify services in Backstage.

In this post, we’ll show you how to:

• Populate the Service Catalog with your Backstage YAML
• Import multiple service definitions to easily create or modify any number of services
• Validate service metadata by viewing schema version information

Populate the Service Catalog with your Backstage YAML

The Service Catalog gives teams throughout your organization a central repository for managing services. When you enable the GitHub integration—as shown in the screenshot below—Datadog will automatically create and maintain Service Catalog entries for your Backstage services.

The integration constantly scans your repositories for Backstage YAML files named service.datadog.yaml and catalog-info.yaml—which you create when you add your service to the Backstage Software Catalog. The code snippet below shows an example of catalog-info.yaml.

apiVersion: backstage.io/v1alpha1
kind: component
metadata:
  annotations:
    backstage.io/techdocs-ref: github://design/doc
    some.annotation: value
  namespace: distribution
  name: delivery-tracker
  title: Delivery App
  description: delivery app component responsible for transit processes
  tags: ["framework:spring", "db:mysql"]
  links:
    - title: Wiki
      url: https://backstage.core.shopist.org/docs/default/component/delivery-cancel
      icon: help
    - title: runbook
      url: https://backstage.core.shopist.org/docs/default/component/rollback
      icon: help
    - title: source
      url: https://gitlab.shopist.org/shopist/delivery-app
      icon: gitlab
    - title: jira
      url: https://shopist.atlassian.net/browse/DA
      icon: help
    - title: diagram
      url: https://shopist.atlassian.net/wiki/spaces/DA
      icon: help
  ignore-attribute:
    id: 1
    value: "ignore_value"
spec:
  type: service
  lifecycle: production
  owner: delivery-app
  system: logistics

Datadog automatically imports these new or revised Backstage files to keep the Service Catalog up to date. Along with the ability to automatically create and maintain Service Catalog entries based on your Datadog-formatted service definition files, this enables you to troubleshoot, investigate, and collaborate across all your services in a single platform.

As an alternative to using the GitHub integration, you can post Backstage YAML files to the Datadog API. This allows you to programmatically send Backstage service definitions that may not exist in your GitHub repositories.

Import multiple Backstage service definitions at once

To quickly get started managing all your Backstage services in the Service Catalog, you can register any number of new services by adding their definitions to a single YAML file and then sending that file to Datadog via the API or adding it to a repository to be imported by our GitHub integration. The code snippet below shows an example of how you could define multiple services in a single YAML file. Each service definition begins with three dashes to delineate it from the other services.

---
apiVersion: backstage.io/v1alpha1
kind: component
metadata:
  annotations:
    backstage.io/techdocs-ref: github://architecture/doc
  namespace: shopping
  name: delivery-transit-prod
  title: Delivery App Transit
  description: delivery-app component responsible for transit 
spec:
  type: service
  lifecycle: production
  owner: delivery-app
  system: logistics

---
apiVersion: backstage.io/v1alpha1
kind: component
metadata:
  annotations:
    backstage.io/techdocs-ref: github://architecture/doc
  namespace: shopping
  name: delivery-aggregate-prod
  title: Delivery App Aggregation
  description: delivery-app component responsible for aggregation spec:
  type: service
  lifecycle: production
  owner: delivery-app
  system: logistics

After you’ve imported your Backstage YAML, all your service data will be centralized in Datadog, alongside observability and security data from your applications and infrastructure.

Ensure valid metadata by viewing schema and version information

Datadog creates and maintains Service Catalog entries from the Backstage YAML you provide. That YAML is based on the Backstage schema, which includes the metadata required by the Backstage Software Catalog. But that schema leaves out some metadata that the Service Catalog can use to help your team manage the complexity of your microservices architecture.

Service Catalog makes it easy to see which service definitions are based on the Backstage schema so that you can easily identify services that might be missing key metadata. In the screenshot below, the Schema Version column is highlighted to show the schema used by each service definition. Datadog has imported Backstage YAML for services such as delivery-aggregate and delivery-cancel-prod to automatically create Service Catalog entries for them. You can see that these services are based on the Backstage schema, which indicates an opportunity to enrich them with additional metadata that the Service Catalog can use, such as a service team’s on-call schedule and contact information.

Once a Backstage service definition has been imported, you can update it to ensure that the service metadata is complete and that it takes full advantage of the Service Catalog’s capabilities to help you organize and communicate information about your services. The screenshot below shows the built-in editor where you can add or modify service metadata. We’ve added a URL to the PagerDuty integration field so that the Service Catalog will show on-call information for this service.

Once you’ve modified metadata that’s been imported from Backstage, it’s automatically converted to use the Datadog schema. In the screenshot below, the Service Catalog shows that the service now uses version 2.1 of the Datadog schema. The on-call column now shows contact information for the team member currently on call.

Manage all your services with the Service Catalog

With support for Backstage YAML, Datadog automatically creates and maintains Service Catalog entries for your Backstage-based services so that you have a single platform that provides visibility into all your services. See our documentation to start importing Backstage service definitions into the Service Catalog.

You can also populate the Service Catalog using APM, USM, service telemetry, and via the Datadog UI. If you’re not already using Datadog, you can start today with a 14-day free trial.