Testing early and often throughout the software development process (shift-left testing) helps teams stay agile and reduce the time it takes to validate and release new updates. Datadog Synthetic CI/CD Testing enables you to implement shift-left testing throughout your CI/CD pipeline so that your team can prevent faulty code deployments from degrading your end-user experience. If you are using GitHub Actions to automatically build and test changes to your codebase, you can now add Synthetic tests to your workflows with Datadog’s new GitHub Action.
By using GitHub Actions in tandem with Datadog Synthetic Monitoring, you can quickly troubleshoot failed tests within your workflows. And to get visibility into the health and performance of the pipelines that are running these tests, you can monitor GitHub Actions with Datadog CI Visibility.
In this post, we’ll show you how to set up the new GitHub Action and monitor your GitHub Actions workflows in Datadog.
Set up the GitHub Action to run Synthetic tests
To start incorporating Synthetic tests into your GitHub workflows, simply add your Datadog API and application keys as secrets in your GitHub repository. You can then start configuring your GitHub Actions workflows to execute Synthetic tests. If you’re using GitHub Actions but haven’t started using Synthetic Monitoring, now is a great time to create your first Synthetic test.
Datadog’s GitHub Action will be displayed as an action to install within a new or existing workflow. A workflow is a configurable automated process (e.g., a build or test), defined by a YAML file in your repository, which will run when triggered by an event of your choosing (e.g., whenever a pull request is created). Configure your workflow to use DataDog/synthetics-ci-github-action@v0.2.0, as shown in the example below.
name: Run Synthetic tests using public IDs of tests
jobs:
e2e_testing:
runs-on: ubuntu-latest
steps:
- name: Run Datadog Synthetics tests
uses: DataDog/synthetics-ci-github-action@v0.2.1
with:
api_key: ${{secrets.DD_API_KEY}}
app_key: ${{secrets.DD_APP_KEY}}
public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'
Say you want to make sure that new code deployments do not degrade the end-user experience on your e-commerce site. To proactively monitor the site, you implement a suite of Synthetic tests to verify the functionality of the checkout process, as configured in the workflow above. This GitHub Actions workflow will trigger the Synthetic tests associated with the specified public_ids.
Alternatively, you could use a search query (e.g., tag:e2e-tests) to configure your GitHub Actions workflow to run tests with a certain tag. If you are using different build environments (e.g., development, staging, and production) you can easily leverage the same unique test suite across them all. For instance, let’s say that you’ve configured a GitHub Actions workflow to trigger a suite of end-to-end tests whenever a pull request is created. The tests pass in your development environment and you’d like to run them again in your staging environment. In this case, you can override the startURL in your global config file so that it points to your staging environment. You can then use the same GitHub Actions workflow to run the same test suite, ensuring uniformity in your testing across different environments.
Monitor GitHub Actions in Datadog
Whenever GitHub Actions executes a Synthetic test, the result will automatically populate within the Datadog CI Results Explorer. Tests are grouped into batches and listed alongside their summary status, duration, and other useful metadata. You can slice and dice by facets, such as ci_provider, test_id, and branch. Additionally, your teams can share pipeline execution data in dashboards and notebooks in order to collaborate more efficiently and determine whether a failing test is the result of an infrastructure misconfiguration, an outdated CDN cache, or test flakiness. Clicking on a batch allows you to pinpoint the specific test(s) in a batch that failed, and compare results across different browsers and devices.
Datadog CI Visibility also offers insights into the performance of your GitHub Actions pipelines, so that you can make them faster and more reliable. Once you’ve instrumented your GitHub workflows with CI Visibility, you’ll be able to troubleshoot problematic builds, stages, and jobs. Additionally, CI Visibility helps you to identify and triage flaky tests that are causing your workflows to fail. In the flame graph below, you can click on the span where the error occurred to see the stack trace and get helpful context around the failure.
Leverage Datadog’s new GitHub Action
Datadog’s GitHub Action is now available in the GitHub Marketplace, making it easy to add Synthetic tests to your GitHub workflows. The CI Results Explorer offers deep visibility into your GitHub Actions test results within the Datadog app. And by instrumenting your CI/CD pipelines with CI Visibility, you can get valuable insight into the health of your pipelines, enabling you to save time, ensure smoother builds, and shift observability to the left. If you’re new to Datadog, sign up for a 14-day free trial.