The modern standards of the web demand that user-facing applications be highly usable and satisfying. When deploying frontends, it’s important to implement a comprehensive testing strategy to ensure your customers are getting the best possible user experience. It can be difficult, however, to gauge the effectiveness of your test suite. For instance, all of your tests may be passing, but they might not cover a specific UI element that is crucial to a critical workflow.
Datadog Synthetic Monitoring and RUM address this challenge by enabling teams to build their testing suites based on real user data, helping answer the following questions:
- What are the most critical workflows to test?
- What workflows are currently untested, and how often do customers use them?
- What percentage of user actions are covered by your testing?
In this post, we’ll show you how the new Test Coverage page can provide crucial insights to help you optimize your test suite and ensure that your application’s most important workflows are covered.
Understand your test coverage
Datadog now leverages user actions, which are tracked by Datadog RUM, to assess your test coverage. A RUM user action refers to a key interaction in your frontend, such as a button click or test input. You can also define and track custom actions, which contain strings of related actions that make up a specific workflow, such as creating a profile, checking out from an e-commerce store, or recovering a lost password. The Test Coverage page enables you to easily compare the total number of actions and custom actions to the number of those covered by your Datadog Synthetic tests, so you can determine the overall effectiveness of your test suite. Your user action testing coverage is calculated both as an overall percentage and also as a percentage weighted by the number of user interactions with each action, so you can understand if your testing appropriately prioritizes your app’s most common user journeys.
Identify untested actions
The Test Coverage page also reports how many tests cover your actions, as well as how many user interactions have been recorded overall. This data, which can be scoped to a specific time frame, can be used to alert you to issues such as test sprawl, in which you have a low percentage of covered actions but a large number of tests covering them. The page also surfaces the top tested and untested actions according to how many user interactions occurred in them within a specified time span. This makes it easy to spot which key actions are missing from your test suite.
To further determine where to focus your test development efforts, you can use the query bar to filter the page’s data to specific actions and views. For instance, if you observe that a particular view has low test coverage, you can query for that view and determine which of its key actions are still untested. You can also toggle between data for all actions and for strictly custom actions, which can help filter out the noise created by smaller sub-actions in the workflows that your custom actions cover. Once you’ve found an action to investigate, you can click it to open a side panel that surfaces relevant user sessions. From each of these sessions, you can pivot to a full simulated replay of the user’s activity to clearly identify the action.
Refine existing tests
In addition to investigating user behavior in actions that aren’t covered in your testing, you can also use the Test Coverage side panel to investigate and refine the tests you’ve already set up. When you open the side panel for a tested action, an additional tab appears that contains all the tests running on that action. From this view, you can quickly determine the uptime and current state of these tests, and then drill down further to edit the test and view additional test performance metrics in the test creation page.
Own your test coverage
By monitoring your Synthetic test coverage with Datadog, you can characterize the effectiveness of your test suite and identify key workflows that are overlooked in your testing. Test coverage monitoring is now generally available for all Datadog RUM and Synthetic Monitoring customers—see our documentation for more details. Or, if you’re brand new to Datadog, sign up for a free trial to get started.
