Alerts help you stay ahead of problems with your applications and infrastructure, but it can be challenging to create alerts that effectively signal potential issues without also producing a lot of noise. This challenge becomes even more pressing as more services, hosts, and conditions are monitored.
Datadog’s new composite monitors let you combine two or more separate monitors using logical operators to further refine your alerts—for actionable insights without the noise.
Actionable alerting made easier
Often, issues that require intervention are identified not by a single metric or event, but by a combination of indicators.
Composite monitors work to weed out unnecessary alerts by constraining the conditions that cause an alert to fire in the first place. You can use composite monitors to alert on any combination of performance indicators, or to add critical nuance to your alert definitions. For instance, you might want to alert when your message queues grow too long, but withhold an alert on service restarts, which can cause temporary queue growth and trigger false alarms. In this case, you might create a composite monitor that triggers only when queue length crosses a threshold and the uptime for the service is greater than 10 minutes. Composite monitors can also include anomaly detection monitors for even more refined alerting and investigating.
Once a composite monitor is set up, you can disable notification from the original singular monitors to reduce the total number of alerts you receive.
Get started
Setting up a composite monitor is as simple as setting up any other monitor type in Datadog, with a step-by-step interface for selecting alert conditions, notification channels, and creating an alert message. You can chain up to 10 of your existing monitors together to create a new composite monitor.
Once you’ve chosen your monitors, you can define the trigger conditions in the box below using the monitors’ corresponding letters and standard logical operators (&&, ||, !) optionally grouped with parentheses.
After defining the trigger conditions, you can add context to the alert by writing a descriptive alert message (and including links to relevant runbooks or docs), and choose which team members or communication tools (such as Slack or PagerDuty) should be notified. And that’s all! Your composite monitor will start evaluating its alerting logic to provide you with fewer, richer, and more actionable alerts.
If you’d like to use composite monitors to optimize alerting for your infrastructure and applications, you can sign up for a free trial of Datadog.
