Docker is being adopted rapidly, and for good reason: it simplifies many aspects of running a service in production. But Docker-powered services typically run many more containers than traditional services run hosts, so monitoring is much more complex. And with orchestrators like Kubernetes, Mesos, and ECS managing your containers, you may not even know where your containers are running at any given moment. When your containers can shift from host to host without your knowledge or intervention, manual monitoring of your services becomes nearly impossible.
How can you monitor a service which is running on a shifting set of hosts?
Datadog’s Autodiscovery feature automatically keeps track of what is running where, and gathers detailed metrics from your containers and services—wherever they may be running. With Autodiscovery, you can now continuously monitor all your services, no matter how dynamic or ephemeral the underlying infrastructure may be.
What sorcery is this?
With Autodiscovery enabled, the Datadog Agent continuously listens to Docker events. Whenever a container is created or started, the Agent identifies which service is running in the container, looks for the appropriate monitoring configuration for that service, and starts collecting and reporting metrics. Whenever a container is stopped or destroyed, the Agent understands that too.
I want it! What do I need to do?
If you followed our standard instructions for deploying Datadog on Kubernetes or DC/OS, you have Autodiscovery enabled by default. Without any further customization, the Datadog Agent will attempt auto-configuration for roughly a dozen services, including Redis and Elasticsearch, which do not require custom credentials or configuration.
To use Autodiscovery on a different container platform, or to enable additional checks, you only need to do two things. First, define the configuration templates for the containers you want to monitor. Then, enable Autodiscovery for the Datadog Agent.
Configuration templates
The config templates for Autodiscovery have a very similar structure to the standard YAML configuration files used by the Datadog Agent, with two exceptions. First, Autodiscovery templates specify Docker image names or labels, telling the Agent which containers the monitoring configuration should apply to. And second, Autodiscovery makes use of variables like %%host%% and %%port%% to apply the configuration to dynamic infrastructure.
Our documentation provides an example config template for Apache (httpd) monitoring.
You can store these templates as local config files, or in a shared configuration store like etcd or Consul. You can also add your Autodiscovery configuration templates using pod annotations in Kubernetes (since Agent version 5.12) or label annotations for Docker containers (since version 6.20).
Configuring the Agent
To configure your Datadog Agents to use Autodiscovery when installed on a host, edit your datadog.yaml configuration file to include the listeners and config_providers keys, as we show in our documentation. For Agents installed as Docker containers, Autodiscovery is enabled automatically.
Use variables to account for dynamic infrastructure
If you’re using a container orchestrator for scheduling, you probably don’t know which host IP address or port number the Agent needs to access to connect to a given service. That’s why our configuration templates support variables like %%host%% and %%port%%, which the Agent will fill in with actual values retrieved from the Docker API. You can also use indices for variables that return a list of values, for example, %%port_4%% if your service exposes multiple ports and you need to select one in particular.
Try it out
It is now easier than ever to monitor highly dynamic, ephemeral container infrastructure!
If you are already a Datadog user, you can start using Autodiscovery right now. Otherwise you can try it out by signing up for a free trial.
Datadog Agent image on the Docker Store
You can get the Docker-certified Datadog Agent image—which includes the new Autodiscovery feature—directly from the Docker Store. In one step you’ll be able to pull the image and immediately begin monitoring your hosts and containerized services, however dynamic or high-scale your infrastructure may be. Docker is a great partner to work with and we’re thrilled to be featured on the store.
