Editor’s note (September 2019): The steps below have been modified for Agent 6. See our documentation for more details about this integration.
David Caplan @davc is the Chief Technology Officer at SolveBio and runs his own blog, davecap.com.
At SolveBio we use AWS Elastic Beanstalk to manage some of our apps on EC2. Our API for biomedical data is critical for running diagnostic tests, so we like to keep an eye on its performance. We started using Datadog a few weeks ago to help us out with monitoring and metrics, but we ran into some issues getting it to play nicely with Elastic Beanstalk. This article describes how we got it all working together.
How SolveBio uses Elastic Beanstalk
Similar to Heroku, Elastic Beanstalk makes it super easy to deploy various web apps to EC2. At SolveBio, we mostly work with Python and Node.js, so the method described here focuses on those stacks. The Datadog Agent is a daemon managed by supervisord, which sends system metrics to Datadog but also works as a statsD proxy.
One kink we initially encountered was that the Datadog Agent and Elastic Beanstalk rely on separate installations of supervisord. This confuses the heck out of Elastic Beanstalk. Luckily, getting past this kink is simple. You just have to tell Elastic Beanstalk to stop Datadog before deploying and start it again when you’re done.
Before continuing, make sure you’ve read the Elastic Beanstalk Getting Started Guide and Customizing and Configuring a Python Container.
Deploy the Datadog Agent on Elastic Beanstalk
Once your project is all set up for Elastic Beanstalk, you can deploy the Datadog Agent by using Beanstalk’s environment configuration files. Datadog provides a 99datadog.config file that you can download and place in a new .ebextensions directory. You can also use the containerized Datadog Agent for monitoring, if you use containers in your Elastic Beanstalk environment.
Once you save your configuration file, commit the changes and then deploy your app using git aws.push. The Agent should be installed once the update finishes, and it should also keep working after further deploys to your Elastic Beanstalk instances. We found it helpful to enable SSH login to our instances for debugging.
Now the Elastic Beanstalk monitoring fun begins
Now that we’re up and running, monitoring AWS Elastic Beanstalk with Datadog has already proven to be very helpful. With that said, we’re still getting acquainted with Datadog so I’ll be back in a few weeks to share my thoughts on key AWS Elastic Beanstalk metrics to monitor and any tips and tricks or best practices I’ve gained. In the meantime, if you’d like to gain visibility into your AWS Elastic Beanstalk performance you can try Datadog for free for 14 days.
