Monitor your Lambda functions with Datadog's Serverless Framework plugin

Since it was released in 2015, the Serverless Framework has become the community-favorite way to manage and deploy serverless applications in the cloud. Similar to Terraform and CloudFormation, it lets you express infrastructure as code, making it easy to share and version-control your entire serverless environment.

However, after deploying your serverless application, it can be challenging to answer questions like: How is my code running? Are there cold starts or bottlenecks? How will I know if my serverless application or one of its dependencies starts throwing errors? To make it easy to collect observability data from your serverless applications, we built the Datadog Serverless plugin. Without any code changes, you can painlessly instrument your AWS Lambda functions to get enhanced, real-time visibility into your serverless functions, applications, and cloud infrastructure in one place.

Monitoring Lambda functions with Datadog

Serverless, the concept, centers around the idea of solving business problems, while letting a cloud provider manage the underlying infrastructure (containers, scaling groups, orchestration, etc.). But serverless environments are fundamentally different than those running on cloud-native infrastructure or on-prem—there is no place to run a monitoring agent, and you get billed only when your application runs. When it comes to monitoring, the “why” stays the same, but the “what” and “how” differ.

From working with our customers, we’ve noticed several common requirements for serverless monitoring. First, performance matters—adding latency to applications is a non-starter. So, we developed the Datadog Lambda Layer, which provides a real-time view into custom metrics from your serverless functions—with minimal overhead.

Another theme is that serverless applications rarely exist in isolation—they’re usually part of a larger application or group of microservices. Datadog’s Lambda Layer ties together Lambda traces from AWS X-Ray and APM traces from hosts and containers, allowing you to get end-to-end visibility into requests as they travel across any type of infrastructure—whether they access Lambda functions, containers, hosts, or any combination thereof.

The Serverless plugin includes built-in support for Datadog’s AWS X-Ray integration and Lambda Layers, so you can track custom business metrics, automatically instrument your Lambda functions, and get end-to-end visibility into requests as they travel across your serverless architecture.

Getting started

The Datadog Serverless plugin supports Lambda functions written in Node.js and Python. For this example, we’ll use Node.js, but the steps are similar for Python. To set up a new project (named my-awesome-service in this example), run:

# install the serverless framework CLI
npm install -g serverless

# create a new service using the CLI's AWS Node.js template
serverless create --template aws-nodejs --path my-awesome-service

# using Python?
# serverless create --template aws-python3 --path my-awesome-service

# switch to the project directory and install the Datadog plugin 
cd my-awesome-service
npm install --save-dev serverless-plugin-datadog

# using Yarn?
# yarn add --dev serverless-plugin-datadog 

In your project directory, you should see a populated configuration file (serverless.yml), as well as a file for your Lambda handlers (handler.js). The simplest configuration file you need to get started with Datadog is:

service: my-awesome-service

plugins:
  - serverless-plugin-datadog

provider:
  name: aws
  runtime: nodejs10.x

functions:
  hello:
    handler: handler.hello
    events:
        - http: GET hello

custom:
  datadog:
    addLayers: true
    apiKey: "<YOUR_DATADOG_API_KEY>"
    flushMetricsToLogs: false

The Serverless Framework’s aws-nodejs template configuration file configures a hello Lambda function by default. Note that addLayers is set to true—this will configure the plugin to automatically add the Datadog Node.js and Python Lambda Layers to your functions, so you can simply import the necessary Layer methods into your function code to collect custom metrics (as described in the next section). If you use other runtimes like Ruby or Go, you can follow the instructions here to install the desired Layers, and configure the Serverless Framework to add them to your service.

flushMetricsToLogs gives you the option to submit custom metrics to Datadog asynchronously. If you wish to enable it, make sure you’ve also set up the most recent version of the Datadog Lambda Log Forwarder.

Substitute your Datadog API key under apiKey (or store your Datadog API key in an environment variable if desired). You’re almost ready to go! If you haven’t done so already, you’ll need to set up your AWS credentials for the Serverless Framework. Lastly, enable the AWS integration in your Datadog account. If you want to collect your logs as well, add the Datadog Lambda Forwarder to your AWS account.

Configure the function to submit a custom metric

After we’ve configured the Serverless Framework, we’re ready to code. Go ahead and copy and paste the following snippet into your handler.js file:

const { datadog, sendDistributionMetric } = require("datadog-lambda-js");

async function hello(event, context) {
  sendDistributionMetric(
    "coffee_house.order_value",       // Metric name
    12.45,                            // Metric value
    "product:latte", "order:online"   // Associated tags
  );
  return {
    statusCode: 200,
    body: "hello, dog!",
  };
}

module.exports.hello = datadog(hello);

This example code configures the hello function to report the value of a custom metric named coffee_house.order_value, and tags the metric with metadata about the order (product:latte and order:online). See the documentation for more details about configuring your functions to report custom metrics.

Deploy and explore

Then simply run serverless deploy to deploy your function. After your package uploads, you’ll see a URL in your terminal output corresponding to the Lambda function. Copy and paste the link into your browser to send an HTTP request.

Now head over to your Datadog account to start exploring key performance metrics (invocations, errors, and duration) from your function. The Serverless view aggregates data from all of the serverless functions running in your environment, enabling you to monitor their performance in one place. You can search and filter by name, AWS account, region, runtime, or any tag—or click on a specific function to inspect its key performance metrics, distributed traces, and logs.

Get plugged in to your serverless environment

If you’re already monitoring your serverless applications and infrastructure with Datadog, you can immediately install the Serverless Framework plugin to collect custom metrics and traces from your Python and Node.js Lambda functions in real time. If you’re new to Datadog, get started with a free trial.