Monitor Your OpenAI Usage | Datadog
Datadog logo home

Monitor your Azure OpenAI usage with Datadog

Track, assess, and optimize your organization’s usage of Azure OpenAI. Get an in-depth overview of cost estimation, prompt and completion sampling, performance metrics, and more.

/azure-open-ai-dashboard-new

800+ Turn-Key Integrations, Including

Product Benefits

Track Azure OpenAI Usage Patterns

  • Easily break down API requests by Azure OpenAI model, service, organization ID, and API key with out-of-the-box dashboards
  • Identify where Azure OpenAI’s models are used in your organization and to what extent with unique organization IDs
  • Enable tracking by API keys to break down usage by specific users, uncover spikes in usage and associated costs, and trace unauthorized API access back to specific keys and users
  • Avoid rate-limit errors or excessive latencies with recommended monitors that proactively measure the rate of your requests and token usage
/blog/monitor-openai-with-datadog/monitor-openai-prompts-completions-logs.png

Monitor and Allocate Costs Based on Token Usage

  • Discover primary cost drivers for your Azure OpenAI usage; track your total token consumption and break it down by prompt and completion tokens
  • Leverage out-of-the-box dashboards to track the average total number of tokens per request & and the average number of prompt and completion tokens per request
  • Deploy and start monitoring without any need for professional services or extensive training
/tokenusageopenaiimage.jpeg

Easily Analyze API Response Times to Troubleshoot and Optimize Performance

  • Seamlessly track latency performance by operation and API, along with performance across rate limit, blocked calls, and total errors
  • Utilize traces to uncover performance issues on individual requests, such as the specific models that were queried and the precise content of prompts and completions
  • Automatically save error and high latency traces for 15 days
  • Gain deeper context to identify the source of unusual changes in performance by collecting Azure OpenAI logs alongside metrics and traces
/azurai-bb-3.png

Real results from Datadog customers

12B Log events each day, managed cost-effectively
EA DICE
<2 min Mean time to resolution (MTTR)
CITIZENS BANK
50% Cost savings on cloud resources
TRAVELSUPERMARKET

The Essential Monitoring and Security Platform for the Cloud Age

Datadog brings together end-to-end traces, metrics, and logs to make your applications, infrastructure, and third-party services entirely observable.

Platform Diagram

Loved & Trusted by Thousands

Washington Post logo 21st Century Fox Home Entertainment logo Peloton logo Samsung logo Comcast logo Nginx logo

Start your free trial Request a Demo