Protect Sensitive Data with Synthetics Private Location Runners
September 25, 2024
Introduction
It is very common for companies to have use cases where they need to monitor internal applications, such as intranet portals and APIs. Testing and monitoring for these applications may need to be geographically specific for global businesses to ensure secure monitoring for industries with strict compliance requirements.
Datadog Synthetics Private Location Workers are self-hosted agents that enable monitoring of internal, firewalled, or geographically specific applications and services. They bridge your internal network with Datadog’s Synthetic monitoring platform, allowing you to test applications not accessible from the public internet, simulate user experiences from specific locations, and maintain control over your testing environment.
Private Location Runners are particularly valuable for organizations that need to keep sensitive data within their network, verify regional content delivery, or maintain complete control over their testing infrastructure.
Explanation of the architecture
The following steps describe the architecture and use cases for running Datadog Synthetics checks on a private network:
Step 1
Datadog Synthetics Private Location Workers can be installed via a Docker container or a Windows MSI package. The Docker container can be used in a standalone Docker environment or in an orchestrator such as Kubernetes. Each Worker is capable of executing API and browser based tests. Resource consumption (CPU, memory, and number of workers) can be estimated by following the scaling guidance here.Step 2
Public-facing applications such as web applications over HTTP/S are checked by Datadog Synthetic Monitoring with checks running from Datadog’s public locations and can only access public endpoints.Step 3
On startup, the Private Location Worker connects to Datadog over a secure in-house protocol based on AWS Signature Version 4. Next, the worker retrieves synthetic test configurations. This can be optionally configured to use a proxy if one is required to connect to Datadog.Step 4
Private Synthetic Checks monitor private APIs and websites by simulating user traffic from within your network. These checks support a variety of protocols, including HTTP/S, DNS, TCP, UDP, ICMP, gRPC, and WebSocket.Step 5
Check results are sent using the same secure in-house protocol in Step 3. For browser-based tests, this includes artifacts such as screenshots, errors, and resources. Traffic to Datadog is sent to the following endpoints depending on the version of the worker:- Synthetics Worker v1.5.0 or later: intake.synthetics.datadoghq.com is the only endpoint you need to configure.
- API test results for the Synthetics Worker > v0.1.6: intake.synthetics.datadoghq.com
- Browser test results for the Synthetics Worker > v0.2.0: intake-v2.synthetics.datadoghq.com
- API test results for the Synthetics Worker < v0.1.5: api.datadoghq.com
Authors
Kennon Kwok, Product Solutions Architect
References
Inspiration and reference documents or existing solutions: