As cyberattacks rise in number and sophistication, many CISOs are pushing their organizations to adopt modern SIEM solutions to better monitor and investigate threats to their applications and infrastructure. Enterprises with a large Microsoft Azure or Windows-based footprint in particular are increasingly eyeing Microsoft Sentinel to consolidate their security stack and workflows.
However, moving to a new SIEM can be highly challenging in practice. Security teams tend to operate on tight budgets and often lack the dedicated IT support needed to complete these migrations. And even more importantly, installing and managing IT infrastructure is not the core responsibility of security teams; these concerns divert their focus from their primary role of detecting and investigating security threats. Without an easy way to standardize, enrich, and route logs, your teams are likely to spend valuable time managing extract, transform, and load (ETL) processes. They also risk being overwhelmed, in this scenario, by noisy false positives—which can cause them to lose visibility across their infrastructure and miss critical Indicators of Compromise (IoCs).
Datadog Observability Pipelines now integrates seamlessly with Microsoft Sentinel, enabling security teams to collect, transform, and route logs to this SIEM solution without requiring custom scripts or preprocessing. With this integration, you can simplify your SIEM migration to Microsoft Sentinel by centralizing log collection and ETL functions within your own infrastructure and routing logs to their destination according to your network policies.
In this post, we’ll cover how Observability Pipelines can help you:
- Process, parse, and enrich logs for threat detection with Microsoft Sentinel
- Simplify migrations and send logs directly to archives
Process, parse, and enrich logs for threat detection with Microsoft Sentinel
Enterprise security teams collect logs from various sources in different formats, requiring normalization for effective threat detection. However, this data needs to be processed in the format required by their preferred vendors to easily write threat detection rules; correlate tactics, techniques, and procedures (TTPs); and investigate malicious attacks. Without a standardized way to centrally manage processing and parsing, security teams face the extra burden of handling ETL pipelines separately at each destination. This is a complex job that takes time and attention away from their main goal of detecting threats.
With Datadog Observability Pipelines, you can centralize log processing and normalize security logs before you send them to Microsoft Sentinel. For example, you can enrich logs with GeoIP information and redact sensitive data before it leaves your environment. Or, to standardize your security logs from sources like Microsoft Office 365 Defender, Okta, GitHub, and more, you can remap them to the Open Cybersecurity Schema Framework (OCSF) format used by Microsoft Sentinel and other top security vendors. By using the Grok Parser, as shown below, you can also handle unstructured logs from 150+ sources, such as Azure IoT Edge, or create custom parsing rules.
Once it transforms your security logs, Observability Pipelines can route your data to Microsoft Sentinel to take advantage of Sentinel’s built-in threat detection and native incident response capabilities—helping to lower your mean time to response (MTTR).
For example, let’s say you’re the CISO at a large telecom services company. Your company uses Elasticsearch for DevOps troubleshooting, along with Google SecOps (formerly known as Chronicle) for security. Let’s also say that you are currently evaluating Microsoft Sentinel and have decided to split your logs to send a copy to all three destinations. Managing this log splitting across different vendors and standardizing log processing at each separate destination requires extra effort for multiple teams. With Observability Pipelines, your security team can easily prioritize and route logs to Microsoft Sentinel alongside the others without losing visibility, sacrificing compliance, or breaking any of your current security or DevOps workflows. The pipeline used in this scenario is configured below:
Log destinations
Alternative log destinations include Splunk, Datadog Cloud SIEM, SentinelOne, Sumo Logic, Dynatrace, and others. (See the complete list here.)
Simplify migrations and send logs directly to archives
Adopting a new SIEM vendor is often time-consuming and disruptive to ongoing security investigations. Planning for this kind of migration requires you to prepare historical log data for your next platform—a process that typically lasts months. Managing this vendor transition across multiple sources and destinations tends to make the process even more complicated, leading to increased costs.
Datadog Observability Pipelines integrates with all major SIEMs, data lakes, logging platforms, and cloud storage providers to enable your security teams to easily evaluate new solutions as needed. During this evaluation period, you still maintain full control and visibility over your data and costs as you continue to route logs to your current vendor. For example, if you’re a CISO managing a potential migration from Splunk to Microsoft Sentinel, you can configure Observability Pipelines to easily dual ship production logs to both destinations. At the same time, you can send full-fidelity logs directly to archives such as Azure Blob Storage for longer-term storage, ensuring a smooth transition and uninterrupted threat detection. A pipeline for this scenario is shown below:
Speed up your Microsoft Sentinel adoption with Datadog Observability Pipelines
Datadog Observability Pipelines enables you to choose the logging platform and security solutions of your choice, including Microsoft Sentinel, so that you can support enhanced analytics, improve your threat detection, and avoid vendor lock-in. To get started sending your logs to Microsoft Sentinel with Observability Pipelines, set up the Microsoft Sentinel destination and environment variables, and configure any processors as needed. For more information, visit our documentation.
If you’re new to Datadog, you can sign up for a 14-day free trial.