Developers and teams who want to deploy new code often and safely leverage feature flags to decouple code deployments from feature releases. Feature flags enable teams to release new features to a subset of users, making it possible to test a new feature’s impact on users and ensuring that developers can easily roll back the feature if it causes downstream issues.
Feature flag tracking is a new Datadog RUM capability that enriches your browser and mobile RUM data with feature flag tags, providing visibility into your application performance and user experience so that you can release new features safely and reliably. With feature flag tracking, your team can isolate and measure the impact that releases tagged with feature flags are having on user experience and make sure new features are not causing performance issues.
Once you’ve enabled feature flag data collection, you will see the Feature Flags page in the RUM explorer. This page provides out-of-the-box analysis that shows you which feature releases have caused a rise in errors or a performance regression, which features are active or inactive, and which have been fully released, helping you pinpoint problematic features and clean up out-of-date feature flags. Additionally, feature flag tracking surfaces flaws arising from your releases before they become widespread issues, enabling you to turn off or roll back the problematic feature as soon as you spot an issue and avoid impacting more users.
In this post, we’ll show you how to use feature flag tracking to:
- Identify problematic features during incident investigation
- Check to see if your newly released feature is causing errors or negatively impacting user experience
Identify problematic features during incident investigation
When an incident occurs, you want to be able to quickly identify the root cause. If the culprit turns out to be, for instance, a newly released feature that is still in beta testing, you want to get that information early so that you can fix it before it negatively impacts more users.
For example, say you are an engineer at an e-commerce site and you notice a Watchdog anomaly alert indicating that there has been a recent spike in latency. You know that a new feature was released in beta, so you navigate to the Feature Flag page in Datadog RUM. Here, you can see high-level, out-of-the-box metrics on all your feature flags, get information about the status of each flag, compare user sessions, and see what views are evaluating your feature flag. This visibility helps you quickly determine if there is an anomaly with your feature flag data.
Because you know the specific feature flag you’re looking for, you type its name into the search bar so that you can view user sessions correlating to that feature flag. Alternatively, you could drill down to see only feature flags associated with the specific app or view you’re working on.
Once you find the feature flag in question, you can investigate further by clicking through to the feature flag analysis page.
Here, you find a wealth of out-of-the-box metrics, including loading times and error rates by variant, which show you if your flagged feature is introducing new errors, causing a spike in errors, increasing latency, or affecting core web vitals. These insights help you confirm that the feature flag from a recent deployment by the infrastructure team is causing the increased latency. With this knowledge in hand, you can turn off your feature flag right away so that it doesn’t affect more users. If you are using LaunchDarkly or Flagsmith to manage your feature flags, you can use the dashboard widget to embed the UI into a Datadog dashboard and turn off your feature flag without leaving the app.
Check to see if your newly released feature is causing errors or negatively impacting user experience
When you deploy a new change to your application, you need to ensure that the rollout is not causing any errors or negatively impacting performance, which could lead to negative impact on business metrics. For example, high latency on your site can cause users to become frustrated and abandon purchases.
Say you released a new checkout flow for a subset of users behind a feature flag, having taken this precaution because checkout experiences can impact customer behavior. While the change should help users complete checkouts smoothly, you risk losing revenue from frustrated customers if the flow doesn’t work as intended or introduces unforeseen problems. As soon as users start using the feature, sessions will display all applicable feature flag values, as in the example below.
The RUM Explorer allows you to perform analytics on your feature flags. You can group your sessions and views by specific feature flags and filter sessions to view only the users who are interacting with your flagged features.
Using the RUM Explorer, you compare your new checkout flow release with the current version and find that the number of purchases, a KPI for your site, is significantly lower on average for the new feature flag release than with the current version.
Once you realize that this release may have a negative impact on revenue if released to all users, you roll back the feature flag and investigate how to resolve this issue with the new feature. To do so, you take a look at user session replays and find that many of them end after seeing ads generated by the new checkout flow. This gives you added context to begin investigating what component of the new feature may be causing the negative user experience that is driving users to end sessions before making a purchase.
Monitor your releases and ensure user satisfaction with feature flag tracking
Feature flag tracking in Datadog RUM enables you to automatically identify bad performance in your releases. This visibility streamlines data collection and analysis, helping you pinpoint any releases that have led to a rise in errors or a performance regression and enabling you to focus on safely and reliably releasing features. And feature flag tracking provides a number of out-of-the-box metrics around errors, latency, and core web vitals to help you troubleshoot.
You can start collecting feature flag data for custom feature flag management solutions or use one of our integration partners, Amplitude, LaunchDarkly, Split, Statsig, Flagsmith, or DevCycle. Feature flag tracking complements other Datadog features, such as Deployment Tracking, which points you to new deployments that may be behind feature flags, and Faulty Deployment Detection, which alerts you to new features that may be causing issues.
If you’re new to Datadog, sign up for a 14-day free trial, and leverage feature flag tracking to ensure safe releases.
