Growing demand for streaming video and interactive experiences
As one of Germany’s leading players in the arena of multi-channel entertainment, Seven.One Entertainment Group is tasked with growing online consumer engagement amid tumultuous market changes. Viewer habits are radically shifting, with users moving away from traditional “linear” TV and toward video-on-demand as their preferred means of consuming media. Audiences for live TV shows are also flocking to more interactive, second-screen experiences and leaving behind traditional, passive consumption. And on top of these developments, competitors in the market are proliferating, as are the device types and software platforms on which companies need to deliver content.
Need for agility drives DevOps
For Pavlo Voznenko, CTO of Seven.One Entertainment Group, competing in such a challenging environment requires never-ending product improvements, driven by deep knowledge about the consumer’s experience.
“Being in a competitive market, it’s really important to understand what our consumers are experiencing,” says Pavlo. “And if we deliver a bad experience or fail to respond as well as our competitors, I don’t think people will use our applications at all. We need to constantly evolve.”
A strong DevOps mindset is key to executing on this strategic goal of ongoing evolution. “An important part of modern software development is releasing as often as possible, as part of DevOps best practices” says Pavlo. According to this strategy, teams must be able to quickly find any weak points in the user experience (such as video stream interruptions) and then deploy fixes on demand.
“ Being in a competitive market, it’s really important to understand what our consumers are experiencing. Datadog’s traceability, durability, and intuitive UI have really helped us adopt the DevOps mindset.”
Pavlo Voznenko
CTO, Seven.One Entertainment Group
Fragmented monitoring held DevOps back
When Pavlo first came to Seven.One Entertainment Group, a lack of shared tooling was crippling the very DevOps practices that could enable teams to quickly spot and fix issues. Each team used its own monitoring solution, so no tool provided visibility over the whole application or enabled engineers to trace requests across services. If users reported poor streaming quality, for example, engineers had no way to pinpoint bottlenecks in the chain. Visibility into live production services was so poor, in fact, that engineers would even dread being on call. “When you use a monitoring tool in some domains only, it doesn’t help the DevOps mindset,” says Pavlo.
Flagship events required “war rooms”
The lack of adequate monitoring also made it challenging for Seven.One Entertainment Group to deliver live interactive shows, an especially popular feature for the company. These shows draw up to 10 million simultaneous viewers online, with many signing in to cast a vote for their favorite singer, for example, or to share other opinions. To handle such an extreme load of brute force traffic, one’s infrastructure needs to be resilient and agile.
But without any distributed tracing to shed light on infrastructure requirements beforehand, or live real-user monitoring during airtime, the engineering team had to manually manage these high-profile events. Engineers could prepare only by spinning up the maximum number of instances in Amazon Web Services (AWS) and warming up all the caches. Then once the show was live, they would gather in a “war room,” monitor the traffic coming in, and be prepared to react quickly if any bottlenecks appeared.
Hosting shows with live voting has always been strategically important for Seven.One Entertainment Group to grow user engagement, but the success of these shows once seemed like a high-stakes gamble. “We want to continually amaze our users,” says Pavlo. “But if one day our site or app fails its live streaming, we don’t get a second chance.”
Global monitoring system gives DevOps a boost
To address these problems, in 2020, Seven.One Entertainment Group turned to Datadog’s application performance monitoring (APM), real-user monitoring (RUM), and infrastructure monitoring. And for the first time, teams now have 360-degree visibility. “Having infrastructure monitoring, APM, and RUM together in one solution has allowed us to trace the entire user experience, find bottlenecks in the infrastructure, and detect anomalies quickly,” says Pavlo. “And that has helped us to evolve in our DevOps practices, to make DevOps part of our DNA, and deliver better and faster from each show.”
Notably, engineers no longer fear being on call for incident management. “Datadog’s seamless observability has helped our teams adopt DevOps practices and be more secure in being on call.”
Datadog’s durability and its intuitive UI have helped its adoption, according to Pavlo. “The level of adoption from the engineering teams has been phenomenal.”
No more war rooms
Datadog has also been particularly helpful both before and during live interactive shows. Even before these shows are broadcast, Seven.One Entertainment Group can now predict bottleneck points in its infrastructure, so it can make any necessary adjustments ahead of time. And during the shows, Datadog RUM provides live analysis, enabling support teams to verify good performance and address any unforeseen bottlenecks that appear.
“Even with millions of users, we feel at ease. We know our bottlenecks, and we try to address everything beforehand,” says Pavlo. “It doesn’t require engineering time to attend each show, create war rooms, or work overtime. We don’t have war rooms anymore. Now it’s business as usual. It’s a relief.”
“ We brought in Datadog to try to consolidate our traceability and observability. Now Datadog is our central tool for virtually everything including incident detection, synthetic testing, and now compliance with the latest security features.”
Impressive cost savings
A final, concrete benefit Seven.One Entertainment Group has experienced from using Datadog has been lowered infrastructure costs. “Datadog has allowed us to be more efficient with our budgeting because now we can set up exactly what we need, and we pay only for what we consume and not some magic number,” says Pavlo. In fact, the company was able to reduce its costs by 78% just from adopting Datadog APM alone.
From DevOps to DevSecOps
Looking to the future, Pavlo expects Seven.One Entertainment Group to lean more on Datadog for visibility into security weaknesses. Security is particularly important for publicly broadcast TV because hacks, mishaps, and inappropriate content can affect so many users. For this reason, Seven.One Entertainment Group plans to use Datadog to help improve collaboration with security teams. “We are really happy with Datadog’s new security features, which allow us to run threat detection on our infrastructure and analyze different threat signals. It is helping us get to the next step of adopting DevSecOps practices,” says Pavlo.
Datadog unlocked the door to a DevOps culture
The adoption of DevOps practices has been the greatest benefit of Datadog, according to Pavlo. “Datadog’s traceability, durability, and intuitive UI have really helped us adopt the DevOps mindset. My development teams love it. It’s helped to improve our business, get ahead of what’s happening, and spot potential issues before they occur.”
And as a business partner, Pavlo gives Datadog high marks. “To succeed in a highly competitive market, you need to have the right partners. I’ve never had such service from other companies on the market, so well done by Datadog. Awesome.”
