We had a great time attending last week’s OSCON conference in Portland, OR. Going stronger and stronger on its 15th edition, OSCON now attracts a large crowd of open-source users and authors.
The diversity of the crowd is a proof to how pervasive open-source software has become. Without the open-source community Datadog would not exist in its current form, so we are happy to contribute and share as much of our software as we can.
Here is a short list of talks we enjoyed last week.
An update on Open Compute
Open Compute is a project, originally started at Facebook to replicate the open-source transformation to the rest of the infrastructure, including physical elements like server racks and data centers.
Jay Parikh shared one particular application to demonstrate the Open Compute approach to solving large-scale computing problems: storing photos at the Facebook scale. He shared an interesting metric: 8% of their photos are accessed 82% of the time so storing all photos using the same storage substrate does not make a lot of financial sense to them.
At their scale the usual tweaks (compression, etc.) don’t make enough $ difference so they went back to the drawing board to design their cold storage fabric.
With a new data center design and new server racks they are able to achieve impressive results: 30% of the original $/GB cost, including an 80% saving on their data center costs compared to their conventional and already-optimized compute-heavy data centers.
To get there, they ripped out all electrical redundancy systems, packed 2PB of storage per rack, with only 2kW of power draw (25% of a normal rack, individual power consumption).
While few companies can boast to have problems at this scale, it is great to see the open-source spirit applied to the rest of the infrastructure.
Go is everywhere
2013 is the year of Go. It certainly felt like to back-end programming what Ruby was to front-end: refreshing, expressive and productive. Google is aggressively pushing the language forward with numerous sessions to win over the hearts and minds of the software community at large.
We have started to get our feet wet with Go and so far we like it.
Daemons, solved
Thanks to Caskey Dickson, SRE at Google, the community now has some proven blueprint to achieve something that is deceivingly hard to get right: making sure that processes stay up and running.
Since the inception of Unix 40 odd years ago, there has been a plethora of methods to start processes on boot and make sure they keep running or restart when they crash. Caskey’s argument is that none of them has completely cracked that nut.
His recipe relies on a few principles:
For more details, head over to the slides.
See you next year!
It was a great show. We will be back.
