Interesting box. With an AFRR of 5 - 7% the box would nominally 'lose' a hard drive or two a year. I wonder how they deal with transient failures (which would be more like 2 - 3 month). If they do the 'triple replica' thing they only get 36T effective per box, but I suppose at about 2G / movie in MPG2 thats still about 18,000 movies 'online' as it were. Now if they did something wild and crazy like a form of RAID6 that would be even cooler (and give them like 100T of actual space or 50,000 movies. Serious video, but really hard to stream to multiple people.
The old SGI/TimeWarner Video-on-demand nightmare seems so quaint now. That had a full cabinet (42U) of storage and servers and it poorly served about 50 streams of a couple of thousand 'titles' and cost > $2M each if my aging brain remembers all the specs. Hmm these guys [1] peg the cost for the "Full Service Network at $100M which sounds about right.
[1] http://www.vizworld.com/2009/04/what-led-to-the-fall-of-sgi-...
I've often wondered if CDNs are going to go out of business in the streaming world, and no longer be relevant, in the world of (rapidly) falling transit costs.
Once transit costs drop below $1/mbit @95th percentile (circa 2014 [1]), it's costing you $1 to transfer approx 300 gigabytes/month. At that point, the management costs associated with all of those distributed CDNs becomes greater than your transit costs. This is somewhat offset by continuously dropping prices of CDNS [2] which is as low as $0.02/Gigabyte on a 500 TByte commit.
What this does tell me is that NetFlix thinks they have an opportunity to save money on CDNs and can beat that $0.02/Gigabyte.
Bandwidth Delay Product(BDP) - keeps CDNs relevant for _downloads_ (iTunes, Software, etc..) - because everybody likes to max out their 100 mbit comcast connection, but you don't need much bandwidth to stream a show. Maybe some value for CDNs in HD or live event streaming.
[1] http://drpeering.net/AskDrPeering/blog/articles/Peering_vs_T...
[2] http://blog.streamingmedia.com/the_business_of_online_vi/201...
Cool, maybe they can deploy it at comcast and deliver content via the same magical content pipe comcast uses for their xfinity service.
Makes me wonder how long until Netflix looks at doing managed services for video delivery along the lines of Amazon's AWS services.
Hopefully they should be able to use these cost savings to purchase more content and get it into their system. Does anyone have any informed speculation about how much they could potentially stand to save? I'm unfamiliar with how peering agreements work and or the cost structure.
Here's the accompanying Netflix blog post:
http://blog.netflix.com/2012/06/announcing-netflix-open-conn...
I have worked on Netflix's servers first hand. After seeing how large of a presence they have in their current CDNs, I can say that they have a hell of a lot of work ahead of them for that to become viable. They are enormous.
Completely off-topic, but I love the redesigned Silverlight player for Netflix on PC. It's gorgeous.
From https://signup.netflix.com/openconnect/software:
Operating System
For the operating system, we use FreeBSD version 9.0. This was selected for its balance of stability and features, a strong development community and staff expertise. We will contribute changes we make as part of our project to the community through the FreeBSD committers on our team.
Web server
We use the nginx web server for its proven scalability and performance. Netflix audio and video is served via HTTP.
Routing intelligence proxy
We use the BIRD Internet routing daemon to enable the transfer of network topology from ISP networks to the Netflix control system that directs clients to sources of content.