US Air Force connects 1,760 Playstation 3's to build supercomputer (2010)

  • This is one of those headlines that initially reads like a joke until you think of the economics of it. "cost is about 5-10% of the cost of an equivalent system built with off-the-shelf computer parts."

    I wonder if this is due to the fact that the Playstation hardware is (was?) competitively priced to encourage revenue generation through games? Or was Sony simply very good at mass-producing these units?

  • > The PS3s are the older, larger variety, since the newer slim models don't allow for the installation of Linux.

    Big caveat there. Sony later removed the ability to install "OtherOS" (a.k.a. Linux or FreeBSD) on PS3s from those models. [0] I wonder if that firmware update basically neutered the supercomputer or if they were able to keep them from updating automatically.

    [0] https://en.wikipedia.org/wiki/OtherOS

  • UGH! All this infrastructure and all this work into a software platform and it's something that will be obsoleted in a few years by off the shelf GPUs built on an standard development environment backed by industry

    That's the problem of building something super-cutting-edge on a grand scale - you run the risk of making a super evolved version of an evolutionary dead end.

    Sure it sounds like a cool idea at the time, the Cell optimized SETI At Home demo around the PS3 launch ripped through work units far more quickly than any Intel system.

    I wonder what became of the cluster once it was decommissioned? Military surplus PS3s anyone?

  • The challenge was not rigging this up - but making things usable & efficient. The Cell Broadband Engine was notoriously hard in architecture & needed lots of optimization.

    Part of the reason why PS4 & Xbox later pivoted towards x86-64 ISA

  • This used to be fairly common, I think.

    I remember in college we had a lab full of PS3s because they were cheap and powerful and there was a course you could take to learn to develop on them. The lab later expanded after a big donation from nvidia of high end gaming machines to teach students CUDA.

    The courses sucked though, because everyone would take them just to get access to the labs.

  • Related: Los Alamos' Roadrunner was based on the CELL as a co-processor, and was #1 in TOP500 in November 2008.

    https://en.wikipedia.org/wiki/Roadrunner_(supercomputer)

  • In the PS2 era I considering buying it and using the "linux kit" instead of buying a desktop. I was prepared for the differences in performance but then I knew that sony didn't release all the drivers or specs and graphic acceleration was not available. Just gave up the idea.

    The PS3 could be used as a computer, this allowed sony to pay less taxes in the EU. Since it is such a closed platform the "install other OS" feature could be disable remotely by the vendor automatically and, I think, without any user intervention. When it became economically "better" for sony, they disable the feature.

    These are good examples of the problems with such closed systems.

  • I worked on a DARPA project a few years before this - where we were using CBE as the core for a polymorphic processor (one with an FPGA attached to every IO). We were also gutting PS3s to make mission computers for early unmanned systems - running Ubuntu on top. USAF wasn't the only one - not only were there commercial supply challenges with the PS3, various components were being horded by various nation states. We were pretty sure they didn't even no what to do with the parts, but was a basic attempt to prevent projects like this from getting off the ground.

  • I'm old enough to remember the rumors about Saddam Hussein fusing clusters of 15 PS2s to make a UAV realtime controller [1].

    To be fair, he could have been more upfront about it. Some marketing managers would have shown that at E3.

    [1] https://www.reddit.com/r/AskHistorians/comments/l3hp2i/did_s...

  • Make me think of this cluster of 200 PS3 that was used in 2009 by EPFL to solve the elliptic curve discrete logarithm problem: https://www.epfl.ch/labs/lacal/articles/112bit_prime/

  • The Cell's SPEs were such an inspired design - if you look at CPU architecture, 99% of the complexity comes from pretending there's a flat memory space - data hazards, cache coherency, latency hiding, prediction etc.

    If you make a beefy processor that works like a microcontroller - reading and writing everything from SRAM, and making all main memory DMA explicit - you get all the speed a fraction of the logic budget.

    If you look at a modern CPU pipeline, it's 20ish stages, with most of them about handling the aforementioned complexities, with fetch-decode-execute taking up at most like 6 of them. In MCUs (and SPEs) it's basically all there is.

    Unfortunately they never did figure out how to write programs for such a curiosity in an accessible manner.

    I'm curious if it's worth taking another crack at this architecture, this time with better tooling.

  • I remember reading about the air force buying a lot of ps3's back in the 2000s, and that just cemented to me that information is power and any government will do all they can to own as much of it as possible. IBM just published some quantum breakthrough[1] and my first thought was that it will be used by the military or intelligence agencies. Hopefully by the time this technology trickles down to more corrupt agencies the people will have caught up.

    You might as well be talking about magic when it comes to quantum computing and me, I have no idea if that new openssh standard really does protect against a quantum computer trying to break it.

    1. https://www.techradar.com/news/ibm-claims-to-have-mapped-out...

  • Is there any evidence of the resulting “supercomputer” being used? It’s one thing to buy two thousand PS3s. Another to deploy them into a data center. And yet another to build processes and software that uses it. The challenges involved seem discouraging.

  • The most surprising aspect of this story to me is that the Air Force was allowed to exercise this level of resourcefulness. I just assumed that the procurement process would stipulate 10 year support contracts or something similar.

  • I was there, but not on that project. It worked well. The reasons it was attractive were the cost savings, but more importantly the unique aspects of the Cell processor in the PS3s.

  • I believe they had to jailbreak these to get Linux installed on them.

    Just imagining an Air Force employee (?) browsing community modding forums and such for older firmware and tools, kinda funny.

  • Just think what they're doing with all our phones now.

  • In semi-related news...

    https://www.techspot.com/news/93980-14800-asrock-mining-rig-...

  • Huh I wonder if that's where Jonathan Nolan got the idea from for his show Person of Interest.

  • Which would be a good alternative to build similar cluster supercomputers nowadays?

  • Someone must have had a pretty amazing internal pitch deck to get this approved.

  • Reminds me the scene in Chappy

  • > that cost is about 5-10% of the cost of an equivalent system

    I bet 1,760 chickens cost roughly 5-10% of the cost of a Clydesdale, but I know which one I'd prefer to pull a cart.