DE-Cix – Power failure leads to Downtime in germany

  • Kinda fun to see the increase in AMS-IX traffic: https://ams-ix.net/technical/statistics

    Keep in mind this is only one of the many DE-CIX datacenters, they exist in at least 8 datacenters in Frankfurt: https://twitter.com/goebelmeier/status/983453047543214081

    Only those peers who are single-homed to Interxion FRA5 / DE-CIX6 will have experienced an outage.

  • Imagine it being 8pm, you sit in a operating room of germanys largest internet knot with up to 6,027 Tbit/s throughput and then the power goes down.

    I am concerned that this even happened in a place important like this.

    pic related: http://img.pr0gramm.com/2018/04/09/9286baa4b2261184.jpg

    The article states internet was down for users at around 10pm (I expirenced it live). So they might have been on generators for two hours (speculation).

    Shouldn't we decentralize more?

  • We were hit by this.

    In fact, we're hit by this again, down again at 22:09:19 EST for me and still down. Their website is broken too (https://www.de-cix.net/) and their email servers are offline which makes contacting them amusing.

    Not much to be done, really, fortunately the impact for us is very trivial.

  • General home connectivity also suffered, at least a few people (and myself) had connectivity issues, if you search for @vodafone_de on twitter you'll see a lot of people complaining (though I assume it was not just vodafone with issues)

  • Just received an email update from INTERXION acknowledging they did indeed lose power again at FRA5. This leaves us in a bit of a mess.