GCP Outage

  • It looks like that it is a central service @ Google called Chemist that is down.

    "Chemist checks the project status, activation status, abuse status, billing status, service status, location restrictions, VPC Service Controls, SuperQuota, and other policies."

    -> This would totally explain the error messages "visibility check (of the API) failed" and "cannot load policy" and the wide amount of services affected.

    cf. https://cloud.google.com/service-infrastructure/docs/service...

    EDIT: Google says "(Google Cloud) is down due to Identity and Access Management Service Issue"

  • Getting a lot of errors for Claude Sonnet 4 (Cursor) and Gemini Pro.

    Nooooo I'm going to have to use my brain again and write 100% of my code like a caveman from December 2024.

  • Cloudflare is down too. From https://www.cloudflarestatus.com:

    Update - We are seeing a number of services suffer intermittent failures. We are continuing to investigate this and we will update this list as we assess the impact on a per-service level.

    Impacted services: Access WARP Durable Objects (SQLite backed Durable Objects only) Workers KV Realtime Workers AI Stream Parts of the Cloudflare dashboard Jun 12, 2025 - 18:48 UTC

    Edit: https://news.ycombinator.com/item?id=44261064

  • Everything appears to be down as of 18:43 UTC... https://downdetector.com/

  • The status page is green, but there are outages reported: https://downdetector.com/status/google-cloud/

  • What's crazy is that RCS messaging is down as a result of this outage. It shows how poorly the technology or infrastructure was designed.

  • Yes Firebase auth is down and affecting many apps, on Discord and Slack groups tons of others are corroborating. A bit disappointing that there is no post on the status page for nearly 30 mins: https://status.firebase.google.com/

  • Cloudflare Outage also just updated

    > Cloudflare’s critical Workers KV service went offline due to an outage of a 3rd party service that is a key dependency. As a result, certain Cloudflare products that rely on KV service to store and disseminate information

  • Does anyone know of a good dashboard to check for such BGP routing anomalies as (apparently) this one? I am currently digging around https://radar.cloudflare.com/routing but it doesn't show which routes were actually leaked.

    I would love if anyone has any good tool recommendations!

  • thank god hn is hosted on a single bare metal server, free of all this bloat.

  • Smells like BGP since there are services people claim have nothing to do with GCP being affected. OpenRouter is down, Lovable is down, etc.

  • Interesting how I landed here. I was having trouble with Nest. Then I went to Down Detector. I noticed many sites having a simultaneous uptick. Then I came to HN, and found this link at the top of the front page.

  • If Google Chat is down per https://www.google.com/appsstatus/dashboard/, the ability for Google engineers to communicate among themselves impaired, despite SREs having IRC as a backup.

  • > No major incidents

    … Proceeds to show worldwide degraded service level alerts.

  • Sorry, after decades of being hard wired, I just installed a PCIe Wifi6 card on my desktop. Internet took a dive the second I got it connected. Must have done something wrong.

  • This is at least why Claude is dead: https://status.anthropic.com/incidents/kn7mvrgb0c8m

    Also spotify isn't working for me so I assume that's also related.

    These are my most important productivity resources! Sad!

  • Incident report published: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

  • Status page is showing green because GCP admins can't login to change it ;)

  • Status pages at cloud providers aren't usually based in reality -- usually requires VP level political games to actually get them changed especially for serious outages.

  • Look like affect to Cloudflare as well [1]

      Update - Cloudflare’s critical Workers KV service went offline due to an outage of a 3rd party service that is a key dependency.
      
      Jun 12, 2025 - 19:57 UTC
    
    1: https://www.cloudflarestatus.com/

  • Would be comedy if one of the progenitors of this took Sundar’s buyout offer yesterday and let the world burn today.

  • Kinda funny that the top post on HN titled "GCP Outage" links to the Google Cloud status page which shows...no outage.

  • They have updated the status page finally https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

  • Does anyone know if it's region-specific? We're experiencing it and are in us-west-1.

  • https://www.cloudflarestatus.com/ is showing outage, which cause google gcp outage, claude outage, firbase outage https://status.firebase.google.com/

  • This appears to be continuing to cascade over an hour later... wow... more and more services mentioned as completely down on the outage page.

    Kind of nice to not be glued to AI chat prompts for a while to be honest.

  • Everyone is down. Cloudflare has problems too. All auth providers broken.

  • Someone must have checked in AI Generated code :-)

  • Super duper frustrating having the status page being green. Why can't Google do this properly?

  • Wish there existed a decentralized network connecting computers around the world

  • https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

    > Multiple GCP products are experiencing impact due to Identity and Access Management Service Issue

    IAM issue huh. The post-mortem should be interesting at least.

  • Google Maps not loading, thought it was my 4g, go to see if my connection works by loading Hacker News, GCP Outage XD

  • console not loading, storage slow, support forms dead, status page green. no fallback, no real-time alert, was just wondering when it'll start working. whole stack feels brittle when basic visibility tools fail too. everyone’s pointing fingers but nobody has root access to truth.

  • Cloudflare speedtest is down too, I assume because of this?

  • Let's say a typical base service (network attached RAM or whatever) has 99.99% reliability. If you have a dependency on 100 of those, you're suddenly closer to 99% reliability. So you switch to higher-level dependencies, and only have 10 dependencies, for a 99.9% reliability. But! It turns out, those dependencies each have dependencies, so they're really already more like 99.9% at best, and you're back at 99% reliability.

    "good enough" is, indeed, just good enough to make it not worthwhile to rip out all the upstreams and roll your own everything from scratch, because the cost of the occasional outages is much lower than the cost of reinventing every single wheel, nut, bolt, axle, bearing, and grease formulation.

  • Looks like I'm about to start learning which of my time-killing websites are hosted on GCP - The Ringer is down, and since Spotify owns them and is a major GCP customer, it looks like they've been hit by this. CRAZY that the GCP status page is still green.

  • One of these days in which the young engineers learn the concept of 'counterparty risk'.

  • I wonder what the damage ($) for having a good portion of the internet down for an hour or two ;)

  • So frustrating, but here's a link to track status of this outage: https://status.anthropic.com/incidents/kn7mvrgb0c8m

  • Actual incident link posted: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

  • Supabase is also down

  • Just our bi-yearly reminder of our over reliance on cloud providers for literally everything. Can't say there's an answer beyond trying to build more independent tech but we know how that goes.

  • When you deploy code generated by Gemini :D

  • Cloudflare KV is also having an outage. I wonder who is reliant on who here.

  • Google denies the outage. https://x.com/Google/status/1933246051512644069

  • Was just about to do a demo, but Google Meet was down. Tried to use Jitsi as a fallback, but couldn't log in because Firebase was down too. Ended up using a Slack Huddle, lol.

  • love how their status page is green with no issues detected!

  • For us Cloud SQL instances are toast but App Engine Standard instances are still serving requests. Google Cloud console is borked too, mostly just erroring out.

  • Seems like a wider issue at Google than just GCP, the Sheets and Chat APIs are also returning similar "Visibility check was unavailable" errors.

  • some core GCP cloud services are down. might be a good time for GCP dependent people to go for a walk, do some stretches, and check back in a couple hours.

  • Haha, I don't ordinarily spend a lot of time in the Google Cloud Console but just now I was debugging a squirrely OAuth issue with reCAPTCHA failing to refresh several days running. I'm getting this weird page error, and I think, "Is this an issue with my organization? [futz futz futz] Hey wait is GCP actually down?" And it turns out to be the top discussion on HN. XD

  • Can't wait to see how charts are going to look like here on the project we have developed for Maintel https://variable.io/maintel-digital-landscape/. It shows availability across multiple services as a landscape. Expecting to see a lot of spikes tomorrow.

  • Experiencing 504s in Google Meet.

    Google Cloud Console won't load.

  • Getting Gateway timeouts on docker hub. Maybe related? I can pull images.

    Example: https://hub.docker.com/layers/library/eclipse-mosquitto/late...

  • Does anyone know if instance-to-instance networking has been affected? My Redis instance has been throwing a lot of connection errors.

  • BigQuery is completely dead

  • Firebase status page has acknowledged it as a "global issue". https://status.firebase.google.com/

    A contact in google mentioned to me that some bad update to Google Cloud Storage service has caused some cascading issues affecting multiple GCP services.

  • The last few times this happened I wouldn't have thought "So this is the day AI takes over".

    But this time...

  • Spotify was not loading, thought my 5G was bad, used YouTube Music instead without issues. Hmmm...

  • Cloudbuild completely down for us. Getting "Visibility check was unavailable" errors.

  • Any chance this is the root being that so many different services are effected? https://github.com/kubernetes/kops/issues/17433

  • GCP Artifact registry still down... Not accepting image push and showing 500 status code

  • Well this explains the issues I've been having with Spotify through the last hour.

  • Twitch was broken too: https://status.twitch.com/incidents/b79nyp1yhxql

    EDIT: Updated link to point to the specific incident.

  • When Google said GCP is "down", did it affect entire availability zones within a region? For people who designed redundant infrastructure, did your backup AZs/regions keep your systems online?

  • Surprised no one else mentioned "it's always DNS" yet :-)

  • "No major incidents" as of 11:37 PDT.

    https://status.cloud.google.com/

    File that in the status pages worth ~0 category.

  • Related ongoing thread:

    Ask HN: Is Firebase Down? - https://news.ycombinator.com/item?id=44260669

  • I wonder how many SLAs Google blew out today with this outage.

  • Damn you Bart Simpson!

    https://en.wikipedia.org/wiki/Bart_Gets_Famous

  • Our GCP workloads are unavailable across several US regions. The GCP console is intermittently unavailable for most pages.

    Crossing my fingers for a quick resolution.

  • My firebase hosting and firestore db are back online, but GCP console and Google SQL instances are still having serious issues as of 7:00pm UTC.

  • Ahhh, explains why some of my apps are going crazy... Couldn't read a message from my kids pre-school

    Thankfully we use AWS at work for everything critical

  • It looks like more than GCP: outages reported across the board including aws

    https://downdetector.com/

  • if all services at down at once, no one is thinking or mentioning a potential attack on US cloud providers ? (China or Russia) Maybe ?

  • GCP status page now reflect the issues, looks like Google Cloud Dataproc, Google Cloud Storage and Identity & Access Management

  • #HugOps

  • Having issues with services in cloud run as well

  • Ah darn it. My Spotify DJ just stopped working.

  • sentry is down https://status.sentry.io/

  • xAI having problems, Supabase down, Discord can't upload images to share in chat. Seems like a major backbone outage.

  • Same here. Even the page to submit support requests is down.

    Cloud console does nothing.

    They should host their support services on AWS and vice-versa.

  • https://downdetector.in/

  • We're in us-west-1 and seeing issues across Cloud Run, Cloud SQL, Cloud Storage and Compute Engine.

  • I'm able to login to the GCP dashboard, but it isn't able to find any of my projects.

  • Yeah their status page is all green nothing to see here (but all production systems are down).

  • Claude Code is down :( too lazy to do manual conversion from Cocoapods dependency to SwiftPM

  • Even though BigQuery is not listed in affected services, we see errors connecting to it

  • I'm having trouble getting any Street View imagery. Can anyone else confirm?

  • We're experiencing intermittent slowness and timeouts on our GCP everything.

  • npm registry happen to be hosted on gcp, because that seems to be down as well.

  • Root cause has been identified and it's being resolved/monitored now

  • My friends and I are even having trouble getting Rcs text messages to send.

  • Everything except us-central1 is back up - it's recovering now though

  • 2 hour outage at this point

  • this aint looking good yall

  • GPay which is a widely used payment service in India is down as well

  • Meet is also down for me right now. Cannot attend any video calls.

  • Yup, intermittent db connection issues and cloud storage problems.

  • Where are the AI agents?

  • reCAPTCHA affected? I couldn't log into my local utilities website due to a reCAPTCHA error. Downdetector agrees, but I interpret that site as dubious.

  • And THAT, Smithers, is why we wear hardhats on the job.

  • Not just GCP, most of Googles services are out of action

  • Is this the new Y2k?

  • Seems recovering now

  • What is this Touchable Grass stuff I keep hearing of?

  • Not just GCP. AWS and Cloudflare too.

    Did someone screw up BGP again?

  • Can't upload discord attachments from mobile.

  • Google Cloud Storage seems to be down or very slow

  • > Waiting for downdetector.com to respond...

  • Guess they used Jules to code their services :)

  • Storage, CloudRun, Firebase...... All down....

  • Yarn package registry also appears to be down.

  • Maybe cloudflare?

  • if everything down at the same time - No one is mentioning an attack on us cloud services ? ( China or Russia ) Maybe ?

  • Text messaging for android is broken as well

  • Gemini API isn't working for me :/

  • identitytoolkit.googleapis.com is 503-ing on us, my whole customer success team is locked out from our platform

  • when its going to be fixed , i am seeing now more and more services getting outage started with IAM ?

  • YouTube was down for me for some time

  • Text messaging on Android is broken

  • mapbox maps seemed to be down for a few minutes about an hour ago. I wonder if it is related.

  • GKE workloads are also affected.

  • Shameless plug for https://rollbar.com

    Good luck out there!

  • YouTube is also very flakey.

  • when its going to be fixed i am seeing now more and more services are down?

  • I just realized that the reason the status isn't updated is cause they can't access it lol.

  • If LLMs are down work grinds to a halt until they return. Just the new era now.

  • is supabase on GCP ? My Supabase projects are down.

  • internal systems at google are currently broken.

  • kaggle not responding correctly, is it related?

  • Our GCP is down

  • Interesting that all Digital Ocean services are fine...

  • i think it'll be disaster.

  • sheesh so many side-affected issues accross all systems, maybe big tech companies like google shouldn't have laid off all those engineers.. https://www.google.com/appsstatus/dashboard/incidents/Eab7zG...

    but no tech bros, just keep following your ketamine addled edgelord when he did this with twitter..

  • "All locations except us-central1 have fully recovered. us-central1 is mostly recovered. We do not have an ETA for full recovery in us-central1."

  • Now my api can not connect to PostreSQL...

    sslv3 alert bad certificate:../deps/openssl/openssl/ssl/record/rec_layer_s3

  • when its going to be fixed i am seeing now more and more services are down?

  • They've now added this as a major incident - before it just was listed under overview

  • Seems recovering now

  • Can't reach my nest thermometer, but their status page says it's fine lol

  • Well, good luck to all googlers dealing with this, that's not fun :(

  • Hola

  • It's completely nuts that Firebase has this: https://status.firebase.google.com/incidents/ZcF1YDUvpdixZ2e...

    "Firebase Data Connect unavailable due to a known Google Cloud global outage"

    While the Google Cloud status page https://status.cloud.google.com/ says "No major incidents" and everything is green. So Google Cloud know there is an outage but just deem it not major enough to show it.

    Edit to add: within 10 minutes of this post Google updated their status page. More curiously the Firebase page I linked to has been edited to remove mention of Google Cloud in the status and now says "Firebase Data Connect is currently experiencing a service disruption. Please check back for status. ".

  • well this explains so much lol

  • @dang could you merge this and https://news.ycombinator.com/item?id=44260669?

  • [dead]

  • [dead]

  • [dead]

  • [dead]

  • [dead]

  • [flagged]

  • [flagged]

  • [flagged]

  • [flagged]

  • [flagged]

  • [flagged]

  • [flagged]

  • Solana is up ¯\_(ツ)_/¯

  • seems recovering

  • Borg and K8s were fighting for resources, so Gemini decided to take out DNS. Now a sysAdmin has to step in.

    * just trying to add a little humour. pretty stressfull outage. grarr!!

  • The cloud enables you to scale. It allows us to distribute systems across multiple regions and data centers. Seems that this is true for outages as well.

    The PHP application I wrote as a student running on a single self-hosted server had a higher uptime than any of the cloud providers or redundant system I have seen so far. If you don’t need the cloud for scalability, do it yourself and save yourself the trouble and money. Most companies would be better off investing into some IT staff instead of giving away their systems in the hands of some proprietary and insanely complex cloud environment. You are becoming dependent on someone you don’t know, have no control over and can’t talk with directly. Also the single point of failure is just shifting: from your system to whatever system is managing the cloud. Guess one advantage is that you can shift the blame to someone else…