Downdetector seems like a great place to advertise. You have a bunch of people who were forced to abandon the provider of whatever type of service that your company may offer. What a great opportunity to show customers your competing product.
I have a 2-hour-long YouTube video still open, it's continuing to load as I continue to watch it without issues, but new videos aren't working. That's an interesting clue: the CDN isn't down as some people are implying.
Edit 45 min later: Everything appears to be working again, including YouTube TV.
It's officially confirmed: https://twitter.com/TeamYouTube/status/1326681978037444608
On the technology side, I think we're pretty close to the point where small startups can create video hosting platforms. For low traffic, running them through ffmpeg and dropping a <video> tag works pretty well. For higher traffic, I think something like BunnyCDN can be used for a reasonable cost.
But the technology isn't the problem. The problem is nobody is going to use your platform. We need to decouple discovery from hosting. We need a slick, simple aggregator that lets people submit video URLs from multiple platforms, then handles recommendations. If your video has an equal chance of blowing up regardless of where it's hosted, it opens the door for competition with YouTube.
doing a `curl -v https://redirector.googlevideo.com` (the host you'll connect to before getting redirected to the actual mp4/m4a streams) opens a tls connection (and gives a cert), but it then hangs for two minutes, before returning a 502.
i'd wager some database storing info about the streams crashed :^D (given that it also crashes on /, it's more likely something else)
Downdetector also shows increased error rates for Search, Play, Facebook, Netflix, Twitter, Spotify, Twitch…
What’s going on?
Was wondering so looked up how much YouTube lost per second.
> YouTube ads generated $15.15 billion in revenue in fiscal year 2019.
So plugged into wolfram alpha:
https://www.wolframalpha.com/input/? i=15.15+billion+dollars+per+number+of+seconds+in+a+year
They lose about $480 per second or $1.7MM per hour. Yikes.
Too bad, the popup for youtube premium still works.
Look at the charts on downdetector homepage - multiple major sites all got hit by something simultaneously and are all trending back down almost identically.
Seems like just the video player is down - homepage loads with thumbnails, and when you click on a video, everything but the video player renders. What I find interesting is that scrolling through a video still displays closed captions and even the thumbnail preview of the timestamp you're hovering over. Curious if anyone know what causes something like this?
Same here in socal, and look at those DownDetector report numbers[0]. Baseline 30, now almost 200,000 reports. I wonder what could be going on to cause this level of global service degradation.
Perhaps related, I uploaded a video earlier today that took nearly 4 hours to process, though it was only ten minutes long.
Reuters: YouTube back up after worldwide outage
"We're so sorry for the interruption. This is fixed across all devices & YouTube services," YouTube said in a tweet https://bit.ly/36r4sjz, without explaining what had caused the outage.
Google did not respond to a Reuters request for comment on the outage.
https://www.reuters.com/article/us-youtube-outages/youtube-b...
YouTube's tweet: https://nitter.net/TeamYouTube/status/1326709701526974464
Sorry... I think I broke YouTube when I converted my Google Music account over to YT. Happened at the exact same time! /s (but really at the same time)
Now that YouTube is back up I take this opportunity to put this thread to a good use and ask what does it take to create an alternative to YouTube?
Would love your opinions/wish list/must haves.
Election production freeze ending?
I wonder how many little children were throwing fits because YouTube was down. And I mean literal children like my friends toddlers with tablets.
I was trying to load a workout video, and it worked on my phone but I wanted to cast it to my TV. I turned off/on my router and Wi-Fi, reset chromecast, restart the phone, restarted the laptop. I was about to just try to do it from my laptop, did a quick survey of the home network, and then the TV started working again. Haha my workout was delayed about 20 or 30 minutes
5 bucks on BGP
This will be interesting and humbling for 'resilient services' advocates.. Nodes can be down but services will not go down..
I'd want to know what caused this. I'm guessing they're running on kubernetes, but it too early to tell what actually must have failed..
clearly there are points of failure and not sufficient backups to that point of failure..
the site loads fine for me but their content CDN seems to have given up on serving videos.
The play store appears to not be serving any downloads. Sounds like some serious CDN issues.
I vpn'd around, this appears to be global. While reading something unrelated my video suddenly started playing.
I think it's load balancer related for whatever is serving their videos
EDIT: Once you have a connection you appear to keep it the entire way through (at least it did for me with youtubedl). Something related to connection queueing...
On client side looks like requests for the video stream return 502 'bad gateway'.
I actually get back a little webpage it says:
502. That’s an error.
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds. That’s all we know.Cool, I came here to confirm that YT broken and not my local setup :-) HN is pretty reliable sometimes.
The most interesting thing about the YouTube outage for me was that attempting to play a video would result in my cooling fans spinning up to max while the loading spinner looped in the background. Closing the tab they spun down to normal levels.
There wasn’t anything unusual about my CPU behavior when this happened, I can only assume it was GPU or some other part of the system (unfortunately I was too busy to do more than basic troubleshooting at the time.)
Youtube music seems to be down too.
Pages load, but videos don't.
This is why youtube-dl is needed.
Does anyone want to try to calculate how much money google is losing every second YouTube is down? It must be astronomical, on the order of Amazon.
BitChute is down also.
Unless BitChute is hosted in Google Cloud and this is just a Google Cloud problem, I have a feeling something major is being censored.
Interesting how Instagram was also down for multiple days just recently [0]..
[0] https://www.businessinsider.in/tech/apps/news/instagram-down...
Same here in socal, and look at those report numbers. Baseline 30, now almost 190,000 reports. I wonder what could be going on to cause this level of global service degradation.
Perhaps related, I uploaded a video earlier today that took nearly 4 hours to process, though it was only ten minutes long.
So we've already broken 5 9's of uptime, will we also break 4 9's as well?
Could this be related to people running a Takeout after the Photos announcement?
With the YouTube app I figured out we can watch new videos by starting a download to the local device... my 3 year old was starting to get very nervous that our bedtime routine was going to be disrupted
Interestingly, trying to install the Chrome browser now will fail.
So is speedtest.net. I wonder how they're related (if at all)
This is why I was having issues watching Youtube 1.5 hours ago. I thought it was my internet, kept doing all kinds of stuff because I was having slow internet in general.
I know there's a whole "infrastructure as code" and "immutable infrastructure" thing happening, but how about "revertible"?
Sometimes I wonder why other big tech companies don't even try to create something to compete with YouTube.
YT have been the only player in the game for far too long.
Honestly, people should be double-uploading videos to BitChute. We all instinctively hate Google, and distrust them already so their demise is sure.
I’m able to successfully get a video to load and play by selecting it from the recommended videos of a video that fails to load.
Edit: looks like it might be back up now.
Huh, weird. I was just able to start a video with youtube-dl + mpv (it took a few tries), but I still can't access anything from the website.
Google Play Music/YouTube Music are slow to load streams for me, but do eventually play.
EDIT: just kidding - playback is intermittently hanging now.
my browser shown an CORS related error, its weird
Oh! And I thought it was something with my setup. Didn't want to fiddle with it atm, and just closed it. Phew! :)
I wonder if it is due to Alibaba's 11.11 indirectly DDOSing them because of so much new livestreaming load.
redirector.googlevideo.com/videoplayback works again, but it looks like they just lost metadata servers. No metadata loads on video pages, but if you happen to have pages opened from the time they still worked you can play videos fine.
EDIT: and its all back up to working
Video playback just spinning here.
Google Play Store also appears unable to download apps, in Maryland at 7:40p EST.
Gmail mails cannot be loaded either. While the frontpage seems working.
So is speedtest.net. I wonder how they're related (if at all)
502 bad gateway (21s response time) for redirector.googlevideo.com
Is just a temporary degradation, probably...
Edit: Err, probably more serious
Still down in TX. Noticing other sites offline as well.
Back up. Back to watching Janey Godley videos..
South East Asia region seems to back to normal
Was just about to post this... pretty crazy
Mixcloud is down as well. CDN is my bet.
socialblade seems to be broken now for all searches of channels not already in their database.
I thought that using kubernetes would have solved these issues of high availability no?
seems like these web services always have to go own eventually.
YouTube music never stopped working.
Youtube DL and PeerTube here we come
Okay, but why was Youtube down?
Down in Thailand at the moment.
Youtube TV is down for me too
it works now edit: it's down again, i can't even see the play button
Down in New Zealand. Perhaps a failed algorithm update with the intention of breaking youtube-dl?
Still dead in California.
Still down in Scotland
Back now in India
down across australia
Is just a temporary degradation
Probably...
Works for me now.
Back now in NZ
Darn, needed to watch Tucker Carlson.
What a moment!
Still down in Serbia.
IT'S BACK
No it is not. Flagged.
And cue all other major services going down in the next few days.
If you’re having trouble watching videos on YouTube right now, you’re not alone –We’ll follow me @adys.7 for updates.
Recalling mr.Stallman. This is a good reminder about what can happen with all of your online projects/ data/disk storage/letters/photos/notes once it stops working ...I never stored things online.
Microservices in action! Can visit the site, see comments, but can't play videos. Go to twitter, check the latest tweets containing "youtube down", thousands of results coming.
Now I know why I used to download all my favorite videos to USB/SSD!
With the amount of attention around this outage, it feels like YouTube has become completely ubiquitous in modern society. I can't imagine a future in which YouTube just ceases to exist. The company has already become tied to the livelihoods of so many people, and has spawned too many significant subcultures.
It's a strange feeling to be alive just after all of the significant technology companies were created and started to gain traction. YouTube has only existed for fifteen years, but it might continue to exist for centuries. The demand that has arisen in people to watch videos online is probably not reversible so long as maintaining a video streaming service is still physically possible. If not YouTube, something else would probably fill in such a void if it ever appears, barring the Apocalypse.
It gives me a lot of conflicting feelings. Twenty years ago, few people would have known they wanted the unique content that YouTube offered that wasn't available at the time, like livestreamers or swathes of content about every single plane crash or minor video game mechanic that can be talked about. But now, thousands have become hooked on them. I can't remember the details, only that I've watched them at some point. I wonder where the time would have went if we were not enterprising enough to invent big data and portable streams and engagement metrics. And for me personally, I can say that I "like" a lot of content, but I'm reasonably certain that's the kind of thing some portion of drug addicts would say about their habits. Not much of it actually helps me.