04-10-2021 19:47 - edited 04-10-2021 19:48
04-10-2021 19:47 - edited 04-10-2021 19:48
on 04-10-2021 21:13
on 04-10-2021 21:13
It's going to be a long day for the Facebook Network team, as they physically have to go to site, as all there remote tools are down @blissgirl
Some is running to the DC's as quick as they can to hopefully get in, and start fixing the problem...
on 04-10-2021 21:47
on 04-10-2021 21:47
04-10-2021 22:09 - edited 04-10-2021 22:11
04-10-2021 22:09 - edited 04-10-2021 22:11
Say bye bye to Facebook Inc
on 04-10-2021 22:33
on 04-10-2021 22:33
on 04-10-2021 22:36
on 04-10-2021 22:36
Facebook Services are showing signs off life, not the public facing bits, but the stuff I like in DNS, and BGP Routes
on 04-10-2021 23:59
Facebook is back on now. Just checked.
Veritas Numquam Perit
on 05-10-2021 00:48
I'm guessing change control and configuration management were absent in today's change along with the reversion plan, probably written somewhere on a post-it pad posted in the data centre toilets wiki. Where they couldn't access as some bright spark decided to link all the electronic doors to a long forgotten steam powered beige DNS box that "just works" located under the security guards desk currently being used as a foot warmer.
I can imagine the risk section copied and pasted from the last one as "some users may experience an outage and reversion will take a couple of minutes. Some users may experience a brief outage in this time."
The more likely scenario is someone took out a cable and randomly put it back in to an empty port.
Or the more conspiracy theory led scenarios: distracts from the whistleblower about to give testimony and not forgetting the panama papers stuff being released which will no doubt feature a certain Mark Zuckerberg's "donations" into the public eye.
on 05-10-2021 12:43
Been trying to find where I saw this, but failing spectatularly...
Please take with a pinch of salt as this is unlikely to be true!
A message was sent to the FB servers basically saying that they no longer exist on the internet.
This meant that trying to remotely turn them back on wouldn't work as they weren't there anymore!
Unfortunately the same server controlled access to the FB building, so no one could use an access card to get in or out.
So the people inside the building who had casued the problem couldn't get out, and the ones outside who knew what the problem was and how to fix it couldn't get in!!
Please note, this is not customer services and we cannot access your account. Do not publish personal details (email, phone number, bank account).
Link to our guide on how to contact them can be found here