Facebook, your uncle’s favourite source for information on how 5G is making everyone communist, experienced a six-hour outage Monday. And it wasn’t just Facebook — the company’s other major platforms, Instagram and WhatsApp, were also down. We finally got a statement from Facebook overnight about what happened, and the culprit was some configuration changes made internally on the company’s backbone routers.
“To all the people and businesses around the world who depend on us, we are sorry for the inconvenience caused by today’s outage across our platforms. We’ve been working as hard as we can to restore access, and our systems are now back up and running,” Facebook said in a statement published online overnight.
“The underlying cause of this outage also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem,” the statement continued.
Those internal tools apparently included the ability to even access the servers, according to multiple reports.
“Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centres caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt,” Facebook said.
But Facebook assures us that everything should be working normally again, and it has no evidence that any user information was taken. Early on in the outage there were rumours of a state-sponsored attack on Facebook, which turned out to be false.
“Our services are now back online and we’re actively working to fully return them to regular operations. We want to make clear at this time we believe the root cause of this outage was a faulty configuration change. We also have no evidence that user data was compromised as a result of this downtime,” Facebook said.
“People and businesses around the world rely on us everyday to stay connected. We understand the impact outages like these have on people’s lives, and our responsibility to keep people informed about disruptions to our services. We apologise to all those affected, and we’re working to understand more about what happened today so we can continue to make our infrastructure more resilient,” Facebook said in the statement.
Facebook is having a really bad week. And it’s only Tuesday. Who knows what the rest of the week might bring? Oh, right. That whistleblower who smuggled tens of thousands of documents out of Facebook is testifying to the Senate at 10 a.m. today.
Best of luck, Zuck. You’re going to need it.