From 'Routine Tests' to 'Bug in the Server': Facebook's Worst Outages

Facebook’s engineering team has issued an apology alongside an announcement of the return of its services after the networking giant and its apps, including Instagram and WhatsApp, was hit with an outage on Monday that lasted approximately six hours.

Three major social media platforms screeched to a halt on Monday afternoon, underscoring yet again how lives and businesses can be disrupted when apps used by billions of people worldwide go dark. Facebook and its apps, including WhatsApp, Instagram, Messenger, and OculusVR stopped working at around 11:45 a.m., ET.

After several hours of error messages and outages, the engineering team of the company that offers messaging, livestreaming, virtual reality and many other digital services, issued an apology, alongside an announcement of the return of its services.

This downtime, attributed on this occasion to a “configuration change to its routers”, is hardly the first experienced by the company over the years.

Facebook went dark in 2008 when it only boasted around 80 million users. A “bug” was blamed for the outage, that at the time affected many of its 80 million users.

The network was down for about a day at the time.

‘Routine Test’

There was a more brief outage that lasted approximately a half-hour on 12 November 2018, which the company attributed to a "routine test". It was on a Monday afternoon that the social network went down, showing users attempting to access its site broken error messages that read "Sorry, something went wrong. We're working on it and we'll get it fixed as soon as we can."

Icon of Instagram social media as seen on a smartphone screen

© Sputnik / Natalia Seliverstova

/

Go to the mediabank

While the incident appeared to have knocked out the company’s main website for some users and affected its Messenger app, Instagram and WhatsApp seemed to dodge the difficulties. After going offline for around a half hour the social network appeared to be working again at around 1:30 p.m. ET.

In a statement, a Facebook spokesperson said the outage was the result of a "routine test". “Earlier today, a routine test caused users to have trouble accessing or posting to all Facebook services including WhatsApp and Instagram. We quickly investigated and restored access for everyone. We’re sorry for the inconvenience," read the statement cited by USA TODAY.

‘Bug in the Server’

Popular social network site Facebook, along with Messenger and Instagram, experienced an outage on 20 November 2018 that rendered its services unavailable to users across the United States as well as parts of Europe and South America.

The outage affecting Facebook and Instagram was caused by a “bug in our server”, the company said at the time.

‘Server Configuration Change’

One of the worst outages experienced by Facebook was in 2019, when service at its various properties, including Instagram, WhatsApp and Messenger, blinked out for almost 24 hours mid-morning. According to Facebook, a “server configuration change” at 11AM ET on 13 March set off a cascading effect of problems through the company’s network, and affecting users around the world.

We’re aware of an issue impacting people's access to Instagram right now. We know this is frustrating, and our team is hard at work to resolve this ASAP.

— Instagram (@instagram) March 13, 2019

While for some the services were completely inaccessible, in other cases just certain features like stories or direct messaging stopped functioning, while the feeds continued to work. Connectivity platform, Workplace, similarly suffered issues, owners of Oculus VR virtual reality gaming headsets were also affected.

Yesterday, as a result of a server configuration change, many people had trouble accessing our apps and services. We've now resolved the issues and our systems are recovering. We’re very sorry for the inconvenience and appreciate everyone’s patience.

— Facebook (@Facebook) March 14, 2019

Sites like Tinder and Spotify, relying on Facebook credentials, were unable to authorise logins. 24 hours later Facebook attributed the incident to a “server configuration change.” “We’ve now resolved the issues and our systems are recovering. We’re very sorry for the inconvenience and appreciate everyone’s patience,” announced the company on Twitter at the time. It also underscored that the downtime was not due to a hacking attempt or a cyberassault.

‘Disruption to Network Traffic’

In the latest incident, Facebook and its apps, including Instagram, WhatsApp, Messenger and Oculus, were inaccessible for hours on 4 October, starting to display error messages around 11:40 a.m. Eastern time, according to reports of users.

Facebook’s community of nearly 3 billion accounts disappeared from the internet for over five hours, before some of the apps gradually began to show signs of life.

*Sincere* apologies to everyone impacted by outages of Facebook powered services right now. We are experiencing networking issues and teams are working as fast as possible to debug and restore as fast as possible

— Mike Schroepfer (@schrep) October 4, 2021

As Facebook Chief Technology Officer Mike Schroepfer confirmed that services were beginning to return, Instagram's public relations team issued a similar statement via Twitter, acknowledging that many people and businesses had been impacted by the outage.

According to the company, the reason for the incident was changes to its underlying internet infrastructure that coordinates the traffic between its data centres. A routine BGP update had apparently gone awry, removing the DNS routing information that Facebook requires to allow other networks to find its sites.

“This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt,” stated Facebook.

The platform eventually restored service after a team reportedly accessed server computers at a data centre in Santa Clara, Calif., according to sources cited by the New York Times. A Facebook update also dismissed speculations that any user data had been comprised during the global downtime incident.

“Sorry for the disruption today — I know how much you rely on our services to stay connected with the people you care about,” read an apology posted by CEO Mark Zuckerberg.

In this Oct. 17, 2019, file photo Facebook CEO Mark Zuckerberg speaks at Georgetown University in Washington.

© AP Photo / Nick Wass

However, the timing of the incident has invariably raised questions as it comes in the wake of a whistleblower report that exposed alleged unethical practices by the company.

Frances Haugen, a former product manager at Facebook, which has 2.8 billion active users as of July 2021, who left the company in May, revealed thousands of pages of corporate inquiry suggesting Facebook concealed evidence of its platforms spreading harmful information.