Facebook’s Chief Technology Officer offered his ‘sincere apologies’ on Monday afternoon as a global outage of the website along with Instagram and WhatsApp stretched into its fifth hour, costing the company $50billion and CEO Mark Zuckerberg $7billion and counting.
The cause of the outage is believed to have been changes made to Facebook’s systems at 11.50am EST.
It is believed they were caused in error and were not a cyber attack, but Facebook is yet to confirm the cause of the outage.
It sparked global outages that are detrimental to online businesses and decimate the company’s ad-wielding power.
At 3.50pm, four hours after the outage began, CTO Mike Schroepfer tweeted: ‘*Sincere* apologies to everyone impacted by outages of Facebook powered services right now. We are experiencing networking issues and teams are working as fast as possible to debug and restore as fast as possible.’
The cause of the outage is believed to be changes made to Facebook’s Border Gateway Protocol.
The BGP allows for the exchange of routing information on the internet and takes people to the websites they want to access by helping them identify them on what is known as a domain naming system – a directory of websites.
Facebook’s changes included withdrawals which removed its properties from the domain naming system it operates and essentially made it impossible for anyone to connect to the sites because they could no longer be found online.
As the company scrambled to fix the issue, share prices fell by more than 5 percent, reducing the company’s total value from $967billion on Friday afternoon to $916billion by Monday’s closing bell. Zuckerberg’s stake shrank by $7billion.
Facebook is yet to confirm the cause of the outage or give any glimpse of how long they think it may last.
A New York Times reporter tweeted that staff at an unspecified Facebook office were unable to use their keycards to gain access to the premises on Monday. It’s unclear if that is related to the outage or not.
NetBlocks, which tracks internet outages and their impact, estimate the outage has already cost the global economy $160m (£117 million).
Experts say that even once the problem is fixed, it will be a nightmare task rebooting the system because it is so large.
The scandal-battered company’s shares had dipped by 5 percent on Monday amid the outage and after a whistleblower went public on Sunday night with claims of how it regularly places profit above morals.
CloudFare’s Chief Technology Office John Graham-Cunningham tweeted on Monday that Facebook accidentally ‘disappeared’ from the internet after making a ‘flurry’ of updates to its BGP – Border Gateway Protocol.
‘Between 15:50 UTC and 15:52 UTC Facebook and related properties disappeared from the Internet in a flurry of BGP updates,’ he said.
Facebook does not use CloudFare but it runs one of the world’s largest DNS resolvers. When sites go down because of failures in DNS systems, CloudFare tries to repair them.
Usman Muzaffar, SVP of engineering at CloudFare, explained to DailyMail.com: ‘Humans access information online through domain names, like facebook.com and DNS converts it into numbers, called an IP address, computers use.
“From what we understand of the actual issue —it is a globalized BGP configuration issue. In our experience, these usually are mistakes, not attacks.
Border Gateway Protocol (BGP) is the routing protocol for the Internet. Much like the post office processing mail, BGP picks the most efficient routes for delivering Internet traffic.
‘Today, the directions for how to get to Facebook’s DNS server’s addresses weren’t available (and seem to still be unavailable).
‘Without being able to contact the DNS servers, visitors trying to reach a Facebook property, like facebook.com, will not get an answer and so the page won’t load.’
Facebook has not commented on the cause of the outage but spokesman Andy Stone said: ‘We’re aware that some people are having trouble accessing our apps and products.
‘We’re working to get things back to normal as quickly as possible, and we apologize for any inconvenience,’ Andy Stone, the company’s director of communications, said.
Initially, there were reports that AT&T, Verizon and T-Mobile were all down too, however those reports stemmed from people being unable to access Facebook-run apps on their mobile devices.
Kevin King, Director of Communications at Verizon, said there was no outage with the network but that it appeared as though some users were having difficulty accessing certain apps or sites on their Verizon devices.
‘There is a known outage affecting some social media platforms, and that may cause some customers to believe they are having a problem with network connectivity. Our network is operating normally,’ a spokesman for AT&T said.
Cyber security specialist Jake Moore said there is a ‘chance’ the issue could be related to a cyber attack but that it is unlikely.
He PA: ‘There have been many reports and I’m struggling to find out exactly what has happened- I’m reading it could be DNS related, which means there is an issue with the connection not knowing where to go to your device.
‘It could well be a human error or a software bug lurking in the shadows but whatever it is Facebook needs to do its best to mitigate the problem of causing more panic about this.
The biggest problem is fears over a cyber attack but as we saw from Fastly in the summer I would hedge my bets on that not being the case as we’re talking about one of the biggest companies in the world, but there’s always a chance.’
Adam Leon Smith, of BCS, The Chartered Institute for IT and a software testing expert, said: ‘The outage is caused by changes made to the Facebook network infrastructure. Many of the recent high-profile outages have been caused by similar network level events.
‘It is reported by unidentified Facebook sources on Reddit that the network changes have also prevented engineers from remotely connecting to resolve the issues, delaying resolution.
‘Notably, many organizations now define their physical infrastructure as code, but most do not apply the same level of testing rigor when they change that code, as they would when changing their core business logic.’
While businesses that rely on Facebook panicked, Twitter rejoiced in a surge in traffic.
Hello literally everyone’ was the tongue-in-cheek message from Twitter tonight as it took a lighthearted approach to its rival Facebook suffering a worldwide outage.
Social media users meanwhile flocked to Twitter to share memes about Facebook being down.
One shared a picture of Pixar superhero Mr Incredible saying ‘It’s showtime’ with the caption: ‘When most social media apps are down, Twitter be like…’
(DailyMail)