In the late 1990s, IT experts worried that a theoretical “Y2K bug” would cause widespread technology outages as the calendar changed from 1999 to 2000. With so many systems linked around the world, many feared that one coding error could end it all. Thankfully, that technology catastrophe never came to pass, but last month a similar cascading failure finally occurred.
On July 19, cybersecurity vendor CrowdStrike pushed out a small update to systems using its hugely popular Falcon platform. The company realized the update contained a coding error and sent out a fixed update just 79 minutes later. By then it was too late. The results were nothing like Y2K, but they sparked what many consider to be the biggest IT outage in history.
While the Y2K failure may have caused email delays and access to ATMs, today’s outages affect everything from health care to food supplies to power grids.
CrowdStrike Falcon is widely used by organizations large and small across industries. The company has earned an excellent reputation thanks to a decade of identifying sophisticated cybercriminals from countries like China, North Korea, and Russia. This has made the company’s platform nearly ubiquitous and vulnerable to critical updates, as has its tight integration with Microsoft Windows OS.
The company inadvertently introduced a logic error that crashed not only Falcon but the entire Windows system. CrowdStrike quickly fixed the issue, but many systems were shut down completely. Offline computers cannot be updated.
Microsoft estimated that the number of Windows devices directly affected was less than 1 percent, but those systems performed critical operations elsewhere and were closely interconnected with other Windows devices.
The cascading failures in the real world mirrored the digital links of the internet. The impact was shocking.
Delta Air Lines, United Airlines and American Airlines had to cancel hundreds of flights, along with many other airlines and airports around the world. Public transportation in New York City, Washington DC and other cities came to a halt. Banks, hospitals and 911 emergency services were shut down. British broadcaster Sky News went off the air.
One insurance company’s analysis shows that a single line of code flaw could cost Fortune 500 companies more than $5 billion in direct losses, and lawsuits are already underway.
A failure of this magnitude should have been one of the most covered stories of the decade, but sandwiched between the assassination attempt on President Trump and Kamala Biden’s quiet coup, it didn’t get much coverage in the media. I read more reports from friends and family members sleeping on airport floors than I did in the traditional media.
The CrowdStrike disaster took many by surprise, but what’s most surprising is that it didn’t happen sooner. The DHS Office of Cyber and Infrastructure Analysis warned in 2016 about how our digital dependency is making us vulnerable. Analysts had envisioned attacks from hostile regimes, but even that assessment was not as dire as the results of the July 19 software update. According to the OCIA report, the most vulnerable systems were:
Cyber-physical technology that allows physical objects to communicate with computer networks. One example is the Colonial Pipeline ransomware attack in 2021, which led to gasoline panic buying due to a perceived gasoline shortage that never actually occurred.
GPS is frequently disrupted in war zones such as Ukraine and the Middle East, where there is growing reliance on the technology for autonomous vehicles, mapping and military targeting.
Smart cities integrate technology and infrastructure to improve environmental and economic efficiency, including functions such as interconnected power grids, traffic management, water supply, waste disposal services, and even government operations.
The Internet of Things connects devices like appliances, cars, and production lines into a larger network. I recently upgraded my house so that my thermostat can send me text messages and my dishwasher can send me emails. Thanks, Silicon Valley.
Cloud technology poses significant security challenges through its myriad entry points. OCIA noted that airlines are especially vulnerable because of their “dependence on cloud systems to schedule passengers, flights, and cargo.” This prediction became reality last month.
While this incident was a disaster for travelers and business revenues, it was a long-awaited wake-up call for the private and public sectors. Our technology must become more resilient, with significantly more backups and redundancies. Large companies must diversify their software portfolios to ensure that one company pushing a single update does not cripple an entire sector.
The CrowdStrike outage revealed the inherent dangers of over-reliance on a single source of technology: putting all your eggs in one basket is never a good idea.
Our adversaries are undoubtedly studying what happened on July 19th and planning accordingly. An attack against our critical infrastructure would play out similarly to the CrowdStrike outage: one failure would set off a cascade of system failures, the sum of which would overwhelm our response capabilities and cause even greater damage.
While the Y2K outage may have slowed email and access to ATMs, today’s outages affect everything from healthcare to food supplies to power grids. While July’s outage was accidental, bad actors quickly exploited it for cybercrime.
CrowdStrike prioritized speed over safety and quality assurance. It took just one global update to bring much of the digital world to its knees. Large companies and governments around the world must use this opportunity to examine how dependent we are on technology, embrace the risks it brings, and prepare for the next cascading outage — because there will be another one.
