Email Storming

Introduction

A recent update from CrowdStrike has caused Windows systems worldwide to crash, displaying the dreaded Blue Screen of Death (BSOD). This widespread issue has left businesses scrambling to restore their systems. If you arrived at work to find your computers down, you’re not alone. Let’s dive into what happened and, most importantly, what you can do next to get back on track.

What Happened

An update from CrowdStrike has caused a significant disruption. The company’s Falcon Sensor product, designed to prevent cyberattacks, inadvertently triggered system crashes on Windows machines. This has affected various organizations globally, including airports, broadcasters like Sky News, and numerous businesses.

Users have flooded forums like Reddit, sharing their experiences. One user lamented, “Wow, stuck in a boot loop, and the entire org taken out.”

The Impact

The fallout from this update is substantial:

  • Airports: Grounded flights and disrupted boarding processes.
  • Businesses: Inoperable systems, leading to halted operations.
  • Broadcasters: Unable to air content, as seen with Sky News.

Official Responses

CrowdStrike engineers are actively addressing the issue. George Kurtz, CEO of CrowdStrike, clarified on X (formerly Twitter) that this is not a cyberattack but a defect in a content update for Windows hosts. Mac and Linux systems remain unaffected. Microsoft has also acknowledged the issue and is working on mitigation actions.

How to Fix the Issue

There’s a workaround, albeit a manual one. Here’s how you can resolve the issue:

  1. Boot into Safe Mode or Windows Recovery Environment (WRE).
  2. Navigate to: C:\Windows\System32\drivers\CrowdStrike
  3. Locate and delete the file: C-00000291*.sys
  4. Reboot your system normally.

What to Do Next

Fixing this problem is no small feat, especially for large organizations with numerous affected systems. Here’s a step-by-step approach to manage the situation:

Immediate Actions

  1. Manual Fix: Apply the workaround manually on each system. This will take time, but it’s currently the only viable solution.
  2. Communication: Keep your teams informed about the status and the steps being taken to resolve the issue.
  3. Support Portal: Regularly check CrowdStrike’s support portal for updates and further instructions.

Long-Term Solutions

  1. Update Policies: Review and possibly revise your update policies to include more stringent testing before deployment.
  2. Backup Systems: Ensure you have robust backup systems in place to revert to a known good state in such situations.
  3. Recovery Plans: Develop comprehensive recovery plans to minimize downtime in future incidents.

Expert Insights

Adam Harrison, managing director at FTI Cybersecurity, notes that manual fixes will be time-consuming but necessary. Ian Thornton-Trump, CISO at Cyjax, emphasizes that while the issue cannot be undone for machines already affected, critical systems might need restoring from backups or using Microsoft’s built-in recovery features.

Conclusion

While the current workaround is labor-intensive, it’s crucial for restoring affected systems. CrowdStrike and Microsoft are working diligently to provide further updates and solutions. Stay informed through official channels and prepare for a meticulous, albeit challenging, recovery process. Remember, you’re not alone in this—many organizations worldwide are navigating the same storm.

For continuous updates and detailed instructions, keep an eye on CrowdStrike’s support portal and communicate directly with their representatives.