On Oct. 9, 2019, Pacific Gas & Electric began intentionally cutting power to hundreds of thousands of customers in an effort to limit wildfires in Northern California. The move, which affected parts of the San Francisco Bay Area, also put a substantial number of mission-critical servers at risk of data loss.
2018 California wildfires. Credit: TheGhostofElizabethShue/Imgur.
“We took this step to ensure safety as a last resort, and we are committed to reducing the risk of catastrophic wildfire events,” Sumeet Singh, vice president of PG&E’s Community Wildfire Safety Program, said Wednesday, per CNN.
The intentional power outages are an unconventional but arguably reasonable response to diablo winds, which are linked with the most destructive wildfires in California history. As those winds subside on Friday, Pacific Gas & Electric intends to begin restoring power after checking its systems for damage — but for some businesses, that’s not much consolation.
How Power Outages Can Cause Server Failures
Over the last two days, Datarecovery.com has received an influx in new server recovery cases, which have failure scenarios directly linked to the intentional outages.
Modern computer servers typically use redundant arrays (RAIDs) to protect crucial data, but they’re also designed to operate continuously. When power resumes after an outage, the servers attempt to start; when frequent brownouts or blackouts cause repetitive startups, the servers can encounter data loss.
“We’re seeing two primary issues,” said Ben Carmitchel, CEO of Datarecovery.com. “First, data corruption, which can occur when read/write operations are interrupted. We’d expect to see those following any power outage. Second, we’re seeing physical hard drive failures and controller damage, likely caused by the power inconsistencies.”
Each year, Datarecovery.com sees more data loss cases during times of inclement weather.
“We’re used to receiving more personal hard drives and servers during the Spring, when thunderstorms cause these types of outages nationwide,” Carmitchel said. “This is the first time I can remember where we’ve seen a noticeable increase in data loss due to an intentional power outage. Then again, intentional power outages are rare, especially on the scale of the Northern California outages.”
Preparing Servers for Power Outages
A hard drive badly burnt in a house fire.
To prevent server damage and data loss, businesses should have secondary and tertiary power supplies for all crucial systems. An appropriate data backup strategy is also important.
“The entire point of disaster recovery is to plan for situations like this,” Carmitchel said. “Make a backup plan and make sure it’s appropriate to your business — if a single day of lost data will cost you $10,000, you can certainly afford to spend $5,000 to set up daily backups. Check that your backup strategy is working, and be prepared to execute it if your area is frequently threatened by natural disasters.”
Server issues can affect any business at any time, and while intentional outages are frustrating, Carmitchel says that these types of data issues are minor compared to the costs of a wildfire.
“We also receive fire-damaged cases when wildfires occur,” he noted. “Those are often recoverable, but given the choice between the two failure scenarios, we’d much rather work on servers with relatively minor issues.”
If your data has been affected by a power outage, call 1-800-237-4200 to discuss data recovery options.
