There are a lot of things to consider when it comes to keeping a data center in operation, and for many tech decision makers natural disasters are low down on the priority list. Then come hurricane seasons like the one that brought us Harvey, and is now bringing us Irma, Jose, and Katia.
Hurricanes aren’t the only thing to worry about, however. Wildfires are raging across the western United States, putting many businesses at risk. Earthquakes are always a potential for Silicon Valley, and plenty of low-lying regions can flood with little notice.
So while hurricanes are a huge concern, these tips for protecting data centers from natural disasters apply equally to everyone.
Businesses that host all their data in one center, whether locally or in the cloud, are only inviting disaster, Forrester infrastructure and operations analyst Naveen Chhabra said.
“Data should be redundantly hosted at multiple locations that are in different geographic risk regions,” Chhabra said. “These don’t need to be across the country,” he added, “but they do need to be far enough apart that a disaster in one center won’t impact another.”
Plan and test
It’s said that the best laid plans often go awry, but the chances of plans failing definitely decrease with practice and testing.
Chhabra says that having a disaster plan is essential, but even moreso is testing it regularly. “40% of businesses never run risk assessment tests, which leaves them vulnerable when the unexpected happens,” Chhabra said.
He added that only 19% of businesses test their emergency plans twice a year or more, which should be a bare minimum.
“Weather outages are becoming the norm,” Chhabra added, “and they don’t need to be.” Were businesses testing and preparing properly, he said, uptime could be much better.
Take physical precautions
In an ideal world your data center would be located in the center of a building, above the bottom floor, and away from any windows. If that isn’t the case you need to make quick decisions to protect your data center.
- If your data center has windows, remove any light objects from nearby that could become airborne and shatter them.
- If you are able to move any hardware to a safer location, do so. This varies by disaster type, of course. For example, if floods are possible, move equipment to higher floors, while those in earthquake zones should keep them lower.
- If data center flooding is possible, be sure there are pumps installed to eliminate water before it can accumulate. Generators should be set up to keep pumps running if power goes down.
- Make sure your fire suppression system is fully functional. If power doesn’t go out, it’s always possible there could be a fire.
- If there isn’t time to send essential data to the cloud, back it up physically in a portable format and entrust it to someone leaving the area.
Be ready to improvise
If you don’t have solid disaster plans, haven’t tested the ones you do have, or are otherwise caught off guard by a coming natural disaster you need to be creative to save machines, protect data, and ensure continuity.
- If time permits, clone servers to a cloud location. They don’t even need to be ready to run if you’re in a huge hurry, but ideally you’ll be able to transfer operations to those machines if something goes wrong in the data center.
- Implement monitoring responsibilities and share them between team members. There should be at least one person monitoring equipment status at all times during a disaster situation.
- If you have a repository or wiki of disaster recovery information (server settings, database info, etc.) give local copies to everyone who will be monitoring from off-site.
- Be sure everyone involved in the disaster recovery change has multiple forms of communication at their disposal. You don’t want a key member to be unavailable if the worst happens.