Did businesses learn anything from Katrina and Rita? You might not think so, judging by reports [1] out of the Midwest of blown continuity plans and desperate last-minute efforts to save IT assets.
Granted, preparing for a radically unexpected event is no cakewalk, but companies seem to have made some pretty basic errors in their risk calculations. "When you hear of a 100-year flood or a 500-year flood, it doesn't mean that regularly once every one hundred or once every five hundred years there's a flood to that level," points out Dr. Louis A. Gritzo, vice president, manager of research, with commercial property insurer FM Global. "What it really means is that every year there is a one-in-a-hundred chance for a 100-year flood, and a one-in-five-hundred chance for a 500-year flood. One of the things this clarifies is that there's a bigger chance of flooding than some businesses expect."
Not all companies were caught flat-footed by the disaster. Telecom and IT provider Verizon Business, which operates an 800-employee global conferencing facility in downtown Cedar Rapids, Iowa, invoked a disaster recovery plan on Wednesday, June 11, after it received a mandatory evacuation notice from city officials. The company immediately started to set up a facility in a hotel on the edge of town, well away from the flood zone. It was under the gun for a 7000-port conference call scheduled for the following Monday.
"The key thing was that we needed to provide satellite interconnect, so we engaged our satellite solutions group to set up three satellite stations on the roof of the hotel," says Dick Price, executive director of business continuance and emergency management with Verizon Business. "We were able to get equipment out to the building, and we had that facility up and operational in about twelve to fourteen hours." The conference call went off without a hitch.
When disaster strikes, companies often end up ditching their emergency response plans because they haven't tested them or updated them, so they don't match the actual conditions on the ground. "That's why it's so important to maintain them," says Price, whose company recently completed an enterprisewide review of some 600 plans. "For instance, we asked a lot of questions about where backup documents are held. Are they on site or offsite? We try to get people to think about what happens if that building is gone when they show up for work tomorrow. Can they still recover, or is everything they need to recover in that building? No plan is any good if you don't have valid, current information and it can't be actually utilized."
Supporting and coordinating the workforce is a major challenge in any disaster. It's not just a question of employees missing work to help the community or to take care of their families and homes, though some organizations fail to plan even for those contingencies. Workers who move out of the disaster zone may simply decide not to come back, especially if they feel that the company has failed to provide the support they need to weather the emergency.
For each of its staffed facilities, Verizon Business designates one person (and a couple of backups) who is responsible for communicating with employees about recovery efforts. In addition, at the national level a disaster team coordinates with the company's environmental health and safety people and the local media to keep employees updated on the status of their workplace and post notices of business closings.
"It's not just a matter of whether our facility is available for them to be able to come to work," adds Price. "We also want to know about their personal issues -- if they're having problems getting hold of their insurance carrier, if they're having to relocate elsewhere because their homes have been flooded or damaged. In Katrina we set up stockpiles of resources -- toiletries, food, water -- for our employees. We try to make sure that they're taken care of."
While not all companies can call on a resource pool as deep as Verizon Business', every organization should ensure that it has a flood emergency response plan in place, says Gritzo. It should do three things:
1. Identify ways to stop the water. What kind of barriers can be put in place? What kind of levees are available or can be temporarily constructed?
2. Prevent the damage. If you can't stop the water, find ways to move valuable equipment and storage to higher levels or areas outside the flood zone.
3. Lay the groundwork for recovery. Decide who's going to do what, and when. And don't overlook the need to coordinate with external disaster relief efforts. Companies should "identify ways they can contribute to the community and ways the community can help the company stay in business, so that people can go back to work as quickly as possible," says Gritzo.
Links:
[1] http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9099558