It’s not just the resiliency of the citizens of New Orleans that has earned the right to be celebrated this hurricane season. Businesses should also be celebrated for their resiliency. I recently spoke with executives from Cooperative Processing Resources (CPR) and LifeShare Blood Centers about the ways they avoided severe downtime during Hurricane Katrina, and bounced back as even more resilient organizations. Kate Campion, president of CPR, based in Dallas, Texas, had offices scattered throughout the New Orleans area. CPR had one office that was impacted directly by Hurricane Katrina; it was underwater and unable to reopen for a few months. Working with Sungard Availability Services, it was able to store, retrieve and recover all company data. Data accessibility was not impacted, and it experienced no downtime due to the storm.
Ric Jones, CIO of LifeShare Blood Centers, is based in Shreveport, Louisiana. During Hurricane Katrina, LifeShare Blood Centers was forced to shut down a number of regional centers in Louisiana. At the same time, blood demand increased substantially while donors were unable to give blood as facilities were closed. Jones says: “It is actually only after we experienced downtime during Hurricane Katrina that we decided it was time to find a solution for our critical IT infrastructure – we found that solution through a small Oklahoma City-based vendor. But years later, data backlogs began to occur, and we realized we needed a better and more reliable solution. We now work with Sungard AS to protect critical applications and to ensure data is constantly backed up and protected.”
What lessons did you learn after Hurricane Katrina?
Kate Campion: Katrina taught us how documenting a Disaster Recovery Plan is one thing, but that does not give you any indication that it will work. The fact that our Disaster Recovery Plan worked extremely well in the case of our New Orleans office, it brought the realization to our member agencies the real importance of having a plan. A Disaster Recovery Plan written on paper is one thing, seeing that it worked seamlessly is another story. When Katrina hit New Orleans and our member agency’s main office, no one knew the full extent of the damage for days. However, we knew immediately that there was no electricity at the main office. This is when the Disaster Recovery Plan kicked in right away, by routing the remote offices to Sungard AS directly instead of running them through the main office since it was not accessible. The re-routing happened in less than one hour, and the remote offices could access and continue to work, helping their customers without very much delay at all. The administration staff that worked out of the main office was switched over to access the database from their homes, and they were able to continue to do disbursements and make critical payments on behalf of their customers without disruption.
How you made your organization more resilient since Katrina?
Kate Campion: CPR’s member agencies that are currently being hosted by Sungard AS all breathed a sigh of relief knowing that the Disaster Recovery Plan they had in place was fully tested and passed with flying colors. You never want to have to use a Disaster Recovery Plan, but when it is required, and put it to the test the result was “business as usual” even though the main office and other areas leading to the office was still underwater for over a month.
What lessons did you learn from Hurricane Katrina?
Ric Jones: That we needed a partner to provide IT resiliency services for our critical infrastructure systems – especially our blood banking programs. These were especially vulnerable – in fact, even the occasional lightning strike would wreak havoc. One such case occurred when a blood bank in Lake Charles, Louisiana, went offline for four days after a lightning strike. We learned that a stable solution was more important than ever as our data systems continued to grow – our donor roster is more than 150,000 deep, and we collect and process about 500 donors a day.
What would you have done differently before Hurricane Katrina hit?
Ric Jones: We would have evaluated stable, reliable vendors who perform annual testing and provide backups. One recent instance that illustrated the value of backups was in 2010, when an unborn baby in England needed rare blood. We were able to find a matching blood donor in their system based on the testing information they had on file. The donor was in Arkansas on vacation but came to Shreveport, donated blood and within 48 hours the blood was on its way to England. This vital research information would be in jeopardy if we didn’t have a disaster recovery backup system.
