Anyone responsible for testing your Disaster Recovery Plan? Did you use a "round table" technique or "simulation"? Any advise?

Our IT folks actually do a simulation and go to the back up site to run work. We never ran a simulation of a branch going down but through experience, have re-routed customer traffic as needed due to flooding and power outages, as examples. Real world testing I suppose.

Roundtable (or "tabletop exercises") are a great way to walk-through the business continuity processes with departmental teams to assess readiness to continue business operations during a disruption. I recommend setting your objectives beforehand and structure a scenario-based exercise (with measurable results) to test. Exercises are a great way to identify problems within the plan, changes that had not been captured and increase internal awareness. You may want to start with simple tests (like responding to a power outage) and increase the complexity over time.

We test in a variety of ways throughout the year. We test the back-up site to ensure data can be restored, we test the off-site item processing facility, we do desktop simulations for the bank as a whole, etc. Each department is responsible for developing procedures and testing them annually. The DR committee reviews/approves the plans and tests. Any actual incidents, such as temporary loss of the core system, are documented and reviewed for lessons learned. Be thorough! Report results to the BOD.

Another thing, when we do our disaster recovery test of our computer systems, we have started having various people (not the CIO, DP Manager, or network admin, but lesser experienced personnel) perform the test. This provides a better evaluation of the actual step-by-step procedures.

New this Fall...our Internal Auditor will select a date and scenario (i.e. this building destroyed, these personnel unavailable, etc.) and implement the disaster recovery test on a surprise basis.

A simulatoin is a great way to "exercise" a plan. I have done it in the following way. I assembled the Emergency Management team, just as they would be in a real emergency. I provided them with information about the "emergency situation" and asked them first to provide information about the status of the Bank's operations at that moment...where were they in the processing day....did they have information about key upcoming deadlines, etc. I tracked their responses and continued to provide them additional informmation, just as it would come in from the Assessment Team during a real emergency. You can let them use a "lifeline" if you want...call out to various managers to get information. The exercise lasted about 4 hours and then we debriefed orally, followed by a written debrief. Along with restoring operations at a hotsite, testing data communications, conducting life safety drills and evacuation drills and testing the calling tree, simulations provide both a means of determining if a Business Continuity Plan will work and identifying potential problems with the plan, as well as providing valuable training.

Are there any good software tools out there to help
automate this Dastardly Disaster Recovery Planning process ??

There are some decent software tools out there the help create and maintain the disaster recovery plan (and a number of not so good ones). Regarding the process of developing the plan, the FFIEC lays out an excellent process for: 1) Business Impact Assessment, 2) Risk Assessment; 3) Risk Management and 4) Risk Monitoring. There are a number software tool vendors listed on the Disaster Recovery Journal website (www.drj.com). I have evaluated a number of them. If you would like to get some "off-line" feedback, please feel free to contact me.