Disaster 101: How to Plan and Run a Disaster Exercise (Part 2)

June 28, 2005
Part 2: The role of the exercise director, plus creating surprise scenarios and using what you learned to improve

[Editor's Note: In the first part of this Disaster 101 tutorial on planning and running your emergency training scenario (be that a security emergency or a natural disaster), William Comtois discussed the pre-exercise preliminaries -- planning, communications, and measurement strategies. In Part 2, he gets down to the nuts-and-bolts of running these exercises and getting the most out of the after-action analysis.]

Running the Exercise

Once your exercise is underway, keeping things on track falls into the hands of the exercise director. Arguably the most important position while the exercise is running, the exercise director should have direct oversight of the participants' activities as well as oversight of the work being done behind the scenes by support resources.

A chain of command should be in place to ensure that issues and vital information about the exercise promptly flow to the exercise director. As issues arise (it's only a matter of time - every exercise will have its share of challenges) the exercise director should be ready and able to act decisively and wisely. Absent such a role, the exercise will invariably be at risk of running astray and missing the objectives behind the exercise's conception.

The exercise director need not be the only role borne by the designee. For instance, in the case of a discussion-based exercise, the facilitator might appropriately serve as the exercise director. Large operations-based exercises are more likely to call for a dedicated exercise director.

One noteworthy function that the exercise director may also perform is the delivery of injections. An injection is a simulated event or information that is presented to one or more of the participants with the intention of adding an element of surprise. Examples would be: "smoke has been detected in the back-up power room", "the nightly news team has just arrived and is demanding an interview with the CEO", "employees on the 12th floor are reporting symptoms of nausea and dizziness", and countless others. Injections may be delivered according to a schedule defined before the exercise commences or on an ad hoc basis at the exercise director's discretion. Injections may be used to initiate the exercise by informing the participants that the first event in the scenario has occurred; Injections can also be used after the exercise has started to steer the course of events in a certain direction or vary the level of difficulty in real time.

In the case of exercises that extend beyond a single room, another important role during the exercise will be that of the observer. Depending on the size and geographic dispersion of the exercise, multiple observers may be needed. As a general rule, an observer should be present at each location where participants are stationed and the actions or decisions performed at that location warrant recording and evaluation by someone other than the participant(s). While such coverage with human observers may be prohibitive, various electronic recording and information technologies might offer a suitable alternative (for more details on technologies for observers, see the SecurityInfoWatch.com article Disaster Exercise Management, Part 3 of 3, Information Technologies).

In many cases the observer may be required to perform not only recording of participant activities but also real-time evaluation. Examples would be:

(1) comparing tasks performed against a checklist of necessary tasks and
(2) rating the participants' skills according to a standardized rating scale.

It is important to ensure that the observer's skills are commensurate with the recording and evaluative tasks required of them. Regardless of observer skill requirements, well before the exercise starts the observers should be identified and assigned to observation stations, given instructions for the observations they will be required to make, and told to whom they should submit their recorded material once the exercise is over.

Related to the observer's evaluative function are performance measures. Performance measures provide a consistent means of evaluating participants' conduct during the exercise. Perhaps the most common set of measures pertain to timeliness. Examples would be "minutes elapsed from the call to the first security guard arriving at the scene", "time to evacuate 90 percent of the building's occupants", and "number of days to restore the company's stock to its pre-disaster level". Other measures can pertain to efficiency, capacity and efficacy. The set of plausible measures can be quite large, and because of this an entire article has been written by the author on this topic (see the SecurityInfoWatch.com article Disaster Exercise Management, Part 1, Performance Measures).

After the Action

Following the exercise, all observations, recordings and other sources of performance measurement should be collected and compiled. Depending on the size of the exercise and how many observations and recordings were made, this task may be quite laborious and should be planned for accordingly. It is generally a good idea to collect the information immediately following the exercise so as to allow a prompt after-action analysis.

The post-exercise analysis and feedback of results is commonly referred to as "after action analysis", "debrief" or a "hotwash". This step may necessitate a meeting of the exercise director, observers and certain support personnel to discuss and distill the findings down to a concise set of actionable recommendations. The findings may comprise assessments relative to expectations of the company's executives, employee's, shareholders, customers, company auditors and/or government officials. In general, the assessment should address performance in terms of timeliness, effectiveness and efficiency (see above cited article on performance measures for examples).

The findings of the analysis should be communicated to the participants in one or more "feedback" sessions. The session(s) should occur as soon as practical after the exercise and give the participants a clear assessment as to how well they performed and what areas they need to improve upon. Various techniques may be used for this purpose. At the discretion of the exercise director, the feedback may be given in reference to individuals, teams or an entire organizational unit. For example, it may be useful to play back audio or video recorded during the exercise and have group discussions with constructive criticism from the exercise director or observers. Summaries of observers' evaluations may also be presented followed by additional commentary from the observers. Self-assessment by the participants may be another effective performance rating tool.

The exercise itself should also be evaluated in terms of how well it met, or did not meet, its objectives. Certainly, the quality of the learning experience would be among the criteria of evaluation. Feedback from the participants will be instrumental in this assessment and should be actively sought by the exercise director and his team.

In cases where significant areas of improvement are identified, it may be appropriate to develop a plan to close these gaps. A variety of gap-closure steps may be advised. Examples could include the need for additional training of select participants, or the procurement of new safety equipment, or even the scheduling of further exercises to zero in on the "problem" areas. A repeat of the exercise might even be included in this plan so that after so many months the participant's will be tested again under similar or identical circumstances to verify closure of the gaps. Agreeing to specific dates by which key milestones will be met is a good idea before the session adjourns.

Conclusion

Exercises offer a powerful tool for preparing your organization for the risks of this day and age. When planned and carried out in a well-orchestrated manner and with a clear purpose they can give the participants a great learning experience that is second only to a live disaster.

About the author: William Comtois is managing director of Varicom, Inc., a consultancy and software company specializing in homeland defense and service logistics. He has over 20 years of experience in applying leading technologies and innovative process management practices to business and defense solutions. Over the past fourteen years, his work has focused on large service companies where he has lead numerous performance improvement, training and process management initiatives that have resulted in major breakthroughs in financial performance, service levels and disaster preparedness. He can be reached by phone at (212) 561-5782 or email at [email protected].

(c) Varicom, Inc.

Related Stories: