Evaluating Defenses in ‘Context’ Part II

(Editor’s Intro) “Welcome to the second part of our two part series on evaluating defenses.  In the first part of this series we presented the concept of the gap between defenses as imagined and defenses as applied.  In this second part of Evaluating Defenses in ‘Context’ the authors will discuss more applications of the concepts and actual results from the field.

Part II: Applying the Concept

T. Shane Bush, CPT
Brian A. Harkins

Applying the Concept to Reviewing Startup Activities Before Startup

This same concept has been successfully adapted to the review of activities before starting new activities or restarting activities after significant changes.  The base concept is that the reviewers determine what defenses are necessary for safe operations.  Once the defenses are identified, the oversight process determines which defenses need to be evaluated in context before startup and which defenses should be should be evaluated in context right after start up.  As before, the intent is to detect and reduce the ‘Gap’ between the ‘defense as imagined’ and ‘defense as applied’ by “Evaluating Defenses in Context”.

Below is the process used to evaluate the start of a tank retrieval system at Hanford in 2009.

  1. Identify defenses in place for the activity: Review the operating procedures and interview subject matter experts to identify the defenses developed of the activity.
  2. Review lessons learned from similar activities: What events, incidents, accidents, have occurred with similar activities in the past?  How have people made mistakes or errors while performing similar activities in the past?  Use the results of this review to evaluate the adequacy of the defenses and to improve the understanding of the reviewer of the context of how the defense could be used.
  3. List the Hazard(s) that the defenses are intended to protect against: For each defense, list the hazards that each defense is intend to guard against.
  4. Review defense in the context that they will be used to determine if the defense will defend against the hazard: Review the defenses in context of how they will be performed.  Review the defenses in the work areas.  Ask users to walk you through how they perform their tasks.  Ask them to describe the hazards associated with their task and to describe the defenses in use that mitigate the hazards.  Ask the workers to describe what could go wrong and what their intended actions would be if there were to an event.
  5. Identify defenses to review in context during startup and operations: Identify defenses to review during startup and operations that context was hard to verify before startup or defenses that are vital to safe operations.  Some defenses may need to be reviewed at specific steps in the startup process or in a specific sequence to properly evaluate them in context of how they will be used during operations.

During Startup and Early Operations:

  1. Review defense in the context that they will be used to determine if the defense will defend against the hazard: Review the defenses in context of how they will be performed.  Review the defenses in the work areas.  Observe workers as they perform their tasks.  Ask the workers to describe what could go wrong and what their intended actions would be if there were to an event.
    1. Note conditions that enable defenses to perform: Note conditions that enable and/or encouraged the worker to adhere to the defenses.  Look for defenses that that workers have developed to perform their jobs for incorporation into the procedures.  Sometimes workers will develop their own defenses that would improve safety if they were proceduralized and used by other workers.
    2. Note conditions that hindered defenses from being successful: Note conditions that may hinder defenses from being used successful.  Ensure that you review the context that the defense will be used both during normal operations and anticipated off-normal conditions.

Examples of Results of Reviews of Defenses in Context:

  1. Defense: Radioactive Waste Transfer shutdown
    1. Defense as it was intended to work – To stop a transfer if a leak was detected the Operator monitoring leak detectors was to phone 2nd Operator stationed near a phone in the pump control building.  The 2nd Operator was expected to walk to the next room and instruct a 3rd Operator to shut down the pump.
    2. Defense as it was found in context – The 2nd Operator only recognized that he had a function every 4 hours (material balance calculations) and was bored.  He requested permission to rove to several stations to give other operators breaks.  This change was made without notifying the Operator monitoring the leak detectors or establishing an alternative communication method to direct the pump to be stopped, negating the defense.
  2. Defense: Use of crane spotter
    1. Defense as it was intended to work – All crane movements were to be directed by a spotter.
    2. Defense as it was found to be used in context – All load movements were to be directed by a spotter but crane operators routinely moved the hook without direction (normalization of deviation or system drift).  The crane operator, while positioning the crane hook in preparation for an upcoming lift without a spotter (spotter was on break), struck and damaged equipment with the hook.
  3. Defense: Field communications
    1. Defense as it was intended to work – Field personnel would be able to communicate unexpected field conditions quickly.
    2. Defense as it was found to be used in context – For several years field personnel had been issued company cell phones while on duty.  Due to cost cutting measures, field personnel had been issued radios instead of phones for communications.  Personnel found that there were significant dead zones where there was no radio reception and had developed work-a-rounds for normal operations.  The use of the radios for communications in the event of an emergency had not been evaluated and was determined to be ineffective.
  4. Defense: Evacuation Route
    1. Defense as it was intended to work – In the event of an emergency personnel were instructed to evacuate up wind from the event scene and exit by a gate on the upwind side of the tank farm.
    2. Defense as it was found to be used in context – When the defense was evaluated in context the wind was blowing from an unusual direction.  The upwind gate which had not been used for several years was blocked by tumbleweeds and drifted sand and was inoperable.

Summary:

This approach was piloted at a large Department of Energy construction site.  Approximately 50 workers participated in the pilot observing numerous work activities including nuclear waste storage and transfer, steel erection, rebar installation, welding, crane operation, fall protection, etc.  In almost every observation ‘Gaps’ were noted.  The participants in the pilot commented that the key to the process was how you approached the worker.  After initial introductions, the approach that brought the best “contextual” response from the workers in the field was;  “I am not familiar with what you do, would you mind walking me through your process and how it works?”

In most cases the workers were anxious to share their knowledge and understanding.  This opened the door for more specific questions about hazards, defenses and how they ‘really got work’ done.  This information was critical in helping the organization better defend against future events.

(Editor’s Note) Thank you again to Shane and Brian for their commitment to safety and the concepts of high reliability.  Please feel free to share similar experiences, or ask questions in the comments section.  The materials from the first part of this series are linked below, to read the first part of this series click here.

HPI Field Observation Worksheet example

HPI Field Observation Worksheet blank

Advertisements

2 Responses to Evaluating Defenses in ‘Context’ Part II

  1. Tom Thome says:

    Both Part I and II are well done, especially considering how complex the whole system is that this process is being applied to. I think Shane and Brian could have gone into more detail to provide more comprehensive coverage without losing the readers. Providing examples is always beneficial and they did well in giving a diverse set of examples. One thing that detracts is there are a number of typos and grammatical errors which, in a few cases, alter the points being made. I suggest running future papers through a technical editor prior to posting. Otherwise, an excellent start to an important additional assistance to approaching event-free performance.

    • David says:

      Tom,

      Thank you for your comment, we appreciate your feedback. One of the great benefits of this form of communication is that we can “fix” errors and make changes at any time. I would ask you, if you could, to please send me an e-mail regarding any of the grammatical errors or typos that you see. We do review these articles at many different levels for both quality and clarity, and I would like to understand how we can make our process more robust. Again, thank you for you comment.

      Thank You

      My E-mail: drisley@pec1.net

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: