Narrowing the Gap Between the Work as Imagined and Work as Performed at SRS

(Editor’s Intro) Continuing with our series of guest bloggers I would like to introduce William L. Rigot, Manager, Engineering Support Services, Savannah River Nuclear Solutions, at DOE’s Savannah River Site (SRS).  Bill has served as a nuclear trained officer in the United States Navy, finishing his sea career as Chief Engineer on USS NIMITZ before retiring, and has worked at SRS since retiring from the Navy.  In addition to his nuclear engineering background, he has a BS in Oceanography and Physics from the United States Naval Academy.  At SRS he has served in a variety of positions including Division Training Manager, Deputy Facility Manager FB Line, and a variety of engineering management positions.  Bill is currently the site’s Subject Matter Expert for Human Performance, the EFCOG co-chair for the EFCOG HPI Task Team, and consults regularly across the DOE complex in the area of Human Performance.  He gave the keynote speech at the 2008 INPO Engineering Human Performance conference.

We would like to thank Bill for this blog!

Narrowing the Gap Between the Work as Imagined and Work as Performed at SRS

By William L. Rigot

BACKGROUND

At the Department of Energy’s (DOE) Savannah River Site (SRS), significant progress has been made implementing the Institute of Nuclear Power Operators (INPO) principles and tools of Human Performance.  The DOE adopted the methodology as the Human Performance Initiative (HPI).  SRS is located near Aiken, SC on a 310 square mile Federal reservation.  Its original purpose was to support the nation’s nuclear weapons programs, and it produced weapons grade plutonium and recycled nuclear fuel from the several reactors used over the years for that production.  Today the site no longer produces plutonium, but is the largest remaining DOE site stabilizing nuclear materials.  A major focus is on improving the environmental legacy of the site.

In the early 1990’s the contractor began sending its new nuclear facility managers through a five week long INPO Plant Managers course.  While the focus of that course is leadership in nuclear operations, each of the managers returned to SRS with a strong belief in the importance of Human Performance as a tool to improve their facility’s performance.  After initial training, performance in those facilities did improve, but because of a lack of a site wide structure to support HPI, success stories were localized.  After the Hanford report on implementation of HPI, both the contractor and the DOE made a decision to support a more structured approach to HPI at the site level.  After five years of effort on implementation, the tools and principles are in wide use.  At this point managers at SRS are beginning to recognize the impact of the gap between the work as imagined by them (or the work as designed by engineers) and the work as done.  This paper describes the journey to understanding and the current path.

IMPLEMENTATION

The SRS contractor began with knowledgeable managers who were unsatisfied with a world class safety program and respectable performance to meet contract milestones.  Most of these managers had been trained by INPO, or had Navy nuclear or commercial nuclear experience.  They started with an INPO recommended Human Performance Gap Analysis, which led to an HPI Strategic Plan that mapped out the major elements of Human Performance that needed to be addressed first.  They implemented a site wide structure consisting of an HPI Steering Committee, HPI Working Group, and HPI Champions Committee.  The HPI Steering Committee was co-chaired by a contractor Operations executive, and a DOE Safety and Health manager.  It serves to establish site level policies on Human Performance and to manage priorities and resources.  They made an early decision to hire a consultant who had recently retired as the Human Performance manager of a long-term INPO I utility.  Early priorities established by the HPI Steering Committee were to focus on Corrective Action programs, selection of HPI tools for workers, managers and engineers, and a focus on establishing a Just Culture.  In addition to the INPO Human Performance Manual, they were guided by the work of James Reason[1], and Sidney Dekker[2][3].  The HPI Working Group was chaired by an INPO Plant Manager course graduate who was an Area Project Manager, and a DOE Safety & Health manager.  This working group consisted mainly of Facility Managers or their deputies from pilot facilities, as well as an HPI Project Manager.  They established HPI implementation as a project, and managed it as a project, projecting cost, schedule and project loading.  Treating implementation as  a project was a major factor in the success of implementation in a large company that was spread over a large geographical area.  The HPI Working Group also chartered and managed several Task Teams responsible for writing procedures, initiating programs, and benchmarking.  The HPI Champions Committee consisted of HPI practitioners working in each of the pilot facilities.  The chair of that committee is also the chair of the HPI Working Group.  The practitioners generally carry out the work listed in the project plan.  They do the training in their respective facilities, coach and mentor, monitor the performance of work, and measure effectiveness of implementation.  The chairs of these committees work together with each other, and with the SRS HPI project manager and the HPI subject matter expert, to assure effective communication of problems with implementation.

HPI PROJECT PLAN

The current project plan is shown in Appendix A.  It lists the HPI tools and programs implemented over a multi-year period.  Due to the size of the organizational structure at SRS, and the limited resources of knowledgeable people, the schedule had to be broken down by facility or programmatic area and by function.  Learning lessons from the Hanford pilot, but recognizing that SRS was much bigger, the schedule was tailored to rolling implementation starting with pilot facilities, and then to the remainder.  Every implementation was evaluated for effectiveness of implementation.  Lessons Learned were incorporated into the training and implementation, and gaps were closed through continuing training and reinforcement.  Training was reinforced through monthly videos and messages in monthly Safety presentations.  The SRS Facility Evaluation Board, an independent oversight group, was formally trained in HPI tools and principles, and they began also to measure effectiveness of implementation.  Implementation methodology generally began with training for the management team at a facility, then training for engineers, changes to the causal analysis methodology in Critiques, Fact Finding investigations, and Post Job Briefs, following finally in training for workers.  We found that methodology to be most effective, because by the time workers received training, they had already begun to see changes in behaviors of their management team and others that made sense to them once they received the training.  When they left the classroom, they stepped back into their facility with a new understanding.

HEURISTICS

As HPI implementation began to take effect at SRS, heuristics (or mental models) became more important in understanding why implementation either worked, or did not.  Some of these and their sources are listed below.

High Reliability Organizations (HRO)[4]

  • Focus on failure
  • Refuse to simplify interpretations
  • Focus on Operations
  • Deference to expertise
  • Commitment to resilience

Other High Reliability Organizations (HRO)[5]

  • Flexibly organized
  • Communicate more than they think they need
  • Learn from everything they do
  • Don’t punish people for making honest mistakes

Todd Conklin’s 4[6]

  • Fixated on failure
  • Seek to reduce complexity in operations
  • Respond to accidents/events with deliberation
  • Respond to near misses with urgency

SRS chose not to pursue HRO principles and activities because the necessity to do so was not urgent, and would confuse the workforce with too many messages.  But many of the HPI leaders were aware of them and tried to be congruent wherever possible.  This was most evident in our approaches to establishing a Just Culture, and subsequent improvements in causal analysis methods.

CULTURE

James Reason discussed a culture of safety in the context of sub-elements that he defined as informed culture, flexible culture, just culture, reporting culture and learning culture.[7] This is pictured in figure 1:

figure 1

Again, informed culture and flexible culture were generally elements of HROs, and SRS chose not to incorporate them into their understanding of culture.  While all three sub-elements of culture were recognized as important, just culture became the principal focus area initially.  It was recognized that the site had a history of sometimes punishing people closest to events, i.e. operators, mechanics, etc., without understanding the organizational drivers for the events.  Over time workers accepted the position that their human errors “caused” the events.  A deeper understanding of Human Performance, and especially INPO’s model of the Anatomy of an Event, led to an awareness that while human error is usually present in most events, blaming workers for doing things that made sense to them at the time was not particularly helpful in preventing events.  The HPI Steering Committee and HPI Working Group made a concerted effort over the entire timeline of HPI implementation to improve just culture as a way to improve reporting and learning cultures.  This would ultimately lead to an improved safety posture.

As part of this effort, SRS incorporated most elements of the Culpability Decision matrix proposed by Reason[8] into the site Discipline procedure.  The chair of the HPI Steering Committee attends all discipline review hearings to ensure that the elements of just culture are met.  The HPI Working Group changed the existing critique process to explicitly incorporate INPO’s Anatomy of an Event model into the investigation tools for the critique.  They also changed the name of the critique to Fact Finding.  As a result, more information is now coming out of investigations where workers are much more forthcoming.

An extension of the Blame Cycle[9] was developed as a learning tool for managers using Peter Senge’s systems thinking  to help understand the additional unintended consequences of blaming workers for human errors.  It is described in figure 2.

figure 2

Edgar Schein’s model for culture, described graphically below in figure 3 was used to help managers gain a better understanding of the gap that existed between the work as they imagined and the work as done.

figure 3

This led to development of a nuclear model taken from the Navy’s Nuclear Power Program and modified by Northrop Grumman at the Newport News Shipyard to specifically relate engineering design work to support Operational Excellence.  It is represented in figure 4.

figure 4

In 2008 the SRS M&O contract was awarded to the Savannah River Nuclear Solutions, LLC (SRNS).  It was the most significant contract transition since the 1989 contract transition from DuPont to Westinghouse Savannah River Co.  As a result of that contract transition, SRNS contracted Gallup to conduct an employee engagement survey to provide some cultural understanding of the workforce.  What they found was that, despite an outstanding safety performance record, the workforce was largely disengaged.  The company is now going through a long term effort to improve engagement with an eye to improving safety.  In 2009, SRNS experienced two significant accidents resulting in worker injuries.  Investigations by DOE identified many management and work control issues that led to unsafe work practices.  SRNS contracted with DuPont to conduct a Safety Culture assessment.  DuPont again identified previously noted gaps.  Despite years of implementation of HPI, gaps still remained between work as imagined and work as done in pockets across the site.

RESULTS OF PILOT ACTIVITIES

The Defense Program’s Tritium facility was the first facility to pilot the more structured HPI implementation, and now has over 5 years of history.  There was strong support from the Vice President at the time to implement HPI.  The Tritium facility was widely regarded as an excellent performer in Conduct of Operations, but had stopped improving, and was receiving regulatory criticism over their compliance to procedures.  Based on input from INPO and our HPI consultant, it was recognized that implementation of HPI would likely cause an increase in reportable events.  The management team was prepared for a rough implementation, and were convinced that successful implementation would result in significantly improved performance.  They were correct on both assumptions.  Their reportable DOE event rate did immediately rise, and stayed high for about a year.  But then it started dropping over the next year, to the point that their reportable rate has been at or near zero for the past eight months.  Additionally their non-reportable event rate has been at zero for the past four months.  The results are shown below in figures 5 and 6.  This facility took the lead in improving causal analysis, and strengthening their Corrective Action program.  A part of their activities included assembling a Management Review Team (MRT) to regularly, during the week, review all problems reported in the facility since the last time they met.  Most items go to a well developed tracking and trending program, but others go through a more rigorous causal analysis method or get focused looks by management coaching teams.  Because all of their lagging indicators have gone to zero, the management team is working to lower their threshold of things that they investigate.

SRS recently was the subject of an HPI benchmarking visit from the HPI Steering Committee chair of another DOE national laboratory.  The visiting executive spent time in the Tritium facility interviewing the management team and walking around the facility to see how they had done their implementation.  The consistent message he got from the facility managers was that none of them believed that they were as good as their numbers indicated, and that they were uneasy with their lengthy success.  In the perspective of work as imagined, this is exactly the management attitude one would expect.

figure 5

figure 6

SUMMARY

Narrowing the gap between what managers imagine is going on and what is really happening can be a challenge.  In particular it forces managers out of their comfort zone to realize that their expectations are not well understood by their employees, and potentially indifferently accomplished.  We have seen this at our pilot facility at SRS.  Our HPI consultant frequently told our managers “you can’t fix what you don’t know.”  By establishing a just culture, improving causal analysis, improving reporting mechanisms, and a host of other Human Performance tools that are well defined by the commercial nuclear industry, SRS has now made measurable progress in narrowing the gap between the work as imagined, and the work as done.  This has not made our managers’ jobs any easier.  In fact, they now have more problems to fix than they have available resources.  But it forces the entire team, including our employees, to focus on what’s really important to make improvements in how we work and get results.  The challenge for SRS is to leverage these lessons learned across a large and diverse population.

Appendix A:


[1] James Reason, Managing the Risks of Organizational Accidents. (Burlington: Ashgate Publishing Company, 1997)

[2] Sidney Dekker,  Just Culture. (Burlington:  Ashgate Publishing Company, 2007)

[3] Sidney Dekker, The Field Guide to Human Error Investigations. (Burlington:  Ashgate Publishing Company, 2002)

[4] Karl Weick and Kathleen Sutcliffe,  Managing the Unexpected. (San Francisco:  Jossy-Bass,  2007)

[5] Robert Pool,  When Failure is not an Option. (Boston:  MIT Technology Review article, July 1997)

[6] Todd Conklin,  EFCOG presentation. November 2009

[7] James Reason,  Managing the Risks of Organizational Accidents. Pp 194-195

[8] Reason,  Managing the Risks of Organizational Accidents. Pp. 209

[9] Reason,  Managing the Risks of Organizational Accidents.  Pp. 127-128

About these ads

9 Responses to Narrowing the Gap Between the Work as Imagined and Work as Performed at SRS

  1. Captain Bill,

    Thanks ever so much for the useful and heart-warming information.

    Would you be so kind as to tell some of the ways that you incorporate the “work performance gap” into issue investigations?

    Thanks ever so much.

    Take care,

    Bill Corcoran
    Mission: Saving lives, pain, assets, and careers through thoughtful inquiry.
    Motto: If you want safety, peace, or justice, then work for competency, integrity, and transparency.

    W. R. Corcoran, Ph.D., P.E.
    Nuclear Safety Review Concepts Corporation

  2. Bill Rigot says:

    Dr. Bill,

    What we did was we explicitly stopped automatically blaming workers for events that they triggered due to “Human Error”. When we did that we began to see how the system set them up to fail.

    For example, we had an electrical event where a couple of E&I mechanics started working on an energized 120V circuit instead of the 24V DC circuit they thought they were working. Because of the sensitivity in DOE to electrical events it became a big deal. We did a formal RCA. The two workers showed up to the first meeting and announced that they didn’t understand why we were going to spend so much time in analysis. They knew what they had done, and that it was wrong; so just send them home for a week without pay. At the end of four hours of intensive scrutiny, we discovered that, given the work package, labeling, and instruction they had been given, they could have done nothing other than what they did. I had lunch with them afterwards and asked them what they’d learned. They acknowledged their error, but they had no idea how much “help” they had had from maintenance planning and engineering to set them up for failure. The organization learned a lot as well, and was better able to understand the “rootier roots” that led to the event in order to craft better solutions. Recognizing that we needed to build a Just Culture first enabled us to start seeing the work performance gaps. Until then, they were intransparent to the management team.

    Warm regards,

    Bill

  3. Captain Bill,

    Thanks for your very helpful reply.

    The event you alluded to sounds quite interesting. Did DOE post the investigation report so that we can all learn from it?

    It sounds like the work as performed was exactly like the work as planned up to the point at which someone (who?) suspected that the mechanics were on the wrong circuit. How do you handle the “work performance gap” when there is no gap, but there is a problem?

    I’m teaching a one-day root cause analysis course at the HPRCT meeting in Baltimore and I’d like to spend some time on the “work performance gap.” It sounds like a great concept, but it might be a little tricky, like in the case you alluded to.

    Any links would be greatly appreciated.

    OBTW: I’m starting a conversation on the “work performance gap” on my flagship e-group.

    Thanks ever so much.

    Take care,

    Bill Corcoran
    Mission: Saving lives, pain, assets, and careers through thoughtful inquiry.
    Motto: If you want safety, peace, or justice, then work for competency, integrity, and transparency.

    W. R. Corcoran, Ph.D., P.E.
    Nuclear Safety Review Concepts Corporation

  4. wroege says:

    In one of the other postings, we got into the issue of how to measure the gap. There are more measurement points than just “management’s” imagination and worker actions. The work planning process seems to be a key mid-point. Perhaps we should look at breaking up the gap into constituant parts. It would certainly help diagnostics to do so.

  5. I would certainly second the idea of measuring the gap. If someone turned me on to that project I would start with a Comparative TimeLine using the micro-steps of the work as planned as the “What Should Have Happened.”

    There is a fairly good example of a Comparative TimeLine for “The Case of the Runaway Filter” at

    http://www.rootcauselive.com/Test%20Site%20Folder/File%20Table%20Contents.htm

    All the best.

    Take care,

    Bill Corcoran
    Mission: Saving lives, pain, assets, and careers through thoughtful inquiry.
    Motto: If you want safety, peace, or justice, then work for competency, integrity, and transparency.

    W. R. Corcoran, Ph.D., P.E.
    Nuclear Safety Review Concepts Corporation

  6. Bill Rigot says:

    Our NNSA Deputy Site Manager was a recent graduate of the INPO Senior Plant Manager Seminar. He visited 2 utilities as part of the 5 week course. One was an undisclosed INPO 1 plant; the other INPO 3. The participants were not told the grades in advance, but they figured it out pretty quickly. At the INPO plant they measured the “vertical alignment” in the message from the site VP down to the shop floor. At the INPO 1 plant the message was undiluted from top to bottom. At the INPO 3 plant, the message got garbled. The interesting thing was that the class was able to figure out where the garble occurred. This would indicate one of the measurement points to measure gaps.

    The key factor in the utilities’ culture is a thirst for feedback; even in the less good utilities. Managers and workers are encouraged to challenge anything they don’t understand or think is wrong. A site VP was observed at a regularly scheduled All Hands meeting with crafts personnel. At the end one of the mechanics challenged the VP because he hadn’t discussed an off the job injury that was recorded the previous day. The VP acknowledged and accepted the challenge and then discussed the injury. It seems the stronger the manager, the more they wanted to hear good critical feedback. Most utilities have a 360 degree feedback performance measurement system that they enforce fairly ruthlessly.

    I think there are many lessons we can learn from the utilities in the area of performance measurement. The problem for the Department of Energy is that they have not figured out how to reward their contractors to report more as the NRC has done with the utilities. In general the contractors are rewarded to report more when they are in PAAA enforcement, at which point, it’s too late. I am, of course speaking personally, and do not speak for my company in this regard.

    Bill Rigot

  7. Frank Coon says:

    I immediately started thinking of “Managing Maintenance Error” (recently read) and then reviewed your bibliography. I have not read the other references. Do you have any specific recommendations or should I just read all of them…?

    I like to think that 99.9999% of skilled workers are quite proud of their work and would never dream of making critical errors. Flaws in engineering, management, or corporate culture/values are much more likely to truly cause these events.

    Thank you for your work.
    Frank

    • wecarnes says:

      Good morning Frank,

      Thank you for joining our blog discussion and for your kind comment. I agree with you that most skilled workers would never intentionally make critical errors. In this blog I hope to reinforce that errors are unintentional and influenced by organizational factors and normal human factors. By understanding and applying the scientific concepts of high reliability, organizations can take steps to help people perform at their best.

      Glad you chose to read Managing Maintenance Error. Dr. Reason’s work has highly influenced this field of high reliability. To your question of what to read next, it depends on the type of work in which you are engaged and if you have any specific performance issues right now. If you do then I’ll be glad to see if I can suggest some things.

      For a recent good overall review of the main areas of the high reliabilty field I recommend ” Managing the Unexpected: Resilient Performance in an Age of Uncertainity” by Weick and Sutcliffe 2007.

      If you wish to discuss your specific interests just reply here and I’ll email you off line.

      Earl

  8. Rebecca Raven says:

    Wonderful post as always, Bill! I have bookmarked it for further study of your links.

    I am particularly interested in the “thirst for feedback” phrase. In my current work, I have noticed the same thing and using that image will help me more effectively communicate.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 97 other followers

%d bloggers like this: