Event analysis and learning from events
Context of the workshop
Accelerating technological development can bring improvements in safety but can also lead to challenges to safety as organizations “push the limits” with more complex, software-intensive systems that require participation from multiple stakeholder groups. The public demands more safety at the same time as it demands more services at less cost. These issues are of greatest importance in so-called “high-hazard” industries where a rare and surprising event can place hundreds or thousands of people at risk.
High-hazard organizations need appropriate structures, rules, and practices to avoid and respond appropriately to safety relevant events, in order to ensure their safety and reliability. These structures and rules are the safety management system that is based on both anticipatory feed-forward models of risk and strategies for feedback control (Rasmussen, 1991). It is never possible completely to specify all risks: even if risks are specified within the operating envelope, in real life organizations operate beyond their operating envelope a non-trivial amount of time (some would say, all the time). Therefore, adequate feedback control relies on learning from operational experience. Organizational weaknesses and latent failures are identified by continual monitoring and systematic analyses of problems, deviations, defects, events, near-misses and organizational surprises. Results of event analyses should lead to new knowledge, new structures, new rules, and new practices with the goal of higher reliability and safety. The systematic analysis of events, using valid and comprehensive methods, is thus a critical starting point for learning with the goal of enhancing safety and reliability.
However, 13 years after the first NeTWork workshop on the topic After the event — from accidents to organizational learning and following more than a decade of experience with various event analysis methods, it seems that the promises of event analysis and organizational learning were only partly kept. We still have to cope with both novel events and so-called recurring events, one recent example being NASA’s loss of the space shuttle Columbia, with causes reminiscent of the Challenger disaster another example the BP Texas City oil refinery explosion. Could it be that our analysis methods do not discover the underlying causes of the events, or does learning from experience not work as it is supposed to do, or is learning happening in the wrong places?
Event analysis
Events may be defined as occurrences of unexpected, undesirable system states, with or without negative consequences. Thus, “event” is a general term including incidents, accidents, near-misses, and other organizational surprises (Koornneef, 2000). Event analysis assumes that the better we understand the factors which led to events and their interactions, the better are the chances to utilize such experience to improve safety.
Event analysis is the later reconstruction of the occurrence of the event as well as of its causes, i.e. the identification of WHAT happened, HOW it happened, and WHY it happened. The answers to the what- and how-questions require a detailed description of the course of the event. For the why-question the analyst has to go beyond the given information, i.e. make causal inferences.
Therefore, it is necessary to base event analysis on an adequate theoretical model. This is where theories of accident genesis come into play (Reason, 1990). Various theoretical models for event analysis have been proposed such as the model of events sequence (Benner, 1975), the energy transfer model (Johnson, 1973), the accident causation model (Reason, 1990), the socio-technical systems model of event genesis (Wilpert & Fahlbruch, 1998) or a systems theory model (Leveson, 2004). Correspondingly, various event analysis methods exist such as root-cause analysis, socio-technical analysis, system theoretic accident analysis, and so forth.
Learning from experience
Beginning with the analysis of major events such as Three Mile Island, Bhopal, Herald of Free Enterprise, and so forth, industry managers and policymakers have sought to avoid future accidents by investigating fully the accidents and near-misses revealed by operating experience. Over time, the threshold of reporting has dropped lower and lower, so that organizations are reporting and investigating more minor incidents in order to correct problems before accidents occur. Increasingly, dissemination of incident investigation reports goes beyond the organization itself: several industries such as nuclear power and commercial aviation have become expert at transferring knowledge across the entire industry. This involves documentation and communities of practice that share explicit and tacit information. Of course, legal liability remains a significant impediment to documenting and disseminating candid analyses.
However, the event analysis process is only one part of how knowledge is created, disseminated and embodied in organizational structures, rules, and practices. One issue is that the transition from written report to organizational learning remains problematic (Carroll et al., 2007). Recommendations get watered down, enacted in ways that do not fully capture the intent of the event analysis, not implemented at all, or implemented with disappointing or unknown results. A second issue is that participation in the event analysis process can provide cross-discipline and cross-organization learning and the development of trustful relationships that build a supportive environment for operations and improvement. Reason (1997) makes reporting and learning from events a critical component of safety culture (see also Flin, Westrum, 2004).
Goals of the workshop
NeTWork, with the generous support of the Foundation for an Industrial Safety Culture, organized a workshop in August 2008 with approximately 30 participants, including both scholars and practitioners. The goals of the workshop were:
-
To discuss and reflect on various approaches to event analysis and learning from operating experience that can enhance safety.
-
To develop new theory, new testable hypotheses, new policies, and new practices that would advance both safety research and practice.
-
To structure an edited book based on the presentations and discussion at the workshop that will capture the insights from the workshop conversations and set a bold agenda for future research, management, and policy.
-
To honor the memory of Bernhard Wilpert, founder and champion of NeTWork, whose many contributions to human factors and social science research included an enduring interest in event analysis and learning from operating experience.
Among the questions discussed during the workshop:
-
Are existing event analysis methods adequate or appropriate for all kind of events (accidents, incidents, and near-misses) or organizations (high-hazard, low-hazard)?
-
Are the results from analyses adequate for learning, i.e. for sensemaking in the organization or industry and for the prevention of future events?
-
How could methods be validated? Or is it enough that the method is practical and leads to reasonable results? What indicators should organizations use to assess the effectiveness of their event analysis and operational systems?
-
How can we treat the huge body of contributing factors/causes? Do we need corrective measures for all of them?
-
Can improvements be made incrementally and bottom-up or does there have to be a comprehensive system model that identifies the leverage points and necessary changes top-down?
-
When do we need external experts for event analyses to provide an outsider viewpoint and when do we need internal staff to contribute their viewpoint and buy into any changes?
-
What kind of knowledge is necessary for event analysis (engineering, human factors, cultural etc.) and how should the knowledge acquisition/transfer be organized?
-
How can event analyses be conducted and supported so that the results are usable and implemented to bring about real improvement?
-
How can we screen events prior to the analysis to allocate scarce resources to those events that will yield the most learning and improvement?
-
What are the enabling and disabling conditions inside and outside of the organization (such as regulatory constraints) that support or prevent learning from experience and dissemination and implementation of lessons learned?
Workshop organizers
-
John S. Caroll, MIT
-
Babette Fahlbruch, TÜV Nord
Outputs
The papers presented during the workshop were published in a special issue of the journal Safety Science.
- Carroll, J., & Fahlbruch, B. (Eds) (2011). The gift of failure: New approaches to analyzing and learning from events and near-misses — Honoring the contributions of Bernhard Wilpert. Safety Science, 49(1). DOI: 10.1016/j.ssci.2010.03.005.
Image credit: Banksy