The illusion of risk control. What would it take to live with uncertainty?

Historical evolution of paradigms of risk and safety

The notion of technological risk was initially associated with the occurrence of circumstances (e.g. leading to accidents) which are possible and probable. “Possible” expresses the fact that these circumstances are known and enumerable; “probable” expresses the fact that these circumstances may arise in the future. These circumstances were described by combinations of events or by sequences of events (scenarios), or both. In the context of safety, the considered circumstances lead to harmful events (accidents) inevitably (deterministic approach). If all the circumstances are known, treatments can be defined a priori to avoid the occurrence of accidents. These treatments were primarily technological barriers which avoid the occurrence of combinations or neutralize the effects sequences. Under this paradigm, safety was defined by the absence of such circumstances and by the absence of accidents, which was guaranteed by the presence of treatments.

A first adaptation of this viewpoint involved a change to the assumption that the circumstances do not always lead to harmful events (called probabilistic approach). In addition, the harm caused was no long assumed to be constant for the same circumstances. For example, the failure of a system component induces, or not, the system’s failure; the consequences of the failure may be more or less serious. Risk, or rather its estimation, was then defined as a combination of the probability of the occurrence of a harmful event and the severity of the harms. Safety was defined by the absence of unacceptable risk, usually expressed as a threshold (for instance, a probability of a crash < 10^-9 per hour). This viewpoint is the most commonly used today. Safety is still achieved by the introduction of barriers, whose availability (function of the probability of failure on demand) intervenes in the evaluation of residual risk. This value is compared with the maximum level of risk allowed (acceptability threshold). Therefore accidents are not avoided, but their risks are controlled.

A second adaptation of this viewpoint introduces the notion of “objectives”, giving meaning to risk-taking. Circumstances may lead to harms, hindering the achievement of certain objectives. However, they may also facilitate the achievement of other objectives. In the field of safety, these multiple objectives are, for instance, the preservation of health and the environment, on the one hand, and value creation, on the other hand. The choice of barriers depends not only on their impact on the occurrence and severity of harms (effectiveness) but also on their costs (efficiency). Risk and safety are now relative concepts, since they lead to the optimization of multiple objectives.

However, these changes of viewpoint on risk and safety are only variations, because they are based on the same paradigm: the circumstances that may lead to accidents are known, as are their possible effects: causes and consequences are enumerable, even if they are probable and not certain. The risk controls (and therefore level of safety) can be defined a priori. If unforeseen circumstances arise, they are handled by “experience feedback” and added to the list of circumstances, considered as a finite set.

We are currently facing a challenge to this founding paradigm of “risk” and “safety”, having to admit that the circumstances (initiating events and scenarios) leading to accidents are uncertain and potentially infinite. Our ignorance of causes (circumstances), effects (harms) and their relationships is primarily quantitative. This aspect concerns, for instance, the difficulty in determining the probability of occurrence of events (causes and effects), the effectiveness of the barriers and the severity of consequences. Our ignorance is also, even more importantly, qualitative. This concerns, for example, our inability to establish an exhaustive list of circumstances which may lead to accidents. Similarly, the nature of the effects of certain circumstances (in particular medium- and long-term effects) cannot be predicted given the state of knowledge (radiation from telecommunication antennas, evolution of cattle feed, emerging risks related to innovation, etc.). This paradigm shift has impacts on the concepts of risk and safety which have to be revisited, but also other fundamental concepts of safety and their operational implementations (models, techniques, processes, practices, cultures, etc.).

Until now, most approaches have aimed at reducing uncertainty, being based on the illusory hope of the contribution of knowledge development. These approaches have proven unsuccessful. The inability to nullify this uncertainty and the need to live with it must be accepted: such is our premise. Acknowledging the existence and impossible eviction of uncertainty as a new paradigm, the NeTWork’2013 seminar aims to re-examine the concepts underlying safety, to identify the nature and scope of changes due to this new paradigm, and to propose ways to address and implement this new paradigm.

The following sections aim to propose an organization of the seminar discussions through three angles: revisiting the concepts, identifying the nature and scope of the necessary changes, and suggesting new trails to address these changes. For each of them, the remainder of the document, without claiming to be exhaustive, provides some illustrative ideas.

Revisiting the concepts

The shift of paradigm underlying safety will likely require revisiting the concepts used to address this area.

Firstly, current definitions of existing concepts can be discussed. For instance, the ISO 31000 standard redefines “risk” as “the effect of uncertainty on objectives”, expressing the relative nature of risk (to objectives) and highlighting the new underlying paradigm, namely “uncertainty”. Reflections on the definition of “uncertainty” are also needed. Thus, among others, Saïna Hassanzadeh defines uncertainty as “a subject’s conscious lack of knowledge about an object, which is not yet clearly known, in a context requiring action or decision”. The definition of “safety” must also be discussed. Moreover, many concepts currently considered as intrinsic to safety could be questioned or could disappear, such as, why not, “risk acceptability”. Other concepts will undoubtedly emerge to address new visions.

Identifying the nature and scope of necessary changes

The paradigm shift will lead to new concepts and new definitions of existing concepts, and thus probably lead to significant changes of practices (models, techniques, organizations, cultures, etc.). We would like at this stage to highlight the nature and scope of changes required. In the remainder of this section, we mention some examples. The ways these changes can be addressed are discussed in the next section.

We noted that if the circumstances causing accidents are known, treatments can be proposed a priori. This approach is clearly inadequate if the set of possible events is unbounded and unanticipated scenarios arise.

Operating permits and product approvals are often provided based on a demonstration that risks are controlled (a safety case, for instance), established a priori. Responsibilities are limited to respecting rules that ensure this control (good design practices, specifications for use, etc.). Similarly, the social acceptance of risk is based on certainties: “everything is under control” or “uncertainty is within a bounded range”. In what ways do uncertainty (qualitative and quantitative) and the impossibility of risk avoidance call into question these principles? In particular, which are the implications for decision-making (approval, acceptance)?

Safety improvement is based on experience feedback from incidents or accidents. Lessons learned improve practices and increase safety. But why do we attempt to learn from experience, when we know that future situations will be different? Why expand the scope of circumstances to be treated a priori if the perimeter of these circumstances is unbounded? What is the nature of post treatments (treatment of unexpected events as they arise)? Or how can organizations ensure that their adaptive capabilities (treatment of unexpected events as they arise) are satisfactory?

The managerial approach to risk and safety is based on certainties on the effectiveness of implemented measures. For example, it assumes the ability of methods used to identify and analyze risks, to provide an exhaustive and ranked list of risks; it also assumes the ability of the models used to represent or accurately measure the risks. But uncertainty also affects these means. What effects on safety assurance?

The objective of this part is to draw a picture of what is being challenged by the paradigm shift.

Suggest new trails

Once the nature and scope of changes has been established, the necessary changes to practices have to be considered. The seminar does not aim to propose specific means, but rather to draw new trails of research leading to new means (principles, approaches, processes, rules, techniques, etc.), taking uncertainty into account.

For example, how to define, if applicable, social and legal acceptability? What new requirements on risk assessment? Are mixed or combined approaches possible despite their differing founding concepts?

Workshop organizers

• Gilles Motet, INSA Toulouse and Foundation for an Industrial Safety Culture
• Corinne Bieder, Airbus

Outputs

The position papers presented during the workshop were improved by authors based on discussion in Sorèze, then published as an open access volume in the SpringerBriefs in Safety Management series. The book can be downloaded for free in PDF and ePub formats.

Return to the list of workshop topics...