ENGR390 Engineering Ethics

Week 6: Safety and Risk

Dr. Thomas Martin

Prof. Saleh Al-Araji

Information Security Research Group

Khalifa University

thomas.martin@kustar.ac.ae
October 11, 2014

Outline

Introduction

Approaches to Risk

Communicating Risk and Liability

Introduction

Approaches to Risk

Communicating Risk and Liability

Introduction

Introduction
Safety is something that engineers need to be constantly aware of.
Decisions made can either improve the healthy and well-being of
many of our society, or else put them in risk. The question is how
should engineers best deal with these questions and concerns?
Further complicating the problem is the constant evolution of technology. Designs that have been tested and proved trustworthy and
reliable can become obsolete as technology changes. Sticking with
the old and known may be safe, but it will lead to being left behind.
In order for there to be progress, there must be experimentation
with new materials, new designs, new machines and compounds.
But along with anything new comes the unknown. Risk is inherent
and dynamic in engineering.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Introduction

Introduction
Public safety is prominent in the code of ethics of any professional
engineering body.
National Society of Professional Engineers: Engineers shall hold
paramount the safety, health, and welfare of the public.
IEEE: Members commit to accept responsibility in making decisions consistent with the safety, health, and welfare of the
public.
ASME: Engineers shall hold paramount the safety, health and
welfare of the public in the performance of their duties.
Not only must an engineer act in accordance with public safety, but if
their professional judgment is being overruled in a way that endangers
the public it is their obligation to bring this to the attention of the
appropriate authority.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Introduction

Introduction
It might be tempting to try to produce something that is completely,
100% safe.

Unfortunately, this is impossible.

There is always a

margin of error, there is always a level of imperfection, there is always

an unknown factor. So when we talk about something being safe, we
are aiming for suciently safe, that the danger is below an acceptable
level of risk.
Increasing safety almost always comes with an increase in costs. If
the price is too high, the public will not pay so it is very important to
nd the right balance between the two trade-os. Designs must meet
the cost constraints, while at the same time avoiding the introduction
of any unacceptable risks.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Introduction

Introduction
How do we decide if the level of risk in a given design is acceptable
or not? We need to nd ways to identify the risks of harm and to be
able to quantify the risks, measure them objectively.
There are unique risks associated with the various tasks of engineering, and they need to be handled in dierent ways. When it comes
to engineering design, risk is managed through the use of developing
design codes, rules proven to produce designs that do not go beyond
the level of acceptable risk.

These make use of basic engineering

principles, such as redundancy and failure modes that give visible

warning.

Risk is managed in the operation of engineering systems

by careful design and continuous review of engineering systems and

processes.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Risk = Probability Harm

A standard engineer's denition of risk is the product of the likelihood
of an event and the magnitude of the resulting harm.
Harm can be anything that limits a person's freedom or well-being.
The types of harm that can (mostly) be quantied include economic
costs and impairments to physical well-being or to the public health,
safety, or welfare. But when we quantiably compare risks, we need
to make sure they are similar risks, that they are using the same units
of harm.
Public perception of risk can be very dierent to actual risks. Consider travel by plane versus car. Air travel is perceived to be much
more dangerous, despite being statistically safer. This can lead to
increased suspicion and criticism by the public, which is not a defendable stance when taking the probability based view of risk.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Acceptable Risk
Using the previous denition, an engineer would determine what is
an acceptable risk as follows:

An acceptable risk is one in which the product of the probability and

magnitude of the harm is equaled or exceeded by the product of the
probability and magnitude of the benet.
Suppose a manufacturing process produces bad-smelling fumes that
might cause health problems.

A cost-benet approach means we

need to compare the costs of mitigating the fumes versus the risks
they pose. Costs of preventing the harm:
Costs of modifying the process
Protective masks
Better ventilation systems
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Acceptable Risk
Costs of tolerating the fumes:
Additional health care
Possible lawsuits
Bad publicity
Loss of income to families of the workers
Other costs due to lives lost
If the total costs of preventing the loss of life is greater than the total
costs of not preventing the deaths, then the current level of risk is
acceptable.
We still have the problems of not always knowing the precise outcomes of each option, nor being able to translate all risks and benets
into monetary terms.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Acceptable Risk
Many have tried to place a monetary value on human life. One way
of doing this is based on likely future earnings, but this does not
value retired people or housewives.
Another approach is to extrapolate from how much more pay people
demand for jobs with higher risk. Or a similar way is to look at how
much people will pay for some safety feature in a car.
All these valuations are simplications. The location the person lived
in and how readily available employment is would certainly impact
how risky a job they would take. And wealthy people are probably
more willing to pay for safety than the poor.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Acceptable Risk
Cost-benet analysis may work ne on balance sheets, but the reasoning is often objectionable to the public. An example of this is the

Ford Pinto . The Pinto was prone to catch re in rear-collisions. As

this was discovered during production, it was estimated to be highly
expensive to stop, redesign and rework (estimated at $137 million).
Ford estimated the likely number of accident to occur due to the
aw, and put a cost of $200,000 per death and $67,000 per serious
burn. They arrived at an estimate of $49 million in liability.
When the lawsuits began, the evidence of their cost-benet analysis
was used against them.

The fact that the aw, and the resulting

burns and deaths, could have been prevented lead to Ford having to
make huge payments. Eventually they had to recall the Pinto and
x the problem.
1

http://auto.howstuffworks.com/1971-1980-ford-pinto12.htm

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Capabilities Approach to Identifying Harm and Benet

There are plenty of limitations of this cost-benet approach:

1 Any secondary or indirect consequences of a hazard are ignored

2 Hazards (both natural and engineering) may create opportunities, which should be taken into consideration

3 We still do not have an accurate, uniform, and consistent metric

to quantify the consequences from a hazard

4 This approach does not make the connection between specic

harms or losses and the diminishment of individual or societal
well-being and quality of life.
An alternative has been suggested of using a capabilities-based approach to risk analysis which focuses on the eects of disasters on
overall human well-being.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Capabilities Approach to Identifying Harm and Benet

Murphy and Gardoni

2 dene

capabilities

as the ability of people

to lead the kind of life they have reason to value.

are distinct from

utilities,

Capabilities

which refer to the mental satisfaction,

pleasure, or happiness of a particular individual.

good indicators for individual's well-being.

Utilities are not

Someone in a poverty-

stricken situation may be making the best of his situation (content

from a utilities standpoint), yet still be objectively deprived.
In capabilities terms, a risk is the probability that individuals' capabilities might be reduced due to some hazard.

http:
//onlinelibrary.wiley.com/doi/10.1111/j.1539-6924.2006.00801.x/abstract
2

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Engineer's Approach to Risk

Capabilities Approach to Identifying Harm and Benet

The benets of this approach are:

1 It captures the adverse eects an opportunities beyond the

consequences traditionally considered

2 Capabilities focus on aspects of individual well-being, which is

core to societal impacts

3 The approach requires considering a few properly selected

capabilities, rather than many complex consequences
A risk is acceptable if the probability is suciently small that the
adverse eect of a hazard will fall below a threshold of the minimal
level of capabilities attainment that is acceptable in principle.

In

practice, it may be tolerable for individuals to temporarily fall below

the acceptable threshold (e.g. after a disaster), as long as this situation is reversible and temporary and the probability is suciently
small.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Public's Approach to Risk

Dierences in approaches to risk

The public often has a very dierent attitude to risk than engineers
and risk experts. People tend to overestimate the likelihood of lowprobability risks associated with causes of death and to underestimate the likelihood of high-probability risks associated with causes
of death. This is made worse through anchoring or cognitive bias.

Anchoring:

tendency to rely too heavily on the rst piece of infor-

mation oered (or estimated).

All other judgments are made by

adjusting away from that anchor.

Cognitive bias:

drawing inuences in an illogical fashion due to indi-

viduals own subjective reality. I.e. ignoring all evidence that conicts
with a position and only remembering the cases that agree with it.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Public's Approach to Risk

Dierences in approaches to risk

There are a number of factors that cause dierences in perspectives
on risk.

The public will often combine the concepts of risk and

acceptable risk. They will also use the more informal adjective risky.
It is often used as a warning sign, that special care is necessary.
This may be because something is new and unfamiliar. Or because
information about it might come from a questionable source.
People use many factors in their own personal risk calculations. Voluntarily assumed risks are more acceptable than risks not voluntarily
assumed (by up to three orders of magnitude). Perceived risk that
has a human origin is 20 times greater than a risk with a natural
origin.

An immediate risk is perceived as being 30 times greater

than an ordinary one.

A regular risk is perceived as being just as

great as an occasional one, and necessary risk is just as great as a

luxury-induced one.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Public's Approach to Risk

Free and Informed Consent

Free and informed consent to all risks we are exposed to is necessary
for ones moral agency. This requires:

1 A person must not be coerced

2 A person must have the relevant information
3 A person must be rational and competent enough to evaluate
the information
Even when the person has a choice in theory, there may be possible
coercion.

If the risk relates to their job, and there are no other

jobs available, do they really have a free choice?

The second and

third points are also tricky due to the public's more subjective view
of risk. Plus they may not have the technical expertise to properly
understand this situation.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Public's Approach to Risk

Equity or Justice
Byssinosis (or brown lung) is a lung disease that eects people working with cotton (without adequate ventilation). Coal worker's pneumoconiosis (or black lung) similarly eects coal miners. These occupational hazards, though quite serious, can be justied from a
utilitarian view.
The great harm to a small number is oset by smaller advantages
to a great number. Protection would be expensive, and costs passed
onto the consumer. Competitively priced good are sold abroad which
improves the economy. Higher costs would remove that benet, and
possibly cost jobs.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

The Public's Approach to Risk

Equity or Justice
From the respect for persons model, the problem is that the benets
and risks have not been fairly spread among the population. Many
enjoy the benets, but only a small number suer the bodily harm.
Applying the Golden Rule, there are few would would want to be in
their position.
As mentioned earlier, the concepts of risk and acceptable risk are
often used interchangeably by the public. Taken that as given, we
will consider the public view of acceptable risk to be a risk in which:

1 Risk is assumed by free and informed consent, or properly

compensated, and in which

2 Risk is justly distributed, or properly compensated.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Communicating Risk and Public Policy

Risk Communication Guidelines

As well as the public and risk-experts having dierent views on risk,
government regulators may have another view entirely.

They are

most concerned with preventing harm to the public, and a claim

of harm that latter turns out to be false may not be considered a
problem. Engineers have an obligation to participate in deliberation
regarding risk by contributing their expertise:

1 Engineers, in communicating risk to the public, should be aware

that the public's approach to risk is not the same as that of
the risk expert. In particular, risk cannot be identied with a
measure of the probability of harm. Thus, engineers should not
say risk when they mean probability of harm. They should
use the two terms independently.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Communicating Risk and Public Policy

Risk Communication Guidelines

2 Engineers should be way of saying, There is no such thing as
zero risk.

The public often uses zero risk to indicate not

that something involves no probability of harm but that it is a

familiar risk that requires no further deliberation.

3 Engineers should be aware that the public does not always trust
experts and believes that experts have sometimes been wrong in
the past. Therefore, engineers, in presenting risks to the public,
should be careful to acknowledge the possible limitations in their
position. They should also be aware that laypeople may rely on
their own values in deciding whether or not to base action on
an expert's prediction of probable outcomes.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Communicating Risk and Public Policy

Risk Communication Guidelines

4 Engineers should be aware that government regulators have a
special obligation to protect the public, and that this obligation
may require them to take into account considerations other than
a strict cost-benet approach.

Although public policy should

take into account cost-benet considerations, it should take into

account the special obligations of government regulators.

5 Professional engineering organizations, such as professional societies, have a special obligation to present information regarding technological risk.

They must present information that is

as objective as possible regarding probabilities of harm. They

should also acknowledge that the public, in thinking about public policy regarding technological risk in controversial areas (e.g.,
nuclear power), may take into consideration factors other than
the probabilities of harm.
Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Engineer's Liability for Risk

Tort Law
The law of tort deals with injuries to one person caused by another,
usually as a result of fault or negligence of the injuring party. It varies
from jurisdiction to jurisdiction, but one standard of proof

3 is:

1 The defendant violated a legal duty

2 The plainti suered injuries compensable in the tort law
3 The defendant's violation of legal duty caused the plainti 's
injuries

4 The defendant's violation of legal duty was the proximate

cause of the plainti 's injuries.

According to the New Jersey Supreme Court

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Engineer's Liability for Risk

Becoming a Responsible Engineer Regarding Risk

The rst step in the process of becoming ethically responsible about
risk is to be aware of the fact that risk is often dicult to estimate
and can be increased in ways that may be subtle and treacherous.
The second step is to be aware that there are dierent approaches
to the determination of acceptable risk.
The third step is to assume their responsibility, as the experts in
technology, to communicate issues regarding risk to the public, with
the full awareness that both the public and government regulators
have a dierent agenda with regard to risk.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Engineer's Liability for Risk

Becoming a Responsible Engineer Regarding Risk

People should be protected from the harmful eects of technology,

especially when the harms are not consented to or when they are
unjustly distributed, except that this protection must sometimes be
balanced against

1 the need to preserve great and irreplaceable benets, and

2 the limitation on our ability to obtain informed consent.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Introduction

Approaches to Risk

Communicating Risk and Liability

Engineer's Liability for Risk

Becoming a Responsible Engineer Regarding Risk

Issues in applying this principle:

1 What exactly does it mean to `protect people from harm?

2 Many disputes can arise as to what constitutes a harm.
3 The determination of what constitutes a great and irreplaceable
benet must be made from the context of a particular situation.

4 Problems arise in determining informed consent and the limitations in obtaining informed consent in may situations.

5 The criterion of unjust distribution of harm is also dicult to

apply.

6 An acceptable risk at a given point in time may not be an

acceptable risk at another point in time.

Thomas Martin
ENGR390 Engineering Ethics

Information Security Research Group

Thank you

Any Questions?