Security in Computing, 4th Ed, Pfleeger

Chapter 3
Program Security

By Mohammed Al-Saleh / JUST

Chapter 3

In this chapter

Programming errors with security implications

Malicious code

viruses, worms, Trojan horses

Program development controls against malicious

code and vulnerabilities

buffer overflows, incomplete access control

software engineering principles and practices

Controls to protect against program flaws in

execution

operating system support and administrative controls

By Mohammed Al-Saleh / JUST

Protecting programs is at the heart of computer

security

Chapter 3

Programs Security

programs constitute so much of a computing system

(the operating system, device drivers, the network
infrastructure, database management systems and
other applications, even executable commands on
web pages); all are called programs

So we need to ask two important questions:

How do we keep programs free from flaws?

How do we protect computing resources against
programs that contain flaws?

By Mohammed Al-Saleh / JUST

Chapter 3

Secure Programs

What we mean when we say that a program is

"secure."

security implies some degree of trust that the program

enforces expected confidentiality, integrity, and
availability.

From the point of view of a program or a

programmer, how can we look at a software
component or code fragment and assess its
security?

similar to the problem of assessing software quality in

general

By Mohammed Al-Saleh / JUST

Chapter 3

Secure Programs
One way to assess security or quality is to ask
people to name the characteristics of software
that contribute to its overall security

different answers from different people because the

importance of the characteristics depends on who is
analyzing the software

By Mohammed Al-Saleh / JUST

one person may decide that code is secure because it takes

too long to break through its security controls
someone else may decide code is secure if it has run for a
period of time with no apparent failures
a third person may decide that any potential fault in meeting
security requirements makes code insecure

Chapter 3

Secure Programs
An assessment of security can also be influenced by
someone's general perspective on software quality

if your manager's idea of quality is conformance to specifications,

then she might consider the code secure if it meets security
requirements, whether or not the requirements are complete or
correct.
This security view played a role when a major computer
manufacturer delivered all its machines with keyed locks

since a keyed lock was written in the requirements

But the machines were not secure, because all locks were configured to use
the same key

Thus, another view of security is fitness for purpose

By Mohammed Al-Saleh / JUST

in this view, the manufacturer (mentioned above) clearly had room for
improvement

Chapter 3

Secure Programs
In general, practitioners often look at quantity
and types of faults for evidence of a product's
quality (or lack of it)

developers track the number of faults found in

requirements, design, and code inspections and use
them as indicators of the likely quality of the final
product

By Mohammed Al-Saleh / JUST

Chapter 3

Fixing Faults
You might argue that a module in which 100
faults were discovered and fixed is better than
another in which only 20 faults were discovered
and fixed

more rigorous analysis and testing had led to the

finding of the larger number of faults

By Mohammed Al-Saleh / JUST

Chapter 3

Fixing Faults
Early work in computer security was based on the
paradigm of "penetrate and patch,"

analysts searched for and repaired faults

test a system's security by attempting to cause it to
fail
The test was considered to be a "proof" of security

if the system withstood the attacks, it was considered secure

Unfortunately, the proof became a counterexample

The problem discovery in turn led to a rapid effort to

"patch" the system to repair or restore the security

By Mohammed Al-Saleh / JUST

However, the patch efforts were largely useless, making the

system less secure rather than more secure because they
frequently introduced new faults
9

Chapter 3

Read about Fuzz Testing

By Mohammed Al-Saleh / JUST

10

Chapter 3

Unexpected Behavior
A better approach than "penetrate and patch," is to
compare the requirements with the behavior

Test whether programs behave as their designers intended or

users expected
unexpected behavior is a program security flaw

Program security flaws can derive from any kind of software

fault

two separate logical categories of program flaws :

Inadvertent/unintentional human errors

malicious, intentionally induced flaws.

we still have to address the flaws effects, regardless of

intention.

By Mohammed Al-Saleh / JUST

They range from a misunderstanding of program requirements to a onecharacter error in coding or even typing

"it doesn't matter whether the stone hits the pitcher or the pitcher hits the
stone, it's going to be bad for the pitcher."
11

Chapter 3

Security Flaws

A system attack often exploits an unintentional security

flaw to perform intentional damage
Regrettably, we do not have techniques to eliminate or
address all program security flaws

security is fundamentally hard and conflicts with usefulness and

performance
Two reasons for this distressing situation

To test a program functionality we test against the should do checklist NOT

the should not do one. It is almost impossible to ensure that a program
does precisely what its designer or user intended, and nothing more.
Programming and software engineering techniques change and evolve far
more rapidly than do computer security techniques

By Mohammed Al-Saleh / JUST

So we often find ourselves trying to secure last year's technology while software developers are
rapidly adopting today's and next year's technology.

12

Chapter 3

Types of Flaws

A list of categories that helps distinguishing one kind of

problem from another and gives us a useful overview of
the ways in which programs can fail to meet their security
requirements

Validation error (incomplete or inconsistent): permission checks

occur when a program fails to check that the parameters

supplied or returned to it conform to its assumptions about
them, or when these checks are misplaced
Domain error: controlled access to data

occur when the intended boundaries between protection

environments are porous including implicit sharing of
privileged/confidential data or when then the lower level
representation of an abstract object, supposed to be hidden
in the current domain, is in fact exposed

By Mohammed Al-Saleh / JUST

13

Chapter 3

Types of Flaws

Serialization: program flow order

Boundary condition violation: failure on first or last case

permit asynchronous behavior of different system components to

be exploited (TOCTTOU)
occur due to omission of checks to assure that constraints (table
size, file allocation, or other resource consumption) are not
exceeded

Inadequate identification and authentication: basis for

authorization

permits operations to be invoked without sufficiently checking the

identity and the authority of the invoking entity

By Mohammed Al-Saleh / JUST

14

Chapter 3

Nonmalicious Program Errors

Unintentional mistakes

cause program malfunctions

but some lead to more serious security vulnerabilities

By Mohammed Al-Saleh / JUST

15

Chapter 3

Buffer Overflows
It is an example on Boundary condition violation
Definition

A buffer (or array or string) is a space in which data

can be held
A buffer resides in memory
Because memory is finite, a buffer's capacity is finite
in many programming languages the programmer
must declare the buffer's maximum size

By Mohammed Al-Saleh / JUST

Then the compiler can set aside that amount of space

16

Chapter 3

Buffer Overflows
Example

char sample[10];
One byte for elements sample[0] through sample[9]
Now we execute the statement:

sample[10] = 'B';

The subscript 10 is out of bounds

The compiler can detect it during the compilation

However, if the statement were

By Mohammed Al-Saleh / JUST

sample[i] = 'B';
we could not identify the problem until i was set during execution

The problem's occurrence depends on what is adjacent to the

array sample

17

Chapter 3

Buffer Overflows

Example

suppose each of the ten elements of the array sample

is filled with the letter A and the erroneous reference
uses the letter B, as follows:
for (i=0; i<=9; i++)
sample[i] = 'A';
sample[10] = 'B'

By Mohammed Al-Saleh / JUST

18

Chapter 3

Buffer Overflows (Example Cont.)

So there are four cases to consider in deciding where

the 'B' goes

By Mohammed Al-Saleh / JUST

19

Chapter 3

Buffer Overflows (Example Cont.)

If the extra character overflows into the user's data space

it simply overwrites an existing variable value

perhaps affecting the program's result

but affecting no other program or data

In the second case, the 'B' goes into the user's program
area

If it overlays an already executed instruction

no effect

If it overlays an instruction that is not yet executed

the machine will try to execute an instruction with operation code 0x42

the internal code for the character 'B

Spilling over into system data or code areas produces

similar results to those for the user's space: computing with
a faulty value or trying to execute an improper operation.

By Mohammed Al-Saleh / JUST

20

By Mohammed Al-Saleh / JUST

Chapter 3

Process Address Space

21

By Mohammed Al-Saleh / JUST

Chapter 3

Call Stack / Activation Record

22

Chapter 3

Integer Overflow

http://www.phrack.org/issues.html?issue=60&id=10
Since an integer is a fixed size (32 bits for the purposes of this
paper), there is a fixed maximum value it can store. When an
attempt is made to store a value greater than this maximum value it
is known as an integer overflow.
Most compilers seem to ignore the overflow, resulting in an
unexpected or erroneous result being stored.
This can get dangerous if the calculation has to do with the size of a
buffer or how far into an array to index.
What happens then!!

a = 0xffffffff
b = 0x1
r = a + b r = (0xffffffff + 0x1) % 0x100000000
r = (0x100000000) % 0x100000000 = 0
This is often called a "wrap around", as the result appears to wrap around to 0.

By Mohammed Al-Saleh / JUST

23

Example 1

#include <stdio.h>
int main(void){
unsigned int num = 0xffffffff;
printf("num = %u (0x%x)\n", num, num);
printf("num + 1 = 0x%x\n", num + 1);
return 0;
}
/* EOF */

The output of this program looks like

this:
num = 4294967295 (0xffffffff)
num + 1 = 0x0

By Mohammed Al-Saleh / JUST

Chapter 3

Integer Overflow Examples

Example 2
#include <stdio.h>
int main(void){
int n;
n = 0x7fffffff;
printf(n = %d (0x%x)\n", n, n);
printf(n + 1 = %d (0x%x)\n", n + 1 , n + 1);
return 0;
}
/* EOF */

The output of which is:

n = 2147483647 (0x7fffffff)
n + 1 = -2147483648 (0x80000000)

24

Chapter 3

Incomplete Mediation (ex. SQL Injection)

John Fiore

SELECT * from CUSTOMERS

WHERE name = 'John Fiore'

By Mohammed Al-Saleh / JUST

25

Chapter 3

Incomplete Mediation (ex. SQL Injection)

John Fiore' or '1'='1

SELECT * from CUSTOMERS

WHERE name = 'John Fiore'
OR '1'='1'

By Mohammed Al-Saleh / JUST

26

Chapter 3

Bring examples on Cross-SiteScripting (CSS)

By Mohammed Al-Saleh / JUST

27

Chapter 3

Time-of-Check-to-Time-of-Use Errors(TOCTTOU)
Real Example:

Buy something that costs $100

The buyer removes five $20 bills from a wallet, carefully counts
them in front of the seller, and lays them on the table
Then the seller turns around to write a receipt
While the seller's back is turned, the buyer takes back one $20
bill
When the seller turns around, the buyer hands over the stack of
bills, takes the receipt, and leaves
Between the time the security was checked (counting the bills)
and the access (exchanging the sculpture for the bills), a
condition changed

By Mohammed Al-Saleh / JUST

What was checked is no longer valid when the object (that is, the sculpture)
is accessed

28

Chapter 3

Time-of-Check-to-Time-of-Use Errors(TOCTTOU)
Computing Example:

file is a symbolic link for a file that can be opened normally

The attacker, after access is called, can change the file

The program with TOCTTOU

vulnerability
if (access("file", W_OK) != 0) {
exit(1);
}
//writing over /etc/passwd
fd = open("file",O_WRONLY);
write(fd, buffer, sizeof(buffer));

By Mohammed Al-Saleh / JUST

Attacker

// After the access check

// and Before the open, "file" points to
the password database
symlink("/etc/passwd", "file");

29

Much of the work done by a program is invisible

to users who are not likely to be aware of any
malicious activity
Can you tell

Chapter 3

Viruses and Other Malicious Code

if a game program does anything in addition to its

expected interaction with you?
Which files are modified by a word processor when
you create a document?
Which programs execute when you start your
computer or open a web page?

Most users cannot answer these questions

By Mohammed Al-Saleh / JUST

30

Chapter 3

Malicious Code

None of us like the unexpected, especially in our

programs

Malicious code behaves in unexpected ways

thanks to a malicious programmer's intention

Malicious code can do anything any other

program can

writing a message on a computer screen, stopping a

running program, generating a sound, or erasing a
stored file.
Or malicious code can do nothing at all right now; it
can be planted to lie dormant, undetected, until some
event triggers the code to act (e.g., based on time)

By Mohammed Al-Saleh / JUST

31

Chapter 3

Malicious Code
malicious code is still around, and its effects are
more pervasive

What it looks like and how it works?

How can malicious code take control of a system?
How can it lodge in a system?
How does malicious code spread?
How can it be recognized?
How can it be detected?
How can it be stopped?
How can it be prevented?

By Mohammed Al-Saleh / JUST

32

Code Type

Characteristics

Virus

Attaches itself to program and

propagates copies of itself to other
programs

Worm

Propagates copies of itself through a

network

Trojan horse

Looks legal/normal programs, but

contains unexpected, additional
functionality

Logic bomb

Triggers action when condition occurs

Time bomb

Triggers action when specified time

occurs

Trapdoor/backdoor

Allows unauthorized access to

functionality

Rabbit

Replicates itself without limit to

exhaust resources

By Mohammed Al-Saleh / JUST

Chapter 3

Kinds of Malicious Code

33

Chapter 3

General Exploit Timeline

The general exploit timeline/scenario follows this sequence:

An attacker discovers a previously unknown vulnerability.

The manufacturer becomes aware of the vulnerability.
Someone develops code (called proof of concept) to demonstrate
the vulnerability in a controlled setting.
The manufacturer develops and distributes a patch or workaround that counters the vulnerability.
Users implement the control.
Someone extends the proof of concept, or the original vulnerability
definition, to an actual attack.

As long as users receive and implement the control

before the actual attack, no harm occurs.
An attack before availability of the control is called a zero
day exploit.

By Mohammed Al-Saleh / JUST

34

Chapter 3

How Viruses Attach

For a virus to do its malicious work and spread itself, it

must be activated by being executed

Many ways to ensure that programs will be executed

E.g., the SETUP program call dozens or hundreds of other
programs

Running an infected program obtained from distribution medium,

such as a CD, or opening an e-mail attachment are common way
for viruses to get activated.

If any one of these programs contains a virus, the virus code could be activated

Also, objects such as graphics or photo images can contain code to be

executed by an editor/viewer

it is a bad idea for programs to perform potentially security-relevant

actions without a user's consent

By Mohammed Al-Saleh / JUST

However, ease-of-use often trumps security, so programs such as browsers, email handlers, and viewers often "helpfully" open files without asking the user
first
35

Chapter 3

Appended Viruses
A program virus attaches itself to a program;
then, whenever the program is run, the virus is
activated.

By Mohammed Al-Saleh / JUST

36

Chapter 3

Appended Viruses
An alternative to the attachment is a virus that
runs the original program but has control before
and after its execution

Virus Surrounding a Program.

By Mohammed Al-Saleh / JUST

37

Chapter 3

Appended Viruses
A third situation occurs when the virus replaces
some of its target, integrating itself into the
original code of the target.

Virus Integrated into a Program

By Mohammed Al-Saleh / JUST

38

By Mohammed Al-Saleh / JUST

Chapter 3

Virus Completely Replacing a Program

39

By Mohammed Al-Saleh / JUST

Chapter 3

Boot Sector Viruses

40

Chapter 3

Memory-Resident Viruses

Some parts of the operating system and most user

programs execute, terminate, and disappear

For very frequently used parts of the operating system and for a
few specialized user programs, it would take too long to reload
the program each time it was needed

Such code remains in memory and is called "resident" code

E.g., a routine that interprets keys pressed on the keyboard

Virus writers like to attach viruses to resident code

A virus can also modify the operating system's table of programs
to run
E.g., changing the startup programs from Windows registry so
the virus starts every boot

Also, viruses like application programs (such as

MS word) and frequently used libraries

By Mohammed Al-Saleh / JUST

41

Chapter 3

Virus Signatures
The pattern which distinguishes a virus is called
a signature.

Anti-viruses (or called virus scanners) look for

signatures to identify a virus
The signature is part of the virus code

By Mohammed Al-Saleh / JUST

42

Chapter 3

Hard-to-Find Viruses
Properties
It is hard to detect.
It is not easily destroyed or deactivated.
It spreads infection widely.
It can reinfect its home program or other programs.
It is easy to create.
It is machine independent and operating system
independent.
Few viruses meet all these criteria. The virus writer
chooses from these objectives when deciding what the
virus will do and where it will reside.

By Mohammed Al-Saleh / JUST

43

Chapter 3

Polymorphic Viruses
A virus that can change its appearance is called
a polymorphic virus. (Poly means "many" and
morph means "form.")

By Mohammed Al-Saleh / JUST

44

Chapter 3

Prevention of Virus Infection

several techniques for building a reasonably safe

community

Use only commercial software acquired from reliable,

well-established vendors
Use virus detectors (often called virus scanners)
regularly and update them daily
Open attachments only when you know them to be
safe
Make a recoverable system image and store it safely
Make and retain backup copies of executable system
files
Test all new software on an isolated computer

By Mohammed Al-Saleh / JUST

45

Chapter 3

The Brain Virus

One of the earliest viruses
one of the most intensively studied
was given its name because it changes the label of any
disk it attacks to the word "BRAIN."
originated in Pakistan
attacks PCs running an old Microsoft operating system
The Brain virus positions itself in the boot sector and in
six other sectors of the disk
The Brain virus appears to have no effect other than
passing its infection

By Mohammed Al-Saleh / JUST

46

Chapter 3

The Internet Worm

2 November 1988
caused serious damage to the network
The perpetrator was Robert T. Morris, Jr., a graduate
student at Cornell University

He was convicted in 1990 of violating the 1986 Computer Fraud

and Abuse Act. He received a fine of $10,000, a three-year
suspended jail sentence, and was required to perform 400 hours
of community service

Morris programmed the Internet worm to accomplish

three main objectives:

Determine where it could spread to.

Spread its infection.
Remain undiscovered and undiscoverable.

By Mohammed Al-Saleh / JUST

47

Chapter 3

The Internet Worm

The worm exploited several known flaws and

configuration failures of Berkeley version 4 of the Unix
operating system
The worm's primary effect was resource exhaustion

Many copies of the worm, all busily attempting to spread the

infection

A second-order effect was the disconnection of many

systems from the Internet
third-order effect: isolation and inability to perform
necessary work
The worm caused an estimated 6,000 installations to shut
down or disconnect from the Internet
cost of the damage range from $100,000 to $97 million.

By Mohammed Al-Saleh / JUST

48

Chapter 3

Code Red

appeared in the middle of 2001

propagates itself on web servers running Microsoft's
Internet Information Server (IIS) software
infected more than 250,000 systems in just nine hours
This spread has the potential to disrupt business and
personal use of the Internet for applications such as ecommerce, e-mail and entertainment
the Code Red worm struck faster than any other worm in
Internet history
overall, 750,000 servers were affected, including 400,000
just in the period from August 1 to 10
Code Red's damage was expected to exceed $2 billion

By Mohammed Al-Saleh / JUST

49

Chapter 3

Keystroke Logging

First, recognize that there is not a direct path between a

key you press on your keyboard and the program (let's
say a word processor) that handles that keystroke.

When you press A

it activates a switch that generates a signal that is received by a device

driver, converted and analyzed and passed along, until finally your word
processor receives the A
there is still more conversion, analysis, and transmission until the A appears
on your screen

A malicious program called a keystroke logger retains a

surreptitious copy of all keys pressed

By Mohammed Al-Saleh / JUST

50

Chapter 3

Man-in-the-Middle Attacks
A keystroke logger is a special form of the more
general man-in-the-middle attack
malicious program interjects itself between two
other programs
One example of a man-in-the-middle attack
could be a program that operated between your
word processor and the file system

each time you thought you were saving your file, the
middle program prevented that, or scrambled your
text or encrypted your file

By Mohammed Al-Saleh / JUST

51

development controls

limit software development activities, making it harder for a

developer to create malicious (or inadvertent) programs
produce better software

operating system controls

Chapter 3

Controls Against Program Threats

limit access to computing system objects and provides safe

sharing of information among programs

administrative controls

limit the kinds of actions people can take

improve system usability, reusability, and maintainability

By Mohammed Al-Saleh / JUST

52