Concepts and Notations for Concurrent Programming

GREGORY R. ANDREWS
Department of Computer Science, University of Arizona, Tucson, Arizona 85721
FRED B. SCHNEIDER
Department of Computer Science, Cornell University, Ithaca, New York 14853 .

Much has been learned in the last decade about concurrent programming..This patmr
identifies the major concepts of concurrent programming and describes some of the more
importam language notations for writing concurrent programs. The roles of processes,
communication, and synchronization are discussed. Language notations for expressing
concurrent execution and for specifying process interaction are surveyed. Synchronization
primitives based on shared variables and on message passing are described. Finally, three
general classes of concurrent programming languages are identified and compared.

Categories and Subject Descriptors: D. 1.3 [Programming Techniques]: Concurrent

Programming; D.3.3 [Programming Languages]: Language Constructs--concurrent
programming structures; coroutines; D.4.1 [Operating Systems]: Process Management; D.4.7
[Operating Systems]: Organization and Design
General Terms: Algorithms, Languages

INTRODUCTION point that there are now undergraduate-

level text books devoted solely to the topic
The complexion of concurrent program- [Holt et al., 1978; Ben-Ari, 1982]. In light of
ming has changed substantially in the past this growing range of applicability, it seems
ten years. First, theoretical advances have appropriate to survey the state of the art.
prompted the definition of new program- This paper describes the concepts central
ming notations that express concurrent to the design and construction of concur-
computations simply, make synchroniza- rent programs and explores notations for
tion requirements explicit, and facilitate describing concurrent computations. Al-
formal correctness proofs. Second, the though this description requires detailed
availability of inexpensive processors has discussions of some c o n c u r r e n t program-
made possible the construction of distrib- ming languages, we restrict attention to
uted systems and multiprocessors that were those whose designs we believe to be influ-
previously economically infeasible. Because ential or conceptually innovative. Not all
of these two developments, concurrent pro- the languages we discuss enjoy widespread
gramming no longer is the sole province of use. Many are experimental efforts that
those who design and implement operating focus on understanding the interactions of
systems; it has become important to pro- a given collection of constructs. Some have
grammers of all kinds of applications, in- not even been implemented; others have
cluding database management systems, been, but with little concern for efficiency,
large-scale parallel scientific computations, access control, data types, and other impor-
and real-time, embedded control systems. tant (though nonconcurrency) issues.
In fact, the discipline has matured to the We proceed as follows. In Section 1 we

Permission to copy without fee all or part of this material is granted provided that the copies are not made or
distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its
dateappear, and notice is given that copying is by permission of the Association for Computing Machinery. To
copy otherwise, or to republish, requires a fee and/or specific permission.
© 1983 ACM 0010-4892/83/0300-0003 $00.75

Computing Surveys, Vol. 15, l ~ ~! March 1988

4 ° G.R. Andrews and F. B. Schneider
CONTENTS structs for performing remote procedure
calls and atomic transactions. In Section 5
we identify and compare three general
classes of concurrent programming lan-
INTRODUCTION guages. Finally, in Section 6, we summarize
1. CONCURRENT PROGRAMS: PROCESSES the major topics and identify directions in
AND PROCESS INTERACTION which the field is headed.
1.1 Processes
1.2 Process Interaction
2. SPECIFYING CONCURRENT EXECUTION
1. CONCURRENT PROGRAMS: PROCESSES
2.1 Coroutines
2.2 The fork and join Statements AND PROCESS INTERACTION
2.3 The cobegin Statement
2.4 Process Declarations 1.1 Processes
3. SYNCHRONIZATION PRIMITIVES BASED
ON SHARED VARIABLES A sequential program specifies sequential
3.1 Busy-Waiting execution of a list of statements; its execu-
3.2 Semaphores
3.3 Conditional Critical Regions tion is called a process. A concurrent pro-
3.4 Monitors gram specifies two or more sequential pro-
3.5 Path Expressions grams that may be executed concurrently
4. SYNCHRONIZATION PRIMITIVES BASED as parallel processes. For example, an air-
ON MESSAGE PASSING line reservation system that involves proc-
4.1 Specifying Channels of Communication
4.2 Synchronization essing transactions from many terminals
has a natural specification as a concurrent
4.3 Higher Level Message-Passing Constructs
4.4 An Axiomatic View of Message Passingprogram in which each terminal is con-
4.5 Programming Notations Based on Message
trolled by its own sequential process. Even
Passing
when processes are not executed simulta-
5. MODELS OF CONCURRENT
PROGRAMMING LANGUAGES neously, it is often easier to structure a
6. CONCLUSION system as a collection of cooperating se-
ACKNOWLEDGMENTS quential processes rather than as a single
REFERENCES sequential program. A simple batch oper-
ating system can be viewed as three proc-
v
esses: a reader process, an executer process,
and a printer process. The reader process
discuss the three issues that underlie all reads cards from a card reader and places
concurrent programming notations: how to card images in an input buffer. The execu-
express concurrent execution, how pro- ter process reads card images from the in-
cesses communicate, and how processes put buffer, performs the specified compu-
synchronize. These issues are treated in tation (perhaps generating line images),
detail in the remainder of the paper. In and stores the results in an output buffer.
Section 2 we take a closer look at various The printer process retrieves line images
ways to specify concurrent execution: co- from the output buffer and writes them to
routines, f o r k and c o b e g i n statements, a printer.
and p r o c e s s declarations. In Section 3 we A concurrent program can be executed
discuss synchronization primitives that are either by allowing processes to share one or
used when communication uses shared more processors or by running each process
variables. Two general types of synchroni- on its own processor. The first approach is
zation are considered--exclusion and con- referred to as multiprogramming; it is sup-
dition synchronizationmand a variety of ported by an operating system kernel [Dijk-
ways to implement them are described: stra, 1968a] that multiplexes the processes
busy-waiting, semaphores, conditional crit- on the processor(s). The second approach
ical regions, monitors, and path expres- is referred to as multiprocessing if the
sions. In Section 4 we discuss message-pass- processors share a common memory (as
ing primitives. We describe methods for in a multiprocessor [Jones and Schwarz,
specifying channels of communication and 1980]), or as distributed processing if the
for synchronization, and higher level con- processors are connected by a communica-

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming * 5

tions network. 1 Hybrid approaches also ex- buffer is used for communication between
i s t - f o r example, processors in a distributed the reader process and the executer proc-
system are often multiprogrammed. ess. These processes must be synchronized
The rate at which processes are executed so that, for example, the executer process
depends on which approach is used. When never attempts to read a card image from
each process is executed on its own pro- the input if the buffer is empty.
cessor, each is executed at a fixed, but per- This view of synchronization follows
haps unknown, rate; when processes share from taking an operational approach to
a processor, it is as if each is executed on program semantics. An execution of a con-
a variable-speed processor. Because we current program can be viewed as a se-
would like to be able to understand a con- quence of atomic actions, each resulting
current program in terms of its component from the execution of an indivisible opera-
sequential processes and their interaction, tion. 2 This sequence will comprise some
without regard for how they are executed, interleaving of t h e sequences of atomic ac-
we make no assumption about execution tions generated by the individual compo-
rates of concurrently executing processes, nent processes. Rarely do all execution in-
except that they all are positive. This is terleavings result in acceptable program be-
called the finite progress assumption. The havior, as is illustrated in the following.
correctness of a program for which only Suppose initially t h a t x ffi 0, that process
finite progress is assumed is thus independ- P1 increments x by 1, and that process P2
ent of whether that program is executed on increments x by 2:
multiple processors or on a single multipro- PI: x :ffix + 1 P2: x:-- x + 2
grammed processor.
It would seem reasonable to expect the final
1.2 Process Interaction value of x, after P1 and P2 have executed
In order to cooperate, concurrently execut- concurrently, to be 3. Unfortunately, this
ing processes must communicate and syn- will not always be the case, because assign-
chronize. Communication allows execution ment statements are not generally imple-
of one process to influence execution of mented as indivisible operations. For ex-
another. Interprocess communication is ample, the above assignments might be
based on the use of shared variables (vari- implemented as a sequence of three indi-
ables that can be referenced by more than visible operations: (i) load a register with
one process) or on message passing. the value of x; (ii) add 1 or 2 to it; and (iii)
Synchronization is often necessary when store the result in x. Thus, in the program
processes communicate. Processes are exe- above, the final value of x might be 1, 2, or
cuted with unpredictable speeds. Yet, to 3. This anomalous behavior can be avoided
communicate, one process must perform by preventing interleaved execution of the
some action that the other detects--an ac- two assignment statements--that is, by
tion such as setting the value of a variable controlling the ordering of the" events cor-
or sending a message. This only works if responding to the atomic actions. (If order-
the events "perform an action" and "detect ing were thus controlled, each assignment
an action" are constrained to happen in statement would be an indivisible opera-
that order. Thus one can view synchroni- tion.) In other words, execution of 1)1 and
zation as a set of constraints on the ordering P2 must be synchronized by enforcing re-
of events. The programmer employs a syn- strictions on possible interleavings.
chronization mechanism to delay execu- The axiomatic approach [Floyd, 1967;
tion of a process in order to satisfy such Hoare, 1969; Dijkstra, 1976] provides a sec-
constraints.
To make the concept of synchronization 2 W e assume t h a t a single m e m o r y reference is indivi-
a bit more concrete, consider the batch sible; ff two processes a t t e m p t to reference the same
m e m o r y cell at t h e same time, t h e result is as if t h e
operating system described above. A shared references were m a d e serially. This is a reasonable
assumption in light of the way m e m o r y is constructed.
1A concurrent program that is executed in this way is See L a m p o r t [19801)] for a discussion of some of t h e
often called a distributed program. implications of relaxing this assumption.

Computing Surveys,Vol. 15,No. I,March 1983

6 * G.R. Andrews a n d F. B. Schneider
ond framework in which to view the role of another. These additional theorems consti-
synchronization. 3 In this approach, the s e - tute a proof of noninterference. To illus-
m a n t i c s of statements are defined, by ax- trate this, consider the following excerpt
ioms and inference rules. This results from a proof outline of two concurrent pro-
in a formal logical system, called a cesses P1 and P2:
"programming logic." Theorems in the logic
PI: ... P2: ...
have the form
{x > O) {x < 0}
{P} S {Q} $1: x := 16 $2: x := - 2
{x -- 16} ...
and specify a relation between statements * * o

(S) and two predicates, a precondition P

and a postcondition Q. The axioms and In order to prove that execution of P2 does
inference rules are chosen so that theorems not interfere with the proof of P1, part of
have the interpretation that if execution of what we must show is that execution of $2
S is started in any state that satisfies the does not invalidate assertions {x > 0} and
precondition, and if execution terminates, {x = 16} in the proof of P1. This is done by
then the postcondition will be true of the proving
resulting state. This allows statements to
be viewed as relations between predicates. (x < 0 a n d x > 0} x :ffi - 2 (x>0}
A proof outline 4 provides one way to and
present a program and its proof. It consists
{ x < 0 a n d x > 0} x :ffi - 2 ( x ffi 16}
of the program text interleaved with asser-
tions so that for each statement S, the triple Both of these are theorems because the
(formed from (I) the assertion that tex- precondition of each, {x < 0 a n d x > 0}, is
tually precedes S in the proof outline, (2) false. What we have shown is that execution
the statement S, and (3) the assertion that of $2 is not possible when either the pre-
textually follows S in the proof outline) is condition or postcondition of $1 holds (and
a theorem in the programming logic. Thus thus $1 and $2 are mutually exclusive).
the appearance of an assertion R in the Hence, $2 cannot invalidate either of these
proof outline signifies that R is true of the assertions.
program state when control reaches that Synchronization mechanisms control in-
point. terference in two ways. First, they can delay
When concurrent execution is possible, execution of a process until a given condi-
the proof of a sequential process is valid tion {assertion) is true. By so doing, they
only if concurrent execution of other pro- ensure that the precondition of the subse-
cesses cannot invalidate assertions that ap- quent statement is guaranteed to be true
pear in the proof [Ashcroft, 1975; Keller, (provided that the assertion is not inter-
1976; Owicki and Gries, 1976a, 1976b; Lam- fered with). Second, a synchronization
port, 1977, 1980a; Lamport and Schneider, mechanism can be used to ensure that a
1982]. One way to establish this is to assume block of statements is an indivisible opera-
that the code between any t w o assertions tion. This eliminates the possibility of state-
in a proof outline is executed atomically s ments in other processes interfering with
and then to prove a series of theorems assertions appearing within the proof of
showing t h a t no statement in one process that block of statements.
invalidates any assertion in the proof of Both views of programs, operational and
axiomatic, are useful. The operational ap-
3 We include brief discussions of axiomatic semantics p r o a c h - v i e w i n g synchronization as an or-
here and elsewhere in the paper because of its impor- dering of events--is well suited to explain-
tance in helping to explain concepts. However, a full ing how synchronization mechanisms work.
discussion of the semantics of concurrent computation For that reason, the operational approach
is beyond the scope of this paper.
a T h i s sometimes is called an asserted program.
is used rather extensively in this survey. It
s T h i s should be construed as specifying what asser- also constitutes the philosophical basis for
tions m u s t be included in the proof rather t h a n as a a family of synchronization mechanisms
restriction on how statements are actually executed. calledpath expressions [Campbell and Ha-

Computing Surveys, Vol. 15,No. I, March 1983

 Concepts and Notations for Concurrent Programming ° 7
bermann, 1974], which are described in Sec- proach, because the s t i f l e of the pro-
tion 3.5. gram itself clarifies the proof obligations for
Unfortunately, the operational approach establishing noninterference.
does not really help one understand the Below, we describe some representative
behavior of a concurrent program or afgne constructs for expressing concurrent exe-
convincingly about its correctness. Al- cution. Each can be used to specify com-
though it has borne fruit for simple concur- putations having a static (fixed) number of
rent programs~such as transactions pro- processes, or can be used in combination
cessed concurrently in a database system with process-creation mechanisms to spec-
[Bernstein and Goodman, 1981]--the op- ify computations having a dynamic (vari-
erational approach has only limited utility able) number of processes.
when applied to more complex concurrent
programs [Akkoyunlu et al., 1978; Bern- 2.1 Coroutines
stein and Schneider, 1978]. This limitation Coroutines are like subroutines, but allow
exists because the number of interleavings transfer of control in a symmetric rather
that must be considered grows exponen- than strictly hierarchical way [Conway,
tially with the size of the component se- 1963a]. Control is transferred between co-
quential processes. Human minds are not routines by means of the r e s u m e state-
good at such extensive case analysis. The ment. Execution of r e s u m e is like execu-
axiomatic approach usually does not have tion of procedure call: it transfers control
this difficulty. It is perhaps the most prom- to the n a m e d routine, saving enough state
ising technique for understanding concur- information for control to return later to
rent programs. Some familiarity with for- the instruction following the resume.
mal logic is required for is use, however, (When a routine is firstresumed, control is
and this has slowed its acceptance. transferred to the beginning of that rou-
To summarize, there are three main is- tine.) However, control is returned to the
sues underlying the design of a notation for original routine by executing another re-
expressing a concurrent computation: s u m e rather than by executing a procedure
6) how to indicate concurrent execution; return. Moreover, any other coroutine can
(ii) which mode of interprocess communi- potentially transfer conWol back tothe orig-
cation to use; inal routine. (For example, coroutine C1
(iii) which synchronization mechanism to could r e s u m e C2, which could r e s u m e C3,
use. which could r e s u m e C1.) Thus r e s u m e
serves as the only way to transfer control
Also, synchronization mechanisms can be between coroutines, and one coroutine can
viewed either as constraining the ordering transfer control to any other coroutine that
of events or as controlling interference. We it chooses.
consider all these topics in depth in the A use of coroutines appears in Figure 1.
remainder of the paper. Note that r e s u m e is used to transfer con-
trol between c0routines A and B, a call is
2. SPECIFYING CONCURRENT EXECUTION used to initiate the coroutine Computation,
Various notations have been proposed for and return is used to transfer control back
specifying concurrent execution. Early pro- to the caller P. The arrows in Figure 1
posals, such as the f o r k statement, are indicate the transfers of control.
marred by a failure to separate process Each coroutine can be viewed as imple-
definition from process synchronization. menting a process. Execution of r e s u m e
Later proposals separate these distinct con- causes process sychronization. W h e n Used
cepts and characteristically possess syntac- with care, coroutines are an acceptable way
tic restrictions that impose some structure to organize concurrent programs that share
on a concurrent program. This structure a single processor. In fact, multiprogram-
allows easy identification of those program ruing can also be implemented using co-
segments that can be executed concur- routines. Coroutines are not adequate for
rently. Consequently, such proposals are true parallel processing, however, because
well suited for use with the axiomatic ap- their semantics allow for execution of only

Computing Su~eys, Vol. lS, No. 1, Mar~h 1983

8 • G . R . A n d r e w s a n d F. B. Schneider
p r o g r a m P,

call `4;
yco?n.ne
resume B;
end
r e s u m e .4;
resume B ~ ...
return

Figure 1. A use of coroutines.

one routine at a time. In essence, coroutines Because f o r k and j o i n can appear in

are concurrent processes in which process conditionals and loops, a detailed under-
switching has been completely specified, standing of program execution is necessary
rather than left to the discretion of the to understand which routines will be exe-
implementation. cuted concurrently. Nevertheless, when
Statements to implement coroutines used in a disciplined manner, the state-
have been included in discrete event simu- ments are practical and powerful. For ex-
lation languages such as SIMULA I [Ny- ample, f o r k provides a direct mechanism
gaard and Dahl, 1978] and its succes- for dynamic process creation, including
sors; the string-processing language SL5 multiple activations of the same program
[Hanson and Griswold, 1978]; and systems text. The U N I X 6 operating system [Ritchie
implementation languages including and Thompson, 1974] makes extensive use
BLISS [Wulf et al., 1971] and most recently of variants of f o r k and join. Similar state-
Modula-2 [Wirth, 1982]. ments have also been included in P L / I and
Mesa [Mitchell et al., 1979].
2.2 The fork and join Statements 2.3 The cobegin Statement
The f o r k statement [Dennis and Van The c o b e g i n statement 7 is a structured
Horn, 1966; Conway, 1963b], like a call or way of denoting concurrent execution of a
resume, specifies that a designated routine set of statements. Execution of
should start executing. However, the invok-
ing routine and the invoked routine proceed cobegin $1 II $2 II "'" USn coend
concurrently. To synchronize with comple- causes concurrent execution of $1, $ 2 , . . . ,
tion of the invoked routine, the invoking Sn. Each of the Si's may be any statement,
routine can execute a j o i n statement. Exe- including a c o b e g i n or a block with local
cuting j o i n delays the invoking routine un- declarations. Execution of a c o b e g i n state-
til the designated invoked routine has ter- ment terminates only when execution of all
minated. (The latter routine is often desig- the Si's have terminated.
nated by a value returned from execution Although c o b e g i n is not as powerful as
of a prior fork.) A use of f o r k and join f o r k / j o i n , s it is sufficient for specifying
follows:
6 UNIX is a trademark of Bell Laboratories.
program P1; program P2; 7 This was first called p a r b e g i n by Dijkstra [1968b].
, . . o , o
s Execution of a concurrent program can be repre-
fork P2; ... sented by a process flow graph: an acyclic, directed
° o . o o .
graph having one node for each process and an arc
from one node to another if the second cannot execute
join P2; end until the fncst has terminated [Shaw, 1974]. Without
o , ,
introducing extra processes or idle time, c o b e g i n
and sequencing can only represent series-parallel
Execution of P2 is initiated when the f o r k (properly nested) process flow graphs. Using f o r k and
in P1 is executed; P1 and 1'2 then execute j o i n , the computation represented by any process flow
concurrently until either P1 executes the graph can be specified directly. Furthermore, f o r k can
be used to create an arbitrary number of concurrent
join statement or P2 terminates. After P1 processes, whereas c o b e g i n as defined in any existing
reaches the j o i n and P2 terminates, P1 language, can be used only to activate a fLxed number
executes the statements following the join. of processes.

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent ]Programming
program OPS YS;

vat input_buffer : array [0.. N - l ] of cardimage;

output_buffer : array [O..N-I] of lineimage;
process reader;
var card " eardimage;
loop
read card from cardreader;
deposit card in input_buffer
end
end;
process executer;
var card : cardimage;
line : lineimage;
loop
fetch card from input_buffer;
process card and generate line;
deposit line in output_buffer
end
end;
process printer;
var line : lineimage;
loop
fetch line from output_buffer;
print line on lineprinter
end
end

end.
Figure 2. Outline of batch operating system.

most concurrent computations. Further- The process declaration provides such a

more, the syntax of the c o b e g i n statement facility.
makes explicit which routines are executed Use of process declarations to structure
concurrently, and provides a single-entry, a concurrent program is illustrated in Fig-
single-exit control structure. This allows ure 2, which outlines the batch operating
the state transformation implemented by a system described earlier. We shall use this
c o b e g i n to be understood by itself, and notation for process declarations in the re-
then to be used to understand the program mainder of this paper to denote collections
in which it appears. of routines that are executed concurrently.
Variants of c o b e g i n have been included In some concurrent programming lan-
in ALGOL68 [van Wijngaarden et al., guages {e.g., Distributed Processes [Brinch
1975], Communicating Sequential Pro- Hansen, 1978] and SR [Andrews, 1981]), a
cesses [Hoare, 1978], Edison [Brinch Han- collection of process declarations is equiv-
sen, 1981], and Argus [Liskov and Scheifler, alent to a single cobegin, where each of
1982]. the declared processes is a component of
the cobegin. This means there is exactly
one instance of each declared process. Al-
2.4 Process Declarations
ternatively, some languages provide an ex-
Large programs are often structured as a plicit m e c h a n i s m - - f o r k or something sim-
collection of sequential routines, which are i l a r - f o r activating instances of process
executed concurrently. Although such rou- declarations. This explicit activation mech-
tines could be declared as procedures and anism can only be used during program
activated by means of c o b e g i n or fork, the initialization in some languages (e.g., Con-
structure of a concurrent program is much current PASCAL [Brinch Hansen, 1975]
clearer if the declaration of a routine states and Modula [Wirth, 1977a]). This leads to
whether it will be executed concurrently. a fixed number of processe~ but allows mul-
Computing $urveys,~ol. 15, No. 1, Ma~h 1983
10 ° G. R. Andrews and F. B. Schneider
tiple instances of each declared process to ing to execute a "deposit" operation on a
be created. By contrast, in other languages buffer (the buffer being a shared data ob-
(e.g., PLITS[Feldman, !979] and Ada ject) should be delayed if the buffer has no
[U. S. Department of Defense, 1981]) pro- space. Similarly, a process attempting to
cesses can be created at any time during "fetch" from a buffer should be delayed if
execution, which makes possible computa- there is nothing in the buffer to remove.
tions having a variable number of pro- Below, we survey various mechanisms for
cesses. implementing these two types of synchro-
nization.
3. SYNCHRONIZATION PRIMITIVES BASED
ON SHARED VARIABLES 3.1 Busy-Waiting
When shared variables are used for inter- One way to implement synchronization is
process communication, two types of syn- to have processes set and test shared vari-
chronization are useful: mutual exclusion ables. This approach works reasonably well
and condition synchronization. M u t u a l ex- for implementing condition synchroniza-
clusion ensures that a sequence of state- tion, but not for implementing mutual ex-
ments is treated as an indivisible operation. clusion, as will be seen. To signal a condi-
Consider, for example, a complex data tion, a process sets the value of a shared
structure manipulated by means of opera- variable; to wait for that condition, a proc-
tions implemented as sequences of state- ess repeatedly tests the variable until it is
ments. If processes concurrently perform found to have a desired value. Because a
operations on the same shared data object, process waiting for a condition must re-
then unintended results might occur. (This peatedly test the shared variable, this tech-
was illustrated earlier where the statement nique to delay a process is called busy-wait-
x :ffi x + 1 had to be executed indivisibly ing and the process is said to be spinning.
for a meaningful computation to result.) A Variables that are used in this way are
sequence of statements that must appear sometimes called spin locks.
to be executed as an indivisible operation is To implement mutual exclusion using
called a critical section. The term "mutual busy-waiting, statements that signal and
exclusion" refers to mutually exclusive ex- wait for conditions are combined into care-
ecution of critical sections. Notice that the fully constructed protocols. Below, we pres-
effects of execution interleavings are visible ent Peterson's solution to the two-pro-
only if two computations access shared var- cess mutual exclusion problem [Peterson,
ibles. If such is the case, one computation 1981]. (This solution is simpler than the so-
can see intermediate results produced by lution proposed by Dekker [Shaw, 1974].)
incomplete execution of the other. If two The solution involves an entry protocol,
routines have no variables in common, then which a process executes before entering its
their execution need not be mutually exclu- critical section, and an exit protocol, which
sive. a process executes after finishing its critical
Another situation in which it is necessary section:
to coordinate execution of concurrent proc- process P1;
esses occurs when a shared data object is in loop
a state inappropriate for executing a partic- Entry Protocol;
ular operation. Any process attempting Critical Section;
such an operation should be delayed until Exit Protocol;
the state of the data object (i.e., the values Noncritical Section
of the variables that comprise the object) end
changes as a result of other processes exe- end
cuting operations. We shall call this type process P2;
of synchronization condition synchroniza- loop
tion. 9 Examples of condition synchroniza- Entry Protocol;
tion appear in the simple batch operating Critical Section;
system discussed above. A process attempt- Exit Protocol;
Noncritical Section
9 U n f o r t u n a t e l y , t h e r e is no ' c o m m o n l y agreed u p o n end
t e r m for this. end
Computing Surveys, Vol. 15, No. 1, March 1983
 Concepts and Notations for Concurrent Programming • 11
Three shared variables are used as follows that the finite progress assuraption is not
to realize the desired synchronization. Boo- invalidated by delays due tO ~synchroniza.
lean variable enteri (i -- I or 2) is true when tion. In general, a synchronization mecha-
process Pi is executing its entry protocol or nism is fair if no process is delayed forever,
its critical section. Variable turn records waiting for a condition that occurs infinitely
the name of the next process to be granted often; it is bounded fair if there exists an
entry into its own critical section; turn is upper bound on how l o n g a process will be
used when both processes execute their re- delayed waiting for a condition that occurs
spective entry protocols at about the same infinitely often. The above protocol is
time. The solution is bounded fair, since a process waiting to
p r o g r a m Mute.,:_ExanTph,;
enter its critical section is delayed for at
most one execution of the other process'
var enter l, enter2 : Boolean initial {false~false);
turn : integer initial ("PI"); { or "P2'" }
critical section; the variable turn ensures
this. Peterson [1981] gives operational
process P1;
loop proofs of mutual exiZlusion, deadlock free-
Entry_Protocol: dom, and fairness; Dijkstra [1981a] gives
enterl := true: { a n n o u n c e intent to enter } axiomatic ones.
turn : = " P 2 " : { set priority to other process } Synchronization protocols that use only
wh,e enter2and turn= "Re" busy-waiting are difficult to design, under-
do skip; { wait if other process is in and it is his turn !
Critical Section; stand, and prove correct. First, although
Exit_Protocol: instructions that make two memory refer-
enterl :=./ah'e; { renounce intent to enter } ences part of a single indivisible operation
Noncritical Section (e.g., the T S (test-and-set) instruction on
end
end:
the IBM 360/370 processors) help, such
instructions do n o t significantly simplify
process P2:
loop
the task of designing synchronization pro-
Entry_Protocol: tocols. Second, busy-waiting wastes pro-
enter2 := true; { a n n o u n c e intent to enter } cessor cycles. A processor executing a spin-
turn := " P I ' ; { set priority to other process ] ning process could usually b e ~employed
while enterl and turn ="PI'"
do skip; { wait if other process is in and it is his turn }
more productively by running other pro-
Critical Section: cesses until the awaited ~condition occurs.
Exit_Protocol: Last, the busy-waiting~approach to syn-
enterl :=.false; { renounce intent to enter } chronization burdens the programmer with
Noncritical Section deciding both what synchronization is re-
end quired and how to provide it. In reading a
end
program that uses busy-waiting, it may not
end.
be clear to the reader which program vari-
In addition to implementing mutual ex- ables are used for implementing synchro-
clusion, this solution has two other desira- nization and which are used for, say, inter-
ble properties. First, it is deadlock free. process communication.
Deadlock is a state of affairs in which two
or more processes are waiting for events
that will never occur. Above, deadlock 3.2 Semaphores
could occur if each process could spin for-
ever in its entry protocol; using turn pre- Dijkstra was one of the first to appreciate
cludes deadlock. The second desirable the difficulties of using low-level mecha-
property is fairness: 1° if a process is trying nisms for process synchronization, and this
to enter its critical section, it will eventually prompted his development of semaphores
be able to do so, provided that the other [Dijkstra, 1968a, 19681)]. A semaphore is a
process exits its critical section. Fairness is nonnegative integer-valued variable on
a desirable property for a synchronization which two operations are defined: P and V.
mechanism because its presence ensures Given a semaphore s, P(s) delays until
s > 0 and then executes s ~ s - 1; the test
~°A more complete discussion of fairness appears in and decrement ar~ executed as an indivis-
L e h m a n n e t al. [ 1 9 8 1 ] . ible operation. V(s) executes s :ffis + 1 as
Compufliig Surveys, VoL 16, No. I, Match 1983
12 • G. R. Andrews and F. B. Schneider
an indivisible operation, n M o s t s e m a p h o r e for some types of condition synchroniza-
implementations are assumed to exhibit tion, n o t a b l y those in which a resource has
fairness: no process delayed while executing only one unit.
P(s) will r e m a i n delayed forever if V(s) A few examples will illustrate uses of
operations are performed infinitely often. semaphores. We show a solution to the two-
T h e need for fairness arises when a n u m b e r process m u t u a l exclusion problem in t e r m s
of processes are simultaneously delayed, all of s e m a p h o r e s in the following:
a t t e m p t i n g to execute a P operation on the program Mutex__Example;
same semaphore. Clearly, the implementa-
tion m u s t choose which one will be allowed var m u t e x : semaphore initial (i);
to proceed w h e n a V is ultimately per- process P1;
formed. A simple way to ensure fairness is loop
to awaken processes in the order in which P(mutex); { Entry Protocol }
t h e y were delayed. Critical Section;
S e m a p h o r e s are a v e r y general tool for V(mutex); [ Exit Protocol }
solving synchronization problems. T o im- Noncritical Section
p l e m e n t a solution to the m u t u a l exclusion end
problem, each critical section is preceded end;
b y a P operation and followed b y a V
operation on the same semaphore. All mu- process P2;
loop
tually exclusive critical sections use the
same semaphore, which is initialized to one.
P(mutex); { Entry Protocol }
Critical Section;
Because such a s e m a p h o r e only takes on
the values zero and one, it is often called a
V(mutex); { Exit Protocol }
Noncritical Section
binary semaphore.
end
T o i m p l e m e n t condition synchronization,
end
shared variables are used to r e p r e s e n t the
condition, and a s e m a p h o r e associated with end.
the condition is used to accomplish the Notice how simple and symmetric the e n t r y
synchronization. After a process has m a d e and exit protocols are in this solution to the
the condition true, it signals t h a t it has m u t u a l exclusion problem. In particular,
done so b y executing a V operation; a pro- this use of P and V ensures b o t h m u t u a l
cess delays until a condition is true b y exclusion and absence of deadlock. Also, if
executing a P operation. A s e m a p h o r e t h a t the s e m a p h o r e i m p l e m e n t a t i o n is fair and
can take any nonnegative value is called a b o t h processes always exit their critical sec-
general or counting semaphore. General tions, each process eventually gets to enter
s e m a p h o r e s are often used for condition its critical section.
synchronization w h e n controlling resource S e m a p h o r e s can also be used to solve
allocation. Such a s e m a p h o r e has as its selective mutual exclusion problems. In the
initial value the initial n u m b e r of units of latter, s h a r e d variables are partitioned into
the resource; a P is used to delay a process disjoint sets. A s e m a p h o r e is associated
until a free resource unit is available; V is with each set and used in the same way as
executed w h e n a unit of the resource is mutex above to control access to the vari-
returned. B i n a r y s e m a p h o r e s are sufficient ables in t h a t set. Critical sections t h a t ref-
erence variables in the same set execute
with m u t u a l exclusion, b u t critical sections
n p is the first letter of the Dutch word "passeren," t h a t reference variables in different sets
which means "to pass"; V is the first letter of execute concurrently. However, if two or
"vrygeven," the Dutch word for "to release" [Dijkstra,
1981b]. Reflecting on the definitions of P and V, m o r e processes require simultaneous access
Dijkstra and his group observed the P might better to variables in two or more sets, the pro-
stand for "prolagen" formed from the Dutch words g r a m m e r m u s t take care or deadlock could
"proberen" (meaning "to try") and "verlagen" (mean-
ing "to decrease") and V for the Dutch word result. Suppose t h a t two processes, P1 and
"verhogen" meaning "to increase." Some authors use P2, each require simultaneous access to sets
wait for P and signal for V. of s h a r e d variables A and B. T h e n , P1 and

ComputingSurveys,Vol.15,No. 1, March 1983

 Concepts and Notations for Concurrent Programming • 13

P2 will deadlock if, for example, P1 acquires program OPSY~,, ....

access to set A, P2 acquires access to set B, vat in_mutex, out.Jnulex : semaphore initial (1, !);
and then both processes try to acquire ac- num_.cards, numJines : semaphoTe initial (0,0);
cess to the set that they do not yet have. free_cards, free-lines : semaphore initial (N,N);
Deadlock is avoided here (and in general) input_buffer : array [O..N-I] of:cardimage;
if processes first try to acquire access to the output..buffer : array [O,.N-I] of lineimage;
same set (e.g., A), and then try to acquire process reader;
access to the other (e.g., B). var card : cardimage;
Figure 3 shows how semaphores can be loop
read card from cardreader;
used for selective mutual exclusion and con- POCree_cards); P(in_mutex);
dition synchronization in an implementa- deposit card in input..buffer;
tion of our simple example operating V(in_mutex); Y(num...cards)
system. Semaphore in___rnutex is used to end
implement mutually exclusive access to end;
input__buffer and out__mutex is used to process executer;
implement mutually exclusive access to var card : eardimage;
output_buffer. 12 Because the buffers line : lineimage;
are disjoint, it is possible for opera- loop
tions on input__buffer and output__buffer P(num_cards); P(in_mutex);
fetch card from input_buffer;
to proceed concurrently. Semaphores V(in_muwx); Y(free-cards);
num cards, num__lines, free__cards, and process card and generate line;
free__lines are used for condition synchro- P(free..lines); P(out_mutex);
nization: num__cards (num__lines) is the deposit line in output_buffer;
number of card images (line images) that V(out_mutex); V(num._lines)
have been deposited but not yet fetched end
end;
from input__buffer (output__buffer);
free__cards (free__lines) is the number of process printer; ~
free slots in input__buffer (output__buffer). var line : lineimage; -~
Executing P(num__cards) delays a process loop
P(num_lines); P(out_mutex);
until there is a card in input___buffer; fetch line from output_buffer;
P(free__cards) delays its invoker until V(out_mutex); VOrree_lines);
there is space to insert a card in in- print line on lineprinter
put__buffer. Semaphores num__lines and end
free__lines play corresponding roles with end
respect to output__buffer. Note that before end.
accessing a buffer, each process first waits
for the condition required for access and Figure 3. Batch operating system with semaphores.
then acquires exclusive access to the buffer.
If this were not the case, deadlock could
result. (The order in which V operations 1974]. At all times, each process is either
are performed after the buffer is accessed is ready to execute on the processor or is
not critical.) blocked, waiting to complete a P operation.
Semaphores can be implemented by us- The kernel maintains a ready list--a queue
ing busy-waiting. More commonly, how- of descriptors for ready processes--and
ever, they are implemented by system calls multiplexes the processor among these
to a kernel. A kernel {sometimes called a processes, running each process for some
supervisor or nucleus) implements proc- period of time. Descriptors for processes
esses on a processor [Dijkstra, 1968a; Shaw, that are blocked on a semaphore are stored
on a queue associated with that semaphore;
~2In this solution, careful implementation of the op- they are not stored on the ready list, and
erations on the buffers obviates the need for sema- hence the processes will not be executed.
phores in_mutex and out_mutex. The semaphores
that implement condition synchronization are suffi- Execution of a P or V operation causes a
cient to ensure mutually exclusive access to individual trap to a kernel routine. For a P operation,
buffer slots. if the semaphore is positive, it is decre-

Computing Surveys, V~l, 15, No. 1, March 1983

14 • G. R. A n d r e w s a n d F. B. S c h n e i d e r

mented; otherwise the descriptor for the since mutually exclusive execution would
executing process is moved to the s e m a - no longer be ensured. Also, when using
p h o r e ' s queue. For a V operation, if the semaphores, a programmer can forget to
semaphore's queue is not empty, one de- include in critical sections all statements
scriptor is moved from that queue to the that reference shared objects. This, too,
ready list; otherwise the semaphore is in- could destroy the mutual exclusion re-
cremented. quired within critical sections. A second
This approach to implementing synchro- difficulty with using semaphores is that
nization mechanisms is quite general and is both condition synchronization and mutual
applicable to the other mechanisms that we exclusion are programmed using the same
shall discuss. Since the kernel is responsible pair of primitives. This makes it difficult to
for allocating processor cycles to processes, identify the purpose of a given P or V
it can implement a synchronization mech- operation without looking at the other op-
anism without using busy-waiting. It does erations on the corresponding semaphore.
this by not running processes that are Since mutual exclusion and condition syn-
blocked. Of course, the names and details chronization are distinct concepts, they
of the kernel calls will differ for each syn- should have distinct notations.
chronization mechanism, but the net effects The c o n d i t i o n a l critical region proposal
of these calls will be similar: to move pro- [Hoare, 1972; Brinch Hansen 1972, 1973b]
cesses on and off a ready list. overcomes these difficulties by providing a
Things are somewhat more complex structured notation for specifying synchro-
when writing a kernel for a multiprocessor nization. Shared variables are explicitly
or distributed system. In a multiprocessor, placed into groups, called resources. Each
either a single processor is responsible for shared variable may be in at most one
maintaining the ready list and assigning resource and may be accessed only in con-
processes to the other processors, or the ditional critical region (CCR) statements
ready list is shared [Jones and Schwarz, that name the resource. Mutual exclusion
1980]. If the ready list is shared, it is subject is provided by guaranteeing that execution
to concurrent access, which requires that of different CCR statements, each naming
mutual exclusion be ensured. Usually, busy- the same resource, is not overlapped. Con-
waiting is used to ensure this mutual exclu- dition synchronization is provided by ex-
sion because operations on the ready list plicit Boolean conditions in CCR state-
are fast and a processor cannot execute any ments.
process until it is able to access the ready A resource r containing variables vl,
list. In a distributed system, although one v 2 , . . . , v N is declared as 13
processor could maintain the ready list, it r e s o u r c e r: v l , v2, . . . . vN
is more common for each processor to have
its own kernel and hence its own ready list. The variables in r may only be accessed
Each kernel manages those processes resid- within CCR statements that name r. Such
ing at one processor; if a process migrates statements have the form
from one processor to another, it comes
region r when B do S
under the control of the other's kernel.
where B is a Boolean expression and S is a
3.3 Conditional Critical Regions statement list. (Variables local to the exe-
cuting process may also appear in the CCR
Although semaphores can be used to pro- statement.) A CCR statement delays the
gram almost any kind of synchronization, P executing process until B is true; S is then
and V are rather unstructured primitives, executed. The evaluation of B and execu-
and so it is easy to err when using them. tion of S are uninterruptible by other CCR
Execution of each critical section must be- statements that name the same resource.
gin with a P and end with a V (on the same
semaphore). Omitting a P or V, or acciden-
tally coding a P on one semaphore and a V ~3Our notation combines aspects of those proposed by
on another can have disastrous effects, Hoare [1972] and by Brinch Hansen [1972, 1973b].

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts a n d N o t a t i o n s for C o n c u r r e n t P r o g r a m m i n g • 15

Thus B is guaranteed to be true when exe- program OPSY$; :~"

cution of S begins. Th e delay mechanism is type buffer(T) = r~'ord
usually assumed to be fair: a process await- slots : array [O..N-I] o f T;
ing a condition B that is repeatedly true head, tail : O..N-I initial (0, 0);
will eventually be allowed to continue. size = O..N initial (0)
end;
One use of conditional critical regions is
vat inp_buff : buffer(cardimage);
shown in Figure 4, which contains another out_buff: buffer(lineimage);
implementation of our batch operating sys-
tem example. Note how condition synchro- resource ib : inp_bufj~ ob : oul..bufJ~
nization has been separated from mutual process reader;
exclusion. Th e Boolean expressions in those var card : cardimage;
CCR statements that access the buffers loop
read card from cardreader~
explicitly specify the conditions required
region ib when inp..buffMz¢ < N d o
for access; thus mutual exclusion of differ- inp_buff.slots[inp_lmff.tail ] ;= card;
ent CCR statements that access the same inp..buff.size := inp_b~fl~iz¢ + |;
buffer is implicit. inp_buff.tail := (inp..buff.~tail -~ 1) rood N
Programs written in terms of conditional end :,
critical regions can be understood quite end
simply by using the axiomatic approach. end;
Each CCR statement implements an oper- process executer;
ation on the resource t ha t it names. Asso- vat card : cardimage;
ciated with each resource r is an i n v a r i a n t line : lineimage;
relation L: a predicate that is true of the loop
region ib when ,inp..buffsize > 0 do
resource's state after the resource is initial-
card := inp_bUffsiots[inp..buffhead]
ized and after execution of any operation inp_buff, size := inp_buff.size -" I;
on the resource. For example, in O P S Y S of inp_buff, head := (inp_lmff.head + 1) rood N
Figure 4, the operations insert and remove end;
items from bounded buffers and the buffers process card and generate line;.
inp__buff and out__buff both satisfy the region ob when out..buffsize < N do
invariant out_.buff, slots[out_.buff.tai~ := line;
out_buff, size := out_buff.size + 1;
IB: out_buff, tail := (out_buff.tail+ !) rood N
0 <_ head, tail _< N - 1 and end
0 <- size < N and end
tail = (head + size) rood N and end;
slots[head] through slots[ (tail - 1) rood N] process printer;
in the circular buffer contain var line : lineimage;
the most recently inserted items loop
in chronological order region ob when out_buffsize > 0 do
fine := out_buff.slots[out.,buff.head];
T h e Boolean expression B in each CCR out...buff size := out_buff.size - !;
statement is chosen so that execution of the out_buffhead := (out_buffhead + 1) rood N
statement list, when started in any state end;
print line on lineprinter
that satisfies Ir a n d B, will terminate in a
end
state t h at satisfies It. Therefore the invar- end
iant is true as long as no process is in the
midst of executing an operation (i.e., exe- end.
cuting in a conditional critical region asso- Figure 4. B a t c h operating s y s t e m with C C R s t a t e -
ciated with the resource). Recall t hat exe- ments.
cution of conditional critical regions asso-
ciated with a given shared data object does
not overlap. Hence the proofs of processes source appear only in assertions within con-
are interference free as long as (1) variables ditional critical regions for t h a t resource.
local to a process appear only in the proof Thus, once appropriate resource invariants
of that process and (2) variables of a re- have been defined, a concurrent program

C o m p u t l ~ L r v e y s , Vol. 15, No. i~ March 1983

16 • G. R. Andrews and F. B. Schneider

can be understood in terms of its compo- rename : monitor;

nent sequential processes. vat declarations of permanent variables;
Although conditional critical regions procedure op l(parameters);
have many virtues, they can be expensive var declarations of variables local to op];
to implement. Because conditions in CCR begin
statements can contain references to local code to implement opl
variables, each process must evaluate its end;
own conditions. 14 On a multiprogrammed
processor, this evaluation results in numer- procedure op N(parameters);
ous context switches (frequent saving and vat declarations of variables local to opN;
restoring of process states), many of which begin
may be unproductive because the activated code to implement opN
process may still find the condition false. If end;
each process is executed on its own pro- begin
cessor and memory is shared, however, code to initialize permanent variables
CCR statements can be implemented quite end
cheaply by using busy-waiting.
CCR statements provide the synchroni- Figure 5. M o n i t o r structure.
zation mechanism in the Edison language
[Brinch Hansen, 1981], which is designed
specifically for multiprocessor systems. source's state, and some procedures, which
Variants have also been used in Distributed implement operations on the resource. A
Processes [Brinch Hansen, 1978] and Argus monitor also has permanent-variable ini-
[Liskov and Scheifler, 1982]. tialization code, which is executed once be-
fore any procedure body is executed. The
3.4 Monitors values of the permanent variables are re-
tained between activations of monitor pro-
Conditional critical regions are costly to cedures and may be accessed only from
implement on single processors. Also, CCR within the monitor. Monitor procedures
statements performing operations on re- can have parameters and local variables,
source variables are dispersed throughout each of which takes on new values for each
the processes. This means that one has to procedure activation. The structure of a
study an entire concurrent program to see monitor with name rename and procedures
all the ways in which a resource is used. opl, . . . , o p N is shown in Figure 5.
Monitors alleviate both these deficiencies. Procedure o p J within monitor mname is
A monitor is formed by encapsulating both invoked by executing
a resource definition and operations that
manipulate it [Dijkstra, 1968b; Brinch Han- call mname.opJ (arguments).
sen, 1973a; Hoare, 1974]. This allows a re-
source subject to concurrent access to be The invocation has the usual semantics
viewed as a module [Parnas, 1972]. Conse- associated with a procedure call. In addi-
quently, a programmer can ignore the im- tion, execution of the procedures in a given
plementation details of the resource when monitor is guaranteed to be mutually exclu-
using it, and can ignore how it is used sive. This ensures that the permanent vari-
when programming the monitor that imple- ables are never accessed concurrently.
ments it. A variety of constructs have been pro-
posed for realizing condition synchroniza-
3.4.1 Definition tion in monitors. We first describe the pro-
A monitor consists of a collection of per- posal made by Hoare [1974] and then con-
manent variables, used to store the re- sider other proposals. A condition variable
is used to delay processes executing in a
~4W h e n delayed, a process could i n s t e a d place condi-
monitor; it may be declared only within a
tion e v a l u a t i n g code in a n a r e a of m e m o r y accessible monitor. Two operations are defined on
to o t h e r processes, b u t this too is costly. condition variables: s i g n a l and wait. If

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming , 17
type buffer(T} = monitor; program OPSY$; ~,
vat { the variables satisfy invariant IB - - see See. 4.3 } type buffer(T} = .,.; { see Figure 5 }
slots : array 10..N-I] of 7~
var i n p . . b u f f : buffer(cardimage);
head, tail : 0..N-I;
o u t _ b u f f : buffer(lineimage);
size : 0.. N;
notfull, n o t e m p t y : condition; process reader;
var card : cardimage;
procedure deposit(p : 7);
loop
begin
read card from ¢ardreader;
if size = N then notfull.wait;
call inp..buffdeposit(card)
slots[tat1] := p;
end
size := size + I;
end;
tail := (tail + 1) rood N;
notempty.signal process executer;
end; var card : cardimage;
line : lineimage;
procedurefetch(var it : 7); loop
begin
call inp_buff.fetch(card);
if size = 0 then notempty.wait;
process card and generate line;
it := slots[head];
call out_buff, deposit(line)
size :=size - 1;
end
h e a d := (head + 1) rood N;
end;
notfull.signal
end; process printer;
var line : lineimage;
begin
loop
size := O; h e a d := O; tail := 0
¢all out..bufffetch(line);
end
print line on lineprinter
end
Figure 6. Bounded buffer monitor.
end
end.
cond is a condition variable, then execution
of Figure 7. B a t c h operating system with monitors.

cond.wait

causes the invoker to be blocked on cond An example of a monitor that defines a

and to relinquish its mutually exclusive bounded buffer type is given in Figure 6.
control of the monitor. Execution of Our batch operating system can be pro-
grammed using two instances of the
cond.signal
bounded buffer in Figure 6; these are shared
works as follows: if no process is blocked on by three processes, as shown in Figure 7.
cond, the invoker continues; otherwise, the At times, a programmer requires more
invoker is temporarily suspended and one control over the order in which delayed
process blocked on cond is r e a c t i v a t e d . A processes are awakened. T o implement
process suspended due to a s i g n a l opera- such medium-term scheduling, 15the prior-
tion continues when there is no other proc- ity w a i t statement can be used. This state-
ess executing in the monitor. Moreover, ment
signalers are given priority over processes cond.wait(p)
trying to commence execution of a monitor
procedure. Condition variables are assumed has the same semantics as cond.wait, ex-
to be fair in the sense that a process will cept that in the former processes blocked
not forever remain suspended on a condi- on condition variable cond are awakened in
tion variable that is signaled infinitely of- ascending order of p. {Consequently, con-
ten. Note t h a t the introduction of condition
variables allows more than one process to
~5 This is in contrast to s h o r t - t e r m s c h e d u l i n g , which
be in the same monitor, although all but is concerned with how processors are assigned to ready
one will be delayed at w a i t or s i g n a l op- processes, and l o n g - t e r m s c h e d u l i n g , which refers to
erations. how jobs are selected to be processed.

Computing Surveys, Vol. 15t No.~i,-~!.areh 1983

18 • G. R. A n d r e w s a n d F. B. Schneider
dition variables used in this way are not The semantics of c o n t i n u e are also slightly
necessarily fair.) different from signal. Executing c o n t i n u e
A common problem involving medium- causes the invoker to return from its mon-
term scheduling is "shortest-job-next" re- itor call, whereas s i g n a l does not. As be-
source allocation. A resource is to be allo- fore, a process blocked on the selected
cated to at most one user at a time; if more queue resumes execution of the monitor
than one user is waiting for the resource procedure within which it was delayed.
when it is released, it is allocated to the It is both cheaper and easier to imple-
user who will use it for the shortest amount ment c o n t i n u e than s i g n a l because s i g n a l
of time. A monitor to implement such an requires code to ensure that processes sus-
allocator is shown below. The monitor has pended by s i g n a l operations reacquire con-
two procedures: ( 1) request (time: integer), trol of the monitor before other, newer
which is called by users to request access processes attempting to begin execution in
to the resource for time units; and (2) t h a t monitor. With both s i g n a l and con-
release, which is called by users to relin- tinue, the objective is to ensure that a
quish access to the resource: condition is not invalidated between the
shortest._next_allocator : monitor; time it is signaled and the time that the
awakened process actually resumes execu-
var free : Boolean; tion. Although c o n t i n u e has speed and
turn : condition; cost advantages, it is less powerful than
procedure request(time : integer); signal. A monitor written using condition
begin variables cannot always be translated di-
if not free then turn.wait(time); rectly into one that uses queues without
free :=false also adding monitor procedures [Howard,
end; 19761)]. Clearly, these additional procedures
complicate the interface provided by the
procedure release; monitor. Fortunately, most synchroniza-
begin tion problems that arise in practice can be
free := true; coded using either discipline.
turn.signal
end; 3.4.2.2 Conditional W a i t a n d A u t o m a t i c
Signal. In contrast to semaphores, s i g n a l s
begin
on condition variables are not saved: a proc-
free := t r u e
end ess always delays after executing w a i t , even
if a previous s i g n a l did not awaken any
process. ~6 This can make s i g n a l and w a i t
3.4.2 Other Approaches to Condition
difficult to use correctly, because other vari-
Synchronization
ables must be used to record that a s i g n a l
was executed. These variables must also be
3.4.2.1 Queues a n d Delay~Continue. In tested by a process, before executing wait,
Concurrent PASCAL [Brinch Hansen, to guard against waiting if the event corre-
1975], a slightly simpler mechanism is pro- sponding to a s i g n a l has already occurred.
vided for implementing condition synchro- Another difficulty is that, in contrast to
nization and medium-term scheduling. conditional critical regions, a Boolean
Variables of type queue can be defined and expression is not syntactically associated
manipulated with the operations d e l a y with s i g n a l and wait, or with the condition
(analogous to wait) and c o n t i n u e (analo- variable itself. Thus, it is not easy to deter-
gous to signal). In contrast to condition mine why a process was delayed on a con-
variables, at most one process can be sus- dition variable, unless s i g n a l and w a i t are
pended on a given queue at any time. This used in a very disciplined manner. It helps
allows medium-term scheduling to be im- if (1) each w a i t on a condition variable is
plemented by (1) defining an array of
queues and (2) performing a c o n t i n u e op-
eration on that queue on which the next- ~eThe limitationsof conditionvariables discussedin
process-to-be-awakened has been delayed. this section also apply to queue variables.

ComputingSurveys,"Col.15,No. 1,March1983
 Concepts and Notations for Concurrent Programming• " 19
. . . . f~:, ?- •~,•

contained in an if statement in which the Boolean expressions associated ~ t h all

Boolean expression is the negation of the conditions for which there are waiting pro-
desired condition synchronization, and (2) cesses. If one of these Boolean expressions
each signal statement on the same condi- is found to be true, one of the waiting pro-
tion variable is contained in an if statement cesses is granted control of the monitor. If
in which the Boolean expression gives the none is found to be true, a n e w invocation
desired condition synchronization. Even so, of one of the monitor's procedures is per-
syntactically identical Boolean expressions mitted.
may have different values if they contain Using Kessels' proposal, the buffer mon-
references to local variables, which they itor in Figure 6 could be recoded as follows.
often do. Thus there is no guarantee that First, the declarations of not__full and
an awakened process will actually see the not._empty are changed to
condition for which it was waiting. A final
not_full : condition size < N;
difficulty with w a i t and signal is that, not__empty : condition size > 0
because signal is preemptive, the state of
permanent variables seen by a signaler can Second, the first statement in deposit is
change between the time a signal is exe- replaced by
cuted and the time that the signaling pro- not__full, wait
cess resumes execution.
To mitigate these difficulties, Hoare and the first statement in fetch is replaced
[1974] proposed the conditional wait state- by
ment not__empty, wait
wait (B) Finally, the signal statements are deleted.
where B is a Boolean expression involving The absence of a s i g n a l primitive is note-
the permanent or local variables of the worthy. The implementation provides an
monitor. Execution of w a i t ( B ) delays the automatic signal, which, though some-
invoker until B becomes true; no signal is what more costly, is less error prone than
required to reactivate processes delayed by explicitly programmed s i g n a l operations.
a conditional wait statement. This synchro- The signal operation cannot be acciden-
nization facility is expensive because it is tally omitted and never signals the wrong
necessary to evaluate B every time any condition. Furthermore, the programmer
process exits the monitor or becomes explicitly specifies the conditions being
blocked at a conditional wait and because awaited. The primary limitation of the pro-
a context switch could be required for each posal is that it cannot be used ~ solve most
evaluation (due to the presence of local scheduling problems, because operation pa-
variables in the condition). However, the rameters, which are not permanent vari-
construct is unquestionably a very clean ables, may not appear in conditions.
one with which to program. 3.4.2.3 Signals as Hints. Mesa [Mitchell
An efficient variant of the conditonal et al., 1979; Lampson and Redell, 1980]
wait was proposed by Kessels [1977] for use employs yet another approach to condition
when only permanent variables appear in synchronization. Condition variables are
B. The buffer monitor, in Figure 6 satisfies provided, but only as a way for a process to
this requirement. In Kessels' proposal, one relinquish control of a monitor. In Mesa,
declares conditions of the form execution of
cname : condition B cond.notify
E x e c u t i n g the s t a t e m e n t c n a m e . w a i t causes a process waiting on condition vari-
causes B, a Boolean expression, to be eval- able cond to resume at some time in the
uated. If B is true, the process continues; future. This is called signal and continue
otherwise the process relinquishes control because the process performing the notify
of the monitor and is delayed on cname. immediately continues execution rather
Whenever a process relinquishes control of than being suspended. 'Performing a notify
the monitor, the system evaluates those merely gives a hint t ~ a waiting process
ComputingSurveys,VoL 15, N0.'l,!~ch 1983
20 • G. R. A n d r e w s a n d F. B. S c h n e i d e r

that it might be able to proceed. 17 There- tion called the m o n i t o r invariant. This
fore, in Mesa one writes predicate should be true of the monitor's
permanent variables whenever no process
while not B do wait cond endloop
is executing in the monitor. Thus a process
instead of must reestablish the monitor invariant be-
fore the process exits the monitor or per-
if not B then cond.wait
forms a w a i t ( d e l a y ) or signal(continue).
as would be done using Hoare's condition The monitor invariant can be assumed to
variables. Boolean condition B is guaran- be true of the permanent variables when-
teed to be true upon termination of the ever a process acquires control of the mon-
loop, as it was in the two conditional-wait/ itor, regardless of whether it acquires con-
automatic-signal proposals. Moreover, the trol by calling a monitor procedure or by
(possible) repeated evaluation of the Boo- being reactivated following a w a i t or sig-
lean expression appears in the actual mon- nal.
itor code--there are no hidden implemen- The fact that monitor procedures are
tation costs. mutually exclusive simplifies noninterfer-
The n o t i f y primitive is especially useful ence proofs. One need not consider inter-
if the executing process has higher priority leaved execution of monitor procedures.
than the waiting processes. It also allows However, interference can arise when pro-
the following extensions to condition vari- gramming condition synchronization. Re-
ables, which are often useful when doing call t h a t a process will delay its progress in
systems programming: order to implement medium-term schedul-
ing, or to await some condition. Mecha-
(i) A time-out interval t can be associated nisms that delay a process cause its execu-
with each condition variable. If a pro- tion to be suspended and control of the
cess is ever suspended on this condition monitor to be relinquished; the process re-
variable for longer than t time units, a sumes execution with the understanding
notify is automatically performed by that both some condition B and the moni-
the system. The awakened process can tor invariant will be true. The truth of B
then decide whether to perform an- when the process awakens can be ensured
other w a i t or to take other action. by checking for it automatically or by re-
(ii) A b r o a d c a s t primitive can be defined. quiring that the programmer build these
Its execution causes all processes wait- tests into the program. If programmed
ing on a condition variable to resume checks are used, they can appear either in
at some time in the future (subject to the process that establishes the condition
the mutual exclusion constraints asso- (for condition variables and queues) or in
ciated with execution in a monitor). the process that performed the wait (the
This primitive is useful ff more than Mesa model).
one process could proceed when a con- If the signaler checks for the condition,
dition becomes true. The broadcast we must ensure that the condition is not
primitive is also useful when a condi- invalidated between the time that the sig-
tion involves local variables because in nal occurs and the time that the blocked
this case the signaler cannot evaluate process actually executes. That is, we must
the condition (B above) for which a ensure that other execution in the monitor
process is waiting. Such a primitive is, does not interfere with the condition. If the
in fact, used in UNIX [Ritchie and signaler does not immediately relinquish
Thompson, 1974]. control of the monitor (e.g., if n o t i f y is
3.4.3 An Axiomatic View used), interference might be caused by the
process that established the condition in
The valid states of a resource protected by the first place. Also, if the signaled process
a monitor can be characterized by an asser- does not get reactivated before new calls of
monitor procedures are allowed, interfer-
17Of course, it is prudent to perform n o t i f y operations
only when there is reason to believe that the awakened
ence might be caused by some process that
process will actually be able to proceed; but the burden executes after the condition has been sig-
of checking the condition is on the waiting process. naled (this can happen in Modula [Wirth,
Computing Surveys, Vol. 15, No. 1, March 1983
 Concepts and Notations for Concurrent Programming • 21
1977a]). Proof rules for monitors and the The manager construct [Silberschatz et al.,
various signaling disciplines are discussed 1977] for handling dynamic resource allo-
by Howard [1976a, 1976b]. cation problems and the scheduler monitor
[Schneider and Bernstein, 1978] for sched-
3.4.4 Nested Monitor Calls uling access to shared resources are both
based on this line of thought.
When structuring a system as a hierarchical The last approach to the nested monitor
collection of monitors, it is likely that mon- call problem, and probably t h e most rea-
itor procedures will be called from within sonable, is one that appreciates that moni-
other monitors. Such nested monitor calls tors are only a structuring tool for resources
have caused much discussion [Haddon, that are subject to concurrent access [An-
1977; Lister, 1977; Parnas, 1978; Wettstein, drews and McGraw, 1977; Paruas, 1978].
1978]. The controversy is over what (if any- Mutual exclusion of monitor procedures is
thing) should be done if a process having only one way tQ preserve the integrity of
made a nested monitor call is suspended in the permanent variables that make up a
another monitor. The mutual exclusion in resource. There are cases in which the op-
the last monitor called will be relinquished erations provided by a given monitor can
by the process, due to the semantics of be executed concurrently without adverse
w a i t and equivalent operations. However, effects, and even cases in which more than
mutual exclusion will not be relinquished one instance of the same monitor procedure
by processes in monitors from which nested can be executed in parallel (e.g., several
calls have been made. Processes that at- activations of a read procedure, in a moni-
tempt to invoke procedures in these moni- tor that encapsulates a database)~. Monitor
tors will become blocked. This has perform- procedures can be executed concurrently,
ance implications, since blockage will de- provided that they do not interfere w i t h
crease the amount of concurrency exhibited each other. Also, there are cases in which
by the system. the monitor invariant can be easily estab-
The nested monitor call problem can be lished before a nested monitor call is made,
approached in a number of ways. One ap- and so mutual exclusion for the monitor
proach is to prohibit nested monitor calls, can be released. Based on such reasoning,
as was done in SIMONE [Kaubisch et al., Andrews and McGraw [1977] defines a
1976], or or to prohibit nested calls to mon- monitorlike construct that allows the pro-
itors that are not lexically nested, as was grammer to specify that certain monitor
done in Modula [Wirth, 1977a]. A second procedures be executed concurrently and
approach is to release the mutual exclusion that mutual exclusion b e released for cer-
on all monitors along the call chain when a tain calls. The Mesa language [Mitchell et
nested call is made and that process be- al., 1979] also provides mechanisms that
comes blocked. TM This release-and-reac- give the programmer control over the gran-
quire approach would require that the mon- ularity of exclusion.
itor invariant be established before any
monitor call that will block the process.
3.4.5 Programming Notations Based
Since the designer cannot know a priori
on Monitors
whether a call will block a process, the
monitor invariant would have to be estab- Numerous programming languages have
lished before every call. A third approach been proposed and implemented that use
is the definition of special-purpose con- monitors for synchronizing access to shared
structs that can be used for particular sit- variables. Below, we very briefly discuss
uations in which nested calls often arise. two of the most important: Concurrent
PASCAL and Modula. These languages
~s Once signaled, t h e process will n e e d to reacquire have received widespread use, introduced
exclusive access to all m o n i t o r s along t h e call c h a i n novel constructs to handle machine-de-
before r e s u m i n g execution. However, if p e r m a n e n t pendent systems-programming issues, and
m o n i t o r variables were n o t p a s s e d as reference p a r a m -
eters in a n y of t h e calls, t h e process could reacquire inspired other language designs, such as
exclusive access incrementally, as it r e t u r n s to e a c h Mesa [Mitchell et al., 1979] and PASCAL-
monitor. Plus [Welsh and Bustard, 1979].
Computing Surveys, V~I. 16, NO; 1, March 1983
J : - .
. ...: .... . ..~:~:: ~.:: :~:i~ ,
22 • G. R. A n d r e w s a n d F. B. Schneider
3.4.5.1 Concurrent P A S C A L . Concur- vice modules, whicli a r e special interface
rent PASCAL [Brinch Hansen, 1975, 1977] modules for programming device drivers.
was the first programming language to sup- The run-time support system for Modula
port monitors. Consequently, it provided a is small and efficient. The kerne~ for a PDP-
vehicle for evaluating monitors as a system- 11/45 requires only 98 words of storage and
structuring device. The language has been is extremely fast [Wirth,1977c]. It does not
used to write several operating systems, time slice the processor among processes,
including Solo, a single-user operating sys- as Concurrent PASCAL does. Rather, cer-
tem [Brinch Hansen, 1976a, 1976b], Job tain kernel-supported operations--wait,
Stream, a batch operating system for pro- for example--always cause the processor to
cessing PASCAL programs, and a real-time be switched. (The programmer must be
process control system [Brinch Hansen, aware of this and design programs accord-
1977]. ingly.) This turnsout to be both a strength
One of the major goals of Concurrent and weakness of Modula. A small and effi-
PASCAL was to ensure that programs ex- cient kernel, where the programmer has
hibited reproducible behavior [Brinch Han- some control over processor switching, al-
sen, 1977]. Monitors ensured that patholog- lows Modula to be used for process control
ical interleavings of concurrently executed applications, as intended. Unfortunately, in
routines that shared data were no longer order to be able to construct such a kernel,
possible (the compiler generates code to some of the constructs in t h e language--
provide the necessary mutual exclusion). notably those concerning multiprogram-
Concurrent execution in other modules ruing--have asso'ciated restrictions that can
(called classes) was not possible, due to only be understood in terms of the kernel's
compile,time restrictions on the dissemi- implementation. A variety of subtle inter-
nation of class names and scope rules for actions between the Vari0us synchroniza-
class declarations. tion constructs must be understood in order
Concurrent PASCAL also succeeded in to program in Modula without experiencing
providing the programmer with a clean ab- unpleasant surprises. Some of these patho-
stract machine, thereby eliminating the logical interactions are described by Bern-
need for coding at the assembly language stein andEnsor [1981].
level. A systems programming language Modula implements an abstract machine
must have facilities to allow access to I/O that is well suited for dealing with inter-
devices and other hardware resources. In rupts and I/O devices on PDP-11 proces-
Concurrent PASCAL, I/O devices and the sors. Unlike Concurrent PASCAL, in which
like are viewed as monitors implemented the run-time kernel handles interrupts and
directly in hardware. To perform an I/O I/O, Modula leaves support for devices in
operation, the corresponding "monitor" is the programmer's domain. Thus new de-
called; the call returns when the I/O has vices can be added without modifying the
completed. Thus the Concurrent PASCAL kernel. An I/O device is considered to be a
run-time system implements synchronous process that is implemented in hardware. A
I/O and "abstracts out" the notion of an software process can start an I/O operation
interrupt. and then execute a doio statement (which
Various aspects of Concurrent PASCAL, is like a w a i t except that it delays the
including its approach to I/O, have been invoker until the kernel receives an inter-
analyzed by Loehr [1977], Silberschatz rupt from the corresponding device). Thus
[1977], and Keedy [1979]. interrupts are viewed as signal (send in
Modula) operations generated by the hard-
3.4.5.2 Modula. Modula was developed ware. Device modules are interface modules
for programming small, dedicated com- that control I/O devices. Each contains, in
puter systems, including process control ap- addition to some procedures, a device pro-
plications [Wirth, 1977a, I977b, 1977c, cess, which starts I/O operations and exe-
1977d]. The language is largely based on cutes doio statements to relinquish control
PASCAL and includes processes, interface of the processor (pending receipt of the
modules, which are like monitors, and de- corresponding I/O interrupt). The address

Computing Surveys, Vot. 15, No. 1, March 1983

 Concepts and Notations for Concurrent P r o g r a m m e . . 23
of the interrupt vector for the device is Another approach to defining a module
declared in the heading of the device mod- subject to concurrent access is to provide a
ule, so that the compiler can do the neces- mechanism with which a programmer spec-
sary binding. Modula also has provisions ifies, in one place in each module, all con-
for controlling the processor priority regis- straints on the execution of operations de-
ter, thus allowing a programmer to exploit fined by that module. Implementation of
the priority interrupt architecture of the the operations is separated from the speci-
processor when structuring programs. fication of the constraints. Moreover, code
A third novel aspect of Modula is that to enforce the constraints is generated by a
variables declared in interface modules can compiler. This is the approach taken in a
be exported. Exported variables can be ref- class of synchronization mechanisms called
erenced (but not modified) from outside the path expressions.
scope of their defining interface module. Path expressions were first defined by
This allows concurrent access to these vari- Campbell and Habermann [1974]. Subse-
ables, which, of course, can lead to difficulty quent extensions and variations have also
unless the programmer ensures that inter- been proposed [Habermann, 1975; Lauer
ference cannot occur. However, when used and Campbell, 1975; Campbell, 1976; Flon
selectively, this feature increases the effi- and Habermann, 1976; Lauer and Shields,
ciency of programs that access such vari- 1978; Andler, 1979]. Below, we describe one
ables. specific proposal [Campbell, 1976] that has
In summary, Modula is less constraining been incorporated into Path PASCAL, an
than Concurrent PASCAL, but requires the implemented systems programming lan-
programmer to be more careful. Its specific guage [Campbell and Kolstad, 1979].
strengths and weaknesses have been eval- When path expressions are used, a mod-
uated by Andrews [1979], Holden and ule that implements a resource has a struc-
Wand [1980], and Bernstein and Ensor ture like that of a monitor. It contains per-
[1981]. Wirth, Modula's designer, has gone manent variables, which store the state of
on to develop Modula-2 [Wirth, 1982]. the resource, and procedures, which realize
Modula-2 retains the basic modular struc- operations on the resource. Path expres-
ture of Modula, but provides more flexible sions in the header of each resource define
facilities for concurrent programming and constraints on the order in which opera-
these facilities have less subtle semantics. tions are executed. No synchronization
In particular, Modula-2 provides coroutines code is programmed in the procedures.
and hence explicit transfer of control be- The syntax of a path expression is
tween processes. Using these, the program-
mer builds support for exclusion and con- path path__list end
dition synchronization, as required. In
particular, the programmer can construct A path list contains operation names
monitorlike modules. and path operators. Path operators include
...., for concurrency, ";" for sequencing,
3.5 Path Expressions "n : {path list)" to specify up to n con-
Operations defined by a monitor are exe- current activations of path--~ist, and
cuted with mutual exclusion. Other syn- "[path list]" to specify an unbounded
chronization of monitor procedures is real- number of concurrent activations of
ized by explicitly performing w a i t and sig- path list.
n a l operations on condition variables (or For example, the path expression
by some similar mechanism). Conse-
quently, synchronization of monitor opera- path deposit, fetch end
tions is realized by code scattered through-
out the monitor. Some of this code, such as places no constraints on the order of exe-
w a i t and signal, is visible to the program- cution of deposit and fetch and no con-
mer. Other code, such as the code ensuring straints on the number of activations of
mutual exclusion of monitor procedures, is either operation. This absence of.synchro-
not. nization constraints is eqm'valent to that
p~

Computi~ Survey~.Vol~15,NO.!~Mareh 1983

. :o .
24 • G.R. Andrews and F. B. Schneider
specified by the path expressions begin
head := O; tail := 0
path [deposit], [ fetch] end end.
or Note that one deposit and one fetch can
path [deposit, fetch] end proceed concurrently, which was not pos-
sible in the buffer monitor given in Figure
(A useful application of the "[... ]" opera- 6. For this reason, there is no variable size
tor will be shown later.) In contrast, because it would have been subject to con-
path deposit; fetch end current access.
As a last example, consider the readers/
specifies that each fetch be preceded by a
writers problem [Courtois et al., 1971]. In
deposit; multiple activations of each oper- this problem, processes read or write rec-
ation can execute concurrently as long as
ords in a shared data base. To ensure that
the number of active or completed fetch
processes read consistent data, either an
operations never exceeds the number of
unbounded number of concurrent reads or
completed deposit operations. A module
a single write may be executed at any time.
implementing a bounded buffer of size one
The path expression
might well contain the path
path 1 : (deposit; fetch) end path 1 : ([read], write) end
to specify that the first invoked operation specifies this constraint. (Actually, this
be a deposit, that each deposit be followed specifies the "weak reader's preference" so-
by a fetch, and that at most one instance of lution to the readers/writers problem: read-
the path "deposit; fetch" be active--in ers can prevent writers from accessing the
short, that deposit and fetch alternate and database.)
are mutually exclusive. Synchronization Path expressions are strongly motivated
constraints for a bounded buffer of size N by, and based on, the operational approach
are specified by to program semantics. A path expression
defines all legal sequences of the operation
path N: (1 : (deposit); 1 : (fetch)) end executions for a resource. This set of se-
This ensures that (i) activations of deposit quences can be viewed as a formal language,
are mutually exclusive, (ii) activations of in which each sentence is a sequence of
fetch are mutually exclusive, (iii) each ac- operation names. In light of this, the re-
tivation of fetch is preceded by a completed semblance between path expressions
deposit, and (iv) the number of completed and regular expressions should not be
deposit operations is never more than N surprising.
greater than the number of completed fetch While path expressions provide an ele-
operations. The bounded buffers we have gant notation for expressing synchroniza-
been using for OPSYS, our batch operating tion constraints described operationally,
system, would be defined by they are poorly suited for specifying con-
module buffer( 7);
dition synchronization [Bloom, 1979].
Whether an operation can be executed
path N:( l:(deposit); l:OCetch) ) end,
might depend on the state of a resource in
var { the variables satisfy the invariant IB (see Sec. 4.3) a way not directly related to the history of
with size equal to the number of executions of
deposit minus the number of executions of fetch }
operations already performed. Certain var-
slots : array [ 0 . . N - I ] of T; iants of the readers/writers problem (e.g.,
head, tail : 0 . . N - l ; writers preference, fair access for readers
procedure deposit(p : 7); and writers) require access to the state of
begin the resourcemin this case, the number of
slots[tail] := p; waiting readers and waiting writers--in
tail := (tail + I) rood N order to implement the desired synchroni-
end;
zation. The shortest__next__allocator mo-
procedure fetch(vat it : 7); nitor of Section 3.4.1 is an example of a
begin resource in which a parameter's value de-
it := slots[head];
head := (head + 1) rood N termines whether execution of an operation
end; (request) should be permitted to continue.
Computing Surveys, Vol. 15, No. 1, March 1983
 Concepts and Notations for Concurrent Programming • 25

In fact, most resources that involve sched- is received by executing

uling require access to parameters and/or receive variable__list
to state information when making synchro- from source__designator
nization decisions. In order to use path
expressions to specify solutions to such where variable__list is a list of variables.
problems, additional mechanisms must be The source~designator gives the program-
introduced. In some cases, definition of ad- mer control over where the message came
ditional operations on the resource is suffi- from, and hence over which statements
cient; in other cases "queue" resources, could have sent it. Receipt of a message
which allow a process to suspend itself and causes, first, assignment of the values in the
be reactivated by a "scheduler," must be message to the variables in variable~list
added. The desire to realize condition syn- and, second, subsequent destruction of the
chronization using path expressions has message. TM
motivated many of the proposed exten- Designing message-passing primitives in-
sions. Regrettably, none of these extensions volves making choices about the form and
have solved the entire problem in a way semantics of these general commands. Two
consistent with the elegance and simplicity main issues must be addressed: How are
of the original proposal. However, path source and destination designators speci-
expressions have proved useful for spec- fied? How is communication synchronized?
ifying the semantics of concurrent com- Common alternative solutions for these is-
putations [Shields, 1979; Shaw, 1980, sues are described in the next two sections.
Best, 1982]. Then higher level message-passing con-
structs, semantic issues, and languages
based on message passing are discussed.
4. SYNCHRONIZATION PRIMITIVES BASED
ON MESSAGE PASSING 4.1 Specifying Channels of Communication
Critical regions, monitors, and path expres- Taken together, the destination and source
sions are one outgrowth of semaphores; designators define a communications chan-
they all provide structured ways to control nel. Various schemes have been proposed
access to shared variables. A different out- for naming channels. The simplest channel-
growth is message passing, which can be naming scheme is for process names to
viewed as extending semaphores to convey serve as source and destination designators.
data as well as to implement synchroniza- We refer to this as direct naming. Thus
tion. When message passing is used for send card to executer
communication and synchronization, pro-
sends a message that can be received only
cesses send and receive messages instead
by the executer process. Similarly,
of reading and writing shared variables.
Communication is accomplished because a receive line from executer
process, upon receiving a message, obtains permits receipt only of a message sent by
values from some sender process. Synchro- the executer process.
nization is accomplished because a message Direct naming is easy to implement and
can be received only after it has been sent, to use. It makes it possible for a process to
which constrains the order in which these control the times at which it receives mes-
two events can occur. sages from each other process. Our simple
A message is sent by executing batch operating system might be pro-
grammed using direct naming as shown in
send expression__list Figure 8.
to destination~designator.
The batch operating system also illus-
The message contains the values of the trates an important paradigm for process
expressions in expression__list at the time
s e n d is executed. The destination~des- ~9A broadcast can be modeledby the concurrentexe-
cution of a collection of sends, each sending the
ignator gives the programmer control over message to a differentdestination.A nondestructive
where the message goes, and hence over receive can be modeled by a receive, immediately
which statements can receive it. A message followedby a send.
Computing Surveys,Vol. 15,No. 1~March 1983
26 * G. R. A n d r e w s a n d F. B. Schneider
program OPSYS; awaits the interrupt signifying completion
process reader; of the I/O operation. Depending on the
var card : cardimage; application, it might also send a completion
loop message to the client after the line has been
read card from cardreader; printed.
send card to execuwr Unfortunately, direct naming is not al-
end
end;
ways well suited for client/server interac-
tion. Ideally, the r e c e i v e in a server should
process executer; allow receipt of a message from any client.
var card : cardimage; line : lineimage;
If there is only one client, then direct nam-
loop
receive card from reader;
ing will work well; difficulties arise if there
process card and generate line; is more than one client because, at the very
send line to printer least, a r e c e i v e would be required for each.
end Similarly, if there is more than one server
end; (and all servers are identical), then the
process printer; send in a client should produce a message
var line : lineimage; t h a t can be received by any server. Again,
loop this cannot be accomplished easily with
receive line from executer; direct naming. Therefore, a more sophisti-
print line on lineprinter
cated scheme for defining communications
end
end
channels is required.
One such scheme is based on the use of
end. global names, sometimes called mailboxes.
Figure 8. Batch operating system with message A mailbox can appear as the destination
passing.
designator in any process' s e n d statements
and as the source designator in any process'
interaction--a pipeline. A pipeline is a col- r e c e i v e statements. Thus messages sent to
lection of concurrent processes in which the a given mailbox can be received by any
output of each process is used as the input process that executes a r e c e i v e naming
to another. Information flows analogously that mailbox.
to the way liquid flows in a pipeline. Here, This scheme is particularly well suited
information flows from the reader process for programming client/server interactions.
to the executer process and then from the Clients send their service requests to a sin-
executer process to the printer process. Di- gle mailbox; servers receive service requests
rect naming is particularly well suited for from that mailbox. Unfortunately, imple-
programming pipelines. menting mailboxes can be quite costly with-
Another important paradigm for process out a specialized communications network
interaction is the client/server relation- [Gelernter and Bernstein, 1982]. When a
ship. Some server processes render a ser- message is sent, it must be relayed to all
vice to some client processes. A client can sites where a r e c e i v e could be performed
request that a service be performed by on the destination mailbox; then, after a
sending a message to one of these servers. message has been received, all these sites
A server repeatedly receives a request for must be notified that the message is no
service from a client, performs that service, longer available for receipt.
and (if necessary) returns a completion The special case of mailboxes, in which
message to that client. a mailbox name can appear as the source
The interaction between an I/O driver designator in r e c e i v e statements in one
process and processes that use it--for ex- process only, does not suffer these imple-
ample, the lineprinter driver and the mentation difficulties. Such mailboxes are
printer process in our operating system often called ports [Balzer, 1971]. Ports are
example--illustrates this paradigm. The simple to implement, since all receives
lineprinter driver is a server; it repeatedly that designate a port occur in the same
receives requests to print a line on the process. Moreover, ports allow a straight-
printer, starts that I/O operation, and then forward solution to the multiple-clients/

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming • 27
single-server problem. (The multiple- execution could cause a delay. A statement
clients/multiple-server problem, however, is nonblocking if its execution never delays
is not easily solved with ports.) its invoker; otherwise the statement is said
To summarize, when direct naming is to be blocking. In some message-passing
used, communication is one to one since schemes, messages are buffered between
each communicating process names the the time they are sent and received. Then,
other. When port naming is used, commu- if the buffer is full w h e n a s e n d is executed,
nication can be many to one since each port there are two options: the s e n d might delay
has one receiver but may have many until there is space in the buffer for the
senders. The most general scheme is global message, or the s e n d might return a code
naming, which can be many to many. Di- to the invoker, indicating that, because the
rect naming and port naming are special buffer was full, the message could not be
cases of global naming; they limit the kinds sent. Similarly, execution of a receive,
of interactions that can be programmed when no message that satisfies the source
directly, but are more efficient to imple- designator is available f o r receipt, might
ment. either cause a delay or terminate with a
Source and destination designators can code, signifying that no message was avail-
be fixed at compile time, called static chan- able.
nel naming, or they can be computed at If the system has an effectively un-
run time, called dynamic channel naming. bounded buffer capacity, then: a process is
Although widely used, static naming pre- never delayed when executing a send. This
sents two problems. First, it precludes a is variously called asynchronous message
program from communicating along chan- passing and send no, wait. Asynchronous
nels not known at compile time, and thus message passing allows a sender to get ar-
limits the program's ability to exist in a bitrarily far ahead of a receiver. Conse-
changing environment. For example, this quently, when a message is received, it con-
would preclude implementing the I/O re- tains information about the sender's state
direction or pipelines provided by U N I X that is not necessarily still its current state.
[Ritchie and Thompson, 1974]. 2° The sec- At the other extreme, with no buffering,
ond problem is this: if a program might ever execution of a s e n d is always delayed until
need access to a channel, it must perma- a corresponding 2~r e c e i v e is executed; then
nently have the access. In many applica- the message is transferred and both pro-
tions, such as file systems, it is more desir- ceed. This is called synchronous message
able to allocate communications channels passing. When synchronous message pass-
to resources (such as files) dynamically. ing is used, a message exchange represents
To support dynamic channel naming, an a synchronization point in the execution of
underlying, static channel-naming scheme both the sender and receiver. Therefore,
could be augmented by variables that con- the message received will always corre-
tain source or destination designators. spond to the sender's current state. More-
These variables can be viewed as contain- over, when the s e n d terminates, the sender
ing capabilities for the communications can make assertions about the state of the
channel [Baskett et al., 1977; Solomon and receiver. Between these two extremes is
Finkel, 1979; Andrews, 1982]. buffered message passing, in which the
buffer has finite bounds. Buffered message
4.2 Synchronization passing allows the sender to get ahead of
Another important property of message- the receiver, but not arbitrarily far ahead.
passing statements concerns whether their The blocking form of the r e c e i v e state-
ment is the most common, because a re-
20Although in UNIX most commands read from and ceiving process often has nothing else to do
write to the user's terminal, one can specify that a while awaiting receipt of a message. How-
command read its input from a file or write its output ever, most languages and operating systems
to a File. Also, one can specify that commands be
connected in a pipeline, These options are provided
by a dynamic channel-naming scheme that is trans- 2~Correspondence is determined .by the source and
parent to the implementation of each command. destination designators.

Computing Surveys, Vol. 15r ~Io~I, MoJ~¢h1983

28 • G. R. A n d r e w s a n d F. B. S c h n e i d e r

also provide a nonblocking r e c e i v e or a i f GI --* S1

m e a n s to test w h e t h e r execution of a r e - ~ G2.--~ $2
c e i v e would block. T h i s enables a process , o .

to receive all available messages and t h e n Q Gn-.->Sn

select one to process (effectively, to sched- fi
ule them). is executed as follows. If at least one guard
Sometimes, further control over which succeeds, one o f them, Gi, is selected
messages can be received is provided. T h e nondeterministically; the message-passing
statement s t a t e m e n t in Gi is executed (if present);
t h e n Si, the s t a t e m e n t following the guard,
receive variable__list is executed. If all guards fail, the c o m m a n d
f r o m source__designator w h e n B aborts. If all guards neither succeed nor fail,
execution is delayed until some guard suc-
permits receipt of only those messages t h a t ceeds. (Obviously, deadlock could result.)
m a k e B true. T h i s allows a process to Execution of the iterative s t a t e m e n t is the
" p e e k " at the contents of a delivered mes- same as for the alternative statement, ex-
sage before receiving it. Although this fa- cept selection and execution of a guarded
cility, is not n e c e s s a r y - - a process can al- c o m m a n d is r e p e a t e d until all guards fail,
ways receive and store copies of messages at which time the iterative s t a t e m e n t ter-
until appropriate to act on them, as shown minates r a t h e r t h a n aborts.
in the shortest-next-allocator example at T o illustrate the use of selective com-
the end of this s e c t i o n - - t h e conditional re- munications, we i m p l e m e n t a buffer pro-
ceive makes possible concise solutions to cess, which stores data produced by a pro-
m a n y synchronization problems. T w o lan- d u c e r process and allows these data to be
guages t h a t provide such a facility, P L I T S retrieved b y a c o n s u m e r process: 22
and SR, are described in Section 4.5.
A blocking r e c e i v e implicitly imple- process buffer;
m e n t s synchronization between sender and var slots : array [0..N-l] of T;
receiver because the receiver is delayed un- head, tail: 0..N-I;
til after the message is sent. T o i m p l e m e n t size : O..N;
such synchronization with nonblocking r e -
c e i v e , busy-waiting is required. However, head := O; tail := O; size := O;
blocking message-passing s t a t e m e n t s can do size<N; receive slots[tail] from producer
achieve the same semantic effects as non- size :=size + 1;
blocking ones by using w h a t we shall call tail := (tail + 1) rood N
selective c o m m u n i c a t i o n s , which is based fl size>O; send slots[head] to consumer --
on Dijkstra's guarded c o m m a n d s [Dijkstra, size := size - 1;
1975]. head := (head + 1) rood N
In a selective-communications state- od
ment, a g u a r d e d c o m m a n d has the form end

guard --~ statement T h e p r o d u c e r and consumer are as follows:

T h e guard consists of a Boolean expression, process producer;

optionally followed b y a message-passing vat stuff: T;
loop
statement. T h e guard s u c c e e d s if the Boo-
lean expression is true and executing the generate stuff',
send stuff to buffer
message-passing s t a t e m e n t would not cause
end
a delay; the guard f a i l s if the Boolean
end;
expression is false; the guard (temporarily)
neither succeeds nor fails if the Boolean
expression is true b u t the message-passing 22Even if message passing is asynchronous, such a
s t a t e m e n t c a n n o t y e t be executed without buffer may still be required if there are multiple pro-
causing delay. T h e alternative s t a t e m e n t ducers or consumers.

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts a n d Notations for Concurrent Programming * 29

process consumer; Some proqess relationships .are inher-

var stuff: T; ently asymmetric. In client/server interac-
loop tions, the server often takes different ac-
receive stuff from buffer; tions in response to different kinds of client
use stuff requests. For example, a shortest-job-next
end allocator (see Section 3.4.1) that receives
end "allocation" requests on a r e q u e s t ~ p o r t
If s e n d statements cannot appear in and "release" requests on a release_port
guards, selective communication is straight- can be programmed using message passing
forward to implement. A delayed process as follows:
determines which Boolean expressions process shortest_next..allocator;
in guards are true, and then awaits ar- var.free : Boolean;
rival of a message that allows execution of time : integer;
elient_id : proeess_id;
the r e c e i v e in one of these guards. (If the declarations of a priority queue and other local variables;
guard did not contain a receive, the pro-
.free := true;
cess would not be delayed.) If both s e n d do true; receive (time, elient_.id) from request, port
and r e c e i v e statements can appear in if.free --
guards, 2~ implementation is much more free :=.false;
send allocation to elient..id
costly because a process needs to negotiate
not .free --
with other processes to determine ff they save elient-Jd on priority queue ordered by time
can communicate, and these processes fi
could also be in the middle of such a nego- I] not free; receive release from ~'elease...port
tiation. For example, three processes could if not priority queue empty --
remove client.dd with smallest time from queue;
be executing selective-communications send allocation to client_Jd
statements in which any pair could com- I] priority queue empty -7'
municate; the problem is to decide which .free := true
fi
pair communicates and which one remains od
delayed. Development of protocols that end
solve this problem in an efficient and dead-
lock-free way remains an active research A client makes a request by executing
area [Schwartz, 1978; Silberschatz, 1979; s e n d {time, my__id) to request_-.port;
Bernstein, 1980; Van de Snepscheut, 1981; receive allocation ,
Schneider, 1982; Reif and Spirakis, 1982]. from shortest.._next__allocator
Unfortunately, if s e n d statements are
not permitted to appear in guards, pro- and indicates that it has finished using the
gramming with blocking s e n d and blocking resource by executing
r e c e i v e becomes somewhat more complex. s e n d release to release__port
In the example above, the buffer process
above would be changed to first wait for a
message from the consumer requesting 4.3 Higher Level Message-Passing
data (a r e c e i v e would appear in the second Constructs
guard instead of the send) and then to send
the data. The difference in the protocol 4.3.1 Remote Procedure Call
used by this new buffer process when inter- The primitives of the previous section are
acting with the consumer and that used sufficient to program a n y type of process
when interacting with the producer process interaction using message passing. To pro-
is misleading; a producer/consumer rela- gram client/server interactions, however,
tionship is inherently symmetric, and the both the client and server execute two mes-
program should mirror this fact. sage-passing statements: the client a s e n d
followed by a receive, and the server a
r e c e i v e followed by a send. Because this
e3 A l s o n o t e t h a t a l l o w i n g o n l y s e n d statements in type of interaction is very common, higher
g u a r d s is n o t v e r y u s e f u l . level statements that directly support it

Computing Surveys,Vol. 15,No. 1,M ~ ¢ h 1983

30 • G. R. Andrews and F. B. Schneider
have been proposed. These are termed re- quentially. Alternatively, a new process can
mote procedure call statements because of be created for each execution of call
the interface that they present: a client [Brinch Hansen, 1978; Cook, 1980; Liskov
"calls" a procedure that is executed on a and Scheifler, 1982]; these could execute
potentially remote machine by a server. concurrently, meaning that the different
When remote procedure calls are used, a instances of the server might need to syn-
client interacts with a server by means of a chronize if they share variables.
call statement. This statement has a form In the second approach to specifying the
similar to that used for a procedure call in server side, the remote procedure is a state-
a sequential language: ment, which can be placed anywhere any
other statement can be placed. Such a
call service(value__args; result__args)
statement has the general form
The service is really the name of a channel. accept service(in value__parameters;
If direct naming is used, service designates out result-_parameters) --* body
the server process; if port or mailbox nam-
ing is used, service might designate the kind Execution of this statement delays the
of service requested. Remote call is exe- server until a message resulting from a call
cuted as follows: the value arguments are to the service has arrived. Then the body is
sent to the appropriate server, and the call- executed, using the values of the value pa-
ing process delays until both the service has rameters and any other variables accessible
been performed and the results have been in the scope of the statement. Upon termi-
returned and assigned to the result argu- nation, a reply message, containing the val-
ments. Thus such a call could be translated ues of the result parameters, is sent to the
into a send, immediately followed by a calling process. The server then continues
receive. Note that the client cannot forget execution.2s
to wait for the results of a requested service. When a c c e p t or similar statements are
There are two basic approaches to spec- used to specify the server side, remote pro-
ifying the server side of a remote procedure cedure call is called a rendezvous [Depart-
call. In the first, the remote procedure is a ment of Defense, 1981] because the client
declaration, like a procedure in a sequential and server "meet" for the duration of the
language:24 execution of the body of the a c c e p t state-
ment and then go their separate ways. One
remote procedure service advantage of the rendezvous approach is
(in value__,parameters; that client calls may be serviced at times
o u t result__parameters)
body of the server's choosing; a c c e p t statements,
end for example, can be interleaved or nested.
A second advantage is that the server can
However, such a procedure declaration is achieve different effects for calls to the
implemented as a process. This process, the same service by using more than one a c -
server, awaits receipt of a message contain- c e p t statement, each with a different body.
ing value arguments from some calling (For example, the first a c c e p t of a service
process, assigns them to the value param- might perform initialization.} The final, and
eters, executes its body, and then returns a most important, advantage is that the
reply message containing the values of the server can provide more than one kind of
result parameters. Note that even if there service. In particular, a c c e p t is often com-
are no value or result parameters, the syn- bined with selective communications to en-
chronization resulting from the implicit
s e n d and r e c e i v e occurs. A remote proce- 25 Different semantics result depending on whether
dure declaration can be implemented as a the reply message is sent by a synchronous or by an
single process that repeatedly loops [An- asynchronous send. A synchronous s e n d delays the
drews, 1982], in which case calls to the server until the results have been received by the
caller. Therefore, when the server continues, it can
same remote procedure would execute se- assert that the reply message has been received and
that the result parameters have been assigned to the
e4 This is another reason this kind of interaction is result arguments. Use of asynchronous s e n d does not
termed "remote procedure call." allow this, but does not delay the server, either.

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent:Programmit~g ° 31
able a server to wait for and select one of ating a differentserveroperation with each
several requests to service [U. S. Depart- priority level. A d a [U. S. Department of
ment of Defense, 1981; Andrews, 1981]. Defense, 1981] supports this nicely by
This is illustrated in the following imple- means of arrays of operations. In general,
mentation of the bounded buffer: however, a mechanism isrequired to enable
process buffer;
a server to accept a call that minimizes
some function of the parameters of the
vat slots : array [O..N-I] of T; called operation. S R [Andrews, 1981] in-
head, tail: O..N-I; cludes such a mechanism (see Section
size : O..N; 4.5.4).
head := O; tail := O; size := O;
4.3.2 Atomic Transactions
do size<N; accept deposit(in value : 7) -- 7

slots[tail] := value; A n often-cited advantage of multiple-pro.

size :=size + 1; cessor systems is that they can be made
tail := (tail + 1) rood N resilient to failures. Designing programs
fl size>O; accept fetch(out value : 7) -- that exhibit this fault tolerance is not a
value := slots[head]; simple matter. While a discussion of how to
size := size - 1; design fault-tolerant programs is beyond
head := (head + 1) rood N the scope of this survey, we comment
od briefly on how fault-tolerance issues have
affected the design of higher level message-
end.
passing statements. 2~
The buffer process implements two opera- Remote procedure call provides a clean
tions: deposit and fetch. The firstis invoked way to program client/server interactions.
by a producer by executing Ideally, we would like a remotecall, like a
call deposit (stuff) procedure callin a sequential programming
notation, to have exactly once semantics:
The second is invoked by a consumer by each remote call should terminate only
executing after the named remote procedure has been
call fetch (stuff) executed exactly once by the server [Nel-
son, 1981; Spector, 1982]. Unfortunately, a
Note that d e p o s i t and f e t c h are handled by failure m a y mean that a client is forever
the buffer process in a symmetric manner, delayed awaiting the response to a remote
even though s e n d statements do not ap- call. This might occur if
pear in guards, because remote procedure
calls always involve two messages, one in (i) the message signifyingthe remote pro-
each direction. Note also that buffer can be cedure invocation is lost by the net-
used by multiple producers and multiple work, or
consumers. (ii) the reply message is lost, or
Although remote procedure call is a use- (iii) the server crashes duringexecution of
ful, high-level mechanism for client/server the remote procedure (but before the
interactions, not all such interactions can reply message is sent).
be directly programmed by using it. For This difficulty can be overcome by attach-
example, the s h o r t e s t _ _ n e x t ~ a l l o c a t o r of ing a time-out interval to the remote call;
the previous section still requires two if no response is received by the client
client/server exchanges to service alloca- before the time-out interval expires, the
tion requests because the allocator must client presumes that the server has failed
look at the parameters of a request in order and takes some action.
to decide if the request should be delayed. Deciding what action to take after a de-
Thus the client must use one operation to tected failure can be difficult. In Case (i)
transmit the request arguments and an- above, the correct action would be to re-
other to wait for an allocation. If there are
a small number of different scheduling 2~For a generaldiscussion,the interestedreader is
priorities, this can be overcome by associ- referredto Kohler 1981.
ComputingSurveys, Vol. 16,No. 1, March 1983
32 * G. R. A n d r e w s a n d F. B. Schneider
transmit the message. In Case (ii), however, an inconsistent state following partial exe-
retransmittal would cause a second execu- cution of a remote procedure. The use of
tion of the remote procedure body. This is atomic transactions is one way to do this,
undesirable unless the procedure is idem- but it is quite expensive [Lampson and
potent, meaning that the repeated execu- Sturgis, 1979; Liskov, 1981]. Other tech-
tion has the same effect as a single execu- niques to ensure the invisibility of incon-
tion. Finally, the correct action in Case (iii) sistent states have been proposed [Lynch,
would depend on exactly how much of the 1981; Schlichting and Schneider, 1981], and
remote procedure body was executed, what this remains an active area of research.
parts of the computation were lost, what
parts must be undone, etc. In some cases, 4.4 An Axiomatic View of Message Passing
this could be handled by saving state infor-
mation, called checkpoints, and program- When message passing is used for commu-
ming special recovery actions. A more gen- nication and synchronization, processes
eral solution would be to view execution of usually do not share variables. Nonetheless,
a remote procedure in terms of atomic interference can still arise. In order to prove
transactions. that a collection of processes achieves a
An atomic transaction [Lomet, 1977; common goal, it is usually necessary to
Reed, 1979; Lampson, 1981] is an all-or- make assertions in one process about the
nothing computation--either it installs a state of others. Processes learn about each
complete collection of changes to some other's state by exchanging messages. In
variables or it installs no changes, even if particular, receipt of a message not only
interrupted by a failure. Moreover, atomic causes the transfer of values from sender to
transactions are assumed to be indivisible receiver but also facilitates the "transfer"
in the sense that partial execution of an of a predicate. This allows the receiver to
atomic transaction is not visible to any make a~sertions about the state of the
concurrently executing atomic transaction. sender, such as about how far the sender
The first attribute is called failure atomic- has progressed in its computation. Clearly,
ity, and the second synchronization atom- subsequent execution by the sender might
icity. invalidate such an assertion. Thus it is pos-
Given atomic transactions, it is possible sible for the sender to interfere with an
to construct a remote procedure call mech- assertion in the receiver.
anism with at most once semantics--re- It turns out that two distinct kinds of
ceipt of a reply message means that the interference must be considered when mes-
remote procedure was executed exactly sage passing is used [Schlichting and
once, and failure to receive a reply message Schneider, 1982a]. The first is similar to
means the remote procedure invocation that occurring when shared variables are
had no (permanent) effect [Liskov and used: assertions made in one process about
Scheifler, 1982; Spector, 1982]. This is done the state of another must not be invalidated
by making execution of a remote procedure by concurrent execution. The second form
an atomic transaction that is allowed to of interference arises only when asynchro-
"commit" only after the reply has ben re- nous or buffered message passing is used. If
ceived by the client. In some circumstances, a sender "transfers" a predicate with a mes-
even more complex mechanisms are useful. sage, the "transferred" predicate must be
For example, when nested remote calls true when the message is received: receipt
occur, failure while executing a higher level of a message reveals information about the
call should cause the effects of lower level state of the sender at the time that the
(i.e., nested) calls to be undone, even if message was sent, which is not necessarily
those calls have already completed [Liskov the sender's current state.
and Scheifler, 1982]. The second type of interference is not
The main consideration in the design of possible when synchronous message pass-
these mechanisms is that may it not be ing is used, because, after sending a mes-
possible for a process to see system data in sage, the sender does not progress until the

Computing Surveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Coneurrent:'Programrning " 33
message has been received. This is a good 4.5 Programming Notations Based
reason to prefer the use of synchronous on Message Passing
s e n d over asynchronous s e n d (and to pre-
fer synchronous s e n d for sending the reply A large number of concurrent programming
message in a remote procedure body). One languages have been proposed that use
often hears the argument that asynchro- message passing for communication and
nous s e n d does not restrict parallelism as synchronization. This should not be too
much as synchronous s e n d and so it is surprising; because the two major message-
preferable. However, the amount of paral- passing design issues--channel naming and
lelism that can be exhibited by a program synchronization--are orthogonal, the var-
is determined by program structure and not ious alternatives for each can be combined
by choice of communications primitives. in many ways. In the following, we sum-
For example, addition of an intervening marize the important characteristics of four
languages: CSP, PLITS, Ada, and SR. Each
buffer process allows the sender to be exe-
cuted concurrently with the receiving proc- is well documented in the literature and
ess. Choosing a communications primitive was innovative in some regard. Also, each
merely establishes whether the program- reflects a different combination of the two
mer will have to do the additional work (of design alternatives. Some other languages
defining more processes) to allow a high that have been influential--Gypsy, Distrib-
uted Processes, StarMod and Argus--are
degree of parallel activity or will have to do
then briefly discussed.
additional work (of using the primitives in
a highly disciplined way) to control the
amount of parallelism. Nevertheless, a va- 4.5.1 Communicating Sequential Processes
riety of "safe" uses of asynchronous mes- Communicating Sequential Processes
sage passing have been identified: the (CSP) [Hoare, 1978] is a programming
"transfer" of monotonic predicates and the notation based on synchronous message
use of "acknowledgment" protocols, for passing and selective communications. The
example. These schemes are studied in concepts embodied in CSP hav e greatly in-
Schlichting and Schneider [1982b], where
fluenced subsequent work in concurrent
they are shown to follow directly from sim-
programming language design and the de-
ple techniques to avoid the second kind of
sign of distributed programs.
interference.
In CSP, processes are denoted by a var-
Formal proof techniques for various
iant of the cobegln statement. Processes
types of message-passing primitives have
may share read-only variables, but use in-
been developed. Axioms for buffered, asyn-
put/output commands for synchronization
chronous message passing were first pro-
and communication. Direct (and static)
posed in connection with Gypsy [Good et
channel naming is used and message pass-
al., 1979]. Several people have developed
ing is synchronous.
proof systems for synchronous message-
A n output command in C S P has the form
passing statementswin particular the input
and output commands in CSP [Apt et al., destination!expression
1980; Cousot and Cousot, 1980; Levin and
Gries, 1981; Misra and Chandy, 1981; Soun- where destination is a process name and
dararajan, 1981; Lamport and Schneider, expression is a simple or structured value.
1982; Schlichting and Schneider, 1982a]. An input command has the form
Also, several people have developed proof source?target
rules for asynchronous message passing
[Misra et al., 1982; Schlichting and where source is a process name and target
Schneider, 1982b], and proof rules for is a simple or structured variable local to
remote procedures and rendezvous [Bar- the process containing the input command.
ringer and Mearns, 1982; Gerth, 1982; The commands
Gerth et al., 1982; Schlichting and Schnei-
der, 1982a]. Pr!expression

Computing b-kwve~s,Vol. 16. No. 1, M~rch 1983

34 • G. R. A n d r e w s a n d F. B. Schneider
in process P s and A P L I T S program consists of a number
Ps?target of modules; active modules are processes.
Message passing is the sole means for in-
in process P r match if target and expres- termodule interaction. So as not to restrict
sion have the same type. Two processes parallelism, message passing is asynchro-
communicate if they execute a matching nous. A module sends a message containing
pair of input/output commands. The result the values of some expressions to a module
of communication is that the expression's m o d n a m e by executing
value is assigned to the target variable; both
send expressions to modname [about key]
processes then proceed independently and
concurrently. The " a b o u t key" phrase is optional. If in-
A restricted form of selective communi- cluded, it attaches an identifying transac-
cations statement is supported by CSP. In- tion key to the message. This key can then
put commands can appear in guards of al- be used to identify the message uniquely,
ternative and iterative statements, but out- or the same key Can be attached to several
put commands may not. This allows an different messages to allow messages to be
efficient implementation, but makes certain grouped.
kinds of process interaction awkward to A module receives messages by executing
express, as was discussed in Section 4.2.
By combining communication com- receive variables [from modname ]
mands with alternative and iterative state- [about key]
ments, CSP provides a powerful mecha- If the last two phrases are omitted, execu-
nism for programming process interaction. tion of r e c e i v e delays the executing mod-
Its strength is that it is based on a simple ule until the arrival of any message. If the
idea--input/output commands--that is phrase " f r o m modname" is included, exe-
carefully integrated with a few other mech- cution is delayed until a message from the
anisms. CSP is not a complete concurrent named module arrives. Finally, if the phrase
programming language, nor was it intended "about key" is included, the module is de-
to be. For example, static direct naming is layed until a message with the indicated
often awkward to use. Fortunately, this de- transaction key has arrived.
ficiency is easily overcome by using ports; By combining the options in s e n d and
how to do so was discussed briefly by Hoare r e c e i v e in different ways, a programmer
[Hoare, 1978] and is described in detail by can exert a variety of controls over com-
Kieburtz and Silberschatz [1979]. Recently, munication. When both the sending and
two languages based on CSP have also been receiving modules name each other, com-
described [Jazayeri et al., 1980; Roper and munication is direct. The effect of port
Barter, 1981]. naming is realized by having a receiving
4.5.2 PLITS
module not name the source module. Fi-
nally, the use of transaction keys allows the
PLITS, an acronym for "Programming receiver to select a particular kind of mes-
Language In The Sky," was developed at sage; this provides a facility almost as pow-
the University of Rochester [Feldman, erful as attaching " w h e n B " to a r e c e i v e
1979]. The design of PLITS is based on the statement.
premise that it is inherently difficult to In PLITS, execution of r e c e i v e can
combine a high degree of parallelism with cause blocking. P L I T S also provides prim-
data sharing and therefore message passing itives to test whether messages with certain
is the appropriate means for process inter- field values or transaction keys are avail-
action in a distributed system. Part of an able for receipt; this enables a process to
ongoing research project in programming avoid blocking when there is no message
language design and distributed computa- available.
tion, PLITS is being used to program ap- P L I T S programs interface to the oper-
plications that are executed on Rochester's ating systems of the processors that make
Intelligent Gateway (RIG) computer net- up RIG. Each host system provides device
work [Ball et al., 1976]. access, a file system, and job control. A

Computing Surveys, Vo|. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming ° 35

communications kernel on each machine In order to allow the programmer to con-

provides the required support for inter- trol I/O devices, Ada allowsentries to be
processor communication. bound to interrupt vector locations. Inter-
rupts become calls to those entries and can
4.5.3 A d a 27 therefore be serviced by a task that receives
the interrupt by means of an a c c e p t state-
Ada [U. S. Department of Defense, 1981] is ment.
a language intended for programming Since its inception, Ada has generated
embedded real-time, process-control sys- controversy [Hoare, 1981], much of which
tems. Because of this, Ada includes facili- is not related to concurrency. However, few
ties for multiprocessing and device control. applications using the concurrent program-
With respect to concurrent .programming, ruing features have been programmed, and
Ada's main innovation is the rendezvous at the time of this writing no compiler for
form of remote procedure call. full Ada has been validated. Implementa-
Processes in Ada are called tasks. A task tion of some of the concurrent program-
is activated when the block containing its ming aspects of Ada is likely to be hard. A
declaration is entered. Tasks may be nested paper by Welsh and Lister [1981] compares
and may interact by using shared variables the concurrency aspects of Ada to CSP and
declared in enclosing blocks. (No special Distributed Processes [Brinch Hansen,
mechanisms for synchronizing access to 1978]; Wegner and Smolka [1983] compare
shared variables are provided.) Ada, CSP, and monitors.
The primary mechanism for process in-
teraction is the remote procedure call. Re- 4.5.4 SR
mote procedures in Ada are called entries;
they are ports into a server process speci- SIR (Synchronizing Resources) [Andrews,
fied by means of an a c c e p t statement, 1981, 1982], like Ada, uses the rendezvous
which is similar in syntax and semantics to form of remote procedure call and port
the a c c e p t statement described in Section naming. However, there are notable differ-
4.3.1. Entries are invoked by execution of a ences between the languages, as described
remote call. Selective communications is below. A compiler for SIR has been imple-
supported using the select statement, mented on PDP-11 processors and the lan-
which is like an alternative statement. guage is being used in the construction of a
Both call and a c c e p t statements are UNIX-like network operating system.
blocking. Since Ada programs might have An SIR program consists of one or more
to meet real-time response constraints, the resources. 2s The resource~ construct sup-
language includes mechanisms to prevent ports both control of process interaction
or control the length of time that a process and data abstraction. (In contrast, Ada has
is delayed when it becomes blocked. Block- two distinct constructs for this--the task
ing on call can be avoided by using the and the package.) Resources contain one or
conditional entry call, which performs a more processes. Processes interact by using
call only if a rendezvous is possible imme- operations, which are similar to Ada en-
diately. Blocking on a c c e p t can be avoided tries. Also, processes in the same resource
by using a mechanism that enables a server may interact by means of shared variables.
to determine the number of waiting calls. Unlike Ada, operations may be invoked
Blocking on select can be avoided by by either send, which is nonblocking, or
means of the else guard, which is true if call, which is blocking. (The server that
none of the other guards are. Finally, a task implements an operation can require a par-
can suspend execution for a time interval ticular form of invocation, if necessary.)
by means of the d e l a y statement. This Thus both asynchronous message passing
statement can be used within a guard of and remote call are supported. Operations
select to ensure that a process is eventually m a y be named either staticallyin the pro-
awakened. gram text or dynamically by means of ca-

~Ada is a trademark of the U. S. Department of 28SR's resourcesare not to be cQnfusedwith resources

Defense. in conditionalcriticalregions.
Corap,t~ Sut*oyo,Vet,~5,N¢~t, MarchLgS3
36 • G. R. A n d r e w s a n d F. B. S c h n e i d e r

pability variables, which are variables hav- fiable systems. It has been used to imple-
ing fields whose values are the names of ment special-purpose systems for singie-
operations. A process can therefore have a and multiprocessor architectures.
changing set of communication channels. Distributed Processes (DP) [Brinch Han-
In SR, operations are specified by the in sen, 1978] was the first language to be based
statement, which also supports selective on remote procedure calls. It can be viewed
communications. Each guard in an in state- as a language that implements monitors by
ment has the form means of active processes rather than col-
lections of passive procedures. In DP, re-
op_.name(parameters) [and B] [by A]
mote procedures are specified as externally
where B is an optional Boolean expression callable procedures declared along with a
and A is an optional arithmetic expression. host process and shared variables. When a
The phrase " a n d B " allows selection of the remote procedure is called, a server process
operation to be dependent on the value of is created to execute the body of the pro-
B, which may contain references to param- cedure. The server processes created for
eters. The phrase " b y A " controls which different calls and the host process execute
invocation of o p _ _ n a m e is selected if more with mutual exclusion. The servers and
than one invocation is pending that satisfies host synchronize by means of a variant of
B. This can be used to express scheduling conditional critical regions. An extension of
constraints succinctly. For example, it per- D P that employs the rendezvous form of
mits a compact solution to the shortest-job- remote procedure call and thus has a more
next allocation problem discussed earlier. efficient implementation is described by
Although somewhat expensive to imple- Mao and Yeh [1980].
ment because it requires reevaluation of A StarMod [Cook, 1980] synthesizes as-
whenever a selection is made, this facility pects of Modula and Distributed Processes:
turns out to be less costly to use than it borrows modularization ideas from Mod-
explicitly programmed scheduling queues, ula and communication ideas from Distrib-
if the expected number of pending invoca- uted Processes. A module contains one or
tions is small (which is usually the case). more processes and, optionally, variables
Operations may also be declared to be shared by those processes. Synchronization
procedures. In SR, a procedure is short- within a module is provided by semaphores.
hand for a process that repeatedly executes Processes in different modules interact by
an in statement. Thus such operations are means of remote procedure call; StarMod
executed sequentially. provides both remote procedures and ren-
To support device control, SR provides dezvous for implementing the server side.
a variant of the resource called a real re- In StarMod, as in SR, both s e n d and call
source. A real resource is similar to a Mod- can be used to initiate communication, the
ula device module: it can contain device- choice being dictated by whether the in-
driver processes and it allows variables to voked operation returns values.
be bound to device-register addresses. Op- Argus [Liskov and Scheifler, 1982] also
erations in real resources can be bound to borrows ideas from Distributed Processes--
interrupt vector locations. A hardware in- remote procedures implemented by dynam-
terrupt is treated as a s e n d to such an ically created processes, which synchronize
operation; interrupts are processed by using critical regions--but goes much fur-
means of in statements. ther. It has extensive support for program-
ming atomic transactions. The language
4.5.5 Some Other Language Notations Based
on Message Passing
also includes exception handling and recov-
ery mechanisms, which are invoked if fail-
Gypsy [Good et al., 1979], one of the first ures occur during execution of atomic trans-
high-level languages based on message actions. Argus is higher level than the other
passing, uses mailbox naming and buffered languages surveyed here in the sense that
message passing. A major focus of Gypsy it attaches more semantics to remote call.
was the development of a programming A prototype implementation of Argus is
language well suited for constructing veri- nearing completion.

ComputingSurveys, Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming • 37
5. MODELS OF CONCURRENT As in a message-oriented language, each
PROGRAMMING LANGUAGES object has a caretaker process associated
with it; as in a procedure-oriented language,
Most of this survey has been devoted to
operations are performed on an object by
mechanisms for process interaction and
calling a procedure. The difference is that
programming languages that use them. De-
the caller of an operation and the caretaker
spite the resulting large variety of lan-
that implements it synchronize while the
guages, each can be viewed as belonging to
operation is executed. Both then proceed
one of three classes: procedure oriented,
asynchronously. Distributed Processes,
message oriented, or operation oriented.
StarMod, Ada, and SR are examples of
Languages in the same class provide the
operation-oriented languages.
same basic kinds of mechanisms for process
Languages in each of these classes are
interaction and have similar attributes.
roughly equivalent in expressive power.
Inprocedure-oriented languages, process
Each can be used to implement various
interaction is based on shared variables.
types of cooperation between concurrently
(Because monitor-based languages are the
executing processes, including client/server
most widely known languages in this
interactions and pipelines. Operation-ori-
class, this is often called the monitor
ented languages are well suited for pro-
model.) These languages contain both ac-
gramming client/server systems, and mes-
tive objects (processes) and shared, passive
sage-oriented languages are well suited for
objects (modules, monitors, etc.). Passive
programming pipelined computations.
objects are represented by shared variables,
Languages in each class can be used to
usually with some procedures that imple-
write concurrent programs for uniproces-
ment the operations on the objects. Pro-
sors, multiprocessors, and distributed sys-
cesses access the objects they require di-
tems. Not all three classes are equally
rectly and thus interact by accessing shared
suited for all three architectures, however.
objects. Because passive objects are shared,
they are subject to concurrent access. Procedure-oriented languages are the most
efficient to implement on contemporary
Therefore, procedure-oriented languages
single processors. Since it is expensive to
provide means for ensuring mutual exclu-
simulate shared memory if none is present,
sion. Concurrent PASCAL, Modula, Mesa,
implementing procedure-oriented lan-
and Edison are examples of such languages.
guages on a distributed system can be
Message- and operation-oriented lan-
costly. Nevertheless, procedure-oriented
guages are both based on message passing,
languages can be used to program a distrib-
but reflect different views of process inter-
uted system--an individual program is
action. Message-oriented languages pro-
written for each processor and the com-
vide s e n d and r e c e i v e as the primary
munications network is viewed as a shared
means for process interaction. In contrast
object. Message-oriented languages can be
to procedure-oriented languages, there are
implemented with or without shared mem-
no shared, passive objects, and so processes
cannot directly access all objects. Instead, ory. In the latter case, the existence of a
communications network is made com-
each object is managed by a single process,
its caretaker, which performs all operations pletely transparent, which frees the pro-
on it. When an operation is to be performed grammer from concerns about how the net-
on an object, a message is sent to its care- work is accessed and where processes are
taker, which performs the operation and located. This is an advantage of message-
then (possibly) responds with a completion oriented languages over procedure-oriented
message. Thus, objects are never subject to languages when programming a distributed
concurrent access. CSP, Gypsy, and PLITS system. Operation-oriented languages en-
are examples of message-oriented lan- joy the advantages of both procedure-ori-
guages. ented and message-oriented languages.
Operation-oriented languages provide When shared memory is available, an op-
remote procedure call as the primary means eration-oriented language can, in many
for process interaction. These languages cases, be implemented like a procedure-ori-
combine aspects of the other two classes. ented language [Habermann and Nassi,
Computing Surveys~Vol. 15, No. 1, March 1983
38 • G. R. A n d r e w s a n d F. B. S c h n e i d e r
Busy-Waiting

Semaphores

Critical Regions

PROCEDURE / MESSA GE
ORIENTED ~{ ORIENTED
Monitors Message Passing

Path Expressions

Remote Procedure Call

OPERA TION ORIENTED

Figure 9. Synchronization techniques and language classes.

1980]; otherwise it can be implemented us- anisms is a bad idea. We, however, favor
ing message passing. Recent research has programming in the style appropriate to
shown that both message- and operation- the language.
oriented languages can be implemented
quite efficiently on distributed systems if 6. CONCLUSION
special software/fnTnware is used in the
implementation of the language's mecha- This paper has discussed two aspects of
nisms [Nelson, 1981; Spector, 1982]. concurrent programming: the key con-
In a recent paper, Lauer and Needham cepts--specification of processes and con-
argued that procedure-oriented and mes- trol of their interaction--and important
sage-oriented languages are equals in terms language notations. Early work on operat-
of expressive power, logical equivalence, ing systems led to the discovery of two
and performance [Lauer and Needham, types of synchronization: mutual exclusion
1979]. (They did not consider operation- and condition synchronization. This stim-
oriented languages, which have only re- ulated development of synchronization
cently come into existence.) Their thesis primitives, a number of which are described
was examined in depth by Reid [1980], who in this paper. The historical and conceptual
reached many conclusions that we share. relationships among these primitives are
At an abstract level, the three types of illustrated in Figure 9.
languages are interchangeable. One c a n The difficulty of designing concurrent
transform any program written using the programs that use busy-waiting and their
mechanisms found in languages of one class inefficiency led to the definition of sema-
into a program using the mechanisms of phores. Semaphores were then extended in
another class without affecting perform- two ways: (1) constructs were defined that
ance. However, the classes emphasize dif- enforced their structured use, resulting in
ferent styles of programmingNthe same critical regions, monitors, and path expres-
program written in languages of different sions; (2) "data" were added to the syn-
classes is often best structured in entirely chronization associated with semaphores,
different ways. Also, each class provides a resulting in message-passing primitives. Fi-
type of flexibility not present in the others. nally, the procedural interface of monitors
Program fragments that are easy to de- was combined with message passing, result-
scribe using the mechanisms of one can be ing in remote procedure call.
awkward to describe using the mechanisms Since the first concurrent programming
of another. One might argue (as do Lauer languages were defined only a decade ago,
and Needham) that such use of these mech- practical experience has increased our un-

Computing Surveys,Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent ProKfamming 39
derstanding of how to engineer such pro- guages (San Antonio, Tex.,.Jal~ 1979). ACM,
grams, and the development of formal tech- New York, 1979, pp: 226-236.
niques has greatly increased our under- ANDREWS,G.R. "The design era message switching
standing of the basic concepts. Although system: An apl~l!cation and evaluation of Mod-
ula." IEEE Trans. 8oftw. Eng. SE-5, 2 (March
there are a variety of different program- 1979), 138-147.
ming languages, there are only three essen- ANDREWS, G.R. "Synchronizing resources." ACM
tially different kinds: procedure oriented, Trans. Prog. Lang. 8yst. 3, 4 (Oct. 1981),405-430.
message oriented, and operation oriented. ANDREWS,G.R. "The distributed programming lan-
This, too, is illustrated in Figure 9. guage SR--Mechanisms, design, and implemen-
At present, many of the basic problems tation." Softw. Pract. Exper. 12, 8 (Aug. 1982),
719-754.
that arise when constructing concurrent
ANDREWS, G. R., AND McGRAw, J . R . "Language
programs have been identified, solutions to features for process interaction." In Prec. ACM
these problems are by and large under- Conf. Language Design for Reliable Software,
stood, and substantial progress has been S[GPLAN Not. 12, 3 (March 1977), 114-127.
made toward the design of notations to APT, K. R., FRANCEZ,N., ANDDE;ROEVER,W.P. "A
express those solutions. Much remains to proof system for communicatingsequential proc-
be done, however. The utility of various esses." ACM Trans. Prog. Lang. Syst. 2, 3 (July
1980), 359-385.
languages--really, combinations of con- ASCHCROFT,E.A. "Proving assertions about parallel
structs--remains to be investigated. This programs." J. Comput. Syst. 10 (Jan. 1975), 110-
requires using the languages to develop sys- 135.
tems and then analyzing how they helped BALL, E., FELDMAN, J., Low, J,, RASHID, R., AND
or hindered the development. In addition, ROVNER, P. "RIG, Rochester's intelligent gate-
the interaction of fault tolerance and con- way: System overview." IEEE Trans. Softw. Eng.
• SE-2, 4 (Dec. 1976), 321-328.
current programming is not well under-
BALZER, R. M. "PORTS--A method for dynamic
stood. Little is known about the design of interprogram communicationand job control." In
distributed (decentralized) concurrent pro- Prec. AFIPS Spring Jt. Computer Conf. (Atlantic
grams. Last, devising formal techniques to City, N. J., May 18-20, 1971), col 38. AFIPS
aid the programmer in constructing correct Press, Arlington, Va., 1971, pp. 485-489.
programs remains an important open prob- BARRINGER,H., ANDMEARNS,L "Axioms and proof
lem. rules for Ada tasks." IEE Prec. 129, Pt. E, 2
(March 1982), 38-48.
BASKETT, F., HOWARD, J. H., AND MONTAGUE,J.
ACKNOWLEDGMENTS T. "Task communicationin DEMOS." In Prec.
6th Syrup. Operating Systems Principles (West
Numerous people have been kind enough to provide
Lafayette, Indiana, Nov. 16-18, 1977). ACM, New
very helpful comments on earlier drafts of this survey: York, 1977, pp. 23-31.
David Gries, Phil Kaslo, Lynn Kivell, Gary Levin, IRon
BEN-ARI, M. Principles of COncurrent Program-
Olsson, Rick Schlichting, and David Wright. Three ming. Prentice-Hall, Englewood Cliffs, N~J., 1982.
referees, and also Eike Best and Michael Scott, pro-
BERNSTEIN, A.J. "Output guards and nondetermin-
vided valuable comments on the penultimate draft. ism in communicating sequential processes."
Tony Wasserman has also provided helpful advice; it ACM Trans. Prog. Lang. Syst. 2, 2 (Apr. 1980),
has been a joy to have him as the editor for this paper. 234-238.
Rachel Rutherford critiqued the ultimate draft and BERNSTEIN,A. J., ANDENSOR,J.R. "A modification
made numerous useful, joyfully picturesque com- of Modula." Softw. Praet. Exper. 11 (1981), 237-
ments. 255.
This work was supported in part by NSF Grants BERNSTEIN, A. J., AND SCHNEIDER, F.B. "On lan-
MCS 80-01668and MCS 82-02869at Arizona and MCS guage restrictionsto ensure deterministic behav-
81-03605 at Cornell. ior in concurrent systems," In J. Moneta (Ed.),
Prec. 3rd Jerusalem Conf. Information Technol-
REFERENCES ogy JCIT3. North-Holland Publ., Amsterdam,
1978, pp. 537-541.
AKKOYUNLU, E. A., BERNSTEIN, A. J., SCHNEIDER, F. BERNSTEIN, P. A., ANDGOODMAN,N. "Concurrency
B., AND SILBERSCHATZ, A. "Conditions for the control in distributed database systems." ACM
equivalence of synchronous and asynchronous Comput. Surv. 13, 2 (June 1981), 185-221.
systems." IEEE Trans. Soflw. Eng. SE-4, 6 (Nov. BEST, E. "Relational semantics of concurrent pro-
1978), 507-516. grams (with some applications)." In Prec. IFIP
ANDLER, S. "Predicate path expressions." In Prec. WG2.2 Conf. North-Holland Publ., Amsterdam,
6th A CM Syrup. Principles of Programming Lan- 1982.

ComputingSurveys,VoL15, No. 1, March 1983

40 • G. R. A n d r e w s a n d F. B. S c h n e i d e r
BLOOM, T. "Evaluating synchronization mecha- DENNXS,J. B., AND VAN HORN,E.C. "Programming
nisms." In Proc. 7th Symp. Operating Systems semantics for multiprogrammed computations."
Principles (Pacific Grove, Calif., Dec. 10-12, Commun. ACM 9, 3 (March 1966), 143-155.
1979). ACM, New York, 1979, pp. 24-32. • DIJKSTRA, E.W. "The structure of the 'THE' mul-
BRINCH HANSEN, P. "Structured multiprogram- tiprogramming system." Commun. ACM 11, 5
ruing." Commun. ACM 15, 7 (July 1972), 574-578. (May 1968),341-346. (a)
BRINCH HANSEN, P. Operating System Principles. DIJKSTRA, E. W. "Cooperating sequential proc-
Prentice-Hall, Englewood Cliffs, N. J., 1973. (a) esses." In F. Genuys (Ed.), Programming Lan-
BRINCHHANSEN,P. "Concurrent programming con- guages. Academic Press, New York, 1968. (b)
cepts." ACM Comput. Surv. 5, 4 (Dec. 1973), 223- DIJKSTRA,E.W. "Guarded commands, nondetermi-
245. (b) nacy, and formal derivation of programs." Com-
BRINCH HANSEN, P. "The programming language mun. ACM 18, 8 (Aug. 1975), 453-457.
Concurrent Pascal." IEEE Trans Softw. Eng. DIJKSTRA, E. W. A Discipline of Programming.
SE-1, 2 (June 1975), 199-206. Prentice-Hall, Englewood Cliffs, N. J., 1976.
BRINCH HANSEN, P. "The Solo operating system: DIJKSTRA,E.W. "An assertional proof of a program
Job interface." Soflw. Pract. Exper. 6 (1976), 151- by G. L. Peterson." EWD 779 (Feb. 1979), Nu-
164. (a) enen, The Netherlands. (a)
BRINCH HANSEN, P. "The Solo operating system: DIJKSTRA, E. W. Personal communication, Oct.
Processes, monitors, and classes." Softw. Pract. 1981. (b)
Exper. 6 (1976), 165-200. (b) FELDMAN, J.A.. "High level programming for dis-
BRINCHHANSEN,P. The Architecture of Concurrent tributed computing." Commun. ACM 22, 6 (June
Programs. Prentice-Hall, Englewood Cliffs, N. J., 1979), 353-368.
1977. FLON, L., AND HABERMANN, A. N. "Towards the
BRINCH HANSEN, P. "Distributed processes: A con- construction of verifiable software systems." In
current programming concept." Commun. ACM Proc. ACM Conf. Data, SIGPLAN Not. 8, 2
21, 11 (Nov. 1978), 934-941. (March 1976), 141-148.
BRINCH HANSEN,P. "Edison: A multiprocessor lan- FLOYD, R.W. "Assigning meanings to programs." In
guage." Softw. Praet. Exper. 11, 4 (Apr. 1981), Proc. Am. Math. Soc. Symp. Applied Mathemat-
325-361. ics, vol. 19, pp. 19-31, 1967.
CAMPBELL,R.H. "Path expressions: A technique for GELERNTER,D., AND BERNSTEIN, A.J. "Distributed
specifying process synchronization."Ph.D. disser- communicationvia global buffer." In Proc. Symp.
tation, Computing Laboratory, University of Principles of Distributed Computing (Ottawa,
Newcastle upon Tyne, Aug. 1976. Canada, Aug. 18-20, 1982). ACM, New York, 1982,
CAMPBELL, R. H., AND HABERMANN, A. N. "The pp. 10-18.
specification of process synchronization by path GERTH, R. "A sound and complete Hoare axiomati-
expressions." Lecture Notes in Computer Science, zation of the Ada-rendezvous." In Proc. 9th Int.
vol. 16. Springer-Verlag, New York, 1974, pp. 89- Colloquium Automata, Languages and Pro-
102. gramming (ICALP82), Lecture Notes in Com-
CAMPBELL,R. H., AND KOLSTAD,R.B. "Path expres- puter Science, vol. 140. Springer-Verlag, New
sions in Pascal." In Proc. 4th Int. Conf. on Soft- York, 1982, pp. 252-264.
ware Eng. (Munich, Sept. 17-19, 1979). IEEE, GERTH, R., DE ROEVER, W. P., AND RONCKEN,
New York, 1979, pp. 212-219. M. "Procedures and concurrency: A study in
CONWAY,M. E. "Design of a separable transition- proof." In 5th Int. Symp. Programming, Lecture
diagram compiler." Commun. ACM 6, 7 (July Notes in Computer Science, vol. 137. Springer-
1963), 396-408. (a) Verlag, New York, 1982, pp. 132-163.
CONWAY, M.E. "A multiprocessor system design." GOOD, D. I., COHEN, R. M., AND KEETON-WILLIAMS,
In Proc. AFIPS Fall Jt. Computer Conf. (Las J. "Principles of proving concurrent programs in
Vegas, Nev., Nov., 1963), vol. 24. Spartan Books, Gypsy." In Proc. 6th ACM Symp. Principles of
Baltimore, Maryland, pp. 139-146. (b) Programming Languages (San Antonio, Texas,
COOK, R. P. "*MOD--A language for distributed Jan. 29-31, 1979). ACM, New York, 1979, pp. 42-
programming." IEEE Trans. Softw. Eng. SE-6, 52.
6 (Nov. 1980), 563-571. HABERMANN, A. N. "Path expressions." Dep. of
COURTOIS, P. J., HEYMANS, F., AND PARNAS, D. Computer Science, Carnegie-Mellon Univ., Pitts-
L. "Concurrent control with 'readers' and burgh, Pennsylvania, June, 1975.
'writers'." Commun. ACM 14, 10 (Oct. 1971), 667- HABERMANN,A. N., AND NASSI, I.R. "Efficient im-
668. plementation of Ada tasks." Tech. Rep. CMU-
CousoT, P., AND COUSOT,R. "Semantic analysis of CS-80-103, Carnegie-Mellon Univ., Jan. 1980.
communicating sequential processes." In Proc. HADDON, B.K. "Nested monitor calls." Oper. Syst.
7th Int. Colloquium Automata, Languages and Rev. 11, 4 (Oct. 1977), 18-23.
Programming (ICALP80), Lecture Notes in HANSON, D. R., AND GRISWOLD, R. E. "The SL5
Computer Science, vol. 85. Springer-Verlag, New procedure mechanism." Commun. ACM 21, 5
York, 1980, pp. 119-133. (May 1978), 392-400.

ComputingSurveys,Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming ~ • 41
HOAHE, C. A.R. "An axiomatic basis for computer 56, SRI International, Menlo Parl~,. Calif., Oct.
programming." Commun. ACM. 12, 10 (Oct. 1980. (b)
• 1969), 576-580, 583. LAMPORT, L., AND SCHNEIDER,~F. B. "The 'Hoare
HOARE, C. A.R. "Towards a theory of parallel pro- logic' of CSP, and all that." Tech. Rep. TR 82-
gramming." In C. A. R. Hoare and R. H. Perrott 490, Dep. Computer Sci., Cornell Univ., May,
(Eds.), Operating Systems Techniques. Academic 1982.
Press, New York, 1972, pp. 61-71. LAMPSON, B.W. "Atomic transactions." In Distrib-
HOARE, C. A. R. "Monitors: An operating system uted Systems--Architecture and Implementa-
structuring concept." Commun. ACM 17, 10 (Oct. tion, Lecture Notes in Computer Science, col. 105.
1974), 549-557. Springer-Verlag, New York, 1981.
HOARE, C. A.R. "Communicating sequential proc- LAMPSON, B. W., AND REDELL, D . D . "Experience
esses." Commun. ACM 21, 8 (Aug. 1978), 666-677. with processes and monitors in Mesa." Commun.
HOARE, C. A.R. "The emperor's old clothes." Com- A C M 23, 2 (Feb. 1980), 105-11.7.
mun. ACM 24, 2 (Feb. 1981), 75-83. LAMPSON, B. W., AND STURGIS, H.E. "Crash recov-
HOLDEN, J., AND WAND, I. C. "An assessment of ery in a distributed data storage system." Xerox
Modula." Softw. Pract. Exper. 10 (1980), 593- Palo Alto Research Center, Apr. 1979.
621. LAUER, H. C.,AND NEEDHAM, R.M. "On the duality
HOLT, R. C., GRAHAM, G. S., LAZOWSKA, E. D., AND of operating system struci~ares."In Proc. 2nd Int.
SCOTT, M.A. Structured Concurrent Program- Syrup. Operating Systems (IRIA, Paris, Oct.
ming with Operating Systems Applications. Ad- 1978); reprinted in Oper. Syst. Rev. 13, 2 (Apr.
dison-Wesley, Reading, Mass., 1978. 1979), 3-19.
HOWARD,J.H. "Proving monitors." Commun. ACM LAURa, P. E., AND CAMPBELL, R . H . "Formal se-
19, 5 (May 1976), 273-279. (a) mantics of a class of high level primitives for
coordinating concurrent processes." Aeta Inform.
HOWARD,J.H. "Signaling in monitors." In Proc. 2nd
5 (1975), 297-332.
Int. Conf. Software Engineering (San Francisco,
Oct. 13-15, 1976). IEEE, New York, 1976, pp. 47- LAUER, P. E., AND SHIELDS,M.W. "Abstract speci-
52. (b) fication of resource accessing disciplines: Ade-
JAZAYERLM., et al. "CSP/80: A language for com- quacy, starvation, priority and interrupts." SIG-
municating processes." In Proc. Fall IEEE P L A N Not. 13, 12 (Dec. 1978), 41-59.
COMPCON80 (Sept. 1980). IEEE, New York, LEHMANN, D., PNUELI, A., AND STAVI, J.
1980, pp. 736-740. "Impartiality, justice and fairness: The ethics of
JONES, A. K., AND SCHWARZ, P. "Experience using concurrent termination." Automata, Languages
multiprocessor systems~A status report." ACM and Programming, Lecture Notes in Computer
Comput. Surv. 12, 2 (June 1980), 121-165. Science, col. 115. Springer-Verlag, New York,
1981, pp. 264-277.
KAUBISCH,W. H., PERROTT, R. H., AND HOARE, C. A.
R. "Quasiparallel programming." Softw. Pract. LEVIN, G. M., ANDGRIES,D. "A proof technique for
Exper. 6 (1976), 341-356. communicating sequential processes." Acta In-
form. 15 (1981), 281-302.
KEEDY,J.L. "On structuring operating systems with
monitors." Aust. Comput. J. 10, 1 (Feb. 1978), 23- LISKOV, B.L. "On linguistic support for distributed
27. Reprinted in Oper. Syst. Rev. 13,1 (Jan. 1979), programs." In Proc. IEEE Syrup. Reliability in
5-9. Distributed Software and Database Systems
(Pittsburgh, July 21-22, 1981). IEEE, New York,
KELLER, R.M. "Formal verification of parallel pro- 1981, pp. 53-60.
grams." Commun. ACM 19, 7 (July 1976), 371-
384. LISKOV, B. L., AND SCHEIFLER, R. "Guardians and
actions: Linguistic support for robust, distributed
KESSELS, J. L.W. "An alternative to event queues programs." In Proc. 9th ACM Symp. Principles
for synchronizationin monitors." Commun. ACM of Programming Languages (Albuquerque, New
20, 7 (July 1977), 500-503. Mexico, Jan. 25-27, 1982). ACM, New York, 1982,
KIEBURTZ, a. S., AND SILBERSCHATZ, A. "Com- pp. 7-19.
ments on 'communicatingsequential processes.'" LISTER, A. "The problem of nested monitor calls."
ACM Trans. Program. Lang. Syst. 1, 2 (Oct. Oper. Syst. Rev. 11, 3 (July 1977), 5-7.
1979), 218-225.
LOEtlR, K.-P. "Beyond Concurrent Pascal." In Proc.
KOHLER,W.H. "A survey of techniques for synchro- 6th ACM Syrup. Operating Systems Principles
nization and recovery in decentralized computer (West Lafayette, Ind., Nov. 16-18, 1977). ACM,
systems." ACM Comput. Surv. 13, 2 (June 1981), New York, 1977, pp. 173-180.
149-183.
LOMET, D.B. "Process structuring,synchronization,
LAMPORT, L. "Proving the correctness of nmlti- and recovery using atomic transactions."In Proc.
process programs." IEEE Trans. Softw. Eng. ACM Conf. Language Design for Reliable Soft-
SE-3, 2 (March 1977), 125-143. ware, SIGPLAN Not. 12, 3 (March 1977), 128-
LAMPORT, L. "The 'Hoare logic' of concurrent pro- 137.
grams." Acta Inform. 14, 21-37. (a) LYNCH,N.A. "Multilevel atomicity--A new correct-
LAMPOHT,L. "The mutual exclusion problem." Op. ness criterion for distributed databases." Tech.

Computing Surveys,Vol.15,No. !,March 1983

42 ° G. R. A n d r e w s a n d F. B. S c h n e i d e r
Rep. GIT-ICS-81/05, School of Information and systems." Tech. Rep. TR 81-479, Dep. of Com-
Computer Sciences, Georgia Tech., May 1981. puter Sci., Cornell Univ., Nov. 1981,
MAD, T. W., ANDYEH, R.T. "Communication port: SCHLICHTING,R. D., AND SCHNEIDER,F.B. "Using
A language concept for concurrent programming." message passing for distributed programming:
IEEE Trans. Softw. Eng. SE-6, 2 (March 1980), Proof rules and disciplines." Tech. Rep. TR 82-
194-204. 491, Dep. of Computer Science, Cornell Univ.,
MISRA, J., AND CHANDY,K. "Proofs of networks of May 1982. (a)
processes." IEEE Trans. Softw. Eng. SE-7, 4 SCHLICHTING, R. D., AND SCHNEIDER, F. B. "Un-
(July 1981), 417-426. derstanding and using asynchronous message
MISRA, J., CHANDY, K. M., AND SMITH, T. passing primitives." In Proc. Syrup. Principles of
"Proving safety and liveness of communicating Distributed Computing (Ottawa, Canada, Aug.
processes with examples." In Proc. Symp, Prin- 18-20, 1982). ACM, New York, 1982, pp. 141-
ciples of Distributed Computing (Ottawa, Can- 147. (b)
ada, Aug. 18-20, 1982). ACM, New York, 1982,pp. SCHNEIDER, F. B. "Synchronization in distributed
201-208. programs." A CM Trans. Program. Lang. Syst. 4,
MITCHELL, J. G., MAYBURY, W., AND SWEET, 2 (Apr. 1982), 125-148.
R. "Mesa language manual, version 5.0." Rep. SCHNEIDER, F. B., AND BERNSTEIN, A. J. "Sched-
CSL-79-3, Xerox Paid Alto Research Center, Apr. uling in Concurrent Pascal." Oper. Syst. ReD. 12,
1979. 2 (Apr. 1978), 15-20.
NELSON, B.J. "Remote procedure call." Ph.D. the- SCHWARTZ, J. S. "Distributed synchronization of
sis. Rep. CMU-CS-81-119, Dep. of Computer Sci- communicatingsequential processes." Tech. Rep.,
ence, Carnegie-Mellon Univ., May 1981. Dep. of Artificial Intelligence, Univ. of Edinburgh,
NYGAARD,K., AND DAHL, O.J. "The development July 1978.
of the SIMULA languages." Preprints ACM SIG- SHAW, A.C. The Logical Design of Operating Sys-
P L A N History of Programming Languages Con- tems. Prentice-Hall, Englewood Cliffs, N. J., 1974.
ference, SIGPLAN Not. 13, 8 (Aug. 1978), 245- SHAW,A.C. "Software specification languages based
272. on regular expressions." In W. E. Riddle and R.
OwIcEI, S, S., AND GRIES, D. "An axiomatic proof E. Fairley (Eds.), Software Development Tools.
technique for parallel programs." Acta Inform. 6 Springer-Verlag, New York, 1980, pp. 148-175.
(1976), 319-340. (a) SHIELDS, M. W. "Adequate path expressions." In
OWICKI, S. S., AND GRIES, D. "Verifying properties Proc. Int. Syrup. Semantics of Concurrent Com-
of parallel programs: an axiomatic approach." putation, Lecture Notes in Computer Science, vol.
Commun. ACM 19, 5 (May 1976), 279-285. (b) 70. Springer-Verlag, New York, pp. 249-265.
PARNAS,D.L. "On the criteria to be used in decom- SILBERSCHATZ,A. "On the input/output mechanism
posing systems into modules." Commun. ACM 15, in Concurrent Pascal." In Proc. COMPSAC '77--
12 (Dec. 1972), 1053-1058. IEEE Computer Society Computer Software and
PARNAS,D.L. "The non-problem of nested monitor Applications Conference (Chicago, Ill., Nov.
calls." Oper. Syst. ReD. 12, 1 (Jan. 1978), 12-14. 1977). IEEE, New York, 1977, pp. 514-518.
PETERSON,G.L. "Myths about the mutual exclusion SILBERSCHATZ,A. "Communication and synchroni-
problem." Inform. Process. Lett. 12, 3 (June zation in distributed programs." IEEE Trans.
1981), 115-116. Softw. Eng. SE-5, 6 (Nov. 1979), 542-546.
REED, D.P. "Implementing atomic actions on de- SILBERSCHATZ,A., KIEBURTZ, R. B., AND BERNSTEIN,
centralized data." ACM Trans. Comput. Syst. 1, A . J . "Extending Concurrent Pascal to allow
1 (Feb. 1983), 3-23. dynamic resource management." IEEE Trans.
REID, L. G. "Control and communication in pro- Softw. Eng. SE°3, 3 (May 1977), 210-217.
grammed systems." Ph.D. thesis, Rep. CMU-CS- SOLOMON, M. H., AND FINKEL, R . A . "The Roscoe
80-142, Dep. of Computer Science, Carnegie-Mel. distributed operating system." In Proc. 7th Symp.
Ion Univ., Sept. 1980. Operating System Principles (Pacific Grove,
REIF, J. H., AND SPIRAKIS, P.G. "Unbounded speed Calif., Dec. 10-12, 1979). ACM, New York, 1979,
variability in distributed communications sys- pp. 108-114.
tems." In Proe. 9th ACM Conf. Principles of SOUNDARARAJAN,N. "Axiomatic semantics of com-
Programming Languages (Albuquerque, N. M., municating sequential processes." Tech. Rep.,
Jan. 25-27, 1982). ACM, New York, 1982, pp. 46- Dep. of Computer and Information Science, Ohio
56. State Univ., 1981.
RITCHIE, D. M., AND THOMPSON, K. "The UNIX SPECTOR, A.Z. "Performing remote operations effi-
timesharing system, Commun. ACM 17, 7 (July ciently on a local computer network." Commun.
1974), 365-375. ACM 25, 4 (Apr. 1982), 246-260.
ROPER, T. J., ANDBARTER,C.J. "A communicating U.S. DEPARTMENT OF DEFENSE. Programming
sequential process language and implementa- Language Ada: Reference Manual, vol. 106, Lec-
tion." Softw. Pract. Exper. 11 (1981), 1215-1234. ture Notes in Computer Science. Springer-Verlag,
SCHLICHTII~G, R. D., AND SCHNEIDER, F. B. "An New York, 1981.
approach to designing fault-tolerant computing VAN DE SNEPSCHEUT, J. L . A . "Synchronous corn-

ComputingSurveys,Vol. 15, No. 1, March 1983

 Concepts and Notations for Concurrent Programming • 43

munication between synchronization compo- WETTSTEIN, H. "The problem of nested monitor

nents." Inform. Process. Lett. 13, 3 (Dec. 1981), cells revisited." Oper. Syst. Rev. 12, 1 (Jan. 1978),
127-130. 19-23.
VANWIJNGAARDEN,A., MAILLOUX,B. J., PECK,J. L., WIRTH, N. "Modula: A language for modular multi-
KOSTER, C. H. A., SINTZOFF,M., LINDSEY,C. H., programming." Softw. Pract. Exper. 7 (1977), 3-
MEERTENS, L. G. L. T., AND FISKER, R. 35. (a)
G. "Revised report on the algorithm language WIRTH, N. "The use of ModulE." Softw. Pract. Ex-
ALGOL68."Acta Inform. 5, 1-3 (1975), 1-236. per. 7 (1977), 37-65. (b)
WEGNER, P., AND SMOLKA,S.A. "Processes, tasks WIRTH,1NT. "Design and implementationof Modula."
and monitors: A comparative study of concurrent Softw. Pract Exper, 7 (1977), 67-84. (c)
programming primitives." IEEE Trans. Softw. WIRTH, N. "Toward a discipline of real-time pro-
Eng., to appear, 1983. gramming." Commun. ACM 20, 8 (Aug. 1977),
WELSH, J., ANDBUSTARD,D.W. "Pascal-Plus--An- 577-583. (d)
other language for modular multiprogramming." WIRTH, N. Programming in Modula-2. Springer-
Softw. Pract. Exper. 9 (1979), 947-957. Verlag, New York, 1982.
WELSH, J., AND LISTER,A. "A comparative study of WULF, W. A., RUSSELL,D. B., AND HABERMANN,A.
task communicationin Ada." Soflw. Pract. Exper. N. BLISS: A language for systems program-
11 (1981), 257-290. ming. Commun. ACM 14, 12 (Dec. 1971), 780-790.

Received September 1982; final revision accepted February 1983

ComputingSurveys,Vol. 15, No. 1,~Mareh 1983