PRINCIPLES PROGRAMMING M. Ben-AriPrinciples of Concurrent Programming M. Ben-Ari Tel-Aviv University Englewood Cliffs, New Jersey London New Delhi Singapore Sydney Tokyo Toronto WellingtonLibrary of Congress Cataloging in Publication Data Ben-Ari, M., 1948— Principles of concurrent programming. Bibliography: p. Includes index. 1. Parallel processing (Electronic computers) I. Title. QA76.6.B437 001.642 82-3650 ISBN 0-13-701078-8 (pbk.) AACR2 British Library Cataloguing in Publication Data Ben-Ari, M. Principles of concurrent programming. 1, Parallel processing (Electronic computers) 2. Operating systems (Computers) |. Title 001.6425 = QA76.6 ISBN 0-13-701078-8 ©1982 by PRENTICE-HALL INTERNATIONAL, INC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of Prentice-Hall International, Inc. For permission within the United States contact Prentice-Hall Inc., Englewood Cliffs, N.J. 07632. ISBN 0-13-701078-8 PRENTICE-HALL INTERNATIONAL INC., London PRENTICE-HALL OF AUSTRALIA PTY., LTD., Sydney PRENTICE-HALL CANADA, INC., Toronto PRENTICE-HALL OF INDIA PRIVATE LIMITED, New Delhi PRENTICE-HALL OF JAPAN, INC., Tokyo PRENTICE-HALL OF SOUTHEAST ASIA PTE., LTD., Singapore PRENTICE-HALL INC., Englewood Cliffs, New Jersey WHITEHALL BOOKS LIMITED, Wellington, New Zealand Printed in the United States of America 109876543To MargalitCONTENTS Preface xiii Acknowledgements xv 1 What Is Concurrent Programming? 1 From Sequential to Concurrent Programming 1 Concurrent Programming 4 Correctness of Concurrent Programs 6 Interleaving 7 The Origin of Operating Systems 8 Operating Systems and Concurrent Programming 10 An Overview of the Book 12 Program Notation 14 Exercises 16 2 The Concurrent Programming Abstraction 18 21 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Introduction 18 Mutual Exclusion 19 Correctness 20 Timing 22 Implementing Primitive Instructions 24 Concurrent Programming in Pascal-S 26 Summary 27 Exercises 28X CONTENTS 3 The Mutual Exclusion Problem 29 3.1 Introduction 29 3.2 First Attempt 30 3.3. Second Attempt 32 3.4 Third Attempt 34 3.5 Fourth Attempt 36 3.6 Dekker’s Algorithm 38 3.7 A Proof of Dekker’s Algorithm 40 3.8 Conclusion 43 3.9 Exercises 43 4 Semaphores 50 4.1. Introduction 50 4.2. Mutual Exclusion 51 4.3 The Producer-Consumer Problem 56 4.4 More on the Producer-Consumer Problem 58 4.5 The Sleeping Barber 62 4.6 The Bounded Buffer 65 4.7. Exercises 68 5 Monitors 73 5.1 Introduction 73 5.2. Definition of Monitors 75 5.3 Simulation of the Semaphore 78 5.4. The Readers and Writers Problem 80 5.5 Proving Properties of Monitors 83 5.6 The Simulation of Monitors by Semaphores 86 5.7 Unrestricted Signals 88 5.8 Exercises 90 6 The AdaRendezvous 93 6.1 Introduction 93 6.2. The “Accept” Statement 94 6.3 The “Select” Statement 99 6.4 Proving Properties of the Rendezvous 105 6.5 Exercises 105CONTENTS. xi 7 ~~ The Dining Philosophers 109 7.1 Introduction 109 7.2 First Attempt 110 7.3, Second Attempt 111 7.4 A Correct Solution 113 7.5 Conditional Critical Regions 115 7.6 Exercises 117 Appendix: Implementation Kit 119 A.1 Introduction 119 A.2. The Compiler 120 A.3 The P-Code 121 A.4 Procedure Calls 124 A.5 Concurrency 129 A.6 Semaphores 132 A.7 Randomization 133 A.8 Program Listing 133 Bibliography 165 “Textbooks 165 Sources 165 Index 171PREFACE Concurrent programming—the programming tools and techniques for deal- ing with parallel processes—has traditionally been a topic in operating systems theory texts. There are several reasons why concurrent program- ming deserves a book of its own and should be the core of an advanced computer science systems course: 1. Concurrent programming is what distinguishes operating systems and real-time systems from other software systems. Computer science stu- dents who have taken courses in programming, data structures, compu- ter architecture and probability can easily master the applications of these disciplines in operating systems, but they need to be introduced to techniques that will enable them to deal with parallelism. 2. I doubt if many of my students will ever design or construct a multipro- cessing time-sharing system where algorithms for paging and scheduling are of prime importance. I am certain that they will be designing and constructing real-time systems for mini- and microcomputers. A sound knowledge of concurrent programming will enable them to cope with real-time systems—in particular with the severe reliability require- ments that are imposed. 3. There is a trend towards increasing use of abstract concurrency that has nothing to do with the parallelism of actual systems. Data flow diagrams used in software engineering are nothing more than networks of concur- rent processes. Traditionally, such a design must be implemented in a sequential language, but UNIX? is a programming system which encourages the use of concurrent processes. Adat, the new language designed for the U.S. Department of Defense, includes concurrent programming features as an integral part of the language. + UNIX is a trademark of Bell Laboratories. $ Ada is a trademark of the United States Dept. of Defense.xiv PREFACE 4, Finally, concurrent programming is an important topic in computer science research. The basic results in the field are still scattered throughout the literature and deserve to be collected into one volume for a newcomer to the field. The book requires no prerequisites as such other than computer science maturity (to borrow the term from mathematics). The book is aimed at advanced undergraduate students of computer science. At the Tel Aviv Univeristy, we arrange the course of study so that a student has four semesters of computer science courses which include extensive program- ming exercises, The material is also appropriate for practicing systems and real-time programmers who are looking for a more formal treatment of the tools of their trade. Thave used the material for half of a weekly four-hour semester course in operating systems—the second half is devoted to the classical subjects: memory management, etc. I should like to see curricula evolve that would devote a quarter or trimester to (theoretical) concurrent programming followed by a project oriented course on operating systems or real-time systems, The book is not oriented to any particular system or technique. I have tried to give equal time to the most widespread and successful tools for concurrent programming: memory arbiters, semaphores, monitors and rendezvous. Only the most elementary features of Pascal and Ada are used; they should be understandable by anyone with experience in a modern programming language. Much of the presentation closely follows the original research articles by E. W. Dijkstra and C. A. R. Hoare. In particular, Dijkstra’s Co-operating Sequential Processes reads like a novel and it was always great fun to lecture the material. This book is an attempt to explain and expand on their work. Verification of concurrent programs is one of the most exc ing areas of research today. Concurrent programs are notorious for the hidden and sophisticated bugs they contain. Formal verification seems indispensable. A novel feature of this book is its attempt to verify concurrent programs rigorously though informally. Hopefully, a student who is introduced early to verification will be prepared to study formal methods at a more advanced level, One cannot learn any programming technique without practicing it. There are several programming systems that can be used for class exercise. For insititutions where no such system is available, the Appendix contains the description and listing of a simple Pascal-S based concurrent program- ming system that I have used for class exercise.ACKNOWLEDGEMENTS Tam grateful to Amiram Yehudai for the many discussions that we have had over the past few years on programming and how to teach it. He was also kind enough to read the entire manuscript and suggest improvements. I should also like to thank Amir Pnueli for teaching me programming logic and verification, and Henry Hirschberg of Prentice/Hall International for his encouragement during the transition from notes to book. Also, special thanks to my wife Margalit for drawing the sketches on which the illustrations were based. xvPrinciples of Concurrent Programming1 WHAT IS CONCURRENT PROGRAMMING? 1.1 FROM SEQUENTIAL TO CONCURRENT PROGRAMMING Figure 1.1 shows an interchange sort program. The program can be com- piled into a set of machine language instructions and then executed on a computer. The program is sequential; for any given input (of 40 integers) the computer will always execute the same sequence of machine instructions. If we suspect that there is a bug in the program then we can debug by tracing (listing the sequence of instructions executed) or by breakpoints and snapshots (suspending the execution of the program to list the values of the variables). There are better sequential sorting algorithms (see Aho et al., 1974) but we are going to improve the performance of this algorithm by exploiting the possibility of executing portions of the sort in parallel. Suppose that (for n=10) the input sequence is: 4, 2, 7, 6, 1, 8, 5, 0, 3, 9. Divide the array into two halves: 4, 2,7, 6, 1 and 8, 5, 0, 3, 9; get two colleagues to sort the halves simultaneously: 1, 2, 4, 6, 7 and 0, 3, 5, 8, 9; and finally, with a brief inspection of the data, merge the two halves: A simple complexity analysis will now show that even without help of colleagues, the parallel algorithm can still be more efficient than the sequen- tial algorithm. In the inner loop of an interchange sort, there are (n-1) + (n—2) +... + 1=n(n—1)/2 comparisons. This is approx. n?/2. To sort n/22. WHAT IS CONCURRENT PROGRAMMING? CHAP. 1 Program sortprogram; const n=40; var a: array[1..n] of integer; k: integer; procedure sort(low,high: integer); var i,j, temp: integer; begin for i := low to high-1 do for j := i+1 to high do if a[j] < a[i] then begin temp := afj); afj] := afi); a[i] := temp end end; begin (+ main program *) for k := 1 to n do read (a[k]); sort (1, n); for k := 1 ton do write (a[k]) end. Fig. 1.1. elements, however, requires only (n/2)?/2 = n2/8 comparisons. Thus the parallel algorithm can perform the entire sort in twice n?/8 = n2/4 compari- sons to sort the two halves plus another n comparisons to merge. The table in Fig. 1.2 demonstrates the superiority of the new algorithm. The last column shows that additional savings can be achieved if the two sorts are performed simultaneously. n m/2 (n2/4)+n (n?/8)+n 40 800 440 140 100 5000 2600 1350 1000 500 000 251 000s 126 000 Fig. 1.2. Figure 1.3 is a sequential program for this algorithm. It can be executed on any computer with a Pascal compiler and it can be easily translated into other computer languages.SECT. 1.1 FROM SEQUENTIAL TO CONCURRENT PROGRAMMING 3, program sortprogram; const n=20; twon=40; var a: array(1..twon] of integer; k: integer; procedure sort(low, high: integer); (* as before *) procedure merge(low, middle, high: integer); var countl, count2: integer; k, index1, index2: integer; while count! < middle do if a[count1] < a[count2] then begin write (a[count1]); count| := counti+1; if count! >= middle then for index2 := count2 to high do write(afindex2}) end else begin write (a[count2]); count2 := count2+1; if count2 > high then begin for index1 := count to middle—1 do write (afindex1}); count\ := middle (* terminate *) end end end; begin (* main program *) for k := 1 to twon do read (alk); sort(1, m); sort(n+1, twon); merge(1, n+1, twon) end. Fig. 1.3,4) WHAT IS CONCURRENT PROGRAMMING? CHAP. 1 Suppose that the program is to be run ona multiprocessor computer—a computer with more than one CPU. Then we need some noiation that can express the fact that the calls sort(1,n) and sort(n+1, twon) can be executed in parallel. Such a notation is the cobegin—coend bracket shown in Fig. 1.4. cobegin p,; . . . ; p, coend means: suspend the execution of the main prog- ram; initiate the execution of procedures p,,.. ., p, on multiple computers; when all of p,, .... p, have terminated then resume the main program. Program sortprogram; ( declarations as before *) begin (* main program *) for k := to twon do read(a{k]); cobegi sort(1, n); sort(n+1, twon) coend; merge(1, n+1, twon) end. Fig. 1.4. The programs of Figs. 1.3 and 1.4 are identical except for the cobegin-coend in Fig. 1.4. There would be no need for both versions if the definition of cobegin-coend was modified. Instead of requiring that the procedures be executed in parallel, cobegin-coend becomes a declaration that the procedures may be executed in parallel. It is left to the implementa- tion—the system hardware and software—to decide if parallel execution will be done. Processors may be added or removed from the system without affecting the correctness of the program—only the time that it would take to execute the program. The word concurrent is used to describe processes that have the poten- tial for parallel execution. We have shown how an algorithm can be improved by identifying procedures that may be executed concurrently. While the greatest improvement is obtained only under true parallel execu- tion, it is possible to ignore this implementation detail without affecting the superiority of the concurrent algorithm over the sequential algorithm. 1.2 CONCURRENT PROGRAMMING Concurrent programming is the name given to programming notations and techniques for expressing potential parallelism and for solving the resulting synchronization and communication problems. Implementation of parallel- ism is a topic in computer systems (hardware and software) that is essentiallySECT. 1.2 CONCURRENT PROGRAMMING 5 independent of concurrent programming. Concurrent programming is important because it provides an abstract setting in which to study parallel- ism without getting bogged down in the implementation details. This abs- traction has proved to be so useful in writing clear, correct software that modern programming languages offer facilities for concurrent program- ming. ‘The basic problem in writing a concurrent program is to identify which activities may be done concurrently. If the merge procedure is also included in the cobegin-coend bracket (Fig. 1.5), the program is no longer correct. If you merge the data in parallel with sorting done by your two colleagues, the scenario of Fig, 1.6 might occur. cobegin sort(1, n); sort(n+1, twon); merge(1, n+1, twon) coend Fig. 1.5. Colleague1 Colleague? You Initially 4, 2,7, 6,1 fea = Colleaguel exchanges 2, 4, 7, 6, 1 8, 5, 0, 3,9 - Colleague? exchanges 2,4, 7,6, 1 5, 8, 0, 3,9 - You merge : 5 2 You merge ie ” 2,4 You merge ‘i 2,4, 5 Fig. 1.6. However, merge could be a concurrent process if there were some way of synchronizing its execution with the execution of the sort processes (Fig. 1.7). while count] < middle do wait until i of procedure call sort(1,n) is greater than count] and i of procedure call sort(n+1, twon) is greater than count2 and only then: if alcount!] < a{count2] then Fig. 1.7. Parallelism is important not only for the improvement that can be achieved in program performance but also in program quality. Consider the6 WHAT IS CONCURRENT PROGRAMMING? CHAP. 1 following problem: Read 80-column cards and print them on 125-character lines. However, every run of 1 = 1 to 9 blank spaces is to be replaced by a single blank followed by the numeral n. This program is difficult to write as a sequential program. There are many interacting special cases: a run of blanks overlapping the end of a card, the pair blank-n overlapping the end ofa line and so on. One way to improve the clarity of the program would be to write three separate programs: one to tead cards and write a stream of characters onto a temporary file; a second program to read this character stream and modify runs of blanks, writing the new stream onto a second temporary file; and a third program to read the second temporary file and print lines of 125 characters each. This solution is not acceptable because of the high overhead of the temporary files. However, if the three programs could be run concurrently (not necessarily in parallel) and communications paths could be established between them, then the programs would be both efficient and elegant. Sequence Sequence lodif\ of of runs of ouTPUT blanks/ characters lines Py Py Processes Py , Pz, P3 may execute concurrently Sequence of INPUT ——»{ cards Sequence of characters Py 1.3 CORRECTNESS OF CONCURRENT PROGRAMS Concurrent programming is much more difficult than sequential program- ming because of the difficulty of ensuring that a concurrent program is correct. Consider the sequential sort programs of Figs. 1.1 and 1.3: if they were tested on several sets of input data then we would feel confident that they are correct. Guidelines for testing would be to include a sorted input, a reversed input, an input all of whose elements are identical and so on. A run-of-the-mill bug (such as an incorrect for-loop limit) would seldom escape detection. The scenario in Fig. 1.6, illustrates that the concurrent program of Fig 1.5 is incorrect. However, this program is not a sequential program and other scenarios exist. If the processors assigned to sort are sufficiently rapid then merge may always be working on sorted data. In that case, no amount of testing would detect any problem. One day (perhaps months after the program has been put into production) an improved component in the computer system causes the merge to speed up and then the program gives incorrect answers as demonstrated in 1.6. Of course the natural reaction is:SECT. 1.4 INTERLEAVING 7 “This program worked yesterday so the new component must be at fault.” A scenario is a description of a possible execution sequence of a pro- gram and shows how a computer might “act out” a program. It is usually used to show that a program is incorrect: since the computer may execute the program in a manner that produces the wrong answer, the program cannot be correct. Conversely, how can we show that the concurrent program in Fig. 1.4 is correct? It no longer makes sense to look for and test paths that can be execution sequences. At times, there may be two such sequences caused by parallel execution of the algorithm. Sequential programming has a well-developed proof theory. Assertions are made about the state of the computer (i.¢. the values of the variables and the program counter) before and after executing an instruction, and these are then combined into a logical proof. In concurrent programming, this method needs to be modified because the programs can interfere with each other. The correctness assertions for procedures sort and merge of the previ- ‘ous sections are elementary to state and prove: sort input assertion: a is an array of integers, sort output assertion: a is “sorted”, i.e. a now contains a permutation of the original elements and they are in ascending order, merge input assertion: the two halves of a are “sorted”, merge output assertion: the elements of a have been written in ascend- ing order. The correctness of the program in Fig. 1.1. is immediate from the correctness of procedure sort. The correctness of 1.3 is easily obtained by concatenating the correctness proofs of sort and merge. The correctness of Fig. 1.4 needs a new technique. We have to be able to express the fact that the two instances of sort do not interfere with one another. The program in Fig. 1.5 is incorrect though the procedures comprising it are correct; unfor- tunately, they interact in a manner which makes the program incorrect. The program in 1.7 is correct but new ideas are needed to be able to reason about synchronization. 1.4 INTERLEAVING Interleaving is a logical device that makes it possible to analyze the correct- ness of concurrent programs. Suppose that a concurrent program P consists of two processes P, and P,. Then we say that P executes any one of the execution sequences that can be obtained by interleaving the execution sequences of the two processes. It is as if some supernatural being were to execute the instructions one at a time, each time flipping a coin to decide whether the next instruction will be from P, or P.8 WHAT IS CONCURRENT PROGRAMMING? CHAP.1 We claim that these execution sequences exhaust the possible behaviors of P. Consider any instructions I, and J, from P, and P,, respectively. If J, and 1, do not access the same memory cell or register then it certainly does not matter if I, isexecuted before /,, after /, or even simultaneously with Z, (if the hardware so allows). Suppose on the other hand that I, is “Store 1 into memory cell M” and that J; is “Store 2 into memory cell M”. If J, and [, are executed simultaneously then the only reasonable assumption is that the result is consistent. That is, cell M will contain either 1 or 2 and the computer does not store another value (such as 3) of its own volition. If this were not true then it would be impossible to reason about concurrent programs. The result of an individual instruction on any given data cannot depend upon the circumstances of its execution. Only the external behavior of the system may change—depending upon the interac- tion of the instructions through the common data. In fact, computer hardware is built so that the result of executing an individual instruction is consistent in the way just defined. Thus, if the result of the simultaneous execution of J, and /, is 1 then this is the same as saying that J, occurred before J, in an interleaving and conversely if the result is 2. Interleaving does not make the analysis of concurrent programs simple. The number of possible execution sequences can be astronomical. Neverthe- less, interleaved execution sequences are amenable to formal methods and will allow us to demonstrate the correctness of concurrent programs. 1.5 THE ORIGIN OF OPERATING SYSTEMS Concurrent programming, though generally applicable, grew out of prob- lems associated with operating systems. This section outlines the develop- ment of such systems so that the background to the growth of concurrent programming can be appreciated. It is not often that an obsolete technology reappears. Programmable pocket calculators have resurrected machine language programming: abso- lute addresses must be used for data and labels. On the other hand, the owner is not constrained to working during the hours that the computer center is open. While the pocket calculator isa marvel of electronics, machine language programming directly on the computer is slow and difficult. In the 1950s, when computers were few and expensive, there was great concern over the waste caused by this method. If you signed up to sit at the computer console from 0130 to 0200 and you spent 25 minutes looking for a bug, this 25 minutes of computer idle time could not be recovered. Nor was your colleague who signed up for 0200-0230 likely to let you start another run at 0158. If we analyze what is happening in the terms of the previous sections weSECT. 1.5 THE ORIGIN OF OPERATING SYSTEMS 9 see that the manual procedures that must be performed—mounting tapes, setting up card decks, or changing places at the console—are disjoint from the actual computation and can be performed concurrently with the compu- ter’s processing. The second generation of computers used a supervisor program to batch jobs. A professional computer operator sat at the console. Program- mers prepared card decks which were concatenated into ‘‘batches” that were fed into the computer once an hour or so. The increase in throughput (a measure of the efficiency of a computer; it is the number of jobs—suitably weighted—that can be run in a given time period) was enormous—the jobs were run one after another with no lost minutes. The programmers, how- ever, lost the ability to dynamically track the progress of their programs since they no longer sat at the computer console. In the event of an error in one job, the computer simply commenced execution of the next job in the batch, leaving the programmer to puzzle out what happened from core dumps. With a turnaround time (the amount of time that elapses between a job being submitted for execution and the results being printed) of hours or days, the task of programming became more difficult even though certain aspects were improved by high-level languages and program libraries. Despite this improvement in throughput, systems designers had noticed another source of inefficiency not apparent to the human eye. Suppose that a computer can execute one million instructions per second and that it is connected to a card reader which can read 300 cards per minute (= one card in 1/S second). Then from the time the read instruction is issued until the time the card has been read, 200 000 instructions could have been executed. A program to read a deck of cards and print the average of the numbers punched in the cards will spend over 99% of its time doing nothing even though 5 cards per second seems very fast. The first solution to this problem was spooling. The 1/O speed of a magnetic tape is much greater than that of the card reader and the line printer that are the interface between the computer and the programmer. We can decompose the operation of the computer into three processes: a process to read cards to tape; a process to execute the programs on the tape and write the results onto a second tape; and a process to print the informa- tion from the second tape. Since these processes are disjoint (except for the exchange of the tapes after processing a batch), the throughput can be greatly increased by running cach process on a separate computer. Since very simple computers can be used to transfer information to and from the magnetic tape, the increase in cost is not very great compared to the savings achieved by more efficient use of the main computer. Later generations of computer systems have attacked these problems by switching the computer among several computations whose programs and data are held simultaneously in memory. This is known as multiprog- ramming. While I/O is in progress for program P, the computer will execute10 WHAT IS CONCURRENT PROGRAMMING? CHAP. 1 several thousand instructions of program P, and then return to process the data obtained for P,. Similarly, while one programmer sitting at the terminal of a time-sharing system+ is thinking, the computer will switch itself to execute the program requested by a second programmer. In fact, modern computer systems are so powerful that they can switch themselves among dozens or even hundreds of I/O devices and terminals. Even a minicompu- ter can deal with a dozen terminals. The importance of the concept of interleaved computations mentioned in the previous section has its roots in these multiprogrammed systems. Rather than attempt to deal with the global behavior of the switched compu- ter, we will consider the actual processor to be merely a means of interleav- ing the computations of several processors. Even though multiprocessor systems—systems with more than one computer working simultane- ously—are becoming more common, the interleaved computation model is still appropriate. The sophisticated software systems that are responsible for multi- programming are called operating systems. The term operating system is often used to cover all manufacturer-provided software such as I/O pro- grams and compilers and not just the software responsible for the multi- programming. While the original concern of operating system designers was to improve throughput, it soon turned out that the throughput was affected by numerous system “‘crashes” when the system stopped functioning as it was supposed to and extensive recovery and restart measures delayed execution of jobs. These defects in the operating systems were caused by our inadequ- ate understanding of how to execute several programs simultaneously and new design and programming techniques are needed to prevent them. 1.6 OPERATING SYSTEMS AND CONCURRENT PROGRAMMING If you could sense the operation of a computer that is switching itself every few milliseconds among dozens of tasks you would certainly agree that the computer seems to be performing these tasks simultaneously even though we know that the computer is interleaving the computations of the various tasks. I now argue that it is more than a useful fiction to assume that the computer is in fact performing its tasks concurrently. To see why this is so, let us consider task switching in greater detail. Most computers use interrupts + A time-sharing system is a computer system that allows many programmers to work simultaneously at terminals. Each programmer may work under the illusion that the computer is working for him alone (though the computer may seem to be working slowly if too many terminals are connected).SECT. 1.6 OPERATING SYSTEMS AND CONCURRENT PROGRAMMING 11 for this purpose. A typical scenario for task switch by interrupts is as follows. Program P, makes a read request and then has its execution suspended. The CPU may now execute program P,. When the read requested by P, has been completed, the I/O device will interrupt the execution of P; to allow the operating system to record the completion of the read. Now the execution of either P, or P, may be resumed. The interrupts occur asynchronously during the execution of programs by the CPU. By this is meant that there is no way of predicting or coordinat- ing the occurence of the interrupt with the execution of any arbitrary instruction by the CPU. For example, if the operator who is mounting a magnetic tape happens to sneeze, it may delay the “tape ready” signal by 8.254387 seconds. However, if he is “slow” with his handkerchief, the delay might be 8.254709 seconds. Insignificant as that difference may seem, it is sufficient for the CPU to execute dozens of instructions. Thus for all practi- cal purposes it makes no sense to ask: “What is the program that the computer is executing?” The computer is executing any one ofa vast number of execution sequences that may be obtained by arbitrarily interleaving the execution of the instructions of a number of computer programs and 1/O device handlers. This reasoning justifies the abstraction that an operating system consists of many processes executing concurrently. The use of the term process rather than program emphasizes the fact that we need not differentiate between ordinary programs and external devices such as terminals. They are all independent processes that may, however, need to communicate with each other. The abstraction will try to ignore as many details of the actual applica- tion as possible. For example, we will study the producer-consumer problem which is an abstraction both of a program producing data for consumption by a printer and of a card reader producing data for consumption by a program. The synchronization and communication requirements are the same for both problems even though the details of programming an input routine are rather different from the details of an output routine. Even as new I/O devices are invented, the input and output routines can be designed within the framework of the general producer-consumer problem. On the other hand, we assume that each process is a sequential process. It is always possible to refine the description of a system until it is given in terms of sequential processes. The concurrent programming paradigm is applicable to a wide range of systems, not just to the large multiprogramming operating systems that gave rise to this viewpoint. Moreover, every computer (except perhaps a cal- culator or the simplest microcomputer) is executing progams that can be considered to be interleaved concurrent processes. Minicomputers are sup- plied with small multiprogramming systems. If not, they may embedded in12. WHAT IS CONCURRENT PROGRAMMING? CHAP.1 real-time systems? where they are expected to concurrently absorb and process dozens of different asynchronous external signals and operator commands. Finally, networks of interconnected computers are becoming common. In this case true parallel processing is occurring. Another term used is distributed processing to emphasize that the connected computers may be physically separated. While the abstract concurrency that models switched systems is now well understood, the behavior of distributed systems is an area of current research. 1.7 AN OVERVIEW OF THE BOOK Within the overall context of writing correct software this book treats the single, but extremely important, technical point of synchronization and communication in concurrent programming. The problems are very subtle; ignoring the details can give rise to spectacular bugs. In Chapter 2 we shall define the concurrent programming abstraction and the arguments that justify each point in the definition. The abstraction is sufficiently general that it can be applied without difficulty to real systems. On the other hand it is sufficiently simple to allow a precise specification of both good and bad behavior of these programs. Formal logics exist which can formulate specifications and prove prop- erties of concurrent programs in this abstraction though we will limit our- selves to informal or at most semi-formal discussions. The fact that the discussion is informal must not be construed as meaning that the discussion is imprecise. A mathematical argument is considered to be precise even if it is not formalized in logic and set theory. The basic concurrent programming problem is that of mutual exclusion. Several processes compete for the use of a certain resource such as a tape drive but the nature of the resource requires that only one process at a time actually accessed the resource. In other words, the use of the resource by one Process excludes other processes from using the resource. Chapter 3 pre- sents a series of attempts to solve this problem culminating in the solution known as Dekker’s algorithm. The unsuccessful attempts will each point out a possible “bad” behavior of a concurrent program and will highlight the differences between concurrent and sequential programs. Dekker’s algorithm is itself too complex to serve as a model for more complex programs. Instead, synchronization primitives are introduced. Just as a disk file can be copied onto tape by a single control language command + Wheareas a time-sharing system gives the user the ability to use all the resources of a computer, the term real-time system is usually restricted to systems that are required to respond to specific pre-defined requests from a user or an external sensor. Examples would be air-traffic control systems and hospital monitoring systems.SECT. 1.7 AN OVERVIEW OF THE BOOK 13 ora file can be read by writing read in a high level language, so we can define programming language constructs for synchronization by their semantic definition—what they are supposed to do—and not by their implementa- tion, We shall indicate in general terms how these primitives can be implemented but the details vary so much from system to system that to fully describe them would defeat our purpose of studying an abstraction. Hope- fully, it should be possible for a “casual” systems programmer to write concurrent programs without knowing how the primitives are implemented. A model implementation is described in the Appendix. Chapter 4 commences the study of high level primitives with E. W. Dijkstra’s semaphore. The semaphore has proved extraordinarily successful as the basic synchronization primitive in terms of which all others can be defined. The semaphore has become the standard of comparison. It is sufficiently powerful that interesting problems have elegant solutions by semaphores and it is sufficiently elementary that it can be successfully studied by formal methods. The chapter is based on the producer-consumet problem mentioned above; the mutual exclusion problem can be trivially solved by semaphores. Most operating systems have been based on monolithic monitors. A central executive, supervisor or kernel program is given sole authority over synchronization. Monitors, a generalization of this concept formalized by Hoare, are the subject of Chapter 5. The monitor is a powerful conceptual notion that aids in the development of well structured, reliable programs. The problem studied in this chapter is the problem of the readers and the writers. This is a variant of the mutual exclusion problem in which there are two classes of processes: writers which need exclusive access to a resource and readers which need not exclude one another (though as a class they must exclude all writers). ‘The advent of distributed systems has posed new problems for concur- rent programming. C. A. R. Hoare has proposed a method of synchroniza- tion by communication (also known as synchronization by. rendezvous) appropriate for this type of system. The designers of the Ada programming language have chosen to incorporate in the language a variant of Hoare’s system. Anticipating the future importance of the Ada language, Chapter 6 studies the Ada rendezvous. A classic problem in concurrent programming is that of the Dining Philosophers. Though the problem is of greater entertainment value than practical value, it is sufficiently difficult to afford a vehicle for the compari- son of synchronization primitives and a standing challenge to proposers of new systems. Chapter 7 reviews the various primitives studied by examining solutions to the problem of the Dining Philosophers. Programming cannot be learned without practice and concurrent pro- gramming is no exception. If you are fortunate enough to have easy access to14 WHAT IS CONCURRENT PROGRAMMING? CHAP. 1 a minicomputer or to a sophisticated simulation program, there may be no difficulty in practicing these new concepts. If not, the Appendix describes in full detail an extremely simple simulator of concurrency that can be used for class exercise. In any case, the Appendix can serve as an introduction to implementation of concurrency. The book ends with an annotated bibliography suggesting further study of concurrent programming. 1.8 PROGRAM NOTATION The examples in the text will be written in a restricted subset of Pascal- S, which is itself a highly restricted subset of Pascal. This subset must of course be augmented by constructs for concurrent programming. It is intended that the examples be legible to any programmer with experience in Pascal, Ada, C, Algol, or PL/I. The implementation kit in the Appendix describes an interpreter for this language that will execute the examples and that can be used to program the exercises. The language in the kit contains more Pascal language features than are used in the text of the book and thus users of the kit are assumed to be able to program in sequential Pascal. These extra features are necessary in order to use the kit to solve the exercises, although the exercises them- selves could be programmed in other languages that provide facilities for concurrent programming. The examples in the chapter on monitors are standard and can be adapted to the many systems that provide the monitor facility such as Concurrent Pascal, Pascal-Plus, or CSP/k. The examples in the chapter on the Ada rendezvous are executable in Ada. We now present a sketch of the language that should be sufficient to enable programmers unfamiliar with Pascal to understand the examples. 1. Comments are inserted between (* and *). 2. The first line in a program should be program name; 3. Symbolic names for constants may be declared by the word const followed by the constant itself: const twon=40; 4. All variables in each procedure in the main program must be declared by the word var followed by the names of the variables and a type: var i, j, temp: integer; found: boolean;SECT, 1.8 PROGRAM NOTATION 15 The available types are : integer, boolean (with constants true and false) and arrays: var a:array[/owindex...highindex] of integer; . Following the declaration of the variables, procedures and functions may be declared: procedure name (formal parameters); and func- tion name (formal parameters _): returntype; . The formal parame- ter definition has the same form as that of a variable list: procedure sort (Jow,high: integer); function /ast(index: integer): boolean; . The body of the main program or of procedures is a sequence of statements separated by semi-colons between begin and end. The main program body is terminated by a period and the procedure bodies by semi-colons, The usual rules on nested scopes apply. The statements are: assignment statement if boolean-expression then statement if boolean-expression then statement else statement for index-variable := lowindex to highindex do statement while boolean-expression do statement repeat sequence-of-statements until boolean-expression The syntactic difference between while and repeat is that while takes a single statement and repeat takes a sequence of statements (separated by semi-colons). The semantic difference is that the while tests before the loop is done and repeat tests afterwards. Thus repeat executes its loop at least once. . A sequence of statements may be substituted for “statement” in the above forms by enclosing the sequence of statements in the bracket begin ... end to form a single “compound” statement: if aj] < a{i] then begin temp := alj); aff] == afi); afi] := temp end In detail this is read: if the boolean expression (a[j] < a[i]) has the value true, then execute the compound statement which is a sequence of three assignment statements. If the expression is false, then the (com- pound) statement is not executed and the execution continues with the next statement.16 WHAT IS CONCURRENT PROGRAMMING? CHAP 1 9. 10. 1.9 1d 1.2 1.3 Assignment statements are written variable expression. The vari- able may be a simple variable or an element of an array: ali]. The type (integer or boolean) of the expression must match that of the variable. Integer expressions are composed of integer variables and constants using the operators: +, —, *, div (integer divide with truncation) and mod. Boolean expressions may be formed from relations between integer expressions: =, <> (not equal), <, >, <= (less than or equal) >= (greater than or equal). The boolean operators and, or and not may be used to form compound boolean expressions. For those who know Pascal we list here the additional features that are defined in the language of the implementation kit some of which will be necessary if you plan to write any programs using the kit. (a) Type declarations. Since there are no scalar, subrange or record types, this is mostly useful for array types: type sortarray = array[1..n] of integer var a: sortarray; (b) Character constants and variables of type char. (c) Multidimensional arrays (arrays of arrays). (d) A parameter may be passed by reference rather than value by prefixing the formal parameter by var. (e) Recursive functions and procedures. (f) I/O may be performed only on the standard textfiles input and output. To ensure that you do not forget this restriction, the declaration of external files in the program card has been removed. read, readin, write, writeln, eoln, eof (all without a file parameter) function as in Pascal. Only the default field widths may be used ina write, which will, however, accept a ‘string’ asa field to be printed: writeln (‘the answer EXERCISES* Write a two-process concurrent program to find the mean of m numbers. Write a three-process concurrent program to multiply 3x3 matrices. Each process of the matrix multiply program executes three multiplications and two additions for each of three rows or altogether 15 instructions. How many execution sequences of the concurrent program may be obtained by interleaving the executions of the three processes? + Slightly harder exercises are marked throughout the book with an asterisk (*)1.4 15 1.6 17 EXERCISES 17 Perform a similar analysis for sortprogram. You will have to make some assumptions on the number of interchanges that will be done. Test the concurrent soriprogram of Fig. 1.3. Test the concurrent sortprogram of Fig. 1.4 which has the merge defined as a third process. Run the program several times with exactly the same data Run the program in Fig. 1.8 several times. Can you explain the results? program increment; const m = 20; var n: integer; procedure incr; var i: integer; begin for i end; begin (* main program *) n cobegin incr; incr coend; writen (' the sum is ', n) end. = 1tomdon:=nt1 Fig. 1.8.2 THE CONCURRENT PROGRAMMING ABSTRACTION 2.1 INTRODUCTION Concurrent programming is not the study of operating systems or real-time systems, but of abstract programming problems posed under certain rules. Concurrent programming was motivated by the problems of constructing operating systems, and its examples are abstract versions of such problems. Most importantly, the rules of concurrent programming are satisfied in many systems and thus its techniques can be used in real systems. Components of a system which are not amenable to concurrent programming techniques should be singled out for extremely careful design and implementation. Chapter 1 gave the definition of a concurrent program. It consists of several sequential processes whose execution sequences are interleaved. The sequential programs are not totally independent — if they were so there would be nothing to study. They must communicate with each other in order to synchronize or to exchange data, The first means of communication that we shall study is the common memory. This is appropriate for the pseudo-parallel switched computers where all processes are running on the same processor and using the same physical memory. It is also used on some truly parallel systems such as the CDC Cyber computers where even though one CPU and ten PP's (peripheral processors) are simultaneously executing separate programs, synchronization is accomplished by having the PP’s read and write the CPU's memory. In our abstraction, common memory will be represented simply by global variables accessible to all processe: Common memory can also be used to hold access-restricted proce- dures. Access to these procedures is, in effect, allocated to a process. This is the way most third generation operating systems were implemented. The “system” programs can only be called by special instructions which ensure that only one process at a time is executing a system program. 18SECT. 2.2 MUTUAL EXCLUSION 19 With the introduction of distributed computing it is no longer valid to assume that a common central memory exists. Chapter 5 discusses concur- rent programming by means of sending and receiving signals instead of reading and writing a common variable or executing a common procedure Synchronization by message-passing has been used on several experimental systems for single-processor computers but this approach has not been widely accepted because of the possible inefficiency of message-passing compared with simpler systems. Of course, distributed systems have no choice 2.2: MUTUAL EXCLUSION Mutual exclusion is one of the two most important problems in concurrent programming because it is the abstraction of many synchronization prob- lems. We say that activity A , of process P, and activity A, of process P; must exclude each other if the execution of A, may not overlap the execution of A,. If P, and P, simultaneously attempt to execute their respective activities, Aj, then we must ensure that only one of them succeeds. The losing process must block; that is, it must not proceed until the winning process completes the execution of its activity A. The most common example of the need for mutual exclusion in real systems is resource allocation. Obviously, two tapes cannot be mounted simultaneously on the same tape drive. Some provision must be made for deciding which process will be allocated a free drive and some provision must be made to block processes which request a drive when none is free. There is an obvious solution: run only one job at a time. But this defeats one of the main aims of concurrent programming — parallel execution of several processes. Meaningful concurrency is possible only if the processes are loosely connected. The loose connection will manifest itself by the need for short and occasional communication. The abstract mutual exclusion problem will be expressed: remainder pre-protocol critical section post-protocol. “remainder” will be assumed to represent some significant processing. Occasionally, i.e. after the completion of remainder, the process needs to enter a short critical section. It will execute certain sequences of instructions,20 THE CONCURRENT PROGRAMMING ABSTRACTION CHAP. 2 called protocols before and possibly after the critical section. These pro- tocols will ensure that the critical section is in fact executed so as to exclude all other critical secitons. Of course, just as the critical section should be short relative to the main program in order to benefit from concurrency, the protocols must also be relatively short. The protocols represent the over- head paid for concurrency. Hopefully, if the critical sections and the pro- tocols are sufficiently short then the significant processing abstracted as remainder can be overlapped thus justifying the design of the multipro- gramming system. There is another, more important, reason for requiring loose connec- tion among concurrent processes and that is to ensure reliability. We want to be assured that if there is a bug in one of the processes, then it will not propagate itself into a system “‘crash’’. It should also be possible to gracefully degrade the performance of a system if an isolated device should fail (“fail- soft”). It would be absurd to have a system crash just because one tape drive became faulty The abstract requirement will be that, if a process abnormally termi- nates outside the critical section then no other process should be affected. (For this purpose the protocols are considered to be part of the critical section.) Since the critical section is where the communication is taking place, it is not reasonable to require the same of the critical sections. We might use the following metaphor. If a runner in a relay race fell after he has passed the baton then the race should not be affected. It is unreasonable to hope that the race is unaffected if the fall occurred at the critical moments during the baton exchange. This restriction is not unreasonable even in practice. Critical sections such as disk I/O will often be executed by common system routines or by compiler-supplied routines which have been written by a competent systems programmer. The probability of a software error in such a routine should be much smaller than in a run-of-the-mill program. 2.3 CORRECTNESS What does it mean for concurrent programs to be correct? An ordinary program is correct if it halts and prints the “right” answer. In general, you will know a “right” answer if you see one. This is also true of some concur- Tent programs such as soriprogram. On the other hand, the single most distinguishing feature of an operat- ing system or real-time system is that it must never halt. The only way to halt a typical operating system is to push the start button on the computer panel. An operating system prints nothing of its own (except some non-essential logging and accounting data). Thus when studying operating systems, weSECT. 2.3 CORRECTNESS 21 must revamp our notions of what it means for a program to be correct. Since most concurrent programming is done within operating systems and real- time systems, this is the area to be studied in this book. An obvious truism is that any program must do what its specifications say that it is supposed to do. This is no less true in the case of concurrent programs though the specifications are radically different from those of sequential programs. In our abstraction, we shall distinguish two types of correctness properties: safety properties and liveness properties. Safety properties are those required by the static portions of the specifi- cations. These are often the only requirements explicitly specified. Mutual exclusion is a safety property: the requirement that critical sections exclude one another is absolute and does not change during the execution of the program. The safety property of the producer-consumer problem is that the consumer must consume every piece of data produced by the producer and that it must do so in the order in which they were produced. Safety properties are akin to what is known in the theory of sequential programs as partial correctness: if the program terminates, the answers must be “correct”. Safety properties are usually relatively easy to show. They are explicitly required by the specifications and programs are designed to meet these specifications. You can always achieve more safety by giving up some concurrency and letting more segments of the process execute sequentially. Violation of mutual exclusion is the cause of most operating system crashes. Dynamic memory allocation is frequently involved: a process may be convinced that it knows the location of a certain table in memory while in fact the table has been removed and its memory allocated to another process. At a computer center known to the Author, an unscrupulous pro- grammer managed to insert into the system a program that, on command, would give his programs the highest priority. However, he did not know that mutual exclusion protocols were required because the scheduling table was not fixed in memory. Thus the execution of his command would often write into some other table of the system causing a crash. Since the crash occurred many minutes later, the memory dumps gave no clue as to what had hap- pened. After several weeks, the problem was solved by noting the strange command on the system log. If such an error had been made in a regular systems program it would have been even more difficult to catch. Liveness, on the other hand, deals with dynamic properties. It is akin to sequential programming’s total correctness: the program terminates and the answer is “correct”. In concurrent systems, liveness means that if something is supposed to happen then eventually it will happen. If a process wishes to enter its critical section then eventually it will do so. If a producer produces data then eventually the consumer will consume it.22. THE CONCURRENT PROGRAMMING ABSTRACTION CHAP. 2 The most serious breach of liveness is the global form known as dead- Jock.+ Deadlock means that the computer is no longer doing any (useful) work. A loop that searches for a free tape drive when none is ever going to be available is not useful work. If all processes are suspended or in such loops, then the computer is said to be deadlocked. This system hang is catastrophic since all users in a multiprogrammed system are affected. The following deadlock scenario actually occurred in an operating system. The system had multiple processors. Each job wrote its accounting data into a common area of memory which was written to a disk file when it filled. The write procedure ran on whatever processor was free. If all the Processors tried to write accounting data at the same time, there was no free Processor to write to disk. Since no program voluntarily relinquished its processor, the system deadlocked. A typical solution in this case is to use the computer console terminal to fool a process into thinking that the memory area is empty. Of course, some data will be lost but this is usually preferable to a deadlocked system and restarting all currently running programs. A local breach of liveness is called lockout or (individual) starvation. In this case there is always some process which can progress but some identifi- able process is being indefinitely delayed. Lockout is less serious than deadlock since the computer is still doing some (presumably) useful work. On the other hand, lockout is difficult to discover and correct because it can happen only in complex scenarios where some processes unwittingly “con- spire” to deny a resource to a hapless process. For reasons to be discussed in the next section, we limit our discussion to qualitative liveness which means that the word “eventually”, used in the definition of liveness, means exactly that: within an unspecified but finite length of time. Between the complete disregard of time by the liveness concept and the introduction of explicit time, is the concept of fairness: a process wishing to progress must get a fair deal relative to all other processes. Fairness is more difficult to define precisely and we will mention it only occasionally. In addition, many systems will consciously be “unfair” by designing a priority scheme to favor some processes over others. 2.4 TIMING We make no assumptions concerning the absolute or relative speeds at which the processes are executed. This statement seems rather shocking at first. However, failure to follow this restriction in systems design can cause + Deadlock is theoretically considered to be a safety property because it is something that should never happen. However, absence of deadlock can be shown by proving that there is always at least one live process.SECT. 2.4 TIMING 23 serious bugs. The statement is shocking because of the naive preoccupation with efficiency that most programmers have. After all, we know that a disk is slower than a CPU and it is tempting to take advantage of that fact in designing not only the structure of the system but also the synchronization details. One reason for ignoring timing is that our intuition is not able to cope with the scale involved. There are only about § million minutes in a year but there are a million microseconds in a second. It is folly to state that: process P, ought to be able to finish its critical section before process P; finishes its non-critical section, unless such a statement is backed up by a detailed calculation A second reason for ignoring timing is that time-dependent bugs are extremely difficult to identify and correct. Qualitative treatments as pre- sented in this book, are insufficient; timing calculations must be included. Each such bug can mean weeks of work for a team of programmers. We shall pay any reasonable penalty in efficiency to obtain a reliable system. You have to save an enormous number of microseconds to make a half-hour system crash worthwhile Finally there is an important practical reason to design a software system that is independent of timing assumptions and that is the dynamic nature of a computer configuration. Even if able to carry out the calculations necessary to ensure reliability in a time-dependent system, one is at the mercy of any configuration change. Theoretically, the addition of a single termtinal could invalidate the reliability of the system. This is not too far- fetched. A computer manufacturer once started to market a modernized component that ran at twice the speed of the original component. The first customers received a notice that in effect said: “Thanks for buying our double-speed component but please run it at the original speed for a few months while we comb the operating system for time dependencies”! A system may certainly require some time-dependent processing but these functions should be clearly identified and isolated to ensure the flexibility and realiability of the system. The advantages that accrue to a time-independent (asynchronous) sys- tem can be demonstrated on the hardware level by the architecture of the DEC PDP-11 computers. Instead of synchronizing access to memory and 1/O devices by a clock, all data in the basic PDP-11 is transferred asyn- chronously according to a protocol that is not too different conceptually from what we will be doing. The overhead of the protocol means that for any given electronic technology, the PDP-11 will be somewhat slower than a computer built to a synchronous design. On the other hand, the architecture has proved to be extremely flexible; for example, memory modules of arbitrary speeds can be freely mixed. If a new technology produces memory 1.267 faster than before, such a module can be added to a current system and24 THE CONCURRENT PROGRAMMING ABSTRACTION CHAP. 2 will respond to the protocol just that much faster. A synchronous system can generally mix only modules whose speeds are multiples of each other. In the abstraction, certain assumptions will be made to avoid meaning- less pathologies. Since infinite delay is indistinguishable from a halt, we will assume (globally) that, if there is at least one process ready to run, then some (unspecified) process is allowed to run within a finite time. We also assume (locally) that if a process is allowed to run in its critical section then it will complete the execution of the critical section in a finite period of time. On the other hand, we allow ourselves to use the adversary approach in checking for possible bugs. A concurrent program suffers from deadlock if it is possible to devise a scenario for deadlock under the sole finiteness assump- tions of the previous paragraph. If someone offers you a concurrent prog- ram, you can tailor your counterscenario specifically to the given program; you are an “adversary” allowed to plot against the program, 2.5 IMPLEMENTING PRIMITIVE INSTRUCTIONS Our solutions to the mutual exclusion problem will always cheat by making use of mutual exclusion provided on a lower level—the hardware level. Just as the user of a high level language need not know how a compiler works as long as he is provided with an accurate description of the syntax and semantics of the language, so we will not concern ourselves with how the hardware is implemented as long as we are supplied with an accurate description of the syntax and semantics of the architecture. Presumably the same thing happens at lower levels—the computer logic designer need not know exactly how an integrated circuit is implemented; the integrated circuit designer need only concern himself with the electronic properties of semi- conductors and need not know all the details of the quantum physics that explain these properties. In common memory systems there is an arbiter which provides for mutual exclusion in the access to an individual memory word. The word “access” is a generic term for read and write or, as they are usually called, Load and Store corresponding to the assembler instructions for these actions. The arbiter ensures that in case of overlap among accesses, mutual exclusion is obtained by executing the accesses one after the other. The order of the accesses is not guaranteed to the programmer. On the other hand, the consistency of the access is ensured as described in Chapter 1. Note that the access to a single word is an action that may not be apparent in a high level language. Suppose that n is a global variable that is initially zero and is used as a counter by several processes executing theSECT. 2.5 IMPLEMENTING PRIMITIVE INSTRUCTIONS 25 instruction: 2 := +1. The compiler compiles such a statement into the three assembler instructions: Load n Add 1 Store n Consider now the following scenario. The value of 1 is 6. P, executes Load n and then P; also executes Load n. P, increments the value of 1 in its internal register to obtain 7. Similarly, P, obtains the value 7 in its internal register. Finally, the two processes execute the Store instruction in succession and the value 7 is stored twice. Hence the final value of n is 7. That is we have incremented the value 6 twice and have obtained 7. Common memory arbiters are found both on multiprocessor systems and on single processor systems whose 1/O equipment is connected for direct memory access (DMA). Normally an I/O device would transfer each data word to the CPU for the CPU to store in the memory. However, this imposes an unacceptable overhead on the CPU. Instead, the I/O device is given the address of a block of memory. It only interrupts the CPU when the transfer of the whole block is completed. There is an arbiter to ensure that only one device (or the CPU) has access to the memory at any one time. In this case we say that DMA is being implemented by cycle stealing. The memory is assumed to be driven at its maximum access speed, say one access per microsecond. Each such access is also called a memory cycle. To implement DMA the CPU is normally allowed to compute and access memory. When a data word arrives from an I/O device the right to access memory is usurped from the CPU and the device is allowed to “steal” a memory cycle. There is no real overhead. The memory cycle is needed anyway to store the word and with cycle stealing the CPU need not concern itself with individual words. The computer hardware will be trusted to function properly. We only concern ourselves with the correctness of the system software. This is not always true of course and in practice one must be alert to hardware malfunc- tion. One of the most spectacular bugs known to the Author was caused by a hardware fault that resulted in mixing two memory addresses instead of interleaving them. The net result was a store of data in the mixed-up address, and the presence of foreign data in these memory addresses was never explained by software specialists. Fortunately this sort of thing rarely hap- pens. Another way of using a common memory system is to define a primitive procedure call that is guaranteed to exclude other calls of the same proce- dure. That is, if two processes try to call the same procedure, only one will succeed and the losing process will have to wait. As usual it is not specified in which order simultaneous requests are granted.26 THE CONCURRENT PROGRAMMING ABSTRACTION CHAP. 2 In multiprogramming systems, the interrupt facility is used. A critical procedure is written as an interrupt routine to be executed when a process causes an interrupt. A hardware flag ensures mutual exclusion by inhibiting the interrupt—placing the computer in an uninterruptable state. Upon completion of the interrupt routine, the flag is reset and another process may now cause an interrupt. Another method of implementing mutual exclusion is polling. Each process is interrogated in turn to see if it requires some service that must be done under mutual exclusion. We shail allow ourselves the luxury of defining primitive instructions and, beyond the sketch in this section, we shall not worry about the implementation. With some experience in computer architecture and data structures it should not be too difficult to implement any of these primitives. However, the details differ widely from computer to computer. A study of the implementation kit may help. In the bibliography we give references to several descriptions of concurrent programming implementations. In addi- tion, the serious student should study the architecture of whatever computer and operating system he is using. 2.6 CONCURRENT PROGRAMMING IN PASCAL-S Sequential Pascal (and the subset used in this book) must be augmented by concurrent programming constructs. The concurrent processes are written as Pascal procedures and their identity as concurrent processes is established by their appearance in the cobegin . . . coend statement cobegin P,; P.;.. .; P, coend. A request for concurrent execution of several processes may appear only in the main program and may not be nested. The semantics of the cobegin . . . coend statement are specified as follows: The cobegin statement is a signal to the system that the enclosed procedures are not to be executed but are to be marked for concurrent execution, When the coend statement is reached, the execution of the main program is suspended and the concurrent processes are executed. The interleaving of the executions of these processes is not predictable and may change from one run to another. When all concurrent pro- cesses have terminated, then the main program is resumed at the statement following the coend. An additional notational device that we make use of is the statement repeat . . . forever which is exactly equivalent in its semantic content with repeat . . . until false. However, the latter is rather obscure and we prefer the more transparent notation.SECT. 2.6 CONCURRENT PROGRAMMING IN PASCAL-S_ 27 The use of repeat ... forever emphasizes that these examples are intended to be prototypes of cyclic programs in operating systems and real-time systems. To execute any of the examples in the book on the implementation kit you will generally have to do the following. (See Fig. 1.5 and, for a larger example, Fig. 4.18). 1. Replace repeat ... forever by a for-loop that will execute each process a fixed number of times or otherwise arrange for termina- tion. 2. Insert write statements in the main program or in the concurrent processes to trace the execution of the program. 3. Often, certain procedures have been left unspecified to emphasize the generality of the algorithms. For example, in the producer-con- sumer problem, we have invoked procedures named produce and consume. These procedures must be specified. One possibility is to produce by incrementing an integer and consume by printing it. 4. Warning The interpreter in the kit is very inefficient, so do not get carried away with the size of the programs that you intend to execute, 2.7 SUMMARY This list summarizes the concurrent programming abstraction. 1. A concurrent program will consist of two or more sequential prog- rams whose execution sequences are interleaved. 2. The processes must be loosely connected. In particular, the failure of any process outside its critical section and protocols must not affect the other processes. 3. Aconcurrent program is correct if it does not suffer from violation of safety properties such as mutual exclusion and of liveness properties such as deadlock and lockout. 4. A concurrent program is incorrect if there exists an interleaved execution sequence which violates a correctness requirement. Hence it is sufficient to construct a scenario to show incorrectness; to show correctness requires a mathematical argument that the prog- ram is correct for all execution sequences. 5. No timing assumptions are made except that no process halts in its critical section and that, if there are ready processes, one is eventu- ally scheduled for execution. We may impose other fairness requirements.28 THE CONCURRENT PROGRAMMING ABSTRACTION CHAP. 2 6. We shall extend our basic programming language with synchroniza- tion primitive instructions. As long as the syntax and semantics of these instructions are clearly defined we do not concern ourselves with their implementation. 2.8 EXERCISES 2.1 Standing Exercise Write formal specifications of the programs in this book. For example: Specification for sortprogram. Input: A sequence of 40 integers: A={a,,... , ago}. Output: A sequence of 40 integers: b={b,,..., bag}. Safety property: When the program terminates then (i) b is a permutation of a, and (ii) b is ordered, ic. for 1 <= i < 40, b, <= bj+1. Liveness property: The program terminates. 2.2, Standing Exercise Test the example programs in the text.3 THE MUTUAL EXCLUSION PROBLEM 3.1 INTRODUCTION A solution to the mutual exclusion problem for two processes P, and P, will now be developed without introducing any primitive instructions (other than the common memory arbiter). The purpose of this chapter is not so much to present Dekker’s elegant solution to this very difficult problem as to present Dijkstra’s step-by-step development of Dekker’s solution. During the development we will encounter most of the possible bugs that a concurrent program can have and thus illustrate the theoretical discussion of the previ- ous chapter. The serious reader will want to try to analyze each attempted solution before reading further. ‘The mutual exclusion problem for two processes is as follows: Two processes P, and P, are cach executing in an infinite loop a program which consists of two sections, critical sections crit] and crit2 and the30 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 remainder of the program, non-critical sections rem and rem2. The execu- tion of critl and crit2 must not overlap. 3.2 FIRST ATTEMPT Let us imagine a “protocol igloo” containing a blackboard (Fig. 3.1). The igloo itself and the entrance tunnel are so small that only one person can be in the igloo at any given time. On the blackboard is written the number of the process whose “turn” it is to enter the critical section. The small size of the igloo is our metaphor for the memory arbiter. A process wishing to enter its critical section crawis into the igloo when it is empty and checks the blackboard. If its number is written, it leaves the igloo and happily proceeds to its critical section. If the number of the other process is written then it sadly leaves the igloo to wait for the other process to finish. Ideally, the unfortunate process would be able to take a nap until its turn arrives but we have no way of expressing this in an ordinary program- ming language. Instead, the process can run laps around the igloo to warm up (this is doing “‘nothing”). Periodically it can re-enter the igloo to check the blackboard. This is known as busy waiting because the energy expended by the waiting process is just purposeless work.-When a process has completed its critical section, it writes the number of the other process on the board. Since only one process is in the igloo at any one time, the board shows either a one or a two (assuming that neither process malfunctions within the igloo). program firstattempt, var turn: integer; prodecure p,; begin repeat while turn=2 do (* nothing *); crit]; turn rem1 forever end; procedure p,; begin repeat while turn=1 do (* nothing +);SECT. 3.2 FIRST ATTEMPT 31 begin (* main program *) turn := 1; cobegin Pis P2 coend end Fig. 3.2. This solution (Fig. 3.2) satisfies the mutual exclusion requirement. A process P, enters its critical section only if turn = i. Since, by the common memory assumption, turn will be consistent (will have either a value of 1 or 2), only one process at a time entersits critical section. Since the value of turn is not changed until the termination of the critical section, a second process will not be able to infiltrate before termination of the first critical section. Deadlock is also impossible. Since turn has either the value 1 or the value 2, exactly one process will always be able to progress or, to put it another way, it is impossible for both processes to be simultaneously stuck at the while loops. Similarly, if every statement of each program takes a finite amount of time then the process whose turn it is to enter the critical section will always do so and will eventually allow the other process to enter also. Thus there is no lockout since neither process can prevent the other from entering its critical section. Even though this solution fulfils our requirements for the correctness of concurrent programs, it does not fulfil one of the design requirements of the abstraction. The processes are not loosely connected. The right to enter the critical section is being explicitly passed from one process to the other. This has two drawbacks. Firstly, if process P, needs to enter its critical section 100 times per day while P, needs to enter only once per day then P, is going to be coerced into working at P,’s pace of once per day. When P, has finished its critical section, then, it chalks up a 2 and until P, decides to re-enter the critical section, P, will be forced to wait. The second drawback is as serious. Suppose P, is waiting for P, to execute a critical section and then change the number written on the board from 2 to 1. If by chance P, is devoured by a polar bear on its way to the igloo then not only is P, terminated but P, is hopelessly deadlocked. This is true even if P, is in rem2 outside the critical section when it terminates, thus conforming with our assumption that a process may terminate only outside its critical section. This explicit passing of control is a programming technique known as coroutines. By executing an instruction such as resume (p,), P; is able to request that its execution be suspended in favor of P,. P; then executes until it in turn returns the control of P; by a resume (p,) statement. Coroutines32. THE MUTUAL EXCLUSION PROBLEM CHAP. 3 are a useful programming technique but a system of coroutines must be designed as a single integrated process and are not a substitute for concur- rent programs. 3.3. SECOND ATTEMPT Fig. 3.3. We try to remedy the previous solution by giving each process its own key to the critical section so if one is devoured by a polar bear then the other can still enter its critical section. ‘There is now (Fig. 3.3) an igloo (global variable) identified with each process. It is worth noting that, while in the solution in Fig. 3.2 the variable turn is both read (Load) and written (Store) by both processes, the present solution may be easier to implement because each process reads but does not write the variable identified with the other process. If P, (say) wishes to enter its critical section, it crawls into P's igloo periodically until it notes that c, is equal to 1 signifying that P, is currently not in its critical section. Having ascertained that fact, P, may enter its critical section after duly registering its entrance by chalking a 0 on its blackboard—c,. When P; has finished, it changes the mark on ¢, to | to notify P, that the critical section is free program secondattempt; var C1, Cp: integer; procedure p,; begin : repeatSECT. 3.3 forever end; procedure p,; begin repeat while c,=0 do; o 5 crit2; © rem2 forever end; begin (+ main program *) a=; & cobegin Pus Pe coend end. Fig. 3.4. This program (Fig. 3.4) does not even satisfy the safety requirement of mutual exclusion. The following scenario gives a counter-example where the first column describes the interleaving and the next columns record the values of the variables. Initially P, checks ¢ P, checks ¢; P, sets cl P, sets c2 P, enters crit1 P, enters crit2 SinceP, and P, are simultaneously in their critical sections, the program is incorrect. cocorreD SECOND ATTEMPT 33 o one:34 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 3.4 THIRD ATTEMPT program thirdattempt; var Cy, Cy! integer; procedure p,; begin repeat rem1 forever end; procedure p,; begin repeat c= 05 while c,=0 dos crit2; Ct rem2 forever end; begin (* main program *) q:= 4h; Fig. 3.5. Analyzing the failure of the second attempt, we note that, once P, has ascertained that P, is not in its critical section, P, is going to charge right into its critical section. Thus, the instant that P, has passed the while statement, P; is in effect in its critical section. This contradicts our intention that c¢,=0 should indicate that P, is in its critical section because there may be an arbitrarily long wait between the while statement and the assignment statement. The third attempt (Fig. 3.5) corrects this by advancing the assignment Statement so that c, = 0 will indicate that P, is in its critical section even before it checks c,. Hence P, is in its critical section the instant that the while has been successfully passed.SECT. 3.4 THIRD ATTEMPT 35 Unfortunately this program easily leads to system deadlock as seen by the following scenario: P, checks ¢, P, checks c; CG Initially il 1 P, sets ¢, 0 1 P, sets cy 0 0 0 0 0 0 The continual checking of the variables can be continued indefinitely and cannot be considered progress. Thus the program is hopelessly deadlocked. Even though this program is unacceptable because of the deadlock, it is instructive to prove that it satisfies the mutual exclusion property. By sym- metry it is sufficient to show that: (P, in crit1) implies (P, is not in crit2). 1. (When P, entered crit1) then (c, was not 0). This follows from the structure of the program, namely the test on cy by Py. 2. (¢> is not 0) implies (P, is not in crit2). crit2 is bracketed between assignments to c, which ensure that this statement is always true. 3. (When P, entered crit1) then (P, was not in crit2) This is a logical consequence of (1) and (2). 4. (P, in crit1) implies (c, is 0). crit1 is bracketed between assignments to c,. 5. (c; is 0) implies (P, does not enter crit2). The test will not allow P, through. 6. (P, in crit1) implies (P, does not enter crit2). A logical consequence of (4) and (5). 7. As long as (P, is in crit1), (P, will never enter crit2). This follows from (6). Since (6) refers to an arbitrary instant of time, then as long as its antecedent (P, in crit1) remains true, so will its consequent (P, does not enter crit2). 8. (P, in crit1) implies (P, is not in crit2). From (3) and (7). Note that the proof has the simple structure of a deduction in the propositional calculus except for the need to express and deduce time- related properties such as “when”, “as long as” etc. There is a formal logic called temporal logic that can express these properties and can be used to formally prove properties of concurrent programs. For example, the reason- ing in this proof can be formalized as an induction on the time that has passed since P, entered crit1. We are trying to prove that mutual exclusion is never violated: (3) ensures the basis of the induction; (6) is an induction step: assuming that P, is now in crit1, we can deduce that P, will not now enter36 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 crit2 so that upon the conclusion of the current instruction, mutual exclusion will still not be violated. 3.5 FOURTH ATTEMPT program fourthattempt; var C,, C2: integer; procedure p,; begin c= 05 while c,=0 do begin e:= 1; forever end; procedure p,; begin repeat ¢):= 0; while c,=0 do begin = 1; (+ do nothing for a few moments *) i= 0 ent crit2; C): rem2 forever end; begin (+ main program *) c= 1; Qs cobegin PAs Pr coend end. Fig. 3.6.SECT. 3.5 FOURTH ATTEMPT 37 In the previous solution, when P, chalks up 0 on c; to indicate its intention to enter its critical section, it also turns out that it is insisting on its right to enter the critical section. It is true that setting c; before checking c. prevents the violation of mutual exclusion but if P, is not ready to yield then P, should yield. In the next attempt (Fig. 3.6) we correct this stubborn behavior by having a process relinquish temporarily its intention to enter its critical section to give the other process a chance to do so. P, enters its igloo and chalks up a 0. If upon checking P,’s igloo, P, finds a 0 there too, it chival- rously returns to its igloo to erase the 0. After a few laps around the igloo it restores the signal c, = 0 and tries again. The comment is there simply to remind you that since arbitrary interleaving is permissible, the sequence of two assignments to the same variable is not meaningless. First note that the previous proof of mutual exclusion holds here. From the above discussion, it should now be clear that there is such a thing as too much chivalry. If both processes continue yielding then neither will enter the critical section. The scenario is as follows: Initially P, sets c, P, sets cy P, checks c, P, checks ¢, P, sets c, P, sets cy P, sets c, P sets cy cConHooocore CrRrecocoHe? It is clear that this could be indefinitely extended and that liveness does not hold because neither process will ever enter its critical section. However, it is extremely unlikely ever to occur. Nevertheless we are forced to reject this solution. The main objection here is not so much that neither process will ever enter the critical section (it is unlikely that perfect synchronization continues indefinitely) but that we have no way of giving an a priori bound on the number of iterations that the loops will execute before they are passed. Thus we have no way of guaranteeing the performance of such a system. Should this bug be classified as deadlock or lockout? On the one hand, both processes are looping on a protocol which is certainly not useful computation and the situation is similar to the previous attempt. However, we prefer to call this lockout to emphasize the following distinction. In the previous attempt the situation is hopeless. From the instant that the program is deadlocked, all future executions sequences remain deadlocked. In this38 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 case however, the slightest aberration of the scenario will free one of the processes and in practice this will eventually happen. The key notion here is the conspiracy between the processes and not the hopelessness of the situa- tion. It is only because we wish to be able to guarantee a worst-case behavior that we reject the current attempt. 3.6 DEKKER’S ALGORITHM program Dekker; var turn: integer; C1, €2: integer; procedure p,; begin repeat ¢ 0; while c, = 0 do if turn = 2 then begin cq := 1; while turn=2 do; i= 0 end; critl; cy: rem forever end; procedure p,: begin repeat 2 3=.0; while c, = 0 do if turn=1 then begin cat —ols while turn=1 dos rem2 forever end;SECT. 3.6 DEKKER’S ALGORITHM 39 begin (* main program *) ee; ct tur) cobegin P13 P2 coend end. Fig. 3.7. Dekker’s solution is an ingenious combination of the first and fourth attempted solutions. Recall that in the first solution we explicitly passed the right to enter the critical section between the processes. Unfortunately, the key to the critical section could be irretrievabley lost if one of the processes is terminated. In the fourth solution we found that keeping separate keys leads to the possibility of infinite deferment of one process to the other. Dekker’s algorithm (Fig. 3.7) is based on the previous solution but solves the problem of lockout by explicitly passing the right to insist on entering the critical solution. Each process has a separate igloo so it can go on processing even if one process is terminated by a polar bear. Note that we are here using the assumption that no process is terminated in its critical section (including the protocol). There is now an “umpire” igloo with a blackboard labelled “turn”, (Fig. 3.8). If P; chalks up a0 onc, and then finds that P, has also chalked up a 0, it goes to consult the umpire. If the umpire has a 1 written upon it, then P; knows that it is its turn to insist and so P, periodically checks P,’s igloo. P, of course notes that it is its turn to defer and chivalrously chalks up a 1 on cy which will eventually be noted by P,. P; meanwhile waits for P, to terminate40 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 its critical section. Upon termination, P, not only frees the critical section by setting c; to 1 but also resets zurn to 2 both to free P from the inner loop and to transfer the right to insist to P3. Mutual exclusion is proved exactly as in Section 3.4 since the value of turn has no effect on the decision to enter the critical section. Proving liveness is somewhat of a challenge. Symmetrically it is suffi- cient to prove that, if P; executes c, := 0 indicating its intention to enter the critical section, then eventually it does so. This is done in two parts. First we prove that if P,; attempts to enter its critical section but cannot do so, eventually the variable «urn is permanently held at the value 1. But if arn is held permanently at 1 then P, can always enter its critical section. 3.7 A PROOF OF DEKKER’S ALGORITHM Let us now prove the liveness of Dekker’s Algorithm. The algorithm is shown in flowchart form in Fig. 3.9. The formal statement that we want to prove is that if the program counter of process P, is at point a, (i.c. P, has left rem] and thus expresses a wish to enter the critical section), eventually the program counter of P, will be at a (i.e. P, may enter the critical section). Of course an exactly symmetrical proof will prove the liveness of P,. Our notation will be more concise: a; will be an abbreviation for the statement that the program counter of P, is at @,. Similarly for P, and £;. Remember the assumption that a process is never terminated in its critical section (including the protocols). Thus if P; is at a, it will eventually reach ag. If P, is at a, it will eventually reach a, or as though without further information we cannot specify which of the two statements will be reached. ‘To simplify the proof, this assumption is extended to remi (a, and B,). In the exercises we indicate the modifications needed if we allow P, to terminate in remi. Note that if P, is at a; and c, = 1, we cannot conclude that eventually P, is at ws. The assumption only guarantees that P, eventually tests the value of 2; by then the value of c, could have been changed. To conclude that a; implies eventually a; we would have to show that a; and that the value of ¢, is held at 1 indefinitely. Thus, when by assumption the test is eventually done, the value of c, will in fact be 1. Since the only assignments to ¢; are in P,, we can deduce the values of c, from the a’s and the f’s, respectively. We express these facts as invariants, i.e. statements that are always true. IL. c, = 0 if and only if ay or ay OF es OF a OF 2. 12. c, = 0 if and only if B; or By or Bs oF Bs or By.A PROOF OF DEKKER’S ALGORITHM 41 SECT. 3.7 “[ wm=%o=!9 sanqen UL ULOTY S49x49q 6°E “BLE42. THE MUTUAL EXCLUSION PROBLEM CHAP. 3 Theorem (a, and never as) is false, that is a) implies eventually as. Proof 1. (a, and never as) and (turn held at 2) imply eventually (c, held at 1). Since (never as), P, eventually passes a; and thence to ay. Since (turn held at 2), P, reaches a, and ag and is then blocked in the loop at ag. By I1, as long as P, is at a, c, must equal 1. 2. (c, held at 1) and (turn held at 2) imply eventually (turn = 1). The truth of the two clauses concerning c, and turn together with the assumption that processes are not terminated implies that P) must eventually reach B, and assign the value 1 to turn. But (turn held at 2) and eventually (turn = 1) means that there will be a point of time when turn is simultaneously 1 and 2. This contradicts the consistency of the values in the common memory. From (a, and never ws) and (turn held at 2) we have deduced a contradiction. Thus it must be that (@, and never a) implies eventually (turn is not 2). Since turn = 2 or turn = 1 we have proved. 3. (a; and never as) implies eventually (turn =1). 4. (a and never as) implies (never as) and (never a,) and (never a). The only way to reach a, or a; from a; is to pass through a. But we assume that we never reach as. 5. (a and never «s) implies eventually (turn held at 1). Once the value of turn is 1 (as ensured by (3)) the only way that the value can change back to 2 is to execute a,. By 4 this will never happen. 6. (a, and never as) implies eventually (P, loops forever at a-a,). By (4) and (5), eventually [(turn held at 1) and (P, isnever at as)]. Hence since P, must reach a;-a, fom a, a, OF ag it will then loop forever at asa. 7. (a; and never as) implies eventually (c, held at 0). By (6), eventually we loop at a;-e, which implies by 11 that c, will be held at 0. 8. (c; held at 0) and (turn held at 1) imply eventually (c; held at 1). Similar to (1): P, must eventually loop at Bg. Then by I2, c; is held at 1. 9. (a and never a) implies eventually (c; held at 1). From (5), (7) and (8). But (8) contradicts (6): if, is held at 1 then P, cannot be loopingSECT. 3.8 CONCLUSION 43 at ay-a,. From (a, and never @;) we have deduced a contradic- tion. Thus (a; and never as) is false. 3.8 CONCLUSION Mutual exclusion of two processes is about the simplest problem in concur- rent programming. The difficulty of obtaining a correct solution to such a simple problem suggests that programming features more powerful than the common memory arbiter will be needed. In the exercises you can explore some other solutions of the type given here. In particular, the solutions of the mutual exclusion problem for n processes are so difficult that they are of more or less academic interest only, especially when compared with the trivial solution to the problem given, say, by semaphores. There is another defect in the common memory arbiter and that is the busy wait that is used to achieve synchronization. The solutions all contain a statement: while condition do (* nothing +). Unless you have a dedicated computer doing the looping this is a waste of CPU computing power. Even if there is no CPU waste (as would be the case if the processes were I/O controllers) there is the severe overhead associated with cycle stealing. Thus the frequent accesses to turn in Dekker’s solution can prevent useful compu- tation from being done by other processes. The primitives discussed in the next chapters uniformly suspend the execution of the blocked processes. This is usually implemented by keeping a queue of processes, ie. a queue of small blocks of memory containing essential information on the blocked processes. Thus the overhead is only a small amount of memory and the small amount of computation needed to manage the queue. A final objection to Dekker’s algorithm is that it uses a common variable which is written into by both processes. In the exercises we discuss Lamport’s algorithms which have the advantage that each variable need only be written by one process. Thus his algorithms are suitable for implementa- tion on distributed systems where the values of the variables can be trans- mitted and received but where each variable is written into only on the computer in which it physically resides. 3.9 EXERCISES 3.1 (Dijkstra) Fig. 3.10 is a solution to the mutual exclusion problem for n processes that is a generalization of Dekker’s solution. (a) *Show that mutual exclusion holds. (b) Show that deadlock does not occur. (c)_ Show that lockout is possible.44 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 program Dijkstra; const on + (* number of processes *) var b,c: array [0 . . 2] of boolean; turn: integer; procedure process(i : integer); var ji: integer; ok: boolean; begin repeat bli] := false; repeat while tun <> i do begin fi] := true; if b[urn] then turn := i false; true; for j:= 1 ton do if j <> i then ok and c{j] s bli] := true; rem forever end; begin (* main program +) for turn := 0 ton do begin Blturn] := true; {turn} := true end; turn cobegin process(1); process(2); process(n) coend end. Fig. 3.10. 3.2 (Lamport) Fig. 3.11 is (what the Author calls) the Dutch Beer version of the Bakery Algorithm, restricted to two processes. (a) Show safety and liveness. (Hint The variables are supposed to represent “ticket” numbers”. The process with the lower ticket number enters its critical section. In case of a tie, it is arbitrarily resolved in favor of P;.) (b) Show that the commands n; := 1 are necessary.EXERCISES 45 (c) *Extend the algorithm to nm processes. (Hint Each process will choose a ticket number greater than the maximum of all outstanding ticket numbers. It will then wait until all processes with lower numbered tickets have completed their critical sections.) program dutchbeer; var ny, Np: integers procedure p,; begin critl; forever end; procedure p2; begin repeat ny = nyt; while (1, <> 0) and (ny <= 13) do; crit? forever end; begin (* main program *) 2 = 05 cobegin Pas P2 coend end. Fig. 3.11. 3.3 Fig. 3.12 is Lamport’s Bakery Algorithm restricted to two processes (a) Show the safety and liveness of this solution. (b) Generalize to n processes. (c) Show that for n > 2 the values of the variables n; are not bounded. (d) *Suppose we allow a read (i.e. Load) of a variable n; to return any value if it takes place simultaneously with a write (Store) of n; by the ith process. Show that the correctness of the algorithm is not affected. Note, however, that we require the write to execute correctly. Similarly, all reads which do not overlap writes to the same variable must return the correct values. (e) Show that the correctness of the Dutch Beer version of the algorithm is not preserved under the malfunction described in (d).46 3.4 THE MUTUAL EXCLUSION PROBLEM CHAP. 3 program bakery; var Cty Cay yy Mg integer; procedure p,; while c, <> 0 do; while (1, <> 0) and (ny < m,) do; forever end; procedure p,; begin repeat while c; <> 0 do; while (1, <> 0) and (ny <= 1) do; crit; forever end; begin (+ main program +) cy = 05 Fig. 3.12. Fig. 3.13 isa solution to the mutual exclusion problem for two processes. Discuss the correctness of the solution: if it is correct, then prove it. If not, write scenarios that show that the solution is incorrect. Several sychronization primitives that have been used are based on hardware instructions that enable several assignment statements to be executed as one (indivisible) primitive instruction. The solutions to the mutual exclusion prob- Jem using these primitives are very simple, but they remain busy wait algorithms in contrast to algorithms using the primitives to be studied in the next chapter.EXERCISES 47 program attempt; var C1, C2! integer; procedure p;; begin repeat rem1; repeat qi i- until c, <> 0; crit; forever end; procedure p2; begin repeat rem2, repeat Qisl-¢ until c, <> 0; crit; C2 forever end; begin (* main program +) ee 1s a! 1 Fig. 3.13. 3.5 The IBM 360/370 computers have an instruction called TST (Test and Set). There is a system global variable called c (Condition Code), Executing 7ST(!) for local variable / is equivalent to the following two assignments: (a) Discuss the correctness (safety, deadlock, lockout) of the solution of the mutual exclusion problem shown in Fig. 3.14. (b) Generalize to n processes. (©) What would happen if the primitive TST instruction were replaced by the two assignments? (4) *Modify the implementation kit to include the TST instruction.