CMSC 838F: Programming Language Techniques for Concucurrent and Distributed Systems

Welcome!
Class mechanics:
Class participation (35%)
Homework and programming projects Paper presentation during the semester Discussion of papers during class

Project (40%) Final exam (25%)

No class on Monday, Jan 29

What is this Class About?

Concurrent and distributed programming becoming mainstream
Multicore processors The Internet

Why Concur. and Distrib. Prog?

Performance
Exploit more resources Reduce data movement

Design
Some tasks can be divided up naturally

How do we program these systems? What do the programs mean? How can be reason about such programs?

Failure isolation Heterogeneous trust

The Plan
Get our feet wet with threading in Java
Project due Wed, Jan 31

Next time go over advanced threading idioms in Java Then back to the basics
Memory consistency models Programming models and language design Transactions Analysis

CMSC 838F: Programming Language Techniques for Concucurrent and Distributed Systems
Java Threads

Computation Abstractions
Processes (e.g., JVMs) t4 t2 t2 p1 t3 p2 CPU 1 t1 t5 p3 p4 CPU 2 Threads

Processes vs. Threads

int x; foo() { x } int x; foo() { x }
int x;

t1

t1

foo() { x }

foo() { x }

A computer
7

Processes do not share data

Threads share data within a process

So, What Is a Thread?

Conceptually: it is a parallel computation occurring within a process

Implementation View
esp esp eip eip esp eip

Implementation view: its a program counter and a stack. The heap and static area are shared among all threads All programs have at least one thread (main)

Per-thread stack and instruction pointer

Saved in memory when thread suspended Put in hardware esp/eip when thread resumes
9 10

Tradeoffs
Threads can increase performance
Parallelism on multiprocessors Concurrency of computation and I/O

Programming Threads
Threads are available in many languages
C, C++, Objective Caml, Java, SmallTalk

Natural fit for some programming patterns

Event processing Simulations

In many languages (e.g., C and C++), threads are a platform specific add-on
Not part of the language specification

But increased complexity

Need to worry about safety, liveness, composition

They're part of the Java language specification

And higher resource usage

11 12

Java Threads
Every application has at least one thread
The main thread, started by the JVM to run the applications main() method

Thread Creation
execution (time) main thread thread starts

main() can create other threads

Explicitly, using the Thread class Implicitly, by calling libraries that create threads as a consequence
RMI, AWT/Swing, Applets, etc.

thread starts thread ends

thread join

13

14

Thread Creation in Java

To explicitly create a thread:
Instantiate a Thread object
An object of class Thread or a subclass of Thread

Running Example: Alarms

Goal: let's set alarms which will be triggered in the future
Input: time t (seconds) and message m Result: well see m printed after t seconds

Invoke the objects start() method

This will start executing the Threads run() method concurrently with the current thread

Thread terminates when its run() method returns

15

16

Example: Synchronous alarms

while (true) { System.out.print("Alarm> "); // read user input String line = b.readLine(); parseInput(line); // sets timeout // wait (in secs) try { Thread.sleep(timeout * 1000); } catch (InterruptedException e) { } System.out.println("("+timeout+") "+msg); }

Making It Threaded (1)

public class AlarmThread extends Thread { private String msg = null; private int timeout = 0; public AlarmThread(String msg, int time) { this.msg = msg; this.timeout = time; } public void run() { try { Thread.sleep(timeout * 1000); } catch (InterruptedException e) { } System.out.println("("+timeout+") "+msg); }
17

18

Making It Threaded (2)

while (true) { System.out.print("Alarm> "); // read user input String line = b.readLine(); parseInput(line); if (m != null) { // start alarm thread Thread t = new AlarmThread(m,tm); t.start(); } }
19

Alternative: The Runnable Interface

Extending Thread prohibits a different parent Instead implement Runnable
Declares that the class has a void run() method

Construct a Thread from the Runnable

Constructor Thread(Runnable target) Constructor Thread(Runnable target, String name)

20

Thread Example Revisited

public class AlarmRunnable implements Runnable { private String msg = null; private int timeout = 0; public AlarmRunnable(String msg, int time) { this.msg = msg; this.timeout = time; } public void run() { try { Thread.sleep(timeout * 1000); } catch (InterruptedException e) { } System.out.println("("+timeout+") "+msg); } }
21

Thread Example Revisited (2)

while (true) { System.out.print("Alarm> "); // read user input String line = b.readLine(); parseInput(line); if (m != null) { // start alarm thread Thread t = new Thread( new AlarmRunnable(m,tm)); t.start(); } }

22

Notes: Passing Parameters

run() doesnt take parameters We pass parameters to the new thread by storing them as private fields
In the extended class Or the Runnable object Example: the time to wait and the message to print in the AlarmThread class

Concurrency
A concurrent program is one that has multiple threads that may be active at the same time
Might run on one CPU
The CPU alternates between running different threads The scheduler takes care of the details Switching between threads might happen at any time

Might run in parallel on a multiprocessor machine

One with more than one CPU May have multiple threads per CPU

Multiprocessor machines are becoming more common

Multi-CPU machines aren't that expensive any more Dual-core CPUs are available now
24

23

Scheduling Example (1)

CPU 1 p1 p2

Scheduling Example (2)

CPU 1 p1 p2

One process per CPU

CPU 2 p1 p2 CPU 2 p1 p2

Threads shared between CPUs

p2 threads:

p1 threads:
25

p2 threads:

p1 threads:
26

Concurrency and Shared Data

Concurrency is easy if threads dont interact
Each thread does its own thing, ignoring other threads Typically, however, threads need to communicate with each other

Data Race Example

public class Example extends Thread { private static int cnt = 0; // shared state public void run() { int y = cnt; cnt = y + 1; } public static void main(String args[]) { Thread t1 = new Example(); Thread t2 = new Example(); t1.start(); t2.start(); } }

Communication is done by sharing data

In Java, different threads may access the heap simultaneously But the scheduler might interleave threads arbitrarily Problems can occur if were not careful.
27

28

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Data Race Example

Shared state cnt = 0
static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Shared state y=0

cnt = 0

Start: both threads ready to run. Each will increment the global cnt.

T1 executes, grabbing the global counter value into its own y.

29

30

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Data Race Example

Shared state cnt = 1
static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Shared state y=0

cnt = 1

y=0

T1 executes again, storing its value of y + 1 into the counter.

y=1

T1 finishes. T2 executes, grabbing the global counter value into its own y.

31

32

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

But When it's Run Again?

Shared state cnt = 2

y=0

y=1

T2 executes, storing its incremented cnt value into the global counter.

33

34

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Data Race Example

Shared state cnt = 0
static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Shared state y=0

cnt = 0

Start: both threads ready to run. Each will increment the global count.

T1 executes, grabbing the global counter value into its own y.

35

36

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Data Race Example

Shared state cnt = 0
static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

Shared state y=0

cnt = 1

y=0

y=0

T1 is preempted. T2 executes, grabbing the global counter value into its own y.

y=0

T2 executes, storing the incremented cnt value.

37

38

Data Race Example

static int cnt = 0; t1.run() { int y = cnt; cnt = y + 1; } t2.run() { int y = cnt; cnt = y + 1; }

What Happened?
Different schedules led to different outcomes
Shared state cnt = 1 This is a data race or race condition

y=0

A thread was preempted in the middle of an operation

Reading and writing cnt was supposed to be atomicto happen with no interference from other threads But the schedule (interleaving of threads) which was chosen allowed atomicity to be violated These bugs can be extremely hard to reproduce, and so hard to debug
Depends on what scheduler chose to do, which is hard to predict
39 40

y=0

T2 completes. T1 executes again, storing the incremented original counter value (1) rather than what the incremented updated value would have been (2)!

Question
If instead of
int y = cnt; cnt = y+1;

Question
If you run a program with a race condition, will you always get an unexpected result?
No! It depends on the scheduler, i.e., which JVM youre running, and on the other threads/processes/etc, that are running on the same CPU

We had written
cnt++;

Would the result be any different? Answer: NO!

Dont depend on your intuition about atomicity

Race conditions are hard to find

41

42

Synchronization
Refers to mechanisms allowing a programmer to control the execution order of some operations across different threads in a concurrent program. Different languages have adopted different mechanisms to allow the programmer to synchronize threads. Java has several mechanisms; we'll look at locks first.

Locks (Java 1.5)

interface Lock { void lock(); void unlock(); ... /* Some more stuff, also */ } class ReentrantLock implements Lock { ... }

Only one thread can hold a lock at once

Other threads that try to acquire it block (or become suspended) until the lock becomes available

Reentrant lock can be reacquired by same thread

As many times as desired No other thread may acquire a lock until has been released same number of times it has been acquired

43

44

Avoiding Interference: Synchronization

public class Example extends Thread { private static int cnt = 0; static Lock lock = new ReentrantLock(); public void run() { lock.lock(); int y = cnt; Lock, for protecting cnt = y + 1; the shared state lock.unlock(); } Acquires the lock; } Only succeeds if not } held by another

Applying Synchronization
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Shared state

cnt = 0

T1 acquires the lock

thread Releases the lock

45 46

Applying Synchronization
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Applying Synchronization
cnt = 0
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Shared state y=0

Shared state y=0

cnt = 0

T1 reads cnt into y

T1 is preempted. T2 attempts to acquire the lock but fails because its held by T1, so it blocks
48

47

Applying Synchronization
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Applying Synchronization
cnt = 1
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Shared state y=0

Shared state y=0

cnt = 1

T1 runs, assigning to cnt

T1 releases the lock and terminates

49

50

Applying Synchronization
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Applying Synchronization
cnt = 1
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Shared state y=0

Shared state y=0

cnt = 1

T2 now can acquire the lock.

T2 reads cnt into y. y=1

51

52

Applying Synchronization
int cnt = 0; t1.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); } t2.run() { lock.lock(); int y = cnt; cnt = y + 1; lock.unlock(); }

Different Locks Dont Interact

cnt = 2
static int cnt = 0; static Lock l = new ReentrantLock(); static Lock m = new ReentrantLock(); void inc() { l.lock(); cnt++; l.unlock(); } void inc() { m.lock(); cnt++; m.unlock(); }

Shared state y=0

y=1

T2 assigns cnt, then releases the lock

This program has a race condition

Threads only block if they try to acquire a lock held by another thread
53 54

Whats Wrong with the Following?

static int cnt = 0; static int x = 0; Thread 1 while (x != 0); x = 1; cnt++; x = 0; Thread 2 while (x != 0); x = 1; cnt++; x = 0;

Reentrant Lock Example

static int cnt = 0; static Lock l = new ReentrantLock(); void inc() { l.lock(); cnt++; l.unlock(); } void returnAndInc() { int temp; l.lock(); temp = cnt; inc(); l.unlock(); }

Threads may be interrupted after the while but before the assignment x = 1
Both may think they hold the lock!

Reentrancy is useful because each method can acquire/release locks as necessary

No need to worry about whether callers have locks Discourages complicated coding practices
55 56

This is busy waiting

Consumes lots of processor cycles

Deadlock
Deadlock occurs when no thread can run because all threads are waiting for a lock
No thread running, so no thread can ever release a lock to enable another thread to run
Lock l = new ReentrantLock(); Lock m = new ReentrantLock(); Thread 1 l.lock(); m.lock(); ... m.unlock(); l.unlock(); Thread 2 m.lock(); l.lock(); ... l.unlock(); m.unlock();
57

Deadlock (contd)
Some schedules work fine
Thread 1 runs to completion, then thread 2

But what if...

Thread 1 acquires lock l The scheduler switches to thread 2 Thread 2 acquires lock m

Deadlock!
Thread 1 is trying to acquire m Thread 2 is trying to acquire l And neither can, because the other thread has it
58

Wait Graphs

Wait Graph Example

l T1

T1

Thread T1 holds lock l

T2

Thread T2 attempting to acquire lock m

T2

Deadlock occurs when there is a cycle in the graph

T1 holds lock on l T2 holds lock on m T1 is trying to acquire a lock on m T2 is trying to acquire a lock on l
59 60

Another Case of Deadlock

static Lock l = new ReentrantLock(); void f () throws Exception { l.lock(); FileInputStream f = new FileInputStream("file.txt"); // Do something with f f.close(); l.unlock(); }

Solution: Use Finally

static Lock l = new ReentrantLock(); void f () throws Exception { l.lock(); try { FileInputStream f = new FileInputStream("file.txt"); // Do something with f f.close(); } finally { // This code executed no matter how we // exit the try block l.unlock(); } }
61 62

l not released if exception thrown

Likely to cause deadlock some time later

Synchronized
This pattern is really common
Acquire lock, do something, release lock under any circumstances after were done
Even if exception was raised etc.

Example
static Object o = new Object(); void f() throws Exception { synchronized (o) { FileInputStream f = new FileInputStream("file.txt"); // Do something with f f.close(); } }

Java has a language construct for this

synchronized (obj) { body } Every Java object has an implicit associated lock

Obtains the lock associated with obj Executes body Release lock when scope is exited
Even in cases of exception or method return
63

Lock associated with o acquired before body executed

Released even if exception thrown
64

Discussion
object o os lock

Example: Synchronizing on this

class C { int cnt; void inc() { synchronized (this) { cnt++; } } } C c = new C(); Thread 1 c.inc(); Thread 2 c.inc();

An object and its associated lock are different!

Holding the lock on an object does not affect what you can do with that object in any way Ex:
synchronized(o) { ... } // acquires lock named o o.f (); // someone else can call os methods o.x = 3; // someone else can read and write os fields
65

Does this program have a data race?

No, both threads acquire locks on the same object before they access shared data
66

Example: Synchronizing on this (contd)

class C { int cnt; void inc() { synchronized (this) { cnt++; } } void dec() { synchronized (this) { cnt--; } } } C c = new C(); Thread 1 c.inc(); Thread 2 c.dec();

Example: Synchronizing on this (contd)

class C { int cnt; void inc() { synchronized (this) { cnt++; } } } C c1 = new C(); C c2 = new C(); Thread 1 c1.inc(); Thread 2 c2.inc();

Data race?
No, threads acquire locks on the same object before they access shared data

Does this program have a data race?

No, threads acquire different locks, but they write to different objects, so thats ok
67 68

Synchronized Methods
Marking method as synchronized same as synchronizing on this in body of the method
The following two programs are the same
class C { int cnt; void inc() { synchronized (this) { cnt++; } } }
69

Synchronized Methods (contd)

class C { int cnt; void inc() { synchronized (this) { cnt++; } } synchronized void dec() { cnt--; } } C c = new C(); Thread 1 c.inc(); Thread 2 c.dec();

class C { int cnt; synchronized void inc(){ cnt++; } }

Data race?
No, both acquire same lock
70

Synchronized Static Methods

Warning: Static methods lock class object
Theres no this object to lock
class C { static int cnt; void inc() { synchronized (this) { cnt++; } } static synchronized void dec() { cnt--; } C c = new C(); Thread 1 c.inc(); Thread 2 C.dec();

Thread Scheduling
When multiple threads share a CPU...
When should the current thread stop running? What thread should run next?

A thread can voluntarily yield() the CPU

Call to yield may be ignored; dont depend on it

Preemptive schedulers can de-schedule the current thread at any time

Not all JVMs use preemptive scheduling, so a thread stuck in a loop may never yield by itself. Therefore, put yield() into loops

Threads are de-scheduled whenever they block (e.g., on a lock or on I/O) or go to sleep
71 72

Thread Lifecycle
While a thread executes, it goes through a number of different phases
New: created but not yet started Runnable: is running, or can run on a free CPU Blocked: waiting for I/O or on a lock Sleeping: paused for a user-specified interval Terminated: completed

Which Thread to Run Next?

Look at all runnable threads
A good choice to run is one that just became unblocked because
A lock was released I/O became available It finished sleeping, etc.

Pick a thread and start running it

Can try to influence this with setPriority(int) Higher-priority threads get preference But you probably dont need to do this
73 74

Some Thread Methods

void join() throws InterruptedException
Waits for a thread to die/finish

Example: Threaded, Sync Alarm

while (true) { System.out.print("Alarm> "); // read user input String line = b.readLine(); parseInput(line); // wait (in secs) asynchronously if (m != null) { // start alarm thread Thread t = new AlarmThread(m,tm); t.start(); // wait for the thread to complete t.join(); }
76

static void yield()

Current thread gives up the CPU

static void sleep(long milliseconds) throws InterruptedException

Current thread sleeps for the given time

static Thread currentThread()

Get Thread object for currently executing thread }
75

Daemon Threads
void setDaemon(boolean on)
Marks thread as a daemon thread Must be set before thread started

Key Ideas
Multiple threads can run simultaneously
Either truly in parallel on a multiprocessor Or can be scheduled on a single processor
A running thread can be pre-empted at any time

By default, thread acquires status of thread that spawned it Program execution terminates when no threads running except daemons

Threads can share data

In Java, only fields can be shared Need to prevent interference
Rule of thumb 1: You must hold a lock when accessing shared data Rule of thumb 2: You must not release a lock until shared data is in a valid state

Overuse use of synchronization can create deadlock

Rule of thumb: No deadlock if only one lock
77 78

Producer/Consumer Design
Suppose we are communicating with a shared variable
E.g., some kind of a buffer holding messages

Conditions (Java 1.5)

interface Lock { Condition newCondition(); ... } interface Condition { void await(); void signalAll(); ... }

One thread produces input to the buffer One thread consumes data from the buffer How do we implement this?
Use condition variables

Condition created from a Lock await called with lock held

Releases the lock
But not any other locks held by this thread

Condition ...

Adds this thread to wait set for lock Blocks the thread

signallAll called with lock held

Resumes all threads on locks wait set Those threads must reacquire lock before continuing
79

wait set
80

(This is part of the function; you dont need to do it explicitly)

Producer/Consumer Example
Lock lock = new ReentrantLock(); Condition ready = lock.newCondition(); boolean valueReady = false; Object value; void produce(Object o) { lock.lock(); while (valueReady) ready.await(); value = o; valueReady = true; ready.signalAll(); lock.unlock(); } Object consume() { lock.lock(); while (!valueReady) ready.await(); Object o = value; valueReady = false; ready.signalAll(); lock.unlock(); }
81

Use This Design

This is the right solution to the problem
Tempting to try to just use locks directly Very hard to get right Problems with other approaches often very subtle
E.g., double-checked locking is broken

82

Broken Producer/Consumer Example

Lock lock = new ReentrantLock(); boolean valueReady = false; Object value; void produce(object o) { lock.lock(); while (valueReady); value = o; valueReady = true; lock.unlock(); } Object consume() { lock.lock(); while (!valueReady); Object o = value; valueReady = false; lock.unlock(); }

Broken Producer/Consumer Example

Lock lock = new ReentrantLock(); boolean valueReady = false; Object value; void produce(object o) { while (valueReady); lock.lock(); value = o; valueReady = true; lock.unlock(); } Object consume() { while (!valueReady); lock.lock(); Object o = value; valueReady = false; lock.unlock(); }

Threads wait with lock held no way to make progress

83

valueReady accessed without a lock held race condition

84

Broken Producer/Consumer Example

Lock lock = new ReentrantLock(); Condition ready = lock.newCondition(); boolean valueReady = false; Object value; void produce(object o) { lock.lock(); if (valueReady) ready.await(); value = o; valueReady = true; ready.signalAll(); lock.unlock(); } Object consume() { lock.lock(); if (!valueReady) ready.await(); Object o = value; valueReady = false; ready.signalAll(); lock.unlock(); }
85

More on the Condition Interface

interface Condition { void await(); boolean await (long time, TimeUnit unit); void signal(); void signalAll(); ... }

away(t, u) waits for time t and then gives up

Result indicates whether woken by signal or timeout

signal() wakes up only one waiting thread

Tricky to use correctly
Have all waiters be equal, handle exceptions correctly

what if there are multiple producers or consumers?

Highly recommended to just use signalAll()

86

Await and SignalAll Gotchas

await must be in a loop
Dont assume that when wait returns conditions are met

Blocking Queues in Java 1.5

Interface for producer/consumer pattern
interface Queue<E> extends Collection<E> { boolean offer(E x); /* produce */ /* waits for queue to have capacity */ E remove(); /* consume */ /* waits for queue to become non-empty */ ... }

Avoid holding other locks when waiting

await only gives up locks on the object you wait on

Two handy implementations

LinkedBlockingQueue (FIFO, may be bounded) ArrayBlockingQueue (FIFO, bounded) (plus a couple more)
87 88

Wait and NotifyAll (Java 1.4)

Recall that in Java 1.4, use synchronize on object to get associated lock
os lock object o os wait set

Wait and NotifyAll (contd)

o.wait()
Must hold lock associated with o Release that lock
And no other locks Adds this thread to wait set for lock Blocks the thread

Objects also have an associated wait set

o.notifyAll()
Must hold lock associated with o Resumes all threads on locks wait set Those threads must reacquire lock before continuing
(This is part of the function; you dont need to do it explicitly)

89

90

Producer/Consumer in Java 1.4

public class ProducerConsumer { private boolean valueReady = false; private Object value; synchronized void produce(Object o) { while (valueReady) wait(); value = o; valueReady = true; notifyAll(); } synchronized Object consume() { while (!valueReady) wait(); valueReady = false; Object o = value; notifyAll(); return o; }

InterruptedException
Exception thrown if interrupted on certain ops
wait, await, sleep, join, and lockInterruptibly Also thrown if call one of these with interrupt flag set

Not thrown when blocked on 1.4 lock or I/O

class Object { void wait() throws IE; ... } interface Lock { void lock(); void lockInterruptibly() throws IE; ... } interface Condition { void await() throws IE; void signalAll(); ... }

91

92