CONCURRENCY IN JAVA

CSC 302
 This lecture slides is based on
 Java Concurrency in Practice by
Goetz, et. al.
http://www.javaconcurrencyinpractice.com
Outline

PART 1
 Introduction
 Basic Thread Safety
 Sharing Objects
 Composing Objects
 Building Blocks
Introduction
 Why concurrency?
 Need for OS
 Resource utilization - why wait? Another program can be running
while another is waiting for resource
 Fairness - why wait? Computer resources can be shared via time
slicing
 Convenience - why wait? Easier and more desirable to write
several programs that each perform a single task
 The needs motivated development of processes
 First concurrency: processes
 The same motivated development of threads
 Threads allow multiple streams of program control flow
within a process
Threads
• Threads execute simultaneously and asynchronously with
respect to one another.
• Benefits of Threads
• Responsiveness (esp. GUIs)
• Exploiting multi-processors
• Simplicity of modeling
• Simplified handling of asynchronous events
Risks of Threads
 Java’s “built-in” threads means that concurrency is NOT an
advanced topic
 Safety hazards (correctness) (operations in multiple
threads may be interleaved arbitrarily by the runtime)
 Liveness hazards (progress) (Indefinite loop where code
that follows the loop never get executed)
 Performance hazards (happiness)(Poor service time,
responsiveness, throughput, scalability, resource
consumption)
 Threads are Everywhere

• Frameworks use threads

• Timer
• Servlets and JSPs
• RMI
• Swing and AWT
Outline

PART 1
 Introduction
 Basic Thread Safety
 Sharing Objects
 Composing Objects
 Building Blocks
Synchronization
 Concurrent programming is about writing thread-safe
code and managing access to shared mutable state
 Informally, object’s state is its data stored in state variable
 Shared implies that a variable could be accessed by
multiple threads
 Mutable implies that variable value could change during
its lifetime
 Thread safety is about protecting data from uncontrolled
concurrent access
 Making an object thread-safe requires using
synchronization to coordinate access to its mutable state
Synchronization…
• Given a mutable variable v
• If multiple threads access v
• AND if one can modify v
• ALL must synchronize access
• This includes read access
• Never attempt to ignore this
• If sync is broken, so is the code (even if it passes tests)
Concurrent Correctness
• Fixing a broken “shared” mutable state var
• Don’t Share var across threads Or
• Make the var immutable! Or
• Synchronize access to var
• Better yet, design thread-safe classes using good object-
oriented techniques
• Encapsulation
• Immutability
• Clear specifications of invariants
What is “Thread-Safe”?
• Definitions are vague and vary
• Problem: what is correctness?
• Thread Safe:
• “Correct” in multi-threaded env
• OR, no more broken in a mult-threaded environment
then in a single-threaded one
• Thread-safe classes encapsulate all synchronization
Example: Unsafe Counter
@NoThreadSafe
public class UnsafeSequence{
private int value;
public int getNext() {
return value++;
}
}
 What’s Wrong with That?

• Invariant:
• getNext must return a sequence
• Unlucky execution timing:

Value->9 9+1->10 Value=10

Value->9 9+1->10 Value=10

 Example: Stateless Servlet
@ThreadSafe
public class StatelessFactorizer implements Servlet{
public void service( ServletRequest req, ServletResponse
resp){
BigInteger i = extractFromRequest(req);
BigInteger[ ] factors = factor(i);
encodeIntoResponse(resp, factors);
}
}
//Stateless objects are always thread-safe
 Atomicity -
Example
@NotThreadSafe
public class UnsafeCountingFactorizer implements Servlet{
private long count = 0;
public long getCount( ) {return count;}
public void service( ServletRequest req, ServletResponse resp){
BigInteger i = extractFromRequest(req);
BigInteger[ ] factors = factor(i);
++count;
encodeIntoResponse(resp, factors);
}}
// Work fine in a single threaded environment
// Not atomic because ++count operation is a read-modify-write
operation
// Hence, not thread-safe
 Race Conditions

• The problem is a race condition

• Def: r.c. occurs when the correctness of a computation
depends on the relative timing or interleaving of multiple
threads by the runtime.
• Often confused for data race
Achieving Safety
• Go stateless!
• Use atomic operations
• Use locking
Using Atomics!
(It’s not a treaty violation)
@ThreadSafe
public class UnsafeSequence{
private final
AtomicLong value = new AtomicLong(0);

public long getNext() {

return value.incrementAndGet();
}
}
Locking
• Solved one mutable variable by making the var atomic
• What if we have 2variables Can we just make them both
atomic?
• NOT if they are dependent
• We have to lock any combined operations
Example
• Suppose we have a dictionary that stores the last (key,value)
pair
• Setting the (k,v) pair must be locked. It is not enough to use
an atomic var for k, and another one for v
 Intrinsic Locks
• Java provides built-in locks
• Intrinsic locks act as a mutex (mutual exclusive lock)
• When thread A attempts to acquire lock held by thread B, A
must wait, or block
• If B never releases the lock, A waits forever!
Intrinsic Locks
• Java provides built-in locks
• Each object is a lock, so is each class
• Marked by synchronized
• Enforces
• Atomicity
• Memory visibility (more later)
Reentrancy
• Requesting a lock held by another causes a block
• Intrinsic locks are reentrant
• A thread that owns a lock, that requests the same lock, will
succeed (good for inheritance!)
Example Locking
@ThreadSafe
Public class LockSafe<K,V> {
@GuardedBy(“this”) private K key;
@GuardedBy(“this”) private V value;

public synchronized setKvPair(K k, V v) {

this.key = k;
this.value = v;
}
}
Locks: “Guarded By”
“For each mutable state variable that may be accessed by
more than one thread, all accesses to that variable must be
performed with the same lock held. In this case, we say that
the variable is guarded by that lock”
Locks: Documentation
“Every shared, mutable variable should be guarded by exactly
one lock. Make it clear to maintainers which lock that is”
Locks: Across Invariants
“For every invariant that involves more that one variable, all
the variables involved in that invariant must be guarded by
the same lock” - (JCP p29)
Critical Concept
• Any mutable state that can be concurrently accessed, must
be synchronized EVERYWHERE else in the program!
• EXAMPLE: TimerTask (JCP p29)
 Why Not Lock Everything?
• Even if every method were synchronized, it doesn’t solve
actions that combine those methods! (example: vector)
• Also, possible liveness and/or performance problems
• Poor concurrency example
Lock Advice
“Avoid holding locks during lengthy computations or
operations at risk of not completing quickly such as network
or console I/O” - (JCP p32)
Outline

PART 1
 Introduction
 Basic Thread Safety
 Sharing Objects
 Composing Objects
 Building Blocks
Sharing Objects
• Memory Visibility
• Shared object modification should be visible to other threads
• Without synchronization, this may not happen
More about Visibility
“In general, there is no guarantee that the reading thread will
see a value written by another thread on a timely basis, or
even at all” - (JCP p33)
Public class NoVisibility {
private static boolean ready;
private static int number;

private static class ReaderThread extends Thread {

public void run() {
while (!ready) Thread.yield()
System.out.println(number)
}
}

public static void main(String[] args) {

new ReaderThread().start();
number = 42;
ready = true;
}
}
NoVisibility :(
• NoVisibility could loop forever!
• NoVisibility could write 0 through reordering
• Other badness and sadness
• Don’t do this!
Synchronization Helps
• NoVisibility demonstrates stale data
• Synchronization must be used on all shared variables, even
just for reads
• Enforces memory visibility
A Word about 64bit
• When a thread reads a var w/o synchronization, at worst it is
stale, but not random
• Out-of-thin-air safety
• One exception: 64 bit numbers (double and long)
• Read/write can be 2 32 bit operations!
Volatile Variables

• Can declare a java var volatile

• Volatile ensures visibility, but not
locking.
• Volatile use can be fragile. Many times,
you shouldn’t use it
• Good uses of volatile
• Ensure visibility of their own state
• That of the object they refer to
• Or indicate that an important event has
occurred (e.g. shutdown)
Use Volatile ONLY when
• Writes to the var do not depend on its current value
or are only written by one thread
• The var does not participate with invariants with
other state vars
• Locking is not required for any other reason when
the var is being accessed
Publication and Escape
• Publishing an object - makes it available outside
current scope
• Many times, objects should not be published at all
• Breaks encapsulation
• Not fully constructed (!)
• Object published when it should not have been is
said to have escaped!
Publication Methods
• Public static fields
• Chained publication
• Special Case: published inner class
• Passing to an alien method
How Escape Happens
• All linked to poor design
• Semantic escapes:
• Return ref instead of copy
• Inadvertent chained publication
• Syntactic escapes:
• Escaped this during construction
• “Object not properly constructed”
This Escape Example

Public class ThisEscape {

// JCP p41
public ThisEscape(Eventsource source) {
source.registerListener(
new EventListener() {
public void onEvent(Event e) {
doSomething(e);
}
});
}
}
Preventing Escape
• Thread Confinement
• Immutability
• Safe Publication
Thread Confinement
• Keep mutable vars confined to a single thread
• Ad-hoc = enforced by impl.
• Stack Confinement
• Local vars
• Violated by publication
• ThreadLocal
• Per thread value holding object
Immutable Objects
• Always thread safe
• Can’t escape after construction
• Definition
• It’s state cannot be modified after construction
• All fields are final*
• Properly constructed
• Can have mutable variables
Final Fields
• Can’t be modified
• AND, have special semantics in the Java Memory model
(initialization safety)
• Make all fields final by default
• “Mostly Immutable” is better than mutable
Using Volatile Again!
• Volatile can be used to publish immutable objects
• Still not locked, but can be safe depending on semantics
• Example - JCP p49-50
Improper Publication
• If synchronization is not used to publish a mutable object,
the object is not properly published
• Without proper publication, there are serious problems with
stale data
Safe Publication
• Object reference and object state must become
visible at the same time!
• Idioms:
• Initializing from static initializer
• Storing ref in volatile or AtomicReference
• Storing ref in final field of properly constructed object
• Storing ref in a lock-guarded field
Good News!
• Java Collections safe publication examples:
• Key or value in Hashtable, synchronizedMap, or
ConcurrentMap
• Insert in Vector, CopyOnWriteArrayList,
CopyOnWriteArraySet, synchronizedList, or
synchronizedSet
• Insert in BlockingQueue or ConcurrentLinkedQueue
More Good News
• Effectively Immutable Objects
• Objects that are used as if they were immutable
• Example: Date
Modification Vs Visibility
• Safe publication ensures visibility
• Synchronization is required if the object can be modified
post-publication
Rules of Engagement
• Thread-Confined: no sharing
• Shared read-only: (effectively) immutable objects
• Shared thread-safe: internal synchronization
• Guarded: only access with the associated guard
Designing Thread-Safe Classes
• Identify the variables that form the object’s state
• Identify the invariants that contain the state variables
• Establish a policy for managing concurrent access to the
object’s state
Java Monitor Pattern
• Principle of instance confinement
• Encapsulate all mutable state and guard it with intrinsic lock
• Good coarse-grained locking
Delegating Thread Safety
• Safety can often be delgated to an internally-used thread-
safe object
• Collections are especially useful
• Only works for 1 state var, or multiple independent vars
Extending Thread-safe Classes
• BE CAREFUL! Extending means that the
synchronization policy is distributed over multiple
separately maintained classes!
• Inheritance requires the base class to publish
enough state
• Wrappers require that the same lock is used
• Composition is often less fragile
Extension Examples:
• Inheritance (JCP p72)
• Wrapping (JCP p72-73)
• Composition (JCP p74)
Outline

PART 1
 Introduction
 Basic Thread Safety
 Sharing Objects
 Composing Objects
 Building Blocks
 Synchronized Collections
• Are thread safe, but
• Composite actions require additional locking for semantic correctness
• Careless use of locking leads to poor performance (think iteration)
Iterators

• The synchronized collections provide iterators

• These iterators are fail-fast
• Throws ConcurrentModificationException
Concurrent Collections
• Designed for concurrent access
• Improved performance with a few minor trade-offs:
• Your assignment: go look up the docs on
ConcurrentHashMap and CopyOnWriteArrayList and try them
out!
Blocking Queues
• Support the producer-consumer pattern
• Can be bounded or unbounded
• Serial thread confinement
• Try it out!
Blocking
• Threads may block
• Wait for I/O
• Wait for Lock
• Wait for Thread.sleep
• Wait for computation
• put and get from BlockingQueue block
Interruption
• interrupt() method
• Request a blocking thread stop
• A blocking method should throw an InterruptedException
• If you catch this
• Propagate the exception OR
• Restore the interrupt
Example:
Try{
someBlockingMethod();
} catch (InterruptedException) {
Thread.currentThread().interrupt();
}
Synchronizers

• Latches: one-time trigger

• e.g. CountDownLatch
• FutureTask: long-running result
• Has a get() method
• Returns value if done
• Blocks until it is done if not
More Synchronizers

• Semaphore: virtual permits

• Now a library class!
• Barriers: wait for enough threads
• Useful for threaded stepping
• Exchanger 2-party barrier
Outline

PART 1
 Introduction
 Basic Thread Safety
 Sharing Objects
 Composing Objects
 Building Blocks
Advanced Execution
• The Executor Framework
• Decouples task submission and task execution
• Simple interface:
void execute(Runnable command)
• Easiest way to implement a producer-consumer design in an
application
Execution Policies
• Answers
• In what thread will tasks be run
• What order should they be run
• How many concurrently?
• How many queued?
• “Victim” selection
• Setup and Teardown
Policy Examples
• Single-thread (JCP p119)
• 1-thread-per-client (JCP p118)
• Thread Pools: some built in
• newFixedThreadPool
• newCachedThreadPool
• newSingleThreadExecutor
• newScheduleThreadPool
 Cancellation & Shutdown
• Reasons for cancellation
• User-requested
• Time-limited
• Events
• Errors
• Shutdown
Cancellation Policy
• “how” “when” and “what” of cancellation
• Not enforced by Java, but generally, use interruption for
cancellation
• E.g. have a cancel() method call the interrupt
Interruption Policy
• Determines how a thread interprets an interruption request
• Tasks are “guests” in a thread
• Preserve interruption status
• Only call interrupt, when you know the interruption policy
Detecting Interrupts
• Polling the thread’s interrupted status
• Detecting an InterruptedException
Non-interruptible Blocks
• Synchronous socket I/O:
• Close the socket
• Asynchronous I/O with Selector:
• Call wakeup
• Lock Acquisition:
• Use the explicit Lock class
• Has a lockInterruptibly() method
Stopping a Thread-Based Service
• ExceutorService extends Executor
• Provides “Lifecycle” options
• shutdown()
• awaitTermination(timeout, unit)
• Any thread based service should provide similar methods
Waiting for Threads
• In the “raw” case, use join()
• A number of Java library classes have advanced “waits” like
ExecutionService’s awaitTermination()
• Obviously, if you implement your own ExecutionService, you’ll
have to use the raw stuff