Parallel

&
Asynchronous Programming
In
Modern Java

Dilip Sundarraj
 About Me
• Dilip

• Building Software’s since 2008

• Teaching in UDEMY Since 2016

What's Covered ?
• Need for Parallel and Asynchronous Programming

• Covers the ParallelStreams and CompletableFuture API

• Techniques to write Fast Performing Code using Functional Style Concurrency

APIs in Java

• Covers Best Practices using ParallelStreams/CompletableFuture API in your code

• NonBlocking RestFul API Client using CompletableFuture API

• Testing ParallelStreams and CompletableFuture API Using JUnit5

Targeted Audience

• Experienced Java Developers

• Developers who has the need to write code that executes faster

• Developers who has the need to write code that executes in Parallel

• Developer who has the need to write asynchronous/non-blocking code

Source Code
Thank You!
Prerequisites
Course Prerequisites
• Java 11 or Higher is needed

• Prior Java knowledge is needed

• Experience working with Lambdas, Streams , Method References

• Experience writing JUnit tests

• Intellij , Eclipse or any other IDE is needed

Why Parallel and Asynchronous
Programming ?
Why Parallel and Asynchronous Programming?

• We are living in a fast paced environment

• In Software Programming:

• Code that we write should execute faster

• Goal of Asynchronous and Parallel Programming

• Provide Techniques to improve the performance of the code

Technology Advancements
Hardware Software

• Devices or computers comes up • MicroServices Architecture style

with Multiple cores

• Blocking I/O calls are common in

• Developer needs to learn MicroServices Architecture. This
programming patterns to maximize also impacts the latency of the
the use of multiple cores
application

• Apply the Parallel Programming • Apply the Asynchronous

concepts
Programming concepts

• Parallel Streams • CompletableFuture

Threads Functional Style of Programming

 Evolution
of
Concurrency and Parallelism
APIs
in Java
 ThreadPool, Parallel Streams
ExecutorService,Futures, CompletableFutures
Fork/Join
Threads ConcurrentCollections etc., Flow API
FrameWork

Lamdas
Data Parallelism Streams API

Java1 Java5 Java7 Java8 Java9

Timeline
1996 2004 2011 2014 2017

Single Core Multi Core

Current Version : Java14

https://en.wikipedia.org/wiki/Java_version_history
Concurrency
vs
Parallelism
Concurrency
• Concurrency is a concept where two or more task can run simultaneously

• In Java, Concurrency is achieved using Threads

• Are the tasks running in interleaved fashion?

• Are the tasks running simultaneously ?

Thread1 Thread2 Thread3 Thread4

Thread1 Thread2 Thread3

Single Core1 Core2 Core3 Core4

Core
MultiCore
Concurrency Example
public class HelloWorldThreadExample {
• In a real application, Threads normally need to private static String result="";
interact with one another
private static void hello(){
delay(500);
• Shared Objects or Messaging Queues
}
result = result.concat("Hello");

private static void world(){

• Issues:
delay(600);
result = result.concat(“ World");
}
• Race Condition
public static void main(String[] args) throws InterruptedException {

• DeadLock and more

1
Thread helloThread = new Thread(()-> hello());
Thread worldThread = new Thread(()-> world()); } Threads

• Tools to handle these issues:

//Starting the thread
helloThread.start();
2 worldThread.start();
• Synchronized Statements/Methods

//Joining the thread (Waiting for the threads to finish)

helloThread.join();
• Reentrant Locks, Semaphores
3 worldThread.join();

• Concurrent Collections
System.out.println("Result is : " + result);

}
• Conditional Objects and More }
Hello World
Parallelism Fork/Join (Parallelism)

Task
• Parallelism is a concept in which
tasks are literally going to run in
1 fork
parallel
SubTask1 SubTask2
fork fork
• Parallelism involves these steps:

SubTask1a SubTask1b SubTask2a SubTask2b

• Decomposing the tasks in to
SubTasks(Forking)
2 Process Sequentially Process Sequentially Process Sequentially Process Sequentially

• Execute the subtasks in sequential

• Joining the results of the tasks(Join)

join join
3
• Whole process is also called Fork/
Join
join
Parallelism Example Fork/Join (Parallelism)

Bob, Jamie, Jill,

Rick
UseCase: Transform to UpperCase

[Bob, Jamie, Jill, Rick] -> [BOB, JAMIE, JILL, RICK] 1 fork
[Bob,Jamie] [Jill,Rick]
fork fork

Bob Jamie Jill Rick

Bob Jamie Jill Rick Process Sequentially Process Sequentially Process Sequentially Process Sequentially

uppercase uppercase uppercase uppercase 2

BOB JAMIE JILL RICK

Core1 Core2 Core3 Core4

[BOB, [JILL, RICK] join

JAMIE] join
BOB JAMIE JILL RICK 3

BOB, JAMIE, JILL, RICK join

Parallelism Example

public class ParallelismExample {

public static void main(String[] args) {

List<String> namesList = List.of("Bob", "Jamie", "Jill", "Rick");

System.out.println("namesList : " + namesList);
List<String> namesListUpperCase = namesList
.parallelStream()
.map(String::toUpperCase)
.collect(Collectors.toList());

System.out.println("namesListUpperCase : " + namesListUpperCase);

}
}
Concurrency vs Parallelism

• Concurrency is a concept where two • Parallelism is a concept where two

or more tasks can run in or more tasks are literally running in
simultaneously
parallel

• Concurrency can be implemented in • Parallelism can only be implemented

single or multiple cores
in a multi-core machine

• Concurrency is about correctly and • Parallelism is about using more

efficiently controlling access to resources to access the result faster
shared resources
Course Project Setup
Section Overview
Section Overview

• Covers Asynchronous and Parallel Programming prior Java 8

• Threads, Futures and ForkJoin Framework and its limitations

• Covers Theory and Hands On

 Overview
of the
Product Service
Product Service

ProductInfo
size, color, price
Service
ProductId

Client Product Service

Product Review
No of reviews,
Service Overall rating
Product
Threads
Threads API
• Threads API got introduced in Java1

• Threads are basically used to offload the blocking tasks as

background tasks

• Threads allowed the developers to write asynchronous style of

code
Thread API Limitations
• Requires a lot of code to introduce asynchrony

• Runnable, Thread

• Require additional properties in Runnable

• Start and Join the thread

• Low level

• Easy to introduce complexity in to our code

 ThreadPool,
ExecutorService
&
Future
Limitations Of Thread
• Limitations of Thread:

• Create the thread

• Start the thread

• Join the thread

• Threads are expensive

• Threads have their own runtime-stack, memory, registers and more

Thread Pool was created specifically to solve this problem

Thread Pool
• Thread Pool is a group of threads created ThreadPool
and readily available

• CPU Intensive Tasks

• ThreadPool Size = No of Cores

• I/O task

• ThreadPool Size > No of Cores

T1 T2

• What are the benefits of thread pool?

• No need to manually create, start and

join the threads
T3 T4

• Achieving Concurrency in your

application
ExecutorService
• Released as part of Java5

• ExecutorService in Java is an Asynchronous Task Execution Engine

• It provides a way to asynchronously execute tasks and provides the results in

a much simpler way compared to threads

• This enabled coarse-grained task based parallelism in Java

ExecutorService

ExecutorService

WorkQueue
ThreadPool

T1 T2

CompletionQueue

T3 T4
Working Of ExecutorService

ExecutorService
WorkQueue ThreadPool

Client Task

T1 T2

Future

CompletionQueue
T3 T4

Result
Limitations of ExecutorService
• Designed to Block the Thread

ProductInfo productInfo = productInfoFuture.get();

Review review = reviewFuture.get();

• No better way to combine futures

ProductInfo productInfo = productInfoFuture.get();

Review review = reviewFuture.get();
return new Product(productId, productInfo, review);
Fork/Join Framework
Fork/Join Framework
• This got introduced as part of Java7

• This is an extension of ExecutorService

• Fork/Join framework is designed to achieve Data Parallelism

• ExecutorService is designed to achieve Task Based Parallelism

Future<ProductInfo> productInfoFuture = executorService.submit(() -> productInfoService.retrieveProductInfo(productId));

Future<Review> reviewFuture = executorService.submit(() -> reviewService.retrieveReviews(productId));

What is Data Parallelism ?
Fork/Join (Parallelism)

Bob, Jamie, Jill,

• Data Parallelism is a concept where a Rick

given Task is recursively split in to 1 fork

SubTasks until it reaches it leaset [Bob,Jamie]
fork
[Jill,Rick]
fork
possible size and execute those tasks
Bob Jamie Jill Rick
in parallel

Process Sequentially Process Sequentially Process Sequentially Process Sequentially

• Basically it uses the divide and 2

BOB JAMIE JILL RICK

conquer approach
[BOB, [JILL, RICK] join
JAMIE] join
3

Watch "Concurrency vs Parallelism”

BOB, JAMIE, JILL, RICK join
How does Fork/Join Framework Works ?

ForkJoin Pool to support Data Parallelism

ForkJoin Pool

ForkJoin Pool
Double Ended Work Queue(deck)

Task Task Task Task T1

Task

WorkStealing
Client ForkJoinTask
Task T2
Worker Threads

Result
Shared Work Queue

Task T3

Task T4
ForkJoin Task
Fork/Join (Parallelism)

Bob, Jamie, Jill,

• ForkJoin Task represents part of the data Rick

and its computation

fork
• Type of tasks to submit to ForkJoin Pool
[Bob,Jamie] [Jill,Rick]
fork fork
• ForkJoinTask

Bob Jamie Jill Rick

• RecursiveTask -> Task that returns a Fork/Join

value
Task Process Sequentially Process Sequentially Process Sequentially Process Sequentially

BOB JAMIE JILL RICK

• RecursiveAction -> Task that does
not return a value
[BOB, [JILL, RICK] join
JAMIE] join

BOB, JAMIE, JILL, RICK join

Fork/Join Example
ForkJoin - UseCase

Input Output

[Bob, Jamie, Jill, Rick] -> [3 - Bob, 5 - Jamie, 4 - Jill, 4 - Rick]

 Streams API
&
Parallel Streams
Streams API
• Streams API got introduced in Java 8

• Streams API is used to process a collection of Objects

• Streams in Java are created by using the stream() method

Intermediate

Terminal
1

3
] Pipeline
ParallelStreams

• This allows your code to run in parallel

• ParallelStreams are designed to solve Data Parallelism

Watch “Concurrency vs Parallelism” & “Fork-Join Framework”

Stream/Parallel Stream

Stream

Parallel Stream
Parallel Streams - UseCase

Input Output

[Bob, Jamie, Jill, Rick] -> [3 - Bob, 5 - Jamie, 4 - Jill, 4 - Rick]

 Unit Testing
Parallel Streams
Using
JUnit5
Why Unit Tests ?

• Unit Testing allows you to programmatically test your code

• Manual Testing slows down the development and delivery

• Unit Testing allows the developer or the app team to make enhancements to the

existing code easily and faster

Sequential/Parallel
Functions
in
Streams API
sequential() and parallel()

• Streams API are sequential by default

• sequential() -> Executes the stream in sequential

• parallel() -> Executes the stream in parallel

• Both the functions() changes the behavior of the whole pipeline

sequential()

• Changing the parallelStream() behavior to sequential

Sequential
parallel()

• Changing the stream() behavior to parallel

Parallel
When to use sequential() and parallel() ?

Used these functions when I would like to evaluate between sequential() and

parallel()
Overview of the
Retail
Checkout
Service
Checkout Service(BackEnd)

UP

N CartItem Price Validator

Checkout Service $
Service
DOWN
ParallelStreams
How it works ?
ParallelStreams - How it works ?
• parallelStream()

• Split the data in to chunks

• Execute the data chunks

• Combine the result

parallelStream() - How it works ?
• Split

• Data Source is split in to small data chunks

• Example - List Collection split into chunks of elements to size 1

• This is done using Spliterators

• For ArrayList, the Spliterator is ArrayListSpliterator

• Execute

• Data chunks are applied to the Stream Pipeline and the Intermediate operations executed in a Common ForkJoin
Pool

• Watch the Fork/Join FrameWork Lectures

• Combine

• Combine the executed results into a final result

• Combine phase in Streams API maps to terminal operations

• Uses collect() and reduce() functions

• collect(toList())
parallelStream() - How it works ?
parallelStreams()

cartItemsList CheckoutService

List<CartItem> priceValidationList = cart.getCartItemList()

//.stream() Split
Split 1
.parallelStream()
cartItemsSplit cartItemsSplit
.map(cartItem Split -> { Split

boolean isPriceValid = priceValidatorService.isCartItemInvalid(cartItem);

cartItem.setExpired(isPriceValid);
cartItem cartItem cartItem cartItem
return cartItem;
Process})Sequentially
Execute 2
Process Sequentially Process Sequentially Process Sequentially
Common ForkJoinPool
.filter(CartItem::isExpired)
.collect(toList()); cartItem
cartItem cartItem cartItem

combine 3 [cartItem, combine

[cartItem,
cartitem]
combine cartitem]

cartItemsList combine
 Comparing
ArrayList vs LinkedList
ParallelStreams
Performance
Spliterator in ParallelStreams

• Data source is split in to multiple chunks by the Spliterator

• Each and every collection has a different Spliterator Implementation

• Performance differ based on the implementation

 Multiply each value in the
collection by a user passed value

[ 1, 2, 3, 4 ] -> [2, 4, 6, 8]
Value * 2
Summary - Spliterator in ParallelStreams

• Invoking parallelStream() does not guarantee faster performance of your code

• Need to perform additional steps compared to sequential

• Splitting , Executing and Combining

Recommendation - Always compare the performance before you use parallelStream()

 Parallel Streams
Final Computation Result Order
Parallel Streams - Final Computation Result Order

[Bob, Jamie, Jill, Rick]

[3 - Bob, 5 - Jamie, 4 - Jill, 4 - Rick]

Parallel Streams - Final Computation Result Order

• The order of the collection depends on:

• Type of Collection

• Spliterator Implementation of the collection

• Example : ArrayList

• Type of Collection - Ordered

• Spliterator Implementation - Ordered Spliterator Implementation

• Example : Set

• Type of Collection - UnOrdered

• Spliterator Implementation - UnOrdered Spliterator Implementation

Collect
&
Reduce
 Collect( ) vs Reduce( )
Collect Reduce

• Part of Streams API

• Part of Streams API

• Used as a terminal operation in Streams • Used as a terminal operation in Streams

API API

• Produces a single result • Produces a single result

• Result is produced in a mutable fashion • Result is produced in a immutable fashion

• Feature rich and used for many different • Reduce the computation into a single value

use cases

• Sum, Multiplication

• Example

• Example

• collect(toList()), collect(toSet())

• Sum -> reduce(0.0, (x , y)->x+y)

• collect(summingDouble(Double::doubleV
alue)); • Multiply -> reduce(1.0, (x , y)->x * y)
How reduce() works ?
[ 1 ,2 ,3 ,4 ]

0+1 => 1

identity 1+2 => 3

10

3+3 => 6

6+4 => 10

The reduce( ) function performs an immutable computation throughout in each and every step.
How reduce() with ParallelStream works ?
Sum of N numbers

Sum of N numbers - reduce() and parallelStream() [1,2,3,4,5,6,7,8]

Split Split

1
[ 1 ,2 ,3 ,4 ] [ 5 ,6 ,7 ,8 ]
Split Split Split Split

0 0 0 0
36 [ 1 ,2 ] [ 3 ,4 ] [ 5 ,6 ] [ 7 ,8 ]

Process Sequentially Process Sequentially Process Sequentially Process Sequentially

2

1 Spliterator
3 7 11 15
reduce()
reduce()
2 ForkJoinPool
3 10 26
reduce() reduce()
3 Reduce

36
Want to learn more ?
Collect() & Reduce()
Hands-On
Identity in reduce()
Identity in reduce()

• Identity gives you the same value

when its used in the computation
Sum of N numbers - reduce() and parallelStream()

• Addition: Identity = 0

• 0 + 1 => 1

• 0 + 20 => 20

• Multiplication : Identity = 1

• 1 * 1 => 1

• 1 * 20 => 20 reduce() is recommended for computations that are associative

Parallel Stream Operations
&
Poor Performance
Parallel Stream Operations & Poor Performance

• Stream Operations that perform poor

• Impact of Boxing and UnBoxing when it comes to parallel Streams

• Boxing -> Converting a Primitive Type to Wrapper class equivalent

• 1 -> new Integer(1)

• UnBoxing -> Converting a Wrapper class to Primitive equivalent

• new Integer(1) -> 1

Common ForkJoin Pool
Common ForkJoin Pool

Execution Engine for Parallel Streams

parallelStream() - How it works ?
parallelStreams()

cartItemsList CheckoutService

Split
Split 1 cartItemsSplit cartItemsSplit
Split Split

cartItem cartItem cartItem cartItem

Execute 2
Process Sequentially Process Sequentially Process Sequentially Process Sequentially
Common ForkJoinPool

cartItem cartItem cartItem cartItem

combine 3 [cartItem, combine

[cartItem,
cartitem]
combine cartitem]

cartItemsList combine
 Common ForkJoin Pool

Common ForkJoin Pool

Double Ended Work Queue(deck)

T1

Task

parallelStreams() T2
Worker Threads

Result
Shared Work Queue

T3

Watch “Fork-Join Framework” Lecture T4

Common ForkJoin Pool

• Common ForkJoin Pool is used by:

• ParallelStreams

• CompletableFuture

• Completable Future have options to use a User-defined ThreadPools

• Common ForkJoin Pool is shared by the whole process

 Parallelism
&
Threads in
Common ForkJoinPool
Parallelism & Threads in Common ForkJoinPool

• parallelStreams()

• Runs your code in parallel

• Improves the performance of the code

• Is there a way to look in to parallelism and threads involved ?

• Yes
 Modifying
Default parallelism
in
Parallel Streams
Modifying Default parallelism

System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", "100");

OR

-Djava.util.concurrent.ForkJoinPool.common.parallelism=100
Parallel Streams - Summary
Parallel Streams - When to use them ?
Parallel Streams - When to use them ?

• Parallel Streams do a lot compared to sequential(default) Streams

• Parallel Streams

• Split

• Execute

• Combine
Parallel Streams - When to use them ?

• Computation takes a longer time to complete

• Lots of data

• More cores in your machine

Always compare the performance between sequential and parallel streams

Parallel Streams - When to use them ?
Parallel Streams - When not to use them ?

• Parallel Streams

• Split

• Execute

• Combine

• Data set is small

• Auto Boxing and Unboxing doesn’t perform better

• Stream API operators -> iterate(), limit()

CompletableFuture
CompletableFuture

• Introduced in Java 8

• CompletableFuture is an Asynchronous Reactive Functional Programming API

• Asynchronous Computations in a functional Style

• CompletableFutures API is created to solve the limitations of Future API

CompletableFuture and Reactive Programming
• Responsive:

1
• Fundamentally Asynchronous

• Call returns immediately and the response will be sent

when its available

• Resilient:

• Exception or error won’t crash the app or code

• Elastic:

3 2
• Asynchronous Computations normally run in a pool of
threads

• No of threads can go up or down based on the need

4
• Message Driven:
• Asynchronous computations interact with each through
messages in a event-driven style
CompletableFuture API

• Factory Methods

• Initiate asynchronous computation

• Completion Stage Methods

• Chain asynchronous computation

• Exception Methods

• Handle Exceptions in an Asynchronous Computation

 Lets Write
our
First CompletableFuture
CompletableFuture
supplyAsync() thenAccept()

• FactoryMethod
• CompletionStage Method

• Initiate Asynchronous • Chain Asynchronous Computation

computation

• Input is Consumer Functional

• Input is Supplier Functional Interface

Interface

• Consumes the result of the

• Returns CompletableFuture<T>() previous

• Returns
CompletableFuture<Void>

• Use it at the end of the

Asynchronous computation
thenApply()
thenApply()

• Completion Stage method

• Transform the data from one form to another

• Input is Function Functional Interface

• Returns CompletableFuture<T>
CompletableFuture

]
1
2

3 Pipeline
 Unit Testing
CompletableFuture
Combing independent
Async Tasks
using
“thenCombine"
thenCombine()
• This is a Completion Stage Method

• Used to Combine Independent Completable Futures

Service1

Latency = maxOf(Service1, Service2)

Client Service

Service2

• Takes two arguments

• CompletionStage , BiFunction

• Returns a CompletableFuture
thenCompose
thenCompose()

• Completion Stage method

• Transform the data from one form to another

public CompletableFuture<String> worldFuture(String input)
{
return CompletableFuture.supplyAsync(()->{
delay(1000);
• Input is Function Functional Interface
return input+" world!";
});
}

• Deals with functions that return CompletableFuture

• thenApply deals with Function that returns a value

• Returns CompletableFuture<T>
Product Service

ProductInfo
Service
ProductId

Client Product Service

Product Review
Service
Product
 Combining
Streams
&
CompletableFuture
Product Service with Inventory

ProductInfo
Service
ProductId

Client Product Service

Inventory
Service
Product

Product

Review
Service
 Exception
Handling
In
CompletableFuture
Exception Handling in Java
• Exception Handling in Java is available since the inception of Java

]
Exception Handling in CompletableFuture
• CompletableFuture is a functional style API
Exception Handling in CompletableFuture
try/catch
Exception Handling in CompletableFuture

• CompletableFuture API has functional style of handling exceptions

• Three options available:

• handle()

Catch Exception and Recover

• exceptionally()

• whenComplete() Catch Exception and Does not Recover

Exception Handling using handle()

hello thenCombine thenCombine thenApply

Success Path
handle handle

Failure Path

handle handle
Exception Handling using exceptionally()

hello thenCombine thenCombine thenApply

Success Path

Failure Path

exceptionally exceptionally
whenHandle()
whenHandle()

• Exception handler in CompletableFuture API

• Catches the Exception but does not recover from the exception
Exception Handling using whenComplete()

hello thenCombine thenCombine thenApply

Success Path
whenComplete whenComplete

Failure Path

whenComplete whenComplete
Product Service with Inventory

!
ProductInfo
Service
ProductId

Client Product Service

Inventory
Service
Product

Product

Review
Service
CompletableFuture
Default ThreadPool
CompletableFuture - ThreadPool
• By default, CompletableFuture uses the Common ForkJoinPool

• The no of threads in the pool == number of cores

 Common ForkJoin Pool

Common ForkJoin Pool

Double Ended Work Queue(deck)

T1

Task

completableFuture() T2
Worker Threads

Result
Shared Work Queue

T3

T4
Watch “Internals of Common ForkJoin Pool” Lecture
 CompletableFuture
&
User Defined ThreadPool
using
ExecutorService
Why use a different ThreadPool ?

• Common ForkJoinPool is shared by

• ParallelStreams

• CompletableFuture

• Its common for applications to use ParallelStreams and CompletableFuture together

• The following issues may occur:

• Thread being blocked by a time consuming task

• Thread not available

Creating a User-Defined ThreadPool

Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
 Threads
In
CompletableFuture
 Async()
Overloaded Functions
In
CompletableFuture
Async Overloaded Functions

• thenAccept()
Async() Overloaded Functions
Regular Functions Async() overloaded Functions

• thenCombine()
• thenCombineAsync()

• thenApply()
• thenApplyAsync()

• thenCompose()
• thenComposeAsync()

• thenAccept() • thenAcceptAsync()
Async() Overloaded Functions

• Using async() functions allows you to change the thread of execution

• Use this when you have blocking operations in your Completablefuture pipeline
 Introduction to
Spring WebClient
and
Overview of the GitHub Jobs API
About this section

Rest Client Github Jobs API

Build Restful API
Using
Spring WebClient
Why Spring WebClient?

• Spring is one of the popular framework in the Java Community

• Spring WebClient is a rest client library that’s got released as part of Spring 5

• Spring WebClient is a functional style RestClient

• Spring WebClient can be used as a blocking or non blocking Rest Client

Github Jobs API
allOf()
allOf() - Dealing with Multiple CompletableFutures

• static method that’s part of CompletableFuture API

• Use allOf() when you are dealing with Multiple CompletableFuture

Service 1

Client Service 2

Service 2
anyOf()
anyOf() - Dealing with Multiple CompletableFutures

• static method that’s part of CompletableFuture API

• Use anyOf() when you are dealing with retrieving data from multiple Data Sources

DB

Client REST API

SOAP Service
 TimeOuts
In
CompletableFuture
Timeouts in CompletableFuture

• Asynchronous tasks may run indefinitely

• Used to timeout a task in CompletableFuture

• orTimeout() in CompletableFutureAPI