0% found this document useful (0 votes)

36 views

Part 0 - Asynchronous Vs Synchronous Programming

- Asynchronous programming helps get away from one thread per request model and improves performance, isolation, and testability compared to synchronous programming. - Key aspects of asynchronous programming include registering callbacks to receive responses asynchronously and composing callbacks by creating stateful handler objects that emit events once required state is received. - Approaches to asynchronous programming in Java include CompletableFutures, actors, and reactive streams.

Uploaded by

zubes78

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

36 views

Part 0 - Asynchronous Vs Synchronous Programming

Uploaded by

zubes78

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 47

Asynchronous

vs
Synchronous
Course Materials

iteratrlearning.com/async-exercises.zip
iteratrlearning.com/async-slides-1.zip
Introduction
Approaches

Programming Styles
SOA and Microservices
● Increasingly popular style of application development

● Relies extensively on communicating over networks

○ RPC
○ REST
○ Message Driven

● Massively increases the use of network communication

Thread Pools with Blocking I/O

Thread 1

Thread 2

Queue
….
One thread per request

E.g. MortgageServlet
Common Threading Model
● Used by Servlets and anything that sits on them
○ Spring
○ Jersey

● JDBC
○ Pool within JDBC connector of threads talking to DB.
○ Often hard to work around this limitation
Quiz: What problems can you think
of with this approach?
Latency

RPC 1 RPC 2 RPC 3

latency

Latency costs money

● Amazon found 1 second of slowdown could cost $1.6 billion in sales
● Google found an extra .5 seconds in search page generation time
dropped traffic by 20%
Isolation from Slow Services
● Blocking I/O holds onto a thread while it blocks

● Slow downstream services, or external APIs can cause hogging.

● Sharing based upon thread pool allocation

Scaling Up
Eg: the C10K problem: can your system/architecture handle 10K connected
clients at once?

● Hard/Impossible to achieve with blocking I/O

● Lots of different approaches
● A good context for core Java improvements
Scalability problems of one thread per
request

● Context Switching
○ OS Scheduler
○ Interruptions

● Locality of reference
○ Cache locality usually the main cost of context switching
Example Asynchrony App

Account Service

Bank

Credit Checking
Service
Introduction to Non-Blocking I/O
Blocking vs Non-blocking I/O
Blocking I/O
● The process performing I/O blocks until the operation is complete
● Need as many threads as concurrent I/O operations
● Scalability issues: spend all your time flipping between threads
● Often simpler to program and debug

Non-Blocking I/O

● The I/O call returns immediately

● Enables N:M threading model
● Scales up to a high number of connections
The benefits are in the asynchronous
threading model
Real World Example: Netflix

From:
https://github.com/Netflix-Skunkworks/WSPerfLab/blob/master/test-results/RxNetty_vs_Tomcat_April2
015.pdf
To the asynchronous Bank Account
Service!
The I/O Stack

Application Code

Servlet Container

NIO

EPoll/KQueue/IO Completion Ports

Asynchronous Servlets

AsyncContext context = request.startAsync();

How to structure your application?

● Anything that performs I/O can’t eat the thread

● Blocking I/O method calls are unusable in an asynchronous

programming context

● How do you split a problem into discrete steps and then pass values
between those steps?
Callbacks
● Fundamental primitive of asynchronous programming.

● Hand your AsyncContext to some code elsewhere:

○ context.complete();

● Where you register your callback can now control its threading model
The Callback Transformation
Result result = otherSystemComponent.performOperation();

// Do something with result

otherSystemComponent.performOperation(result ->

// Do something with result

});

// Eg: AsyncCurrentAccountServlet
Exercise
● Implement a new asynchronous service
○ Grant a mortgage if
■ Requested amount to borrow <= 4 * current account balance
○ Return SC_OK (200) if granting mortgage, SC_FORBIDDEN (403)
otherwise

● Mortgage Application Service

com.iteratrlearning.problems.asynchronous.SimpleMortgageApplicationServlet

com.iteratrlearning.problems.asynchronous.SimpleMortgageApplicationServiceTest
Composing Callbacks
● Callbacks can compose - but it’s not that easy

● You can register for the result of multiple callbacks.

● Manually build a handler for combining these results.

Eg: AsyncMortgageServlet
Summary
● Register callbacks to receive responses asynchronously.

● To compose callbacks
○ create some kind of stateful handler object.
○ Have a method to for each event you want to receive
○ Emit event once you’ve got all the required state.
Summary
Summary

● Asynchronous Programming helps us get away from one thread per

request

● Great Java support

● Improved performance, isolation and testability

Hard to Read
Hard to Adopt
Hard to Debug
Approaches to Concurrency
CompletableFuture
● Promises in Java

● How do I combine and compose the results of different services?

● An I-owe-you for a value

Actors
● Concurrent Objects

● Only hold local state and communicate through message passing

● How do I structure my application to reduce the chance of concurrency

bugs?
Reactive Streams
● Extend the promises model to Streams rather than single values

● How do I combine and compose multiple results from different services, or

synchronous code?

● Also incorporate reactive-pull based back pressure

The End
Timeouts and Circuit Breakers
Timeouts and Retries Summary
● Timeout asynchronous requests to avoid leaks

● Retry a bounded number of times to resolve transient failures

● Avoid unbounded retries as this leads to non-termination issues

Servlet Timeouts

// Set a timeout on the your current operation

AsyncContext context = request.startAsync();

context.setTimeout(5000L);
AsyncHttpClient Timeouts

// Set a timeout on an HTTP request to an

external service

AsyncHttpClient client = …

client.setRequestTimeout(5000L);
Example
● Modify your MortgageApplicationServlet to retry the getBalance()
○ Retry 10 times then if things are still going wrong return Internal
Server Error (500)