Asynchronous
vs
Synchronous
Course Materials
iteratrlearning.com/async-exercises.zip
iteratrlearning.com/async-slides-1.zip
Introduction
Approaches
Programming Styles
SOA and Microservices
● Increasingly popular style of application development
● Relies extensively on communicating over networks
○ RPC
○ REST
○ Message Driven
● Massively increases the use of network communication
Thread Pools with Blocking I/O
Thread 1
Thread 2
Queue
….
One thread per request
E.g. MortgageServlet
Common Threading Model
● Used by Servlets and anything that sits on them
○ Spring
○ Jersey
● JDBC
○ Pool within JDBC connector of threads talking to DB.
○ Often hard to work around this limitation
Quiz: What problems can you think
of with this approach?
Latency
RPC 1 RPC 2 RPC 3
latency
Latency costs money
● Amazon found 1 second of slowdown could cost $1.6 billion in sales
● Google found an extra .5 seconds in search page generation time
dropped traffic by 20%
Isolation from Slow Services
● Blocking I/O holds onto a thread while it blocks
● Slow downstream services, or external APIs can cause hogging.
● Sharing based upon thread pool allocation
Scaling Up
Eg: the C10K problem: can your system/architecture handle 10K connected
clients at once?
● Hard/Impossible to achieve with blocking I/O
● Lots of different approaches
● A good context for core Java improvements
Scalability problems of one thread per
request
● Context Switching
○ OS Scheduler
○ Interruptions
● Locality of reference
○ Cache locality usually the main cost of context switching
Example Asynchrony App
Account Service
Bank
Credit Checking
Service
Introduction to Non-Blocking I/O
Blocking vs Non-blocking I/O
Blocking I/O
● The process performing I/O blocks until the operation is complete
● Need as many threads as concurrent I/O operations
● Scalability issues: spend all your time flipping between threads
● Often simpler to program and debug
Non-Blocking I/O
● The I/O call returns immediately
● Enables N:M threading model
● Scales up to a high number of connections
The benefits are in the asynchronous
threading model
Real World Example: Netflix
From:
https://github.com/Netflix-Skunkworks/WSPerfLab/blob/master/test-results/RxNetty_vs_Tomcat_April2
015.pdf
To the asynchronous Bank Account
Service!
The I/O Stack
Application Code
Servlet Container
NIO
EPoll/KQueue/IO Completion Ports
Asynchronous Servlets
AsyncContext context = request.startAsync();
How to structure your application?
● Anything that performs I/O can’t eat the thread
● Blocking I/O method calls are unusable in an asynchronous
programming context
● How do you split a problem into discrete steps and then pass values
between those steps?
Callbacks
● Fundamental primitive of asynchronous programming.
● Hand your AsyncContext to some code elsewhere:
○ context.complete();
● Where you register your callback can now control its threading model
The Callback Transformation
Result result = otherSystemComponent.performOperation();
// Do something with result
otherSystemComponent.performOperation(result ->
// Do something with result
});
// Eg: AsyncCurrentAccountServlet
Exercise
● Implement a new asynchronous service
○ Grant a mortgage if
■ Requested amount to borrow <= 4 * current account balance
○ Return SC_OK (200) if granting mortgage, SC_FORBIDDEN (403)
otherwise
● Mortgage Application Service
com.iteratrlearning.problems.asynchronous.SimpleMortgageApplicationServlet
com.iteratrlearning.problems.asynchronous.SimpleMortgageApplicationServiceTest
Composing Callbacks
● Callbacks can compose - but it’s not that easy
● You can register for the result of multiple callbacks.
● Manually build a handler for combining these results.
Eg: AsyncMortgageServlet
Summary
● Register callbacks to receive responses asynchronously.
● To compose callbacks
○ create some kind of stateful handler object.
○ Have a method to for each event you want to receive
○ Emit event once you’ve got all the required state.
Summary
Summary
● Asynchronous Programming helps us get away from one thread per
request
● Great Java support
● Improved performance, isolation and testability
Hard to Read
Hard to Adopt
Hard to Debug
Approaches to Concurrency
CompletableFuture
● Promises in Java
● How do I combine and compose the results of different services?
● An I-owe-you for a value
Actors
● Concurrent Objects
● Only hold local state and communicate through message passing
● How do I structure my application to reduce the chance of concurrency
bugs?
Reactive Streams
● Extend the promises model to Streams rather than single values
● How do I combine and compose multiple results from different services, or
synchronous code?
● Also incorporate reactive-pull based back pressure
The End
Timeouts and Circuit Breakers
Timeouts and Retries Summary
● Timeout asynchronous requests to avoid leaks
● Retry a bounded number of times to resolve transient failures
● Avoid unbounded retries as this leads to non-termination issues
Servlet Timeouts
// Set a timeout on the your current operation
AsyncContext context = request.startAsync();
context.setTimeout(5000L);
AsyncHttpClient Timeouts
// Set a timeout on an HTTP request to an
external service
AsyncHttpClient client = …
client.setRequestTimeout(5000L);
Example
● Modify your MortgageApplicationServlet to retry the getBalance()
○ Retry 10 times then if things are still going wrong return Internal
Server Error (500)
● Mortgage Application Service
com.iteratrlearning.problems.asynchronous.RetryingMortgageApplicationServlet
com.iteratrlearning.problems.asynchronous.RetryingMortgageApplicationServiceTest
