Java developers tend to use ‘java.util.UUID#randomUUID()’ API, to generate a UUID (Universally Unique Identifier) number (i.e., ‘b8bbcbed-ca07-490c-8711-5118ee0af2f9’). Under certain circumstances, using this API can affect your application’s availability. Let’s discuss this API in this post with a real-world example.

How does ‘java.util.UUID#randomUUID()’ API works?
java.util.UUID#randomUUID() API internally uses ‘entropy‘ in the operating system to generate a unique number. What does ‘entropy’ mean? Linux kernel uses certain techniques like user’s mouse movements, variance in the hardware fan noise, variance in the noise of the device drivers … to generate random numbers. When there is a lack of ‘entropy’ in the operating system then random number generation will slow down. When there is a slowdown, application threads which are calling this ‘java.util.UUID#randomUUID()’ API call will be put in a BLOCKED state, and they wouldn’t be able to progress further.

If your application uses ‘java.util.UUID#randomUUID()’ API in a critical code path and there is a lack of entropy in the operating system, then multiple threads can enter into this BLOCKED state bringing your entire application to a grinding halt.

Real world application – 50 threads BLOCKED in java.util.UUID#randomUUID() API
Here is a real-world thread dump report of an application that was suffering from this problem. If you haven’t clicked on the hyperlink in the previous sentence, we request you do so. It would give the better context of the problem. (Note: in the thread dump report, we have changed the package name to ‘buggycompany’ to hide the identity of the application).

In the thread dump report, you can notice that there are 102 threads in total. In these 102 threads 50 threads are in the BLOCKED state due to ‘java.util.UUID#randomUUID()’ API call. Below is the stack trace of one of that 50 threads:

"[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock 
java.security.SecureRandom@20a56b2b BLOCKED
 
java.security.SecureRandom.nextBytes(SecureRandom.  java:433)
java.util.UUID.randomUUID(UUID.java:159)
com.buggycompany.jtm.bp.<init>(bp.java:185)
com.buggycompany.jtm.a4.f(a4.java:94)
com.buggycompany.agent.trace.RootTracer.topCompone  ntMethodBbuggycompanyin(RootTracer.java:439)
weblogicx.servlet.gzip.filter.GZIPFilter.doFilter(  GZIPFilter.java)
weblogic.servlet.internal.FilterChainImpl.doFilter  (FilterChainImpl.java:56)
weblogic.servlet.internal.WebAppServletContext$Ser  vletInvocationAction.wrapRun(WebAppServletContext.  java:3730)
weblogic.servlet.internal.WebAppServletContext$Ser  vletInvocationAction.run(WebAppServletContext.java  :3696)
weblogic.security.acl.internal.AuthenticatedSubjec  t.doAs(AuthenticatedSubject.java:321)
weblogic.security.service.SecurityManager.runAs(Se  curityManager.java:120)
weblogic.servlet.internal.WebAppServletContext.sec  uredExecute(WebAppServletContext.java:2273)
weblogic.servlet.internal.WebAppServletContext.exe  cute(WebAppServletContext.java:2179)
weblogic.servlet.internal.ServletRequestImpl.run(S  ervletRequestImpl.java:1490)
weblogic.work.ExecuteThread.execute(ExecuteThread.  java:256)
weblogic.work.ExecuteThread.run(ExecuteThread.java  :221)

Fig: Stack trace of a thread stuck while making ‘java.util.UUID#randomUUID()’ API call

You can notice that the thread got into a BLOCKED state when invoking ‘java.util.UUID#randomUUID()’ due to a lack of ‘entropy’ and unable to progress forward. Due to that 50 threads got stuck. Thus it was making the application unresponsive.

Checking entropy status in Linux
To check the availability of entropy in Linux execute the below command:

cat /proc/sys/kernel/random/entropy_avail

If you see the value to be less than 1000, then it indicates there is a lack of entropy. It may lead to BLOCKED threads in the applications.

Potential Solutions
If this problem surfaces in your application, the following are the potential solutions to address them:

1. RHEL
This problem has been resolved in RHEL 7 and above versions. If you can upgrade to RHEL 7 or above version, please do so.

If you are running on the older version of RHEL, you can follow the recommendations given here to fix this problem

2. Install Haveged in Linux
If your application is running in Linux, then you consider installing the ‘haveged’ library. The ‘haveged project‘ is meant to provide an easy-to-use, unpredictable random number generator based upon an adaptation of the HAVEGE algorithm. Here is the ‘Haveged’ project GIT repository page Here is how you can install it:

On Debian based platforms (Debian, Ubuntu):

sudo apt-get install rng-tools
sudo update-rc.d haveged defaults

On Redhat platforms (RHEL, Fedora, CentOS):

sudo yum install rng-tools
sudo chkconfig haveged on

3. Use /dev/urandom instead of /dev/random
Unix-like operating systems come up with special file ‘/dev/random’ that serve as pseudorandom number generators. Java uses this file to generate random numbers. You can configure it to use ‘/dev/urandom’ instead of ‘/dev/random’.

‘/dev/urandom’ is another special file that is capable of generating random numbers. However, it has the downside of reduced security due to less randomness. You can achieve it by passing the following JVM argument to your application during startup:

-Djava.security.egd=file:/dev/./urandom

Video
https://youtu.be/8Q_injWDm9M