‘java.lang.System.getProperty()’ is a common API used by Java developers to read the System properties that are configured during application startup time. i.e. when you pass “-DappName=buggyApp” as your application’s startup JVM argument, the value of the ‘appName’ system property can be read by invoking the ‘java.lang.System.getProperty()’. Example:

public static String getAppName() {    
 
    String app = System.getProperty("appName"); * 
    return app;
}

When the above method is invoked, ‘buggyApp’ will be returned. However, if ‘java.lang.System.getProperty()’ is used in a critical code path, it has the potential to degrade the application’s performance. In this post let’s discuss what is the performance impact of invoking ‘java.lang.System.getProperty()’, how to mitigate it, and a real-world production problem triggered by this API.

What Is the Performance Impact of Using ‘java.lang.System.getProperty()’ API?
‘java.lang.System.getProperty()’ API underlyingly uses ‘java.util.Hashtable.get()’ API. Please be advised that ‘java.util.Hashtable.get()’ is a synchronized API. It means only one thread can invoke the ‘java.util.Hashtable.get()’ method at any given time. If a new thread tries to invoke ‘java.util.Hashtable.get()’ API when the first thread is still executing it, the new thread will be put in a BLOCKED state. When a thread is in the BLOCKED state, it won’t be able to progress forward. Only when the first thread completes executing the ‘java.util.Hashtable.get()’ API, a new thread will be able to progress forward. Thus if ‘java.lang.System.getProperty()’ or ‘java.util.Hashtable.get()’ is invoked in critical code paths, it will impact the response time of the transaction.

Real-World problem in Atlassian SDK
Recently this type of degradation was observed in Atlassian SDK. The thread dump was captured from this application and analyzed using the thread dump analysis tool fastThread

According to the thread dump analysis report, 189 threads were in the BLOCKED state. Below is the transitive dependency graph from the thread dump report showing the names of the threads that are in the BLOCKED state. When you click on the thread name in the graph, that particular thread’s stack trace will be reported in the tool.


All these threads entered into the BLOCKED state because of the ‘Camel Thread #6 – backboneThreadPool’ (i.e., Red color node in the graph). Here is the initial few lines of this thread’s stack trace:

Camel Thread #6 – 
backboneThreadPool 
Stack Trace is: 
at java.util.Hashtable.get(Hashtable.java:362) 
- locked <0x0000000080f5e118> (a java.util.Properties) 
at java.util.Properties.getProperty(Properties.java:969) 
at java.util.Properties.getProperty(Properties.java:988) 
at java.lang.System.getProperty(System.java:756) 
at net.java.ao.atlassian.ConverterUtils.enforceLength(ConverterUtils.java:16) 
at net.java.ao.atlassian.ConverterUtils.checkLength(ConverterUtils.java:9) 
:
:

Fig: Stack trace of the thread which acquired the LOCK

From the stack trace, you can notice that this thread was invoking the ‘java.lang.System.getProperty()’ API. Since ‘java.lang.System.getProperty()’ API underlyingly uses ‘java.util.Hashtable.get()’ API (which is a synchronized API call). Thus ‘Camel Thread #6 – backboneThreadPool’ will be the only thread allowed to enter this method. Below are the initial few lines of couple of threads (out of 189 threads) who are in BLOCKED state because they are waiting to enter the ‘java.util.Hashtable.get()’ API.

http-nio-8080-exec-293 
Stack Trace is: 
java.lang.Thread.State: BLOCKED (on object monitor) 
at java.util.Hashtable.get(Hashtable.java:362)
 - waiting to lock <0x0000000080f5e118> (a java.util.Properties) 
at java.util.Properties.getProperty(Properties.java:969) 
at java.util.Properties.getProperty(Properties.java:988) 
at java.lang.System.getProperty(System.java:756) 
at net.java.ao.atlassian.ConverterUtils.enforceLength(ConverterUtils.java:16) 
at net.java.ao.atlassian.ConverterUtils.checkLength(ConverterUtils.java:9) 
:
:

Fig: Stack trace of one of the BLOCKED threads waiting for the LOCK

http-nio-8080-exec-279 
Stack Trace is: 
java.lang.Thread.State: BLOCKED (on object monitor) 
at java.util.Hashtable.get(Hashtable.java:362) 
- waiting to lock <0x0000000080f5e118> (a 
java.util.Properties) 
at java.util.Properties.getProperty(Properties.java:969) 
at java.util.Properties.getProperty(Properties.java:988) 
at java.lang.System.getProperty(System.java:756) 
at org.ofbiz.core.entity.EntityFindOptions.<init>(EntityFindOptions.java:124) 
:
:

Fig: Stack trace of another BLOCKED thread waiting for the LOCK

Since this ‘java.lang.System.getProperty()’ API was present in the critical code path, multiple threads were trying to invoke it. Thus, all those 189 threads which were trying to invoke this API were put into the BLOCKED state. Ultimately, the overall application response time was degraded.

What is the Solution?
Here are the potential solutions to address this problem:

1. Upgrade to JDK 11:

Starting from JDK 11, Synchronized ‘HashTable’ has been replaced with ‘ConcurrentHashMap’ in java.util.Properties. Thus when you upgrade to JDK11, you will not run into this problem.

2. Caching the values:

In a significant number of applications, System properties don’t change during runtime. In such circumstances, we don’t have to keep invoking ‘java.lang.System.getProperty()’ API on every single transaction. Rather, we can invoke ‘java.lang.System.getProperty()’ API once, cache its value, and return the cached value on all future calls, as shown in the below code snippet.

private static String app = System.getProperty("appName"); *
 
public static String getAppName() { * *
    return app;
}

If you notice the above code, ‘’java.lang.System.getProperty()’ is now assigned to a static member variable. It means this API will be called during application startup time, that too only once. From that point, if anyone invokes the getAppName() API, he will be returned the cached value. Thus, application threads will not be put into the BLOCKED state at runtime. This simple change can improve the application’s overall response time.