Building a Robust Java Server

Last updated Dec 2, 2005. Building a stand-alone Java Server is a simple task, but building a robust stand-alone Java Server is more complicated. Java has greatly simplified the process of building servers through the advent of the java.net.ServerSocket class. You can write a couple dozen lines of code and have a Java server up and running in about ten minutes. The process encompasses: 1. Create a ServerSocket class instance, providing it a port to listen on, and a number of backlog connections to allow to pend before rejecting connections. 2. Call the ServerSocket class's accept() method to wait for connections on the specified port. 3. The accept() method returns a Socket through which you can communicate with your client. This approach is sufficient for handling single client requests, but as soon as your server needs to accept multiple connections, the problem becomes a little more difficult. The process for handling multiple requests is fundamentally simple. It revolves around keeping the ServerSocket's accept() loop tight. Consider that when the ServerSocket's accept() method is invoked, it listens on the ServerSocket's port, occupying the thread that called it. When it receives a request, then the time spent processing the request is taken away from the threads ability to accept another connection. Therefore the answer to this problem is to very quickly pass the accepted Socket to another thread for processing. For example:
while( true ) { Socket socket = serverSocket.accept(); HandlerThread handler = new HandlerThread( socket ); handler.start(); }

In this case, the listening loop consists of three lines of code: 1. Listen for a connection. 2. Once a connection is received, create a HandlerThread, passing it the accepted Socket. 3. Start the handler thread. The loop iterates, and causes the ServerSocket to listen for the next connection. In theory, this is a good implementation. In practice, it suffices for low-traffic servers, but it won't stand up to load for two reasons:

1. The cost of creating a thread is expensive. 2. There is no upper limit on threads, so high traffic would yield an unmanageable number of threads. Consider running this code on a single CPU box trying to support 500 simultaneous users; the context switching between 500 threads would destroy performance.

Robust Java Server Architecture

Figure 78 Figure 78. The server request handling logic moves requests to a request queue that is serviced by a set of request threads. The architecture works as follows: 1. A client makes a request of the server. 2. The server receives the request, adds the request to the request queue, and then listens for the next request. 3. The request queue adds the request to an internal linked list of requests (adds to the end and removes from the beginning). 4. A waiting request thread wakes up, extracts the request from the linked list and processes it. 5. The request thread passes the request to a request handler that interacts directly with the client.

The theory is straightforward, but the implementation details get a little messy. Namely, we need to create a set of request threads and force them to wait on the queue for a new request. This is implemented using wait and notify object thread management. Listing 1 shows the code for the AbstractServer class. Listing 1. AbstractServer.java
package com.javasrc.server; // Import the Java classes import java.net.*; import javax.net.*; import java.io.*; import java.util.*; /** * Abstract super class for creating servers */ public abstract class AbstractServer extends Thread { /** * Server sock that will listen for incoming connections */ protected ServerSocket serverSocket; /** * Boolean that controls whether or not this server is listening */ protected boolean running; /** * The port that this server is listening on */ protected int port; /** * The number of requests to backlog if we are busy */ protected int backlog; /** * A Request Queue used for high throughput servers */ protected RequestQueue requestQueue; /** * Creates a new AbstractServer */ public AbstractServer( int port, int backlog, String requestHandlerClassName, int maxQueueLength, int minThreads, int maxThreads ) {

// Save our socket parameters this.port = port; this.backlog = backlog; // Create our request queue this.requestQueue = new RequestQueue( requestHandlerClassName, maxQueueLength, minThreads, maxThreads );

/** * Starts this server */ public void startServer() { try { // Create our Server Socket ServerSocketFactory ssf = ServerSocketFactory.getDefault(); serverSocket = ssf.createServerSocket( this.port, this.backlog ); // Start our thread this.start();

} catch( Exception e ) { e.printStackTrace(); } } /** * Stops this server */ public void stopServer() { try { this.running = false; this.serverSocket.close(); } catch( Exception e ) { e.printStackTrace(); } } /** * Body of the server: listens in a tight loop for incoming requests */ public void run() { // Start the server System.out.println( "Server Started, listening on port: " + this.port ); this.running = true;

while( running ) { try { // Accept the next connection Socket s = serverSocket.accept(); // Log some debugging information InetAddress addr = s.getInetAddress(); System.out.println( "Received a new connection from (" + addr.getHostAddress() + "): " + addr.getHostName() ); // Add the socket to the new RequestQueue this.requestQueue.add( s ); } catch( SocketException se ) { // We are closing the ServerSocket in order to shutdown the server, so if // we are not currently running then ignore the exception. if( this.running ) { se.printStackTrace(); } } catch( Exception e ) { e.printStackTrace(); } } System.out.println( "Shutting down..." ); // Shutdown our request queue this.requestQueue.shutdown();

} }

The AbstractServer initializes the RequestQueue with the RequestHandler implementation class name, the minimum and maximum number of request threads to create, and the maximum size to which the queue can grow. When its startServer() method is called, it uses the ServerSocketFactory to create a new ServerSocket and then starts the AbstractServer's thread, which invokes its run() method. The run() method sits in a tight loop calling the ServerSocket's accept() method and then adding the Socket that it receives to the request queue. This brings us to the RequestQueue class, shown in listing 2. Listing 2. RequestQueue.java
package com.javasrc.server; import java.util.*;

// Import our exceptions import com.javasrc.server.exception.*; /** * A Request Queue accepts new requests and processes them with its associated * thread pool */ public class RequestQueue { /** * Request queue */ private LinkedList queue = new LinkedList(); /** * The maximum length that the queue can grow to */ private int maxQueueLength; /** * The minimum number of threads in this queues associated thread */ private int minThreads; /** * The maximum number of threads that can be in this queues associated thread pool */ private int maxThreads; /** * The current number of threads */ private int currentThreads = 0; /** * The name of the request handler implementation class */ private String requestHandlerClassName; /** * The thread pool that is servicing this request */ private List threadPool = new ArrayList(); private boolean running = true; /** * Creates a new RequestQueue */ public RequestQueue( String requestHandlerClassName, int maxQueueLength, int minThreads, int maxThreads )

pool

// Initialize our parameters this.requestHandlerClassName = requestHandlerClassName; this.maxQueueLength = maxQueueLength; this.minThreads = minThreads; this.maxThreads = maxThreads; this.currentThreads = this.minThreads;

// Create the minimum number of threads for( int i=0; i<this.minThreads; i++ ) { RequestThread thread = new RequestThread( this, i, requestHandlerClassName ); thread.start(); this.threadPool.add( thread ); } } /** * Returns the name of the RequestHandler implementation class */ public String getRequestHandlerClassName() { return this.requestHandlerClassName; } /** * Adds a new object to the end of the queue * * @param o Adds the specified object to the Request Queue */ public synchronized void add( Object o ) throws RequestQueueException { // Validate that we have room of the object before we add it to the queue if( queue.size() > this.maxQueueLength ) { throw new RequestQueueException( "The Request Queue is full. Max size = " + this.maxQueueLength ); } // Add the new object to the end of the queue queue.addLast( o ); // See if we have an available thread to process the request boolean availableThread = false; for( Iterator i=this.threadPool.iterator(); i.hasNext(); ) { RequestThread rt = ( RequestThread )i.next(); if( !rt.isProcessing() ) { System.out.println( "Found an available thread" ); availableThread = true; break; } System.out.println( "Thread is busy" );

} // See if we have an available thread if( !availableThread ) { if( this.currentThreads < this.maxThreads ) { System.out.println( "Creating a new thread to satisfy the incoming request" ); RequestThread thread = new RequestThread( this, currentThreads++, this.requestHandlerClassName ); thread.start(); this.threadPool.add( thread ); } else { System.out.println( "Whoops, cant grow the thread pool, guess you have to wait" ); } } // Wake someone up notifyAll();

/** * Returns the first object in the queue */ public synchronized Object getNextObject() { // Setup waiting on the Request Queue while( queue.isEmpty() ) { try { if( !running ) { // Exit criteria for stopping threads return null; } wait(); } catch( InterruptedException ie ) {} } // Return the item at the head of the queue return queue.removeFirst();

/** * Shuts down the request queue and kills all of the request threads */ public synchronized void shutdown() { System.out.println( "Shutting down request threads..." );

// Mark the queue as not running so that we will free up our request threads this.running = false; // Tell each thread to kill itself for( Iterator i=this.threadPool.iterator(); i.hasNext(); ) { RequestThread rt = ( RequestThread )i.next(); rt.killThread(); } // Wake up all threads and let them die notifyAll();

} }

The RequestQueue initializes its thread pool with the minimum number of RequestThreads and then provides two primary methods to interacting with its internal linked list queue:
add(): adds an object to the queue getNextObject(): returns the next

item from the head of the queue

The key to making the RequestThreads work interactively with the RequestQueue is the wait() construct in the getNextObject() method. When the RequestThread starts, it calls the synchronized getNextObject() method to retrieve a request to process. While the queue is empty, the getNextObject() method calls its wait() method. When a new request is added to the queue in the synchronized add() method, it calls notifyAll() to wake up the threads and tell them to check the queue for the new request. As a bit of Java theory, the synchronized method modifier allows only one thread to execute the method at a time. Each object has a lock; to access a synchronized method, the calling thread must obtain the lock before it is allowed into the method. If another thread already has the lock, subsequent threads wait for the lock to become available. If a calling thread executes the wait() method, then it relinquishes the object's lock and goes into a waiting state. The waiting state can only be broken by a call to notify() or notifyAll() on the object upon which the thread is waiting. When the thread wakes up, it obtains the object's lock and then continues processing on the line of code after the wait() call. In this case, the request threads are waiting on the RequestQueue's lock, so in the add() method, the call to notifyAll() wakes up all threads and processing continues in the getNextObject() method by executing the queue's isEmpty() method. If an object is in the queue, it is returned to the thread, the thread relinquishes the RequestQueue's lock, and subsequent threads can check for additional objects in the queue. Listing 3 shows the code for the RequestThread class.

Listing 3. RequestThread.java
package com.javasrc.server; import java.net.*; /** * A Request thread handles incoming requests */ public class RequestThread extends Thread { /** * A reference to our request queue */ private RequestQueue queue; /** * Our state: are we running or not? */ private boolean running; /** * Our processing state: are we currently processing a request? */ private boolean processing = false; /** * Our thread number, used for accounting purposes */ private int threadNumber; /** * Our request handler */ private RequestHandler requestHandler; /** * Creates a new Request Thread * * @param queue The queue that we are associated with * @param threadNumber Our thread number */ public RequestThread( RequestQueue queue, int threadNumber, String requestHandlerClassName ) { this.queue = queue; this.threadNumber = threadNumber; try { // Create our request handler this.requestHandler = ( RequestHandler ) ( Class.forName( requestHandlerClassName ).newInstance() ); } catch( Exception e ) { e.printStackTrace();

/** * Returns true if we are currently processing a request, false otherwise */ public boolean isProcessing() { return this.processing; } /** * If a thread is waiting, then wake it up and tell it to die */ public void killThread() { System.out.println( "[" + threadNumber + "]: Attempting to kill thread..." ); this.running = false; } /** * The threads main processing loop */ public void run() { this.running = true; while( running ) { try { // Obtain the next pending socket from the queue; only process requests if // we are still running. The shutdown mechanism will wake up our threads at this // point, so our state could have changed to not running here. Object o = queue.getNextObject(); if( running ) { // Cast the object to a Socket Socket socket = ( Socket )o; // Mark ourselves as processing a request this.processing = true; System.out.println( "[" + threadNumber + "]: Processing request..." ); // Handle the request this.requestHandler.handleRequest( socket ); // Weve finished processing, so make ourselves available for the next request this.processing = false; System.out.println( "[" + threadNumber + "]: Finished Processing request..." );

} } catch( Exception e ) { e.printStackTrace(); } } System.out.println( "[" + threadNumber + "]: Thread shutting down..." ); } }

The RequestThread creates an instance of the RequestHandler to which it is going to delegate request processing. Its core functionality is in its run() method: it obtains the next object from the queue, marks itself as processing a request, passes the request to the RequestHandler, unmarks itself as a processing a request, and tries to obtain the next object from the queue. In actuality, it is just infrastructure to forward a request to its RequestHandler. This brings us to the RequestHandler interface, shown in listing 4. Listing 4. RequestHandler.java
package com.javasrc.server; /** * RequestHandler interface */ public interface RequestHandler { /** * Handles the incoming request * * @param socket The socket communication back to the client */ public void handleRequest( java.net.Socket socket ); }

And as you would guess, the RequestHandler has a single method that it uses to handle requests.

Summary
Thus far we have looked at creating the infrastructure for a robust Java server, which inclues the introduction of a request queue that is serviced by a set of request threads. Next week, we will create two sample servers built on top of this infrastructure: a simple echo server, and a more robust chat server.