In this blog post I'd like to describe the way that I used recently when solving a problem with limiting allowed concurrent requests for JAX-WS service.
The problem
The application that I was developing together with my team was exhibiting an interface with operation that was expected to consume quite a lot of time. The client of this interface was expecting the response to arrive in some defined time or it was not interested in response. From the multitude of the clients, no client was issuing concurrent requests, always one at a time. For the realization of the interface we chose SOAP-based JAX-WS web service where the response was sent to the client synchronously. This means, that the client was waiting for the response or the request timed out. To not overwhelm server with long lasting requests, there was defined maximum number of requests that could be handled concurrently. All the requests over that number shall be rejected imediatelly, informing client to try again later.
The environment
The environment in which the solution was provided consisted of JBoss EAP 5 with JBoss Native WS. The requests were going through JBoss Web Server. In addition to this web service, there were other services based on HTTP and passing the same route. This disallowed me to use request thread limitation in JBoss Web Server as this would effectivelly lead to limitations on all web interfaces.
The solution design
The solution that I decided for was to use JAX-WS Handler Framework to intercept all requests and calculate actual throughput. The flow of messages was designed to be:
- Request comes to server
- Request is is passed to Throughput Limitation Handler that checks actual number of processed requests
- if it is lower than maximum, increases request counter and passes the request further to endpoint indicating non-blocked request
- if it is higher than maximum, increases request counter and passes the request further to endpoint indicating blocked request
- The enpoint receives request and checks the blocking indication
- if the request is blocked, it constructs failure message by throwing appropriate exception (Fault)
- if the request is not blocked, it is processed and
- response is returned
- failure message is constructed when expiration occurs (Fault)
- For both success or failure, the response passes Throughput Limitation Handler on way back. It decreases the actual request counter.
The Throughput Limitation Handler does not constructs responses when limit is hit. The responsibility of this component is only to guard actual requests and indicate whether limit was reached. It is responsibility of the endpoint to construct appropriate response based on all factors that might influence it (they can be possibly other handlers guarding different aspects).
The solution implementation
The implementation of this solution is based on existing WSDL file that forms contract between clients and server. So the basic artifacts are generated by wsconsume utility. The artifacts that I need to add are:
- Throughput Limitation Handler - guards number of requests processed
- WS Endpoint (Limited Service Session) - processes the request and constructs responses.
I am going to present the solution on an artificial service called Limited Service that actually does nothing but transforms string input to all-upper-case.
Throughput Limitation Handler
The handler as noted above is implemented using JAX-WS handler framework. The (simplified) code for it is listed here. The code does not need much further explanations besides those in the code comments. To sumarize it:
- The handler implements LogicalHandler user to install logical interceptor on the message processing route. It is logical, because it does not want to modify message contents.
- In constructor, it initializes the actual request counter.
- It implements 2 important methods
- handleMessage - this method handles both incomming and outgoing messages when no error ocurs. First, it must be determined what direction given message has and do appropriate handling (counter increase/decrease, put indication into message context for application)
- handleFault - this method handle only cases when error occurs. In this case, handleMessage is skipped for outgoing direction. The logic here only decreases the request counter.
Complete ThroughputLimitHandler source
public class ThroughputLimitHandler implements
LogicalHandler<LogicalMessageContext> {
/**
* The name for parameter that indicates state of traffic limitation to web
* service endpoint.
*/
public static final String LIMIT_REACHED = "LIMIT_REACHED";
...
/** The actual maximum parallel requests. */
private int maxParallelRequests = DEFAULT_MAX_PARALLEL_REQUESTS;
/** The actual parallel requests. */
private AtomicInteger actualParallelRequests;
/** The lock that guards changes to actual parallel requests. */
private final Lock lock;
/**
* Instantiates a new throughput limit handler.
*/
public ThroughputLimitHandler() {
actualParallelRequests = new AtomicInteger(0);
lock = new ReentrantLock();
...
}
@Override
public boolean handleMessage(LogicalMessageContext context) {
// Get the direction of the message
Boolean outbound = (Boolean) context
.get(LogicalMessageContext.MESSAGE_OUTBOUND_PROPERTY);
if (outbound) {
// Response, decrease counter only
try {
lock.lock(); // block until condition holds
int actual = actualParallelRequests.decrementAndGet();
...
} finally {
lock.unlock();
}
} else {
// Request, do checks
lock.lock(); // block until condition holds
try {
if (actualParallelRequests.get() < maxParallelRequests) {
// We could pass the request further for processing
context.put(LIMIT_REACHED, Boolean.FALSE);
context.setScope(LIMIT_REACHED, Scope.APPLICATION);
} else {
context.put(LIMIT_REACHED, Boolean.TRUE);
context.setScope(LIMIT_REACHED, Scope.APPLICATION);
}
// Increment counter, it must be known how many requests are
// actually handled
actualParallelRequests.incrementAndGet();
} finally {
lock.unlock();
}
}
return true;
}
@Override
public boolean handleFault(LogicalMessageContext context) {
// Failure response, decrease counter only
try {
lock.lock(); // block until condition holds
int actual = actualParallelRequests.decrementAndGet();
} finally {
lock.unlock();
}
return true;
}
}
In the complete code that you can download (see Takeaway) there is more than what is listed here. In addition to this, there is added internal marking of requests to see which requests were accepted or rejected and also some logging to see what is happening. It also contains configuration for maximum throughput using system parameter named request.limit, otherwise default maximum is used.
Limited Service Session
The implementation of the endpoint is quite straightforward. Lets see the code first.
Complete LimitedServiceSession source
@WebService(name = "LimitedService", ...)
@HandlerChain(file = "resource://META-INF/jaxws-handlers.xml")
@Stateless
public class LimitedServiceSession implements LimitedService {
/** Web service context. */
@Resource
WebServiceContext wsCtx;
public String limitedServiceRequest(String parameter) {
Boolean limit = (Boolean) wsCtx.getMessageContext().get(
ThroughputLimitHandler.LIMIT_REACHED);
if (limit) {
return null;
}
String upperCase = parameter.toUpperCase();
return upperCase;
}
}
So, here we have web service endpoint. Note important annotation @HandlerChain that defines what handlers shall be used to intercept message targeted to here. Apparently, it contains reference to Throughput Limitation Handler. I'll show the file later. Then, it contains the only method defined by WSDL which first uses web service contect to ask for LIMIT_REACHED parameter. Based on that it decides what to do next. If it is true, then it ends the processing. If it is false, it goes for possibly long lasting processing and returns result of it. The expiration of the processing is not illustrated here as it is not important for the traffic limitation now.
Handlers configuration
You saw above, that the endpoint referenced configuration file name jaxws-handlers.xml. The contents of this file is following:
<?xml version="1.0" encoding="UTF-8"?> <jws:handler-chains xmlns:jws="http://java.sun.com/xml/ns/javaee"> <jws:handler-chain name="HandlerChain"> <jws:protocol-bindings>##SOAP11_HTTP</jws:protocol-bindings> <jws:handler> <jws:handler-name>Throughput Limit Handler</jws:handler-name> <jws:handler-class>martinhynar.blog.ThroughputLimitHandler </jws:handler-class> </jws:handler> </jws:handler-chain> </jws:handler-chains>
It defines handler chain for SOAP binding where our Throughput Limitation Handler is mentioned. Web service provider processes this file and passes all requests through it when Limited Service is contacted.
Other artifacts in the companion code
In addition to this traffic limitation implementation, there is also implementation of the client provided in the attached source codes. The client has form of JBoss bean and is published also as MBean which can be accessed using JMX Console. It contains single method to invoke it and send given amount of requests to Limited Service endpoint. The messages are of course sent in parallel to see traffic limitation effect.
Takeaway
In this post, complete codes were not listed, but you can still get them. You can checkout complete codes as mavenized Eclipse project here. The built example project is verified to work on JBoss AS 6.1.0 with CXF web service stack. (This is little different than I was mentioning in original environment description, but as you can see in the code, the implementation does not use any JBoss web service extensions except for WebContext annotation to bind endpoint to given context URL). After deploying the application, you can find client MBean in JMX Console under name martinhynar.blog:service=LimitedServiceClient.