I've stress tested under a much larger load (200 concurrent) and didn't lose anything. Of course I'm using 3.0.3, not 2.4.4.
I upgraded to the latest version of JBoss. I also switched servlet containers from Tomact to Jetty(the default web container in Jboss 3.0.3). I still get the same problem with 100 concurrent users. I have no idea whats going on. The same exact load script works perfect with 50 concurrent users. I also inspected the results of the HTTP requests and they are whats expected for 50 concurrent users.
what does your load script do in particular? Generally jboss will handle this load no problems, so possibly it is some specific problem
Basically one iteration of the script has the following actions:
1) Login to the system(This is where the user's session gets established).
2) Pull the logged in user's tasks.
Thats it. Its the most basic function in the system.
I am now thinking maybe I have a non-serializable object stored in the session. I will have to bind a HTTPSessionListener and see if that happens during this script. Would that make a difference? Will JBOss invalidate the entire session if a non-serializable object is added. But it seems like the session lives, only the session variables are set to null. Are they getting garbage collected?
Also, the JBoss clustering service is on with one node. Don't know if that matters.
hmm there sounds like there is a lot of things that could be going on here. I would recommend cutting out features and retesting, then adding them back in gradually until you find out which step exactly is causing the problems.
Also turn any features like clustering off and try again. A bit painful to test maybe, but it seems there are a lot of variables there.
I stumbled on a solution. The problem is not with the Application Server.
I was not accurate when I said that there are only two requests in the script.
1) The first request authenticates the user in the database. This is an SSL request. This is where the session is established.
2) The second request is just a router that goes from the Secure to the non-secure domain.This request hydrates the user's session with information.
3)And the third request actually pulls the user's tasks from the database.
Now my load tester OpenSTA spawns multiple users that execute this same script. Each iteration of the script is in the context of a session. As the load on the server increase, the responsiveness of the http requests sent by the tester decreases. There is a specific wait period betwwen each request. What happens is that the second hydrating request is in execution when the tester sends the third request to check the tasks. When this third request arrives we got a problem. Essetially the third thread expects certain data to be present in session, but its not there cause the earlier thread is not done. This would not happen in a real world scenario cause the request to check the tasks cannot be sent till the page with that link is loaded. Anyways to make the long explanation short- Here is the solution I came up with-
1) Have a filter make sure that there is no other thread already busy with a particular session. This makes the session single threaded.. Again the user's session is single threaded and so the third thread cannot be processed. There is no performance penalty as each user's session should have only one request being processed anyways.
I'd be interested to know how high you can push the users now then?
shouldn't your script just ensure that requests that form part of the same session are not executed in parallel, but one after the other, as in a normal browser/servlet interaction ?
Not testing in this way, and adding filters and extra contention will skew any results that you might get, so that they end up telling you little about how your app will behave in the real world.
You are absolutely right. I am trying to figure out how to do that in OpenSTA. So far I can only introduce a WAIT between each request that is part of a session. This however, fails cause if the response time for the request exceeds that of the wait, you got parallel threads in the same session.
Also, the filter is not really a solution, its a hack for load testing purposes. But I just found out that parallel threads is only one of the reasons for this failure. Another is the one where if the login request times out which it does for 100 concurrent users, cause Windows 2000 Professional just stops accepting connections(is that a normal threshold or I have some settings messed up?), then the script is not smart enough to terminate the session requests but keeps making subsequent requests which ofcourse fail since the login never happened.... Bottom line is that I need to fix my load testing tool.
Do you recommend any open source load testing tools. It has to be free of charge!!!!
Tried JMeter but I like the reporting capabilities of OpenSTA better. I don't even know if JMeter will solve this problem.
Any help will be greatly appreciated...