Active Sessions count is insanely high

[mrgordonz]

Hi JBoss Gurus,

I manage a number of hosted instances of an enterprise application which uses JBoss 4.2.2 as the application server.  I occasionally get asked how many concurrent users there are for a particular customer, in which case I generally log on to the server and browse to:

http://localhost:8180/jmx-console/HtmlAdaptor?action=inspectMBean&name=jboss.web:host=localhost,path=/<context-root-masked>,type=Manager

This gives me the activeSessions, as well as a number of other useful details regarding sessions.  However, for one particular customer, the activeSession count has gone completely mad.  This particular customer has a total user population count of roughly 3000, and I would normally expect to see roughly 200 concurrent users at any one time.  Instead, activeSessions is showing values like 12,000 plus.  Which is impossible because it is 4 times the total user population.  In addition, I use ManageEngine App Manager to generate reports each month on availability, active sessions, etc.  App Manager simply uses the above MBean to get the activeSession count every 5 minutes, and as you can imagine the activeSession reports for this customer are all screwed up at the moment.

This weird behaviour all started on October 25, 2010.  I applied a number of application patches, and ever since then, the session count has been stuffed.  I have asked the application vendor to investigate the patches, and they assure me that the patches are not responsible.

I have attached a few screen shots of the active session count - you can see that for most of October it looks pretty normal, then in November, it all goes pear-shaped.  The change started from October 25 - you can see in the screen shot called activeSessions.png a clear jump right before October 26.

I would appreciate any suggestions/advice/clues which will help me to figure out why JBoss thinks there are many thousands of activeSessions when I know there can't be.  Are there any settings in JBoss which might be causing this weirdness?  None of the other customers for whom I manage the hosted environments are experiencing this issue, and their environments are substantially the same (same OS, same version of the application, same version of Java).  The only substantial difference between any two customer environments is the specific application patches which have been applied.

In case it is relevant, the environment details are:

Windows Server 2003 SP2 (32-bit)

JBoss 4.2.2

Apache 2.2.11

Sun JDK 1.5.0_18

4 GB RAM

Thanks in advance,

Paul Hobbs

[nickarls]

Are you aware of any network reconfiguration around that time (firewalls leaving connections hanging etc)?

[mrgordonz]

Hi Nicklas,

None that I am aware of.  All the hosted customers share the same network (firewalls, etc).  Each customer has dedicated virtual servers, but the underlying physical infrastructure is shared.  If it was a network issue I would expect the same issue to be present in other customer environments.

Cheers,

Paul