We are experiencing what seems to be the same problem. The environment is also similar:
Red Hat AS Linux kernel 2.6.12-10-386
Java HotSpot(TM) Server VM 1.5.0_06-b05, Sun Microsystems
JBoss 4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231752)
The MDB stops processing messages after processing 1,000 or so messages. After that, the queue has 0 listerners.
It would be great if someone could shed some light.
Does it stop processing during normal operation or are there any reboots between the time the first message is created and the processing stops?
After a few thousand messages processed, I see the queue with 0 listeners:
$ ./twiddle.sh get jboss.mq.destination:name=taskQueue,service=Queue ReceiversCount ReceiversCount=0 $
No relevant exceptions or errors.
A ThreadDump shows me all my workers threads are waiting:
Thread: JMS SessionPool Worker-22 : priority:5, demon:true, threadId:161, threadState:TIMED_WAITING, threadLockName:EDU.oswego.cs.dl.util.concurrent.LinkedNode@147aeec java.lang.Object.wait(Native Method) EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353) EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723) org.jboss.jms.asf.StdServerSessionPool$MyPooledExecutor.getTask (StdServerSessionPool.java:397) EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747) java.lang.Thread.run(Thread.java:595)
That's all I can see right now (without the flashlight). I'll try to put some tracing, as Scott suggested.
We're using jboss 4.0.3SP1, that uses concurrent.jar version 1.3.4, as shown in its manifest:
$ jar xf concurrent.jar $ cat META-INF/MANIFEST.MF Manifest-Version: 1.0 Specification-Title: util.concurrent Built-By: JBoss Inc. (www.jboss.org) Ant-Version: Apache Ant 1.6.2 Created-By: 1.4.2_06-b03 (Sun Microsystems Inc.) Implementation-Title: util.concurrent Specification-Version: 1.3.4 Specification-Vendor: oswego.edu Implementation-Vendor-Id: http://www.jboss.org/ Implementation-Version: 1.3.4 compiled: February 17 2005 Implementation-Vendor: http://gee.cs.oswego.edu/dl/classes/EDU/oswego/ cs/dl/util/concurrent/intro.html Implementation-URL: http://www.jboss.org/
So, this doesn't seem to be it. But thanks!
BTW, we've put tracing in jms and mq. It generates lot's of log, I'll post if I find something relevant.
We downgraded to jboss 3.2.8SP1. It seems to be working.
We'll be using this version, for now.
Thanks for the help, anyway.