-
1. Re: Messages get stuck in queue
byorn Nov 29, 2006 1:22 AM (in response to victoriaonsand)We are facing the same problem.
If anyone has an idea about this, can you please suggest something? -
2. Re: Messages get stuck in queue
rhino247365 Nov 29, 2006 5:31 AM (in response to victoriaonsand)Did you check the logs to determine if the MDB was still active. In extreme cases, like deadlock on database or huge load due to redelivery, it can freeze and fallover..?
-
3. Re: Messages get stuck in queue
byorn Nov 29, 2006 6:06 AM (in response to victoriaonsand)Thanks for the reply.
JBoss runs in cluster mode as this issue exists in the production environment. We often have noticed that the jms queue is increasing and gets stuck, especially when there is a high load of traffic.
We use JMS to send emails to our customers. We use ftp server to store the attachements. When our queue is stuck, / or when the jms msgs have increaased we restart the ftp service.
When we restart the ftp service the queue gets cleared.
Do you suspect anything?
Please advise. -
4. Re: Messages get stuck in queue
victoriaonsand Nov 29, 2006 6:11 AM (in response to victoriaonsand)The server.log looked fine. The last debug output from the relevant MDB was a message of an succesfully processed message. There wasn't any indication that anything went wrong.
Should I use special settings or log levels in order to get more information about the status of the MDB? -
5. Re: Messages get stuck in queue
byorn Nov 29, 2006 7:50 AM (in response to victoriaonsand)Victoria for what perpose do you use JMS?
We use it to send emails. We are also accessing an FTP server.
What does your MDB do? -
6. Re: Messages get stuck in queue
victoriaonsand Nov 29, 2006 9:47 AM (in response to victoriaonsand)The MDB takes the messages, parses them and calls a Corba-Services with message data as argument. It is simliar as an remote procedure call of the Corba-Service.
The messages transfered via JMS are of type javax.jms.MapMessage. We do not use a ftp server.
There are 7 similar MDBs and 7 queues plus some additional queues like DLQ and error queue. -
7. Re: Messages get stuck in queue
byorn Nov 30, 2006 12:20 AM (in response to victoriaonsand)Have you tested the remote procedure call to the Corba service with multiple threads?
i.e Simulataneous calls to the rpc Corba service ???
Our MDS all access the ftp service, and then our JMS Queue gets stuck.
But when we restart the ftp service , the messages get clear. We suspect it could be due to simulataneos accessing of the FTP Server, that creates a locking situation, thus making the JMS Queue to hang. Nevertheless, ideally the JMS Queue should not hang.
Presently we are writing tests programs with multiple threads, each creating a JMS Message. And we have discovered that the JMS Queue is getting blocked and not getting cleared??
Are you performing any tests to rectify this issue?? If so could you share it with us too ??? -
8. Re: Messages get stuck in queue
victoriaonsand Nov 30, 2006 7:27 AM (in response to victoriaonsand)I am quite sure the Corba-Services are thread safe. They are not the origin of the Problem.
We made some tests with thousand of messages sent to the message queues from multiple Clients in parallel. Most of the time all went fine. In let's say 1 of 20 bulk tests the failure situation occurs. There is no indication why this happens. There has to be a buggy module inside the JMS subsystem.
We have to investigate more effort in order to solve or workaround this problem. Meanwhile we have to restart the server when the problem occurs. -
9. Re: Messages get stuck in queue
byorn Nov 30, 2006 7:37 AM (in response to victoriaonsand)Are you testing this in a Clustered Environment?
-
10. Re: Messages get stuck in queue
victoriaonsand Nov 30, 2006 8:16 AM (in response to victoriaonsand)Yes, we use a cluster of 2 nodes. But I'am quite sure that the problem will occur in a non clustered environment.
-
11. Re: Messages get stuck in queue
mukoline Feb 23, 2007 4:14 AM (in response to victoriaonsand)I have the same problem and I'm currenlty able to continue processing by only stopping and restarting delivery of the queue.
That's not an acceptable solution for us though.
Is there an entry in JIRA for that? Or may be upgrading to a higher jboss version or downgrading to former concurrent.jar fixes it? -
12. Re: Messages get stuck in queue
mukoline Feb 27, 2007 2:36 AM (in response to victoriaonsand)In order to help other people facing the same problem, we got it working by ONLY swithching to Jboss Messaging (instead of JBOSS MQ).
-
13. Re: Messages get stuck in queue
dpabhay Feb 19, 2008 9:43 AM (in response to victoriaonsand)Similar Issue
Using DAO in MDB processing[dao connection pool different than jms connection pool]:
MDBs receive some messages initially and then stop receiving messages from JMS. And after some time there is application deadlock when the MDB that are processing message attempts to acquire DB connection through the DAO layer.
org.jboss.util.NestedSQLException: Transaction is not active: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=Contentuat2/177580, BranchQual=, localId=177580]; - nested throwable: (javax.resource.ResourceException: Transaction is not active: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=Contentuat2/177580, BranchQual=, localId=177580])
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:79)
at com.fxcd.dao.impl.oracle.BaseOracleDaoImpl.getTxConnection(Unknown Source)
at com.fxcd.dao.impl.oracle.RepositoryDocumentDaoImpl.updateMessageForQueuedDocument(Unknown Source)
at com.fxcd.service.RepositoryServiceEJB.updateMessageForQueuedDocument(Unknown Source)
at sun.reflect.GeneratedMethodAccessor218.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.invocation.Invocation.performCall(Invocation.java:345)
at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:214)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:149)
at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:154)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:106)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:363)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:166)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:153)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
at org.jboss.ejb.plugins.CleanShutdownInterceptor.invoke(CleanShutdownInterceptor.java:264)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:624)
at org.jboss.ejb.Container.invoke(Container.java:873)
at org.jboss.ejb.plugins.local.BaseLocalProxyFactory.invoke(BaseLocalProxyFactory.java:415)
at org.jboss.ejb.plugins.local.StatelessSessionProxy.invoke(StatelessSessionProxy.java:88)
at $Proxy301.updateMessageForQueuedDocument(Unknown Source)
at com.fxcd.mdb.RepositoryServiceMDB.onMessage(Unknown Source)
at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.invocation.Invocation.performCall(Invocation.java:345)
at org.jboss.ejb.MessageDrivenContainer$ContainerInterceptor.invoke(MessageDrivenContainer.java:475)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:149)
at org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor.invoke(MessageDrivenInstanceInterceptor.java:101)
at org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:48)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:106)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:335)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:166)
at org.jboss.ejb.plugins.RunAsSecurityInterceptor.invoke(RunAsSecurityInterceptor.java:94)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.MessageDrivenContainer.internalInvoke(MessageDrivenContainer.java:389)
at org.jboss.ejb.Container.invoke(Container.java:873)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.invoke(JMSContainerInvoker.java:1077)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker$MessageListenerImpl.onMessage(JMSContainerInvoker.java:1379)
at org.jboss.jms.asf.StdServerSession.onMessage(StdServerSession.java:256)
at org.jboss.mq.SpyMessageConsumer.sessionConsumerProcessMessage(SpyMessageConsumer.java:904)
at org.jboss.mq.SpyMessageConsumer.addMessage(SpyMessageConsumer.java:160)
at org.jboss.mq.SpySession.run(SpySession.java:333)
at org.jboss.jms.asf.StdServerSession.run(StdServerSession.java:180)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
at java.lang.Thread.run(Thread.java:534)
Caused by: javax.resource.ResourceException: Transaction is not active: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=Contentuat2/177580, BranchQual=, localId=177580]
at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:331)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:462)
at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:894)
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:73)
--------------------------
DAO Datasource configuration
----------------------------
JBoss 4.0.3
JMS (Singleton across 2 cluster nodes)
Sun X2100 Dual core
Oracle 9i
<no-tx-datasource>
<jndi-name>dao-datasource</jndi-name>
<connection-url>@oracle.database.dburl@</connection-url>
<driver-class>oracle.jdbc.driver.OracleDriver</driver-class>
<user-name>@oracle.database.username@</user-name>
@oracle.database.password@
<max-pool-size>100</max-pool-size>
<blocking-timeout-millis>50000</blocking-timeout-millis>
<valid-connection-checker-class-name>
org.jboss.resource.adapter.jdbc.vendor.OracleValidConnectionChecker
</valid-connection-checker-class-name>
<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</exception-sorter-class-name>
<track-statements>false</track-statements>
</no-tx-datasource>