Transaction Timeout and Locking issue
armorris007 Mar 15, 2005 6:23 AMMy first posting on this forum - so please be nice! :)
I'm currently trying to debug our application, and I'm having problems interpreting the condition under which the following stack trace occured. The stack trace is enclosed below. We are using JBoss 2.4.3 in an EJB environment - using BMP and CMT.
The first point: At various times during the execution of the application, a message in the logs is displayed showing that an EJB transaction has timed out - as seen in the first 4 lines of the stack trace. This message can appear on it's own multiple times throughout the lifetime of the application OR in this instance can be accompanied by a transaction rollback message.
The log messages seem to implicate the "LoginP" session bean as being the owner of the transaction HOWEVER, I do not believe that this bean really is the owner of the transaction that is timing out. Having done some testing around this area - I'm able to create a test session bean running with "Required" type transactions - call it directly from a JSP page and have the transaction run beyond the TransactionTimeout period. I then see that the transaction times out YET the log message implicates a different bean altogether which was in no way involved in the thread of execution that I invoked. I'm guessing Log4J has screwed up here somehow?
Second issue - the rolled back stack trace. I'm having difficulty understanding the underlying mechanism that led to this occurance. Obviously from the trace transaction 20790 timed out and rolled back - and we get a nice stack trace explaining where in the code it was. YET, if I follow the code through from the tag that invoked the process through to the point at which the call to DocumentE.getObject() is made, I see no new transactions started - so why was it waiting on another transaction?
I guess the line in the stack trace "QueuedPessimisticEJBLock.waitForTx" - implies that the transaction that rolled back was in the queue - awaiting the completion of another "transaction". If this is the case then is it true to say that between the start of my thread of execution "doStartTag" and the point at which the code getObject() on DocumentE was called - there has been another transaction started which has not completed?
OR could this transaction be timing out because it wants to access a bean which is already in use but in use by another thread perhaps?
My understanding of this queue and how it is used in a multithreaded request scenary is somewhat sparse (can anyone point me to a good reference as to how it works and it's correlation with database locking).
My key question is - what COULD have occured in order for the transaction to be in the queue and in the waitForTx method?
I guess the main problem I am having is that because the timedout log messages implicate the wrong bean (i.e. LoginP) - I've no way of knowing which bean started the transaction that led to the second transaction timing out AND then rolling back. Is it safe for me to assume that the bean that is at the point of contention is the DocumentE bean? i.e. it was in the waitForTX state BECAUSE it was trying to access a DocumentE bean which already had an associated transaction.
Can anyone shed any light on how to interpret this stack trace.
One other side-effect we are seeing - the database connection pool is returning NULLs after running for a period of time. Could this be related to the transactions that are timing out. I'm guessing that the transactions are timing out in the EJB world YET the underlying database transaction NEVER times out and thus stays active on one of the 100 database connections in the pool. Once we have 100 of these TIMEDOUT EJB transactions - then we may run out of useable database connections - which then leads to the pool returning NULLs.
I can't for the life of me figure out how to enable debug logging on the JBossPool in JBoss 2.4.3
Thanks for reading - ANY constructive information would be greatly appreciated.
Cheers,
Andy
2005-03-14 14:00:44,625 [Thread-49] WARN LoginP - Transaction XidImpl [FormatId=257, GlobalId=knowledge1//20790, BranchQual=] timed out. status=STATUS_ACTIVE
2005-03-14 14:00:45,531 [Thread-50] WARN LoginP - Transaction XidImpl [FormatId=257, GlobalId=knowledge1//20810, BranchQual=] timed out. status=STATUS_ACTIVE
2005-03-14 14:00:46,265 [Thread-51] WARN LoginP - Transaction XidImpl [FormatId=257, GlobalId=knowledge1//20820, BranchQual=] timed out. status=STATUS_ACTIVE
2005-03-14 14:00:46,328 [Thread-52] WARN LoginP - Transaction XidImpl [FormatId=257, GlobalId=knowledge1//20825, BranchQual=] timed out. status=STATUS_ACTIVE
2005-03-14 14:00:46,671 [Thread-18] ERROR org.jboss.ejb.BeanLock SearchCommunitiesP DocumentFilterP DocumentE - Thread[Thread-18,5,main]Saw rolled back tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=knowledge1//20790, BranchQual=] waiting for txLock
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - TRANSACTION ROLLBACK EXCEPTION:Transaction marked for rollback, possibly a timeout; nested exception is:
java.lang.RuntimeException: Transaction marked for rollback, possibly a timeout
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - java.lang.RuntimeException: Transaction marked for rollback, possibly a timeout
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.waitForTx(QueuedPessimisticEJBLock.java:297)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.doSchedule(QueuedPessimisticEJBLock.java:175)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.schedule(QueuedPessimisticEJBLock.java:143)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.EntityLockInterceptor.invoke(EntityLockInterceptor.java:132)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invokeNext(TxInterceptorCMT.java:133)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:307)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:99)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:128)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:195)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.EntityContainer.invoke(EntityContainer.java:427)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.server.JRMPContainerInvoker.invoke(JRMPContainerInvoker.java:489)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.GenericProxy.invokeContainer(GenericProxy.java:335)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.EntityProxy.invoke(EntityProxy.java:133)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at $Proxy170.getObject(Unknown Source)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.documente.CDocumentEEJB.getObject(CDocumentEEJB.java:63)
2005-03-14 14:00:46,671 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.documentfilterp.CDocumentFilterPBL.getDocument(CDocumentFilterPBL.java:197)
2005-03-14 14:00:46,671 [Thread-17] ERROR org.jboss.ejb.BeanLock SearchCommunitiesP DocumentFilterP DocumentE - Thread[Thread-17,5,main]Saw rolled back tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=knowledge1//20810, BranchQual=] waiting for txLock
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.documentfilterp.CDocumentFilterPBL.filterDocumentErrors(CDocumentFilterPBL.java:93)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.documentfilterp.CDocumentFilterPBean.filterDocumentErrors(CDocumentFilterPBean.java:72)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at java.lang.reflect.Method.invoke(Method.java:324)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:543)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:87)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invokeNext(TxInterceptorCMT.java:133)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:307)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:99)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:128)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:195)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.StatelessSessionContainer.invoke(StatelessSessionContainer.java:286)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.server.JRMPContainerInvoker.invoke(JRMPContainerInvoker.java:489)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.GenericProxy.invokeContainer(GenericProxy.java:335)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.StatelessSessionProxy.invoke(StatelessSessionProxy.java:123)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at $Proxy43.filterDocumentErrors(Unknown Source)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.documentfilterp.CDocumentFilterPEJB.filterDocumentErrors(CDocumentFilterPEJB.java:73)
2005-03-14 14:00:47,015 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.filterDocumentErrors(CSearchCommunitiesPBL.java:989)
2005-03-14 14:00:47,328 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.statisticalSearch(CSearchCommunitiesPBL.java:698)
2005-03-14 14:00:47,328 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.basicCommunitySearch(CSearchCommunitiesPBL.java:564)
2005-03-14 14:00:47,328 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.basicCommunitySearch(CSearchCommunitiesPBL.java:577)
2005-03-14 14:00:47,328 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.singleCommunitySearch(CSearchCommunitiesPBL.java:290)
2005-03-14 14:00:47,328 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBL.search(CSearchCommunitiesPBL.java:111)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPBean.search(CSearchCommunitiesPBean.java:68)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at java.lang.reflect.Method.invoke(Method.java:324)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:543)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:87)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invokeNext(TxInterceptorCMT.java:133)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:431)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:99)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:128)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:195)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.StatelessSessionContainer.invoke(StatelessSessionContainer.java:286)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.server.JRMPContainerInvoker.invoke(JRMPContainerInvoker.java:489)
2005-03-14 14:00:47,343 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.GenericProxy.invokeContainer(GenericProxy.java:335)
2005-03-14 14:00:47,406 [Thread-12] ERROR org.jboss.ejb.BeanLock SearchCommunitiesP DocumentFilterP DocumentE - Thread[Thread-12,5,main]Saw rolled back tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=knowledge1//20820, BranchQual=] waiting for txLock
2005-03-14 14:00:47,406 [Thread-53] WARN LoginP - Transaction XidImpl [FormatId=257, GlobalId=knowledge1//20859, BranchQual=] timed out. status=STATUS_ACTIVE
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.jboss.ejb.plugins.jrmp.interfaces.StatelessSessionProxy.invoke(StatelessSessionProxy.java:123)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at $Proxy116.search(Unknown Source)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.searchcommunitiesp.CSearchCommunitiesPEJB.search(CSearchCommunitiesPEJB.java:71)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.common.search.SearchManager.search(SearchManager.java:134)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.common.search.SearchManager.search(SearchManager.java:63)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.customer.anon.tag.search.MultiSelectSubmitSearchButtonTagB.doCommunitySearch(MultiSelectSubmitSearchButtonTagB.java:197)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.customer.anon.tag.search.MultiSelectSubmitSearchButtonTagB.performAction(MultiSelectSubmitSearchButtonTagB.java:108)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at com.kmsoftware.tag.common.CActionTag.doStartTag(CActionTag.java:98)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at search._0002fsearch_0002fsearch_0002ejspsearch_jsp_0._jspService(_0002fsearch_0002fsearch_0002ejspsearch_jsp_0.java:1208)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.jasper.servlet.JspServlet$JspCountedServlet.service(JspServlet.java:130)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java:282)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:429)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:500)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
2005-03-14 14:00:47,640 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.core.ServletWrapper.doService(ServletWrapper.java:405)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.core.Handler.service(Handler.java:287)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.core.ServletWrapper.service(ServletWrapper.java:372)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.core.ContextManager.internalService(ContextManager.java:812)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.core.ContextManager.service(ContextManager.java:758)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.service.http.HttpConnectionHandler.processConnection(HttpConnectionHandler.java:213)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.service.TcpWorkerThread.runIt(PoolTcpEndpoint.java:416)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at org.apache.tomcat.util.ThreadPool$ControlRunnable.run(ThreadPool.java:501)
2005-03-14 14:00:47,968 [Thread-18] ERROR DocumentE SearchCommunitiesP DocumentFilterP DocumentE - at java.lang.Thread.run(Thread.java:534)