Hi,


We have been struggling with deadlocking situations on our live server for some time, and I think I have isolated where the problem occurs. We are using JBoss 3.2.1, MVCSoft persistence manager 1.1 and MS SQL Server 2000. Although I believe I have now isolated the problem I am a bit confused about what is happening and why. In a nutshell the problem seems to be that a session facade method is writing to a few tables, returning and then it just hangs in the container before control is returned to the client. The write locks not being released cause other connections to not gain access to the tables, causing our site to "freeze". I have a job that is running every minute in SQL Server monitoring locks that are never released, and if a write lock is held on two consecutive runs the connection is killed. Not the "best" solution, but it has been good for gathering data and ensuring that the site is not "frozen" for too long when these situations arise. Our logging shows that the killed connection ALWAYS has all its locks granted, and that it is not waiting for anything.



The method of our session facade (using CMT) looks as follows:


private void logFormSaving(String sMethod, String sMsg)
{
LOGCAT.info("$$FORM$$ [INSTANCE" + m_instanceCounter.getInstance()+ "] - " + sMethod + " - " + sMsg);
}

public long saveForm(ExhibitorLoginInfo exhibitorLoginInfo, ProductGroupValue productGroup, Collection showProducts, ProductGroupOrderValue productGroupOrderValue, long lProdGroupID, Map formSubmissionValues) throws InvalidPermissionException
{
logFormSaving("saveForm()", " ==== START el:" + exhibitorLoginInfo.getExhibitorLoginID() + "; pg:" + lProdGroupID + "; pgo:" + productGroupOrderValue.getID());

long lProductGroupOrderID = 0;
try
{
//Call (a lot of) methods on CMP entity beans. All are set up to use 'Entity per Transaction" with commit option C
}
catch (InvalidPermissionException e)
{
logFormSaving("saveForm()", " ==== END (InvalidPermissionException) el:" + exhibitorLoginInfo.getExhibitorLoginID() + "; pg:" + lProdGroupID + "; pgo:" + productGroupOrderValue.getID());
super.setRollbackOnly();
throw e;
}

logFormSaving("saveForm()", " ==== END el:" + exhibitorLoginInfo.getExhibitorLoginID() + "; pg:" + lProdGroupID + "; pgo:" + productGroupOrderValue.getID());


return lProductGroupOrderID;
}


From the logs I have found that the method always completes, but control is never returned to the client. I have pasted in a snippet of today's logs, during a quiet time (people using our site are businesses and it is now Sunday afternoon), showing what is happening in one of these situations. The method is called and returns at 15:08:15,718, and the locks are not released. The SQL job runs at 15:09 and 15:10, and sees that the locks for a connection are not released and terminates the connection at 15:10. The 'Optimistic Rollback' EJBException at 15:10:03,156 and XAException at 15:10:03,421 indicate that the connection has been closed, and it always seems to happen in the session facade method shown. What could the reason be for the method "hanging" in the container and the locks not being released? In the example below the method is taking 1.75 minutes between the session facade method returning and the client never gets control again, and the database connection has obtained all the locks it needs to perform the transaction.


Thanks,

Kab




2003-12-14 15:08:14,859 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - ==== START el:6271; pg:591; pgo:6468 2003-12-14 15:08:14,859 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - Current product() 3377 2003-12-14 15:08:14,859 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - calling saveProductOrderForGroup() 3377 2003-12-14 15:08:14,875 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - START 2003-12-14 15:08:14,875 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - call getProductGroupCMPForProductID:3377
2003-12-14 15:08:14,890 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - called getProductGroupCMPForProductID 2003-12-14 15:08:14,890 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - call loadProductGroupOrder:6468 2003-12-14 15:08:14,906 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - call loadProductGroupOrder 2003-12-14 15:08:14,937 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - call obtainProductOrderValueAndSetFields()
2003-12-14 15:08:14,937 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - obtainProductOrderValueAndSetFields() - START 2003-12-14 15:08:14,968 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - obtainProductOrderValueAndSetFields() - Calling getShowProductValue():3377 2003-12-14 15:08:15,140 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - obtainProductOrderValueAndSetFields() - Called getShowProductValue() 2003-12-14 15:08:15,140 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - obtainProductOrderValueAndSetFields() - END 2003-12-14 15:08:15,156 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - called obtainProductOrderValueAndSetFields()
2003-12-14 15:08:15,156 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - call saveProductGroupOrderCreateHistoryIfNeeded()
2003-12-14 15:08:15,156 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - START 2003-12-14 15:08:15,171 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - creating/getting productgrouporder value 2003-12-14 15:08:15,203 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - calling hasProductOrderChanged 2003-12-14 15:08:15,203 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - hasProductOrderChanged() - START 2003-12-14 15:08:15,203 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - hasProductOrderChanged() - loading stored productorder 11212 2003-12-14 15:08:15,203 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - hasProductOrderChanged() - loaded stored productorder 11212 2003-12-14 15:08:15,218 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - hasProductOrderChanged() - END 2003-12-14 15:08:15,218 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - called hasProductOrderChanged 2003-12-14 15:08:15,234 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - deleting product order 2003-12-14 15:08:15,234 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - deleted product order 2003-12-14 15:08:15,234 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - resetOrderStatus 2003-12-14 15:08:15,234 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - have resetOrderStatus 2003-12-14 15:08:15,250 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - create order with history 2003-12-14 15:08:15,390 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - created order with history 2003-12-14 15:08:15,390 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductGroupOrderCreateHistoryIfNeeded() - END 2003-12-14 15:08:15,390 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - called saveProductGroupOrderCreateHistoryIfNeeded()
2003-12-14 15:08:15,406 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveProductOrderForGroup B() - END 2003-12-14 15:08:15,406 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - called saveProductOrderForGroup() 3377 2003-12-14 15:08:15,406 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - calling saveDateOrderedForGroupOrder() 3377 2003-12-14 15:08:15,406 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - START 2003-12-14 15:08:15,421 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - lookup product group order 2003-12-14 15:08:15,421 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - looked up product group order 2003-12-14 15:08:15,421 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - check email required 2003-12-14 15:08:15,437 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - getProductGroupValue() - START:591 2003-12-14 15:08:15,687 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - getProductGroupValue() - END:591 2003-12-14 15:08:15,687 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - email required 2003-12-14 15:08:15,703 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - putting message on queue 2003-12-14 15:08:15,703 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - email done 2003-12-14 15:08:15,703 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveDateOrderedForGroupOrder() - END 2003-12-14 15:08:15,718 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - called saveDateOrderedForGroupOrder() 3377 2003-12-14 15:08:15,718 INFO [com.exhibitormanual.ejb.product.order.OrderProductFacadeBean] $$FORM$$ [INSTANCE487] - saveForm() - ===== END 2003-12-14 15:10:03,156 ERROR [STDERR] javax.ejb.EJBException: Optimistic rollback 2003-12-14 15:10:03,156 ERROR [STDERR] at com.mvcsoft.pm.bean_behavior.Strategy.storeData(Strategy.java:1042)
2003-12-14 15:10:03,156 ERROR [STDERR] at com.mvcsoft.pm.bean_behavior.ActionModelAdd.perform(ActionModelAdd.java:25)
2003-12-14 15:10:03,156 ERROR [STDERR] at com.mvcsoft.pm.bean_behavior.DefaultStorageBehavior.store(DefaultStorageBehavior.java:146)
2003-12-14 15:10:03,171 ERROR [STDERR] at com.mvcsoft.pm.transaction_data.DefaultTransactionData.updatePersistentStore(DefaultTransactionData.java:207)
2003-12-14 15:10:03,171 ERROR [STDERR] at com.mvcsoft.pm.transaction_data.DefaultTransactionData.access$000(DefaultTransactionData.java:31)
2003-12-14 15:10:03,171 ERROR [STDERR] at com.mvcsoft.pm.transaction_data.DefaultTransactionData$PersistenceSynchronization.beforeCompletion(DefaultTransactionData.java:381)
2003-12-14 15:10:03,187 ERROR [STDERR] at org.jboss.tm.TransactionImpl.doBeforeCompletion(TransactionImpl.java:1297)
2003-12-14 15:10:03,187 ERROR [STDERR] at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:338)
2003-12-14 15:10:03,187 ERROR [STDERR] at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:369)
2003-12-14 15:10:03,203 ERROR [STDERR] at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:253)
2003-12-14 15:10:03,203 ERROR [STDERR] at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:104)
2003-12-14 15:10:03,203 ERROR [STDERR] at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:117)
2003-12-14 15:10:03,218 ERROR [STDERR] at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
2003-12-14 15:10:03,218 ERROR [STDERR] at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
2003-12-14 15:10:03,218 ERROR [STDERR] at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:322)
2003-12-14 15:10:03,234 ERROR [STDERR] at org.jboss.ejb.Container.invoke(Container.java:674)
2003-12-14 15:10:03,234 ERROR [STDERR] at java.lang.reflect.Method.invoke(Native Method) 2003-12-14 15:10:03,234 ERROR [STDERR] at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
2003-12-14 15:10:03,250 ERROR [STDERR] at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:549)
2003-12-14 15:10:03,250 ERROR [STDERR] at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:101)
2003-12-14 15:10:03,250 ERROR [STDERR] at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:83)
2003-12-14 15:10:03,250 ERROR [STDERR] at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
2003-12-14 15:10:03,265 ERROR [STDERR] at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
2003-12-14 15:10:03,265 ERROR [STDERR] at org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:100)
2003-12-14 15:10:03,265 ERROR [STDERR] at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
2003-12-14 15:10:03,281 ERROR [STDERR] at $Proxy245.saveForm(Unknown Source) 2003-12-14 15:10:03,281 ERROR [STDERR] at com.exhibitormanual.web.product.form.FormOnlyAction.saveForm(FormOnlyAction.java:215)
2003-12-14 15:10:03,281 ERROR [STDERR] at com.exhibitormanual.web.product.form.FormOnlyAction.submitForm(FormOnlyAction.java:319)
2003-12-14 15:10:03,281 ERROR [STDERR] at com.exhibitormanual.web.product.form.FormOnlyAction.actionPerform(FormOnlyAction.java:98)
2003-12-14 15:10:03,296 ERROR [STDERR] at com.exhibitormanual.web.action.ValidateLoginInfoAction.perform(ValidateLoginInfoAction.java:162)
2003-12-14 15:10:03,296 ERROR [STDERR] at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1813)
2003-12-14 15:10:03,296 ERROR [STDERR] at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1587)
2003-12-14 15:10:03,312 ERROR [STDERR] at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:510)
2003-12-14 15:10:03,312 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
2003-12-14 15:10:03,312 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
2003-12-14 15:10:03,312 WARN [com.exhibitormanual.ejb.mail.receiver.MailQueueReceiverBean] FinderException in MailQueueReceiverBean getProductGroupLocal(): 0 2003-12-14 15:10:03,328 ERROR [STDERR] at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360)
2003-12-14 15:10:03,328 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:342)
2003-12-14 15:10:03,343 ERROR [STDERR] at com.oreilly.javaxslt.util.XSLTFilter.doFilter(XSLTFilter.java:95)
2003-12-14 15:10:03,343 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:334)
2003-12-14 15:10:03,343 ERROR [STDERR] at com.exhibitormanual.web.filters.BrowserFilter.doFilter(BrowserFilter.java:160)
2003-12-14 15:10:03,343 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:334)
2003-12-14 15:10:03,359 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:286)
2003-12-14 15:10:03,359 ERROR [STDERR] at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:558)
2003-12-14 15:10:03,359 ERROR [STDERR] at org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
2003-12-14 15:10:03,375 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:507)
2003-12-14 15:10:03,375 ERROR [STDERR] at org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
2003-12-14 15:10:03,375 ERROR [STDERR] at org.mortbay.http.HttpServer.service(HttpServer.java:863)
2003-12-14 15:10:03,390 ERROR [STDERR] at org.jboss.jetty.Jetty.service(Jetty.java:460)
2003-12-14 15:10:03,390 ERROR [STDERR] at org.mortbay.http.HttpConnection.service(HttpConnection.java:775)
2003-12-14 15:10:03,390 ERROR [STDERR] at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:939)
2003-12-14 15:10:03,406 ERROR [STDERR] at org.mortbay.http.HttpConnection.handle(HttpConnection.java:792)
2003-12-14 15:10:03,406 ERROR [STDERR] at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:201)
2003-12-14 15:10:03,406 ERROR [STDERR] at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
2003-12-14 15:10:03,421 ERROR [STDERR] at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:455)
2003-12-14 15:10:03,421 INFO [org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener] Unregistered handle that was not registered! null for managedConnection: org.jboss.resource.adapter.jdbc.xa.XAManagedConnection@155c9f
2003-12-14 15:10:03,421 WARN [org.jboss.tm.TransactionImpl] XAException: tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=emcws02p2//63715, BranchQual=] errorCode=XA_UNKNOWN(0)
javax.transaction.xa.XAException: Connection reset by peer: socket write error at com.inet.tds.k.if(Unknown Source) at com.inet.tds.k.rollback(Unknown Source) at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.rollback(XAManagedConnection.java:185)
at org.jboss.tm.TransactionImpl.rollbackResources(TransactionImpl.java:1648)
at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:394)
at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:369)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:253)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:104)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:117)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:322)
at org.jboss.ejb.Container.invoke(Container.java:674)
at java.lang.reflect.Method.invoke(Native Method)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:549)
at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:101)
at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:83)
at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
at org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:100)
at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
at $Proxy245.saveForm(Unknown Source)
at com.exhibitormanual.web.product.form.FormOnlyAction.saveForm(FormOnlyAction.java:215)
at com.exhibitormanual.web.product.form.FormOnlyAction.submitForm(FormOnlyAction.java:319)
at com.exhibitormanual.web.product.form.FormOnlyAction.actionPerform(FormOnlyAction.java:98)
at com.exhibitormanual.web.action.ValidateLoginInfoAction.perform(ValidateLoginInfoAction.java:162)
at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1813)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1587)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:510)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360)
at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:342)
at com.oreilly.javaxslt.util.XSLTFilter.doFilter(XSLTFilter.java:95)
at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:334)
at com.exhibitormanual.web.filters.BrowserFilter.doFilter(BrowserFilter.java:160)
at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:334)
at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:286)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:558)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:507)
at org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
at org.mortbay.http.HttpServer.service(HttpServer.java:863)
at org.jboss.jetty.Jetty.service(Jetty.java:460)
at org.mortbay.http.HttpConnection.service(HttpConnection.java:775)
at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:939)
at org.mortbay.http.HttpConnection.handle(HttpConnection.java:792)
at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:201)
at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:455)