5 Replies Latest reply on Jul 21, 2003 3:16 PM by adrian.brock

    BMP findByPrimaryKey hanging after multiple lookups

    heismann

      Hello,

      I'm in the process of porting from Weblogic 5.1 to JBoss.

      I'm running into a problem with an BMP Entity Bean when calling home.findByPrimaryKey(). After findByPrimaryKey() is called on that instance a multiple number of times (it varies) the calling the thread is blocked until an UndeclaredThrowableException is thrown (caused by a java.lang.InterruptedException). The full stacktrace is below.

      I'm accessing the Entity Bean using a session facade. The Entity Bean is configured with the "Standard BMP EntityBean" container-configuration, including the commit-option A.

      The order of events is as follows:

      1. lookup the Bean
      2. delete the Bean
      3. recreate the Bean using the same data with any updates using the same primary key.

      Steps 1-3 are repeated 6 times without fail, on the 7th time the thread blocks on the lookup.

      Since the Entity instance is in the cache and has not been passivated then it is, I believe, safe to assume that we don't have a table/row locking issue. I've verified this conclusion is correct.

      The InterruptedException appears to have been thrown from EDU.oswego.cs.dl.util.concurrent.QueuedSemaphore$WaitNode.doTimedWait

      After the container gets into this state I am then prevented from accessing any other instances of the Entity Bean.

      Any ideas on why this is happening will be appreciated.


      Thanks for your time.

      Zac

      ----------------------------------------------------------------

      JBoss Version: 3.2.1; build: CVSTag=JBoss_3_2_1 date=200305041533.

      JDK: Sun 1.4.2
      OS: Solaris.

      java.lang.reflect.UndeclaredThrowableException
      at $Proxy98.findByPrimaryKey(Unknown Source)
      at com.mycompany.dataaccess.overseer.OverSeerBean.saveOrder(Unknown Source)
      at com.mycompany.dataaccess.overseer.OverSeerBean.saveOrder(Unknown Source)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:629)
      at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186)
      at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
      at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
      at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:296)
      at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:104)
      at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:117)
      at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
      at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
      at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:322)
      at org.jboss.ejb.Container.invoke(Container.java:674)
      at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
      at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:549)
      at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:101)
      at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:83)
      at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
      at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
      at org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:100)
      at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
      at $Proxy90.saveOrder(Unknown Source)
      at org.apache.jsp.saveOrder_jsp.completeService(saveOrder_jsp.java:67)
      at com.mycompany.snp.servlets.ValidatingServlet.doService(Unknown Source)
      at com.mycompany.snp.servlets.ValidatingServlet.doPost(Unknown Source)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
      at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
      at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
      at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360)
      at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:294)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:558)
      at org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
      at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:507)
      at org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
      at org.mortbay.http.HttpServer.service(HttpServer.java:863)
      at org.jboss.jetty.Jetty.service(Jetty.java:460)
      at org.mortbay.http.HttpConnection.service(HttpConnection.java:775)
      at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:939)
      at org.mortbay.http.HttpConnection.handle(HttpConnection.java:792)
      at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:201)
      at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
      at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:455)
      Caused by: java.lang.InterruptedException
      at java.lang.Object.wait(Native Method)
      at EDU.oswego.cs.dl.util.concurrent.QueuedSemaphore$WaitQueue$WaitNode.doTimedWait(QueuedSemaphore.java:123)

      at EDU.oswego.cs.dl.util.concurrent.QueuedSemaphore.attempt(QueuedSemaphore.java:47)
      at org.jboss.ejb.plugins.AbstractInstancePool.get(AbstractInstancePool.java:127)
      at org.jboss.ejb.plugins.EntityInstanceInterceptor.invokeHome(EntityInstanceInterceptor.java:79)
      at org.jboss.ejb.plugins.EntityLockInterceptor.invokeHome(EntityLockInterceptor.java:61)
      at org.jboss.ejb.plugins.EntityCreationInterceptor.invokeHome(EntityCreationInterceptor.java:28)
      at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:88)
      at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:325)
      at org.jboss.ejb.plugins.TxInterceptorCMT.invokeHome(TxInterceptorCMT.java:74)
      at org.jboss.ejb.plugins.SecurityInterceptor.invokeHome(SecurityInterceptor.java:92)
      at org.jboss.ejb.plugins.LogInterceptor.invokeHome(LogInterceptor.java:120)
      at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invokeHome(ProxyFactoryFinderInterceptor.java:93)
      at org.jboss.ejb.EntityContainer.internalInvokeHome(EntityContainer.java:477)
      at org.jboss.ejb.Container.invoke(Container.java:694)
      at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
      at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:549)
      at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:101)
      at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:83)
      at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
      at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
      at org.jboss.proxy.ejb.HomeInterceptor.invoke(HomeInterceptor.java:173)
      at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
      ... 53 more

        • 1. Re: BMP findByPrimaryKey hanging after multiple lookups

          You will have to show a threaddump and the full
          stacktrace.

          Deleting a bean should remove it from the cache.

          I suspect you are getting a transaction timeout
          because of an undetected deadlock in the db?

          Regards,
          Adrian

          • 2. Re: BMP findByPrimaryKey hanging after multiple lookups
            heismann

            Hi Adrian,

            Thank you for your reply.

            I've attached a threaddump. I believe it contains the full stacktrace you want.

            I don't believe a deadlock is occurring. I believe I've checked this by running a query against the row/table that may be locked using the txn-datasource that the EntityBean is using.

            Thanks,
            Zac

            • 3. Re: BMP findByPrimaryKey hanging after multiple lookups

              The stacktrace shows the entity's pool waiting
              for an available instance, are you using strict pooling?

              Regards,
              Adrian

              • 4. Re: BMP findByPrimaryKey hanging after multiple lookups
                heismann

                I am using strict pooling and when I remove it the issue seems to go away.

                I still question why it was happening in the first place.

                I've extended the Standard BMP EntityBean container configuration.

                <container-configuration extends="Standard BMP EntityBean">
                <container-name>MY BMP EntityBean</container-name>
                <container-pool-conf>
                1
                20
                true
                </container-pool-conf>
                </container-configuration>


                /** A FIFO semaphore that is set when the strict max size behavior is in effect. When set, only maxSize instances may be active and any attempt to get an instance will block until an instance is freed.
                */
                private FIFOSemaphore strictMaxSize;


                So based upon the above comment from the org.jboss.ejb.AbstractInstancePool class should I conclude that the Entity instances are not being freed for some reason?

                Zac

                • 5. Re: BMP findByPrimaryKey hanging after multiple lookups

                  Correct, once 20 entities are in the cache, your strict
                  pool is exhausted.

                  Regards,
                  Adrian