8 Replies Latest reply on Jun 22, 2008 3:31 AM by barbacena

    Not acquiring lock - @Synchronized

    barbacena

      Hi,


      Am I doig something wrong putting a POJO in session scope? I am receving this:


      java.lang.IllegalStateException: could not acquire lock on @Synchronized component: eloUserContext
              at org.jboss.seam.core.SynchronizationInterceptor.aroundInvoke(SynchronizationInterceptor.java:41)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
              at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)
              at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:166)
              at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:102)
              at br.gov.tse.secad.util.EloUserContext$$javassist10.toString(EloUserContext$$javassist10.java)
              at com.sun.facelets.util.DevTools.writeVariables(DevTools.java:168)
              at com.sun.facelets.util.DevTools.writeVariables(DevTools.java:147)
              at com.sun.facelets.util.DevTools.debugHtml(DevTools.java:135)
              at com.sun.facelets.tag.ui.UIDebug.writeDebugOutput(UIDebug.java:92)
              at com.sun.facelets.tag.ui.UIDebug.encodeBegin(UIDebug.java:81)
              at javax.faces.component.UIComponent.encodeAll(UIComponent.java:884)
              at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892)
              at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892)
              at com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:592)
              at org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:108)
              at org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:189)
              at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:106)
              at com.sun.faces.lifecycle.LifecycleImpl.phase(LifecycleImpl.java:251)
              at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:144)
              at javax.faces.webapp.FacesServlet.service(FacesServlet.java:245)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
              at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:85)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:154)
              at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:260)
              at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:366)
              at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:493)
              at org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:60)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:58)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
              at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
              at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
              at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
              at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
              at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
              at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:241)
              at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
              at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:580)
              at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
              at java.lang.Thread.run(Thread.java:619)

        • 1. Re: Not acquiring lock - @Synchronized
          barbacena

          I've got this error when I was loading testing. I think the timeout of the tryLock is expiring therefore the exception is being throw.


          I think this is not the best way to reject a connection, or is it?


          Is it possible to configure this timeout without having to re-compile the class?


          Ty.

          • 2. Re: Not acquiring lock - @Synchronized
            monkeyden

            I don't know what your code looks like, since you didn't post any, but this is from the API docs:




            Specifies that a stateful component has multiple concurrent clients, and so access to the component must be synchronized. This annotation is not required for session scoped components, which are synchronized by default.
            • 3. Re: Not acquiring lock - @Synchronized
              svetzal

              Marcell Barbacena wrote on Jun 20, 2008 04:25:


              I've got this error when I was loading testing. I think the timeout of the tryLock is expiring therefore the exception is being throw.


              I think you'll find that if you're throwing more requests at your component than it can handle, you'll be able to reproduce this every time.


              If you measure the time taken to service a single request, say it's 100ms, and divide that into the default @Synchronized timeout value (or whatever value you're setting it to) say it's 30,000ms (30 seconds), you'll find your component will handle a max burst of 300 requests (30000 / 100) within that 30 second period before it starts throwing this.


              You might find you want to trap this exception in your pages.xml file something like this:


              <exception class="java.lang.IllegalStateException">
                  <redirect view-id="/toobusy.xhtml">
                      <message>Sorry, I'm too busy to do that right now.</message>
                  </redirect>
              </exception>


              Increasing the Timeout value using say @Synchronized(timeout = 60000L) will only try to even out your load over a longer period and may simply delay the amount of time it takes for a user to get the too busy error.


              If your load is bursty then increasing the timeout might work well. If it's steady however you need a way like above to give your app some breathing room or you may find things just start cascade failing in your app as various timeouts get hit.



              • 4. Re: Not acquiring lock - @Synchronized
                barbacena

                hi Steve,


                That is exactly what is happening. But I found that the default timeout on the tryLock (in SynchronizationInterceptor.java:41) is only 1s. My component is like a entity POJO:



                @Name("eloUserContext")
                @Scope(SESSION)
                public class UserContext {
                  ...
                }
                



                Should I raise the timeout placing the @Synchronized annotation along side with the scope annotation?


                Because I am loading testing this is a expected burst as I am forcing everything to struggle. I got your idea to redirect the user to a busy page, but I think that IllegalStateException is not a good trap for this, do you agree?


                p.s.: I am already redirecting the user to a default error page trapping java.lang.Exception.
                p.s.s.: I am planning to put a front load balancer, therefore I think the responsibility to reject user should be provided by this and not from Seam, then I think that this tryLock might not be reasonable for this.


                • 5. Re: Not acquiring lock - @Synchronized
                  svetzal

                  I'm sorry Marcell - you're absolutely right. The default is 1 second not 30 :(


                  Wow, I should have remembered that :) sorry to mislead.


                  Yes, use the @Synchronized annotation to push your timeout a bit but only if you expect your load to be bursty...


                  Unfortunately from what I've seen, the IllegalStateException is what's thrown - I agree I'd like it to be a more specific exception as well. I should dig out a recent stack trace and follow it down to the final root cause to verify. I'll probably have another today.


                  Also, I can almost understand why the timeout is set so low by default - an added second onto a response time while waiting for a lock is bad, 30 seconds which I'm using is terrible. Users don't have that kind of patience :) But in my case I can't get much more optimization out of our methods on that class (some of them take several seconds already) so I had to push this out.


                  That said, perhaps exploring how to optimize the response time of your methods in your class will also help your scalability.


                  We hook a method interceptor in to all our Session beans to time all method calls so we can get a composite of what time is spent where and what most needs to be optimized.

                  • 6. Re: Not acquiring lock - @Synchronized
                    barbacena

                    Hi Steven,


                    My methods is very very optimized. It is taking a top of 80 milliseconds. What is hurting a lot is JSF rendering. The worst page is taking about 1.3s just to render. Are you getting same times as this? Are you using facelets and Sun RI? Did you provide any optimization parameters?


                    Ty

                    • 7. Re: Not acquiring lock - @Synchronized
                      svetzal

                      Marcell Barbacena wrote on Jun 20, 2008 15:56:

                      My methods is very very optimized. It is taking a top of 80 milliseconds. What is hurting a lot is JSF rendering. The worst page is taking about 1.3s just to render. Are you getting same times as this? Are you using facelets and Sun RI? Did you provide any optimization parameters?


                      I've seen exactly this in a lot of our apps.


                      One thing to be careful - you'll get different page render times reported by different sources. This caught me.


                      For example, I had a filter servlet that would report server-side page render times. But this never corresponded to observation or tools like Firebug / YSlow. I found that on non-trivial pages we'd see say 500ms on the server, and 3s on the browser.


                      We've done many things to improve performance in this respect.


                      First is to NEVER EVER USE s:graphicImage. It is evil, destroys performance. One day we'll get around to rewriting it - in the meanwhile for customers we've built explicit phase listeners that  do the right thing like allow us to send out cache-control headers, correct mimetype etc.


                      Second is to always specify width and height of all images on the page. The browser often won't even attempt to render the page until it has key images loaded, which depending on your page design might even be all of them. And this can take some time, give browsers may only send 2-4 concurrent requests.


                      We use a lot of Seam/RichFaces so we get killed on a very high quantity of JavaScript, CSS, and image downloads. We use a Servlet filter to inject cache-control headers to help the browser cache these things. It still will request them but at least it won't wait to receive them to render the page.


                      Not sure if you're hitting the same thing, hopefully this can be of help to you.

                      • 8. Re: Not acquiring lock - @Synchronized
                        barbacena

                        Hi Steven,


                        I did not time the static part of the application (things as images, css, js). Our application has a 2-tier web cache and I expected then to take this load off. I don't really do anything like filters to set cache headers up to now to improve anything. Could you share this code?


                        I was really hoping you had a silver bullet configuration at web.xml ... I guess I can't count on this anymore ...


                        My current problem is that my search result page is taking more then 2s to the server do the render phase... add to this the network time and you end killing the response time.


                        Did you test changing Sun RI to Myfaces? What about wicket? Do you think JSF is killing the performance of our apps?