4 Replies Latest reply on Aug 9, 2013 9:20 AM by nishant_ag14

    Jboss AS 7 http thread pool problem

    nishant_ag14

      Hi,

       

      We are using Jboss AS 7 in domain mode with following threads and web subsystem configuration

       

      Thread subsystem:

       

             <subsystem xmlns="urn:jboss:domain:threads:1.1">

                      <bounded-queue-thread-pool name="http-executor">

                          <core-threads count="100"/>

                          <queue-length count="10"/>

                          <max-threads count="2000"/>

                          <keepalive-time time="20" unit="seconds"/>

                      </bounded-queue-thread-pool>

                  </subsystem>

       

      Web Subsystem:

       

       

       

           <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">

                      <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http" redirect-port="8443" executor="http-executor" max-connections="100000"/>

                     ...

           </subsystem>

       

       

       

      There are 2 nodes which are exposed for incoming http request using the above configuration.

       

      There is also a CITRIX load balancer (LB) which is getting traffic from application clients and forwarding the requesting to these jboss nodes.

       

      The problem is that we see that even though we have configured 100 core threads in the threadpool the request is processed by a single thread for a long duration and then another thread is picked from the pool.

       

      This creates the problem that clients see long acknowledgement time to serve their request. Any idea why this is happening and how can we tell jboss to use different thread for different http requests.

       

      Please see our application logs with this problem:

       

      21:57:47,107 INFO  [PerfLog.DTServlet] (http-executor-threads - 1) The message from UUID[cc9f1b9c-7443-4177-93ea-eecf550325a3]  processed in [88] ms

      21:57:47,859 INFO  [PerfLog.DTServlet] (http-executor-threads - 1) The message from UUID[cc9f1b9c-7443-4177-93ea-eecf550325a3]  processed in [88] ms

      21:57:48,624 INFO  [PerfLog.DTServlet] (http-executor-threads - 1) The message from UUID[cc9f1b9c-7443-4177-93ea-eecf550325a3]  processed in [92] ms

      21:57:49,631 INFO  [PerfLog.DTServlet] (http-executor-threads - 1) The message from UUID[cc9f1b9c-7443-4177-93ea-eecf550325a3]  processed in [79] ms

       

       

      I also observe that the TCP connection between Jboss nodes and LB remains same and is closed after 60 sec of inactivity. If i send a request after the TCP connection has closed then a new thread is picked from the pool.

       

      We see similar TCP connection behaviour in weblogic version of application but we observe that in weblogic the same thread is not used for processing the request that gives better performance.

       

      Any help in this regards is greatly appreciated as this is a major problem for us migrating to Jboss from weblogic.

        • 1. Re: Jboss AS 7 http thread pool problem
          sfcoy

          How are you testing this?

          • 2. Re: Jboss AS 7 http thread pool problem
            nishant_ag14

            we have clients that are sending http traffic to LB then to Jboss nodes. we have logs in the application (servlet) that are printing the Thread id as well. And we see that when there is a LB-Jboss node TCP connection opened then the single thread is utilized. if we hit the Jboss node directly from the client (without LB) which does not have sticky TCP connection then each request is served by new thread by Jboss node until max number of threads reached and then restart with the first thread.

             

            Did that answer your question?

            • 3. Re: Jboss AS 7 http thread pool problem
              nishant_ag14

              the issue is resolved.

               

              the configuration in LB was such that it was doing TCP multiplexing hence sending all requests over preexisting TCP connection. After disabling it the things are working as expected.

              • 4. Re: Jboss AS 7 http thread pool problem
                nishant_ag14

                I would like to ask another question that TCP Multiplexing is one of the main features of LBs. So shouldn't Jboss use different threads for different HTTP requests in the same TCP connection? or this is not technically possible?