4 Replies Latest reply on Jul 11, 2018 11:38 AM by chatur

    Hornetq connections capped at 4036

    chatur
    chatur Jul 11, 2018 9:04 AM

      hi All,

       

      We have  been using hornetq successfully for few years in our applications. We are still on version 2.4.2 ( I know its really old and we are long overdue for an upgrade).

       

      Recently there was an application code change that increased the number of connections made to Hornetq. We are seeing that hornetq is refusing any connection after it reaches a count of 4036. I am checking this via the following command.

       

      netstat -an |grep 54451 |wc -l

       

      once it reaches a count of 4036 - and new connections attempts are made we see one of the following two behaviors

       

      (1) An existing connection is dropped and the new connection succeeds

          (OR)

      (2) New connection hangs and timeouts.

       

      I have researched for a very long time and I am not sure where this limit is being imposed. We do not have any explicit hornetq configuration file that specifies this limit. I also checked ulimit and other OS level configs and didnt find any limits there either.

       

      Can someone please point in the direction that can help me find where this limit may be coming from and how I can up this limit ?

       

      Thanks

      • 212 Views
      • Tags:
        • 1. Re: Hornetq connections capped at 4036
          jbertram
          jbertram Jul 11, 2018 10:41 AM (in response to chatur)

          To my knowledge (as a HornetQ developer) I am not aware of any arbitrary limit which the broker enforces on the number of connections.  My feeling is this is an environmental issue.  Have you tried to reproduce this problem in a different environment?

            • Actions
          • 2. Re: Hornetq connections capped at 4036
            chatur
            chatur Jul 11, 2018 11:38 AM (in response to jbertram)

            Hi Justin,

             

            Thanks for your response.

             

            I was able to simulate this issue in UAT yesterday, via the following ( I used apache bench tool )

             

            ab -n 2900 -c 2900 http://UATSERVERNAME:UATPORT/

             

             

            I got the below response when I ran the above - notice the too many open files !

             

            This is ApacheBench, Version 2.3 <$Revision: 655654 $>

            Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

            Licensed to The Apache Software Foundation, http://www.apache.org/

             

            socket: Too many open files (24)

             

             

             

            When I saw open files error , I looked for the max open files allowed on the OS - and got

             

            cat /proc/sys/fs/file-max -- 601850

             

            I also checked for the number of open file handlers using -- lsof |wc -l .

             

            Is there anything I could check ?

              • Actions
            • 3. Re: Hornetq connections capped at 4036
              jbertram
              jbertram Jul 11, 2018 11:28 AM (in response to chatur)

              Unfortunately as we move further from the broker my ability to help declines.  Perhaps a Linux specific forum or mailing list would be a better place to ask such questions.

                • Actions