4 Replies Latest reply on Jul 17, 2009 1:39 PM by Ron Sigal

    javax.jms.JMSException: Failure on underlying remoting conne

    Kris Fraley Newbie

      JBoss EAP 4.3.0.GA_CP02
      JBoss Messaging 1.4.0.SP3-CP03
      JBoss Remoting 2.2.2.SP9

      My client code is setup to consume messages from the testQueue in the out-of-the-box installation of the aforementioned EAP. Also, I have a javax.jms.ExceptionListener callback setup to monitor the connection. Another client begins sending messages to testQueue and my client consumes them ... all is working well. However, the exception listener's onException callback appears to be spuriously called for no apparent reason. If I ignore the spurious callbacks, I can just continue consuming messages as if no problem ever occurred. However, I would like to use the callback to indicate that I really do have a lost connection which cannot be ignored! I have logged these callbacks and they are coming randomly every 30 seconds to 10 minutes apart. remoting-bisocket-service.xml contains ...



      <!-- Standard bisocket connector - the bisocket transport only opens connection from client->server
      so can be used with firewalls where only outgoing connections are allowed.
      For examples of HTTP and SSL transports see docs/examples -->
      <mbean code="org.jboss.remoting.transport.Connector"
      name="jboss.messaging:service=Connector,transport=bisocket"
      display-name="Bisocket Transport Connector">




      <!-- There should be no reason to change these parameters - warning!
      Changing them may stop JBoss Messaging working correctly -->
      org.jboss.jms.wireformat.JMSWireFormat
      org.jboss.jms.wireformat.JMSWireFormat
      jms
      false
      ${jboss.bind.address}
      4457
      org.jboss.jms.client.remoting.ClientSocketWrapper
      org.jboss.jms.server.remoting.ServerSocketWrapper

      1
      214748364
      10
      org.jboss.jms.server.remoting.DirectThreadPool
      <!-- End immutable parameters -->

      true


      <!-- Periodicity of client pings. Server window by default is twice this figure -->
      <!-- 10000 -->
      3600000


      <!-- added by KMF -->
      10000
      5000
      false


      0

      <!-- Number of seconds to wait for a connection in the client pool to become free -->
      10

      <!-- Max Number of connections in client pool. This should be significantly higher than
      the max number of sessions/consumers you expect -->
      200

      <!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->
      10000

      <!-- Use these parameters to specify values for binding and connecting control connections to
      work with your firewall/NAT configuration
      xyz
      abc
      -->



      org.jboss.jms.server.remoting.JMSServerInvocationHandler







      Being a JBoss novice, I wouldn't be surprised if this is just a configuration issue because this seems like basic funtionality. As you might observe in the configuration above, I have tried the workaround given here ... https://jira.jboss.org/jira/browse/SOA-835

      Is there anyone that can give me some advice on how to make these exceptions go away? Thanks for your help!
      Kris

        • 1. Re: javax.jms.JMSException: Failure on underlying remoting c
          Kris Fraley Newbie

          Oops, how about this ...

          <server>
          
           <!-- Standard bisocket connector - the bisocket transport only opens connection from client->server
           so can be used with firewalls where only outgoing connections are allowed.
           For examples of HTTP and SSL transports see docs/examples -->
           <mbean code="org.jboss.remoting.transport.Connector"
           name="jboss.messaging:service=Connector,transport=bisocket"
           display-name="Bisocket Transport Connector">
           <attribute name="Configuration">
           <config>
           <invoker transport="bisocket">
          
           <!-- There should be no reason to change these parameters - warning!
           Changing them may stop JBoss Messaging working correctly -->
           <attribute name="marshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
           <attribute name="unmarshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
           <attribute name="dataType" isParam="true">jms</attribute>
           <attribute name="socket.check_connection" isParam="true">false</attribute>
           <attribute name="serverBindAddress">${jboss.bind.address}</attribute>
           <attribute name="serverBindPort">4457</attribute>
           <attribute name="clientSocketClass" isParam="true">org.jboss.jms.client.remoting.ClientSocketWrapper</attribute>
           <attribute name="serverSocketClass">org.jboss.jms.server.remoting.ServerSocketWrapper</attribute>
          
           <attribute name="numberOfCallRetries" isParam="true">1</attribute>
           <attribute name="pingFrequency" isParam="true">214748364</attribute>
           <attribute name="pingWindowFactor" isParam="true">10</attribute>
           <attribute name="onewayThreadPool">org.jboss.jms.server.remoting.DirectThreadPool</attribute>
           <!-- End immutable parameters -->
          
           <attribute name="stopLeaseOnFailure" isParam="true">true</attribute>
          
           <!-- Periodicity of client pings. Server window by default is twice this figure -->
           <!-- <attribute name="clientLeasePeriod" isParam="true">10000</attribute> -->
           <attribute name="clientLeasePeriod" isParam="true">3600000</attribute>
          
          
           <!-- added by KMF -->
           <attribute name="validatorPingPeriod" isParam="true">10000</attribute>
           <attribute name="validatorPingTimeout" isParam="true">5000</attribute>
           <attribute name="registerCallbackListener">false</attribute>
          
          
           <attribute name="timeout" isParam="true">0</attribute>
          
           <!-- Number of seconds to wait for a connection in the client pool to become free -->
           <attribute name="numberOfRetries" isParam="true">10</attribute>
          
           <!-- Max Number of connections in client pool. This should be significantly higher than
           the max number of sessions/consumers you expect -->
           <attribute name="JBM_clientMaxPoolSize" isParam="true">200</attribute>
          
           <!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->
           <attribute name="callbackTimeout">10000</attribute>
          
           <!-- Use these parameters to specify values for binding and connecting control connections to
           work with your firewall/NAT configuration
           <attribute name="secondaryBindPort">xyz</attribute>
           <attribute name="secondaryConnectPort">abc</attribute>
           -->
          
           </invoker>
           <handlers>
           <handler subsystem="JMS">org.jboss.jms.server.remoting.JMSServerInvocationHandler</handler>
           </handlers>
           </config>
           </attribute>
           </mbean>
          
          </server>



          • 2. Re: javax.jms.JMSException: Failure on underlying remoting c
            Ron Sigal Master

            Hi Kris,

            I don't know enough about JBossMessaging to comment on these ExceptionListener callbacks. Could you tell me what they are saying?

            Thanks,
            Ron

            • 3. Re: javax.jms.JMSException: Failure on underlying remoting c
              Kris Fraley Newbie

              Yes, when exception listener callback occurs, the exception being reported is essentially the subject (title) of this post, that is, javax.jms.JMSException: Failure on underlying remoting connection. So, I concluded that I have a remoting problem ... made sense to me.

              I do have somewhat of an update. Further investigation into the spurious exceptions related to the underlying remoting connection ultimately led me to the topic Unexplained ConnectionValidator Timeout (see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=151168 for details). After reading the description and noting that the topic poster was using Red Hat (RH) JBoss EAP 4.3.0.GA_CP03, I did the obvious … I updated from RH JBoss EAP 4.3.0.GA_CP02 to 4.3.0.GA_CP04 (latest/greatest non-alpha install). Then, I re-ran my test. The test ran overnight with over 150K message transfers to the JMS queue and over 150K from JMS queue without any random exceptions "heard" on the client connections. The spurious exceptions appeared to have completely disappeared!

              Unfortunately, I ran my test again several days later, executed roughly the same number of message transfers, but observed one failure.

              So, it appears the problem may still be there, but has been greatly reduced.

              • 4. Re: javax.jms.JMSException: Failure on underlying remoting c
                Ron Sigal Master

                Hi Kris,

                Well, that just shows that every day, in every way, we get better and better. :)

                I have two suggestions.

                1. On a busy network, or busy hosts, a spurious failure is always possible. If you lengthen the ping cycle, e.g.,

                 <attribute name="validatorPingPeriod" isParam="true">20000</attribute>
                 <attribute name="validatorPingTimeout" isParam="true">10000</attribute>
                


                you might reduce the number of occurrences.

                2. We just added some major fixes on the JBossRemoting / JBossMessaging boundary in EAP 4.3.CP05 (Remoting 2.2.3 / JBM 1.4.0.SP3_CP08), which seems to be available now. I'm not sure if they will affect your situation, but I think that anyone using JBossMessaging should probably upgrade to these new jars.

                Of course, if you have a support contract, you would have to do the whole upgrade.

                -Ron