1 Reply Latest reply on Sep 4, 2009 7:53 PM by Ron Sigal

    Weird port number on connection fail

    Chip Schoch Novice

      We are running JBossAS 4.2.2 with JBM 1.4.4. Recently we experience a problem and I am trying to track it down. It appeared to be a network issue in that we got the following in the log:

      2009-08-25 10:45:22,902 ERROR | Thread-24 | org.jboss.remoting.callback.ServerInvokerCa
      llbackHandler | Error handling callback
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing so
      cket connection for InvokerLocator [bisocket://10.33.32.22
      :1810191292/callback?callbackServerHost=10.33.32.22&callbackServerPort=1810191292&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jb
      oss.jms.client.remoting.ClientSocketWrapper&datatype=jms&guid=axwm-58z961-fy0jth8v-1-fy0jumsb-4y&isCallbackServer=true&onewayThreadPool=org.jboss.jms.server.r
      emoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper]
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientI
      nvoker.java:548)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvok
      er.java:422)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
       at org.jboss.remoting.Client.invoke(Client.java:1858)
       at org.jboss.remoting.Client.invoke(Client.java:718)
       at org.jboss.remoting.Client.invokeOneway(Client.java:769)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCall
      backHandler.java:829)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvok
      erCallbackHandler.java:700)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint
      .java:1467)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.
      java:1379)
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:3
      27)
       at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119
      )
       at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.jav
      a:583)
       at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistri
      butor.java:79)
       at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:670)
       at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:506)
       at org.jboss.messaging.core.impl.ChannelSupport.handle(ChannelSupport.java:251)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostO
      ffice.java:2228)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.ja
      va:515)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoi
      nt.java:769)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:399)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advise
      d$SessionAdvised$send$aop(SessionAdvised.java:87)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(
      SessionAdvised$send_7280680627620114891.java)
       at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
       at sun.reflect.GeneratedMethodAccessor114.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(
      SessionAdvised$send_7280680627620114891.java)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
       at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandle
      r.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:862)
       at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
      
       at org.jboss.remoting.Client.invoke(Client.java:1858)
       at org.jboss.remoting.Client.invoke(Client.java:718)
       at org.jboss.remoting.Client.invoke(Client.java:706)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientS
      essionDelegate$send$aop(ClientSessionDelegate.java:499)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(C
      lientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
       at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect28.invoke(SessionAspect2
      8.java)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(C
      lientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.j
      ava:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(C
      lientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(C
      lientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
       at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:276)
       at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect39.invoke(ProducerAspec
      t39.java)
       at org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(
      ClientProducerDelegate$send_3961598017717988886.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      


      I am particularly curious about the line


      callbackServerPort=1810191292

      Does this port number look to be correct? If not, could this be the cause of the failure to get a connection?

      Thanks alot.

        • 1. Re: Weird port number on connection fail
          Ron Sigal Master

          Actually, that number, strange as it seems, is fine. Normally, that number would be the port where the bisocket://10.33.32.22:1810191292/callback? ...
          callback server (on the client side) would be listening, but the bisocket transport doesn't open a port on the client, so this number is just a unique value to distinguish this InvokerLocator from any other InvokerLocators.

          When a callback client (on the server side), which JBossMessaging uses to sends messages to a consumer, wants a new socket, it writes a message to the client, and the exception you're seeing means that that process failed. It could be a problem with the connection over which that message is written, but it's more likely that there's a problem on the client side. Take a look at your client side logs. If you see a complaint from ConnectionValidator about a failed connection, try increasing the values of

           <attribute name="validatorPingPeriod" isParam="true">10000</attribute>
           <attribute name="validatorPingTimeout" isParam="true">5000</attribute>
          


          in your $JBOSS_HOME/server/$CONFIG/jboss-messaging.sar/remoting-bisocket-service.xml. That will reduce the likelihood of spurious connection failures on a busy network or between busy hosts.