2 Replies Latest reply on Mar 1, 2009 2:58 AM by ron_sigal

    Timeout waiting for a free socket when an ejb publish a jms

    slimamar

      Hello,

      We have an (service) ejb which publish jms message to 3 topics.
      On one topic, it publishes periodically (every 10 seconds) and with this topic
      we have sometimes an error which occurs (see logs below) at server side
      and after the jms clients don't receive any message from this topic.
      We have no error from the client side.

      What's wrong.

      Thanks in advance.

      2009-02-16 18:29:55,576 DEBUG [org.jboss.remoting.transport.socket.SocketClientInvoker] (Pulseur) SocketClientInvoker[1403c543, bisocket://163.104.9.59:199620612]
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://163.104.9.59:199620612/callback?callbackServerHost=163.104.9.59&callbackServerPort=199620612&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&datatype=jms&guid=4j2w91n-726q7f-fr9akbzt-1-fr9akdue-f&isCallbackServer=true&onewayThreadPool=org.jboss.jms.server.remoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper]
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:771)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:423)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:160)
       at org.jboss.remoting.Client.invoke(Client.java:1708)
       at org.jboss.remoting.Client.invoke(Client.java:612)
       at org.jboss.remoting.Client.invokeOneway(Client.java:660)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:832)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:705)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1446)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1358)
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:327)
       at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
       at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:583)
       at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
       at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:610)
       at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:506)
       at org.jboss.messaging.core.impl.ChannelSupport.handle(ChannelSupport.java:238)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostOffice.java:2212)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.java:499)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:763)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:397)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeTarget(SessionAdvised$send_7280680627620114891.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
       at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
       at sun.reflect.GeneratedMethodAccessor298.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
       at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:908)
       at org.jboss.remoting.ServerInvoker$1.run(ServerInvoker.java:2009)
       at org.jboss.jms.server.remoting.DirectThreadPool.run(DirectThreadPool.java:63)
       at org.jboss.remoting.ServerInvoker.handleOnewayInvocation(ServerInvoker.java:2020)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:857)
       at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:106)
       at org.jboss.remoting.Client.invoke(Client.java:1708)
       at org.jboss.remoting.Client.invoke(Client.java:612)
       at org.jboss.remoting.Client.invokeOneway(Client.java:660)
       at org.jboss.remoting.Client.invokeOneway(Client.java:848)
       at org.jboss.remoting.Client.invokeOneway(Client.java:838)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:180)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvokeOneway(DelegateSupport.java:165)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(ClientSessionDelegate.java:495)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeTarget(ClientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
       at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
       at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect_z_handleSend_1523917841.invoke(SessionAspect_z_handleSend_1523917841.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
       at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:269)
       at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect_z_handleSend_1523917841.invoke(ProducerAspect_z_handleSend_1523917841.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:164)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:207)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:145)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:136)
       at org.jboss.jms.client.JBossMessageProducer.publish(JBossMessageProducer.java:177)
       at rte.gsr.app.serveur.ServeurProcessus$Pulseur.run(ServeurProcessus.java:103)
      Caused by: java.lang.IllegalStateException: Timeout waiting for a free socket
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1048)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:757)
       ... 71 more
      2009-02-16 18:29:55,577 DEBUG [com.arjuna.ats.jta.logging.loggerI18N] (Thread-25) [com.arjuna.ats.internal.jta.recovery.info.firstpass] Local XARecoveryModule - first pass
      2009-02-16 18:29:59,485 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] (Pulseur) DefaultCallbackErrorHandler[SocketServerInvoker[163.104.9.161:4457]] handling org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://163.104.9.59:199620612/callback?callbackServerHost=163.104.9.59&callbackServerPort=199620612&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&datatype=jms&guid=4j2w91n-726q7f-fr9akbzt-1-fr9akdue-f&isCallbackServer=true&onewayThreadPool=org.jboss.jms.server.remoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper]. Number of errors so far 1
      2009-02-16 18:29:59,485 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] (Pulseur) DefaultCallbackErrorHandler[SocketServerInvoker[163.104.9.161:4457]] ignoring the callback error
      2009-02-16 18:29:59,485 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] (Pulseur) Error handling callback
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://163.104.9.59:199620612/callback?callbackServerHost=163.104.9.59&callbackServerPort=199620612&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&datatype=jms&guid=4j2w91n-726q7f-fr9akbzt-1-fr9akdue-f&isCallbackServer=true&onewayThreadPool=org.jboss.jms.server.remoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper]
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:771)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:423)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:160)
       at org.jboss.remoting.Client.invoke(Client.java:1708)
       at org.jboss.remoting.Client.invoke(Client.java:612)
       at org.jboss.remoting.Client.invokeOneway(Client.java:660)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:832)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:705)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1446)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1358)
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:327)
       at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
       at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:583)
       at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
       at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:610)
       at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:506)
       at org.jboss.messaging.core.impl.ChannelSupport.handle(ChannelSupport.java:238)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostOffice.java:2212)
       at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.java:499)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:763)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:397)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeTarget(SessionAdvised$send_7280680627620114891.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
       at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
       at sun.reflect.GeneratedMethodAccessor298.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
       at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:908)
       at org.jboss.remoting.ServerInvoker$1.run(ServerInvoker.java:2009)
       at org.jboss.jms.server.remoting.DirectThreadPool.run(DirectThreadPool.java:63)
       at org.jboss.remoting.ServerInvoker.handleOnewayInvocation(ServerInvoker.java:2020)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:857)
       at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:106)
       at org.jboss.remoting.Client.invoke(Client.java:1708)
       at org.jboss.remoting.Client.invoke(Client.java:612)
       at org.jboss.remoting.Client.invokeOneway(Client.java:660)
       at org.jboss.remoting.Client.invokeOneway(Client.java:848)
       at org.jboss.remoting.Client.invokeOneway(Client.java:838)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:180)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvokeOneway(DelegateSupport.java:165)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(ClientSessionDelegate.java:495)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeTarget(ClientSessionDelegate$send_6145266547759487588.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
       at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
       at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect_z_handleSend_1523917841.invoke(SessionAspect_z_handleSend_1523917841.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
       at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:269)
       at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect_z_handleSend_1523917841.invoke(ProducerAspect_z_handleSend_1523917841.java)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
       at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:164)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:207)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:145)
       at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:136)
       at org.jboss.jms.client.JBossMessageProducer.publish(JBossMessageProducer.java:177)
       at rte.gsr.app.serveur.ServeurProcessus$Pulseur.run(ServeurProcessus.java:103)
      Caused by: java.lang.IllegalStateException: Timeout waiting for a free socket
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1048)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:757)
       ... 71 more
      




        • 1. Re: Timeout waiting for a free socket when an ejb publish a
          slimamar

          A topic has also created from the messaging forum :
          http://www.jboss.org/index.html?module=bb&op=viewtopic&t=150732

          • 2. Re: Timeout waiting for a free socket when an ejb publish a
            ron_sigal

            The message

            Caused by: java.lang.IllegalStateException: Timeout waiting for a free socket
             at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInv
            oker.java:1048)
            


            indicates that all of the pooled connections in the callback client invoker (the Remoting invoker on the server side that sends messages to the client side) are in use, and the pool has reached its maximum size. Now, in this particular case, JBossMessaging explicitly configures the callback client invoker to have a maximum pool size of one, which they do to prevent messages from being delivered out of order. So you don't have the ordinarily available option of increasing the pool size.

            The suggestion made by Andy Taylor on the JBM thread, increasing the 'connectionWait' parameter, is a good one, but note that it is implemented only in Remoting 2.4.0.GA and above.