2 Replies Latest reply on Sep 1, 2010 5:23 AM by Jesus Salvo

    On shutdown of a node, SFSB client gets "org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://x.x.x.x:3873/]"

    Jesus Salvo Newbie

      JBoss 5.0.1 GA

       

      On a 2-node cluster, I am getting the exception below from the SFSB client, not in the JBoss AS itself.

       

      A) This exception does __NOT__ occur when:

       

      1) Start nodeA

      2) Start nodeB

      3) EJB client connects and calls methods on SFSB ( say client has stickyness on nodeA ).

      4) Shutdown nodeA

      5) EJB client calls methods on SFSB. Client detects failure, now connects and failsover to nodeB.

       

       

      B) However, it happens when:

       

      1) Start nodeA

      2) EJB client connects and calls methods on SFSB ( say client has stickyness on nodeA ). Works OK.

      3) Start nodeB

      4) Shutdown nodeA ( ensure nodeB is completely started )

      5) EJB client calls methods on SFSB. Client detects failure, but does NOT failover to node B.

       

      The exception for the above scenario will have "Problem establishing socket connection for InvokerLocator [socket://nodeA:3873/] ".

      Is this because the client stub obtained at the time did not know of the changed cluster topology after it connected ?

       

       

      C) The exception also happens when:

       

      1) Start nodeA

      2) Start nodeB

      3) EJB client connects and calls methods on SFSB ( say client has stickyness on nodeA ).

      4) Shutdown nodeA

      5) EJB client calls methods on SFSB. Client detects failure, now connects and failsover to nodeB.

      6) Start nodeA

      7) EJB client calls methods on SFSB. Still works OK ( stickyness on nodeB )

      8) Shutdown nodeB

      9) EJB client calls methods on SFSB. Client detects failure, but does NOT failover to nodeA.

       

      The exception for the above scenario will have "Problem establishing socket connection for InvokerLocator [socket://nodeB:3873/] ".

       

      Does the remoting client ever get updated with the changed cluster topology, or is it only obtained once at the time of connecting ?

      If so, then why for the third scenario (C) above, in step [9], it did not failover to nodeA ? Or does the client stub also simply remove a node from its list once it is unreachable even after the same node is started up again later ?

       

       

      {noformat}

      java.lang.RuntimeException: cluster invocation failed, last exception was:
          at org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:173)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
          at $Proxy3.invoke(Unknown Source)
          at org.jboss.ejb3.proxy.handler.ProxyInvocationHandlerBase.invoke(ProxyInvocationHandlerBase.java:261)
          at org.jboss.ejb3.proxy.handler.session.SessionSpecProxyInvocationHandlerBase.invoke(SessionSpecProxyInvocationHandlerBase.java:101)
          at $Proxy2.putMessagingBuffer(Unknown Source)

      < ...snipped..>

          at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1849)
          at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2169)
          at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:420)
          at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:258)
          at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
          at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:231)
          at java.awt.Component.processMouseEvent(Component.java:5517)
          at javax.swing.JComponent.processMouseEvent(JComponent.java:3135)
          at java.awt.Component.processEvent(Component.java:5282)
          at java.awt.Container.processEvent(Container.java:1966)
          at java.awt.Component.dispatchEventImpl(Component.java:3984)
          at java.awt.Container.dispatchEventImpl(Container.java:2024)
          at java.awt.Component.dispatchEvent(Component.java:3819)
          at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4212)
          at java.awt.LightweightDispatcher.processMouseEvent(Container.java:3892)
          at java.awt.LightweightDispatcher.dispatchEvent(Container.java:3822)
          at java.awt.Container.dispatchEventImpl(Container.java:2010)
          at java.awt.Window.dispatchEventImpl(Window.java:1791)
          at java.awt.Component.dispatchEvent(Component.java:3819)
          at java.awt.EventQueue.dispatchEvent(EventQueue.java:463)
          at java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:242)
          at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:163)
          at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:157)
          at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:149)
          at java.awt.EventDispatchThread.run(EventDispatchThread.java:110)
      Caused by: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://aas023571:3873/]
          at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:771)
          at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:160)
          at org.jboss.remoting.Client.invoke(Client.java:1708)
          at org.jboss.remoting.Client.invoke(Client.java:612)
          at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
          ... 44 more
      Caused by: java.net.ConnectException: Connection refused: connect
          at java.net.PlainSocketImpl.socketConnect(Native Method)
          at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
          at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
          at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:367)
          at java.net.Socket.connect(Socket.java:524)
          at org.jboss.remoting.util.SecurityUtility.connect(SecurityUtility.java:1037)
          at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:203)
          at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1084)
          at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:757)
          at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:160)
          at org.jboss.remoting.Client.invoke(Client.java:1708)
          at org.jboss.remoting.Client.invoke(Client.java:612)
          at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
          at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
          at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
          at $Proxy3.invoke(Unknown Source)
          at org.jboss.ejb3.proxy.handler.ProxyInvocationHandlerBase.invoke(ProxyInvocationHandlerBase.java:261)
          at org.jboss.ejb3.proxy.handler.session.SessionSpecProxyInvocationHandlerBase.invoke(SessionSpecProxyInvocationHandlerBase.java:101)
          at $Proxy2.putMessagingBuffer(Unknown Source)
      < ...snipped..>
          at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1849)
          at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2169)
          at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:420)
          at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:258)
          at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
          at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:231)
          at java.awt.Component.processMouseEvent(Component.java:5517)
          at javax.swing.JComponent.processMouseEvent(JComponent.java:3135)
          at java.awt.Component.processEvent(Component.java:5282)
          at java.awt.Container.processEvent(Container.java:1966)
          at java.awt.Component.dispatchEventImpl(Component.java:3984)
          at java.awt.Container.dispatchEventImpl(Container.java:2024)
          at java.awt.Component.dispatchEvent(Component.java:3819)
          at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4212)
          at java.awt.LightweightDispatcher.processMouseEvent(Container.java:3892)
          at java.awt.LightweightDispatcher.dispatchEvent(Container.java:3822)
          at java.awt.Container.dispatchEventImpl(Container.java:2010)
          at java.awt.Window.dispatchEventImpl(Window.java:1791)
          at java.awt.Component.dispatchEvent(Component.java:3819)
          at java.awt.EventQueue.dispatchEvent(EventQueue.java:463)
          at java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:242)
          at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:163)
          at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:157)
          at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:149)
          at java.awt.EventDispatchThread.run(EventDispatchThread.java:110)
          at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
          ... 46 more
      {noformat}