11 Replies Latest reply on Jan 3, 2014 1:05 AM by mike just

    bridge related: not working between source server and target server from a Vitual machine

    mike just Master

      The bridge is working well between two servers on same machine, but not working when I use a target server which is a Vitual machine running in source server.

       

      The error message from target server:

      2013-12-22 22:06:12,687;[MSC service thread 1-1];ERROR;org.hornetq.core.server;HQ224002: Failure in initialisation: org.jboss.netty.channel.ChannelException: Failed to bind to: /192.168.1.2:5447
        at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
        at org.hornetq.core.remoting.impl.netty.NettyAcceptor.startServerChannels(NettyAcceptor.java:505)
        at org.hornetq.core.remoting.impl.netty.NettyAcceptor.start(NettyAcceptor.java:458)
        at org.hornetq.core.remoting.server.impl.RemotingServiceImpl.start(RemotingServiceImpl.java:256)
        at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1542)
        at org.hornetq.core.server.impl.HornetQServerImpl.access$1300(HornetQServerImpl.java:164)
        at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingLiveActivation.run(HornetQServerImpl.java:2498)
        at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:416)
        at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:464)
        at org.jboss.as.messaging.jms.JMSService.start(JMSService.java:74)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
      Caused by: java.net.BindException: Cannot assign requested address: JVM_Bind
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
        at java.net.ServerSocket.bind(ServerSocket.java:328)
        at org.jboss.netty.channel.socket.oio.OioServerSocketPipelineSink.bind(OioServerSocketPipelineSink.java:128)
        at org.jboss.netty.channel.socket.oio.OioServerSocketPipelineSink.handleServerSocket(OioServerSocketPipelineSink.java:79)
        at org.jboss.netty.channel.socket.oio.OioServerSocketPipelineSink.eventSunk(OioServerSocketPipelineSink.java:53)
        at org.jboss.netty.channel.Channels.bind(Channels.java:561)
        at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:189)
        at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:382)
        at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
        at org.jboss.netty.channel.socket.oio.OioServerSocketChannel.<init>(OioServerSocketChannel.java:78)
        at org.jboss.netty.channel.socket.oio.OioServerSocketChannelFactory.newChannel(OioServerSocketChannelFactory.java:127)
        at org.jboss.netty.channel.socket.oio.OioServerSocketChannelFactory.newChannel(OioServerSocketChannelFactory.java:87)
        at org.jboss.netty.bootstrap.ServerBootstrap.bindAsync(ServerBootstrap.java:329)
        at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:266)
        ... 14 more
      
      

       

      Another error message in source server:

      2013-12-23 14:07:16,941;[Old I/O client worker ([id: 0xa1f1a0d2, /192.168.1.2:59223 => /192.168.1.4:5447])];ERROR;org.hornetq.core.client;HQ214036: Failed to decode packet: java.lang.IllegalArgumentException: HQ119074: Invalid type: 0
        at org.hornetq.core.protocol.core.impl.PacketDecoder.decode(PacketDecoder.java:427)
        at org.hornetq.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:64)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:485)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1658)
        at org.hornetq.core.remoting.impl.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:72)
        at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:787)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:281)
        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:169)
        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:134)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.oio.OioWorker.process(OioWorker.java:71)
        at org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73)
        at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:51)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
        at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:175)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)
      
      
      

       

      Settings on source server:

      <connectors>
           <netty-connector name="netty" socket-binding="messaging"/>
            <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                 <param key="batch-delay" value="50"/>
            </netty-connector>
            <in-vm-connector name="in-vm" server-id="0"/>
                 <connector name="remote-connector">
                      <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
                      <param key="host" value="192.168.1.4"/>
                      <param key="port" value="5447"/>
                 </connector>
      </connectors>
      
      

       

      192.168.1.4 is the ip address of the VM server.

       

      settings on target server from the VM:

      <acceptors>
          <netty-acceptor name="netty" socket-binding="messaging"/>
          <n
      etty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
               <param key="batch-delay" value="50"/>
               <param key="direct-deliver" value="false"/>
          </netty-acceptor>
          <in-vm-acceptor name="in-vm" server-id="0"/>
          <acceptor name="remote-connector">
               <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
               <param key="host" value="192.168.1.2"/>    
               <param key="port" value="5447"/>
          </acceptor>
      </acceptors>
      
      

      192.168.1.2 is the ip address of the source server.

       

      The bridge works fine on the same machine (two servers) if the "<param key="host" value="192.168.1.x"/>" is not configured.

        • 1. Re: bridge related: not working between source server and target server from a Vitual machine
          Justin Bertram Master

          You don't need to configure a special acceptor on the "target" server, and even if you did it wouldn't use the IP address of the "source" server.  An acceptor can only bind to a local address - an address it can use to accept connections.

           

          The connector on the source server simply needs to point to the target server on the normal HornetQ port (i.e. 5445).

           

          Furthermore, it's better to use a netty-connector with a socket-binding rather than a generic connector.  For example:

           

          <netty-connector name="remote-connector" socket-binding="remote-socket-binding"/> 
          ...
          <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
            ...
              <outbound-socket-binding name="remote-socket-binding">
                  <remote-destination host="192.168.1.4" port="5445"/>
              </outbound-socket-binding>
          </socket-binding-group>
          
          • 2. Re: Re: bridge related: not working between source server and target server from a Vitual machine
            mike just Master

            Thanks. Do you mean an accepter named like "remote-connector" is not needed on "target" server? I have removed the accepter named "remote-connector" on "target" server and added the netty-connector and outbound-socket-binding as you mentioned on "source" server, but no good result.

             

            Here is the connect time out error on "source" server. I have started both "source" and "target" server.

            2013-12-25 11:21:02,550;[Thread-0 (HornetQ-server-HornetQServerImpl::serverUUID=a3f8e6e8-6b84-11e3-a261-0b6a9beedf18-1203129835)];ERROR;org.hornetq.core.client;HQ214045: Failed to create netty connection: java.net.SocketTimeoutException: connect timed out
              at java.net.PlainSocketImpl.socketConnect(Native Method)
              at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
              at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
              at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
              at java.net.Socket.connect(Socket.java:519)
              at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.connect(OioClientSocketPipelineSink.java:105)
              at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.eventSunk(OioClientSocketPipelineSink.java:65)
              at org.jboss.netty.channel.Channels.connect(Channels.java:634)
              at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:207)
              at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:229)
              at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
              at org.hornetq.core.remoting.impl.netty.NettyConnector.createConnection(NettyConnector.java:505)
              at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnection(ClientSessionFactoryImpl.java:1225)
              at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:1071)
              at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:246)
              at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:826)
              at org.hornetq.core.server.cluster.impl.BridgeImpl.createSessionFactory(BridgeImpl.java:766)
              at org.hornetq.core.server.cluster.impl.BridgeImpl.connect(BridgeImpl.java:795)
              at org.hornetq.core.server.cluster.impl.BridgeImpl$ConnectRunnable.run(BridgeImpl.java:1101)
              at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:106)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
              at java.lang.Thread.run(Thread.java:619)
            
            

             

            Anything still missed to make it run?

             

            BTW, I have not found any schema definition for "outbound-socket-binding" in jboss-as-messaging_1_3.xsd.

            • 3. Re: bridge related: not working between source server and target server from a Vitual machine
              mike just Master

              I have also tried with the "target" server running on the same host machine with "source" server and also not working if no acceptor is defined on "target" server. The only difference is there is no "connect time out" error.

              • 4. Re: bridge related: not working between source server and target server from a Vitual machine
                Justin Bertram Master

                Do you mean an accepter named like "remote-connector" is not needed on "target" server?

                That's correct.

                 

                Anything still missed to make it run?

                Since it isn't working, I would say yes, there is still something missing. 

                 

                When you start the "target" server are you binding it to the interface to which the "source" server is trying to connect (i.e. 192.168.1.4)?  Is the "target" server using any kind of port binding offset?

                 

                The whole connector/acceptor concept is pretty simple.  A connector just needs to point to a host:port where there is an acceptor listening for connections.  It's as simple as that.

                 

                BTW, I have not found any schema definition for "outbound-socket-binding" in jboss-as-messaging_1_3.xsd.

                That's because it isn't in there.  The <socket-binding-group> is not part of the messaging subsystem.  Look in, e.g. jboss-as-config_1_3.xsd.

                • 5. Re: Re: bridge related: not working between source server and target server from a Vitual machine
                  mike just Master

                  When you start the "target" server are you binding it to the interface to which the "source" server is trying to connect (i.e. 192.168.1.4)?

                  Not sure how to do this? Can you adive a little bit more? I do not think I have done anything related to the binding for now.

                   

                  Is the "target" server using any kind of port binding offset?

                  I do not think so.

                  • 6. Re: Re: bridge related: not working between source server and target server from a Vitual machine
                    Justin Bertram Master

                    Not sure how to do this? Can you adive a little bit more? I do not think I have done anything related to the binding for now.

                    When just about any piece of software that is going to accept network connections starts it binds to one (or more) of the machines network interfaces so it can actually receive connections.  Most machines on a network have several interfaces - at least one "loopback" interface (e.g. 127.0.0.1) that can only be used by local clients and at least one remote interface that can be used by remote clients connecting to the machine.  By default JBossAS/Wildfly will bind to 127.0.0.1 which means that no remote clients can connect to it.  This default behavior can be overridden by using a command line switch (e.g. "-b" or "-Djboss.bind.address=...") or by modifying the "public" interface in the <interfaces> subsystem in standalone*.xml or domain.xml.  If you're not doing this on your "target" server then nobody will be able to connect to it since it will be bound by default to a loopback interface.

                    • 7. Re: Re: bridge related: not working between source server and target server from a Vitual machine
                      mike just Master

                      Understood. Thanks for the explanation. And I just tried and confirmed it works fine now. Thanks so much.

                       

                      But I am wondering if there is a way to use a new interface with the address (192.168.1.4) bound and then specify this interface to the remote-socket-binding the "source" server defined instead of modifying the "public" interface in "target" server standalone*.xml, like

                      <socket-binding name="remote-socket-binding" interface="for-remote-connect" port="${xxxxxxxx:xxxx}"/>
                      
                      
                      
                      

                       

                      Can this be done?

                       

                      Or is there any other method to configure this beside setting the outbound-socket-binding?

                      • 8. Re: Re: bridge related: not working between source server and target server from a Vitual machine
                        Justin Bertram Master

                        What you want to do should be possible.  Just look through the jboss-as-config_1_3.xsd schema for the configuration options for <socket-binding> and <interface>.

                        • 9. Re: Re: Re: bridge related: not working between source server and target server from a Vitual machine
                          mike just Master

                          Hi Justin,

                           

                          I tried following settings in "target" server's standalone.xml and it seems working fine. One question is why the port bridge uses has to be 5445 (the same number with the "target" server's messaging socket-binding port)? I used other port number but did not work. Can you explain it a little bit?

                           

                          
                          <interface name="bridge">
                                 <inet-address value="${jboss.bind.address.unsecure:192.168.1.4}"/>
                          </interface>
                           
                          <socket-binding name="bridge" interface="bridge" port="${jboss.bridge.port:5445}"/>
                           

                           

                          The outbound-socket-binding in "source" server is

                           

                          
                          <outbound-socket-binding name="remote-socket-binding">
                               <remote-destination host="192.168.1.4" port="5445"/>
                          </outbound-socket-binding>
                           

                           

                           

                          As I see there are two socket-bindings related to the netty connector: one is "messaging", the other is called "messaging-throughput". What is the difference between them and how are they used? I tried both "5445" and "5455" port numbers in above setting and they both work fine.

                           

                          <netty-connector name="netty" socket-binding="messaging"/>
                          <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                               <param key="batch-delay" value="50"/>
                          </netty-connector>
                           
                          <socket-binding name="messaging" port="5445"/>
                          <socket-binding name="messaging-throughput" port="5455"/>
                          
                           

                           

                          If above method I tried has no problem, then does this grammar look perfect enough or is there any other better setting to satisfy the same target?

                           

                          <socket-binding name="bridge" interface="bridge" port="${jboss.bridge.port:5445}"/>
                           
                          • 10. Re: Re: Re: bridge related: not working between source server and target server from a Vitual machine
                            Justin Bertram Master

                            One question is why the port bridge uses has to be 5445 (the same number with the "target" server's messaging socket-binding port)? I used other port number but did not work. Can you explain it a little bit?

                            The bridge has to connect to a port where the server is actually listening (e.g. 5445 or 5455 by default).  I'm not sure I can explain it any more simply than that.  I believe we've discussed this already on this thread and it's certainly covered in the documentation (e.g. here and here).

                             

                            As I see there are two socket-bindings related to the netty connector: one is "messaging", the other is called "messaging-throughput". What is the difference between them and how are they used? I tried both "5445" and "5455" port numbers in above setting and they both work fine.

                            Each acceptor has a matching connector so that, for example, a JMS connection factory can be configured with a specific connector to connect to a specific acceptor.  The "messaging" connector/acceptor pair has the most basic configuration possible.  Every configuration value uses the default.  The "messaging-throughput" connector/acceptor pair is slightly more advanced.  It has a few settings to enable greater overall throughput (hence the name).  You can read more about all the configuration options (including those already set for "messaging-throughput") in the documentation.

                             

                            In general, I recommend you thoroughly review the HornetQ User Guide as I think it would help clarify many areas of confusion for you.

                            • 11. Re: Re: Re: bridge related: not working between source server and target server from a Vitual machine
                              mike just Master

                              Thanks for all your help Justin. So far it is running so good. I could continue bother you if I meet with more problems..