8 Replies Latest reply on Dec 4, 2012 4:17 PM by mpgong

    Jboss 7.1 final clustering - destination address may not be null

    mpgong

      Hello,

       

      I'm trying to create a simple cluster on one machine with a domain and after changing the interface definition to the ip address of my machine, the domain is starting with the following errors.

       

       

      [Server:server-two] 12:59:08,274 INFO  [org.xnio] (MSC service thread 1-4) XNIO Version 3.0.3.GA
      [Server:server-two] 12:59:08,284 INFO  [org.xnio.nio] (MSC service thread 1-4) XNIO NIO Implementation Version 3.0.3.GA
      [Server:server-one] 12:59:08,286 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC00001: Failed to start service jboss.host.controller.client: org.jboss.msc.service.StartException in service jboss.host.controller.client: java.lang.IllegalArgumentException: destination address may not b
      e null
      [Server:server-one]     at org.jboss.as.server.mgmt.domain.HostControllerServerClient.start(HostControllerServerClient.java:161) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
      [Server:server-one]     at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
      [Server:server-one]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_01]
      [Server:server-one]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_01]
      [Server:server-one]     at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_01]
      [Server:server-one] Caused by: java.lang.IllegalArgumentException: destination address may not be null
      [Server:server-one]     at org.jboss.remoting3.remote.RemoteConnectionProvider.connect(RemoteConnectionProvider.java:98) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-one]     at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:296) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-one]     at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:251) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-one]     at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:349) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-one]     at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:337) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-one]     at org.jboss.as.protocol.ProtocolChannelClient.connect(ProtocolChannelClient.java:125) [jboss-as-protocol-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     at org.jboss.as.protocol.ProtocolChannelClient.connectSync(ProtocolChannelClient.java:134) [jboss-as-protocol-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     at org.jboss.as.server.mgmt.domain.HostControllerServerConnection.openChannel(HostControllerServerConnection.java:158) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     at org.jboss.as.server.mgmt.domain.HostControllerServerConnection.connect(HostControllerServerConnection.java:86) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     at org.jboss.as.server.mgmt.domain.HostControllerServerClient.start(HostControllerServerClient.java:135) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-one]     ... 5 more
      [Server:server-one]
      [Server:server-two] 12:59:08,494 INFO  [org.jboss.remoting] (MSC service thread 1-4) JBoss Remoting version 3.2.2.GA
      [Server:server-two] 12:59:08,947 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-4) MSC00001: Failed to start service jboss.host.controller.client: org.jboss.msc.service.StartException in service jboss.host.controller.client: java.lang.IllegalArgumentException: destination address may not b
      e null
      [Server:server-two]     at org.jboss.as.server.mgmt.domain.HostControllerServerClient.start(HostControllerServerClient.java:161) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
      [Server:server-two]     at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
      [Server:server-two]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_01]
      [Server:server-two]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_01]
      [Server:server-two]     at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_01]
      [Server:server-two] Caused by: java.lang.IllegalArgumentException: destination address may not be null
      [Server:server-two]     at org.jboss.remoting3.remote.RemoteConnectionProvider.connect(RemoteConnectionProvider.java:98) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-two]     at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:296) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-two]     at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:251) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-two]     at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:349) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-two]     at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:337) [jboss-remoting-3.2.2.GA.jar:3.2.2.GA]
      [Server:server-two]     at org.jboss.as.protocol.ProtocolChannelClient.connect(ProtocolChannelClient.java:125) [jboss-as-protocol-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     at org.jboss.as.protocol.ProtocolChannelClient.connectSync(ProtocolChannelClient.java:134) [jboss-as-protocol-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     at org.jboss.as.server.mgmt.domain.HostControllerServerConnection.openChannel(HostControllerServerConnection.java:158) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     at org.jboss.as.server.mgmt.domain.HostControllerServerConnection.connect(HostControllerServerConnection.java:86) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     at org.jboss.as.server.mgmt.domain.HostControllerServerClient.start(HostControllerServerClient.java:135) [jboss-as-server-7.1.0.Final.jar:7.1.0.Final]
      [Server:server-two]     ... 5 more
      [Server:server-two]
      
      

       

      I've been following the AS7 cluster howto and the book by F. Marchioni, but can't get the cluster to work.  If i use the address 127.0.0.1 it seems to work fine, but that isn't what we want. 

      My env is running ipv4 and ipv6 which i think is the issue, but the server is started using the preferipv4 property.

       

      Any idea what i might be doing wrong? 

      Thanks.

        • 1. Re: Jboss 7.1 final clustering - destination address may not be null
          rhusar

          How did you configure the host? You can paste the XML.

           

          BTW 7.1.1 is downloadable here, http://www.jboss.org/jbossas/downloads/ might wanna give it a try

          • 2. Re: Jboss 7.1 final clustering - destination address may not be null
            mpgong

            I tried something else and it seemed to work ok, but not sure if it will work ok when i set up mod_cluster.

             

            I changed the interface to use <any-ipv4-address/>.

             

            Other than that my host file is unchanged.

             

             

            <?xml version='1.0' encoding='UTF-8'?>
            
            <host name="master" xmlns="urn:jboss:domain:1.1">
            
                <management>
                    <security-realms>
                        <security-realm name="ManagementRealm">
                            <authentication>
                                <properties path="mgmt-users.properties" relative-to="jboss.domain.config.dir"/>
                            </authentication>
                        </security-realm>
                        <security-realm name="ApplicationRealm">
                            <authentication>
                                <properties path="application-users.properties" relative-to="jboss.domain.config.dir" />
                            </authentication>
                        </security-realm>
                    </security-realms>
                    <management-interfaces>
                        <native-interface security-realm="ManagementRealm">
                            <socket interface="management" port="${jboss.management.native.port:9999}"/>
                        </native-interface>
                        <http-interface security-realm="ManagementRealm">
                            <socket interface="management" port="${jboss.management.http.port:9990}"/>
                        </http-interface>
                    </management-interfaces>
                </management>
            
                <domain-controller>
                   <local/>
                   <!-- Alternative remote domain controller configuration with a host and port -->
                   <!-- <remote host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}"/> -->
                </domain-controller>
            
                <interfaces>
                    <interface name="management">
                        <any-ipv4-address/>
                    </interface>
                    <interface name="public">
                       <any-ipv4-address/>
                    </interface>
                    <interface name="unsecure">
                        <!-- Used for IIOP sockets in the standarad configuration.
                             To secure JacORB you need to setup SSL -->
                        <any-ipv4-address/>
                    </interface>
                </interfaces>
            
                 <jvms>
                    <jvm name="default">
                      <heap size="64m" max-size="256m"/>
                      <permgen size="256m" max-size="256m"/>
                        <jvm-options>
                            <option value="-server"/>
                        </jvm-options>
                   </jvm>
                 </jvms>
            
                <servers>
                    <server name="server-one" group="main-server-group">
                        <!-- Remote JPDA debugging for a specific server
                        <jvm name="default">
                          <jvm-options>
                            <option value="-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"/>
                          </jvm-options>
                       </jvm>
                       -->
                    </server>
                    <server name="server-two" group="main-server-group" auto-start="true">
                        <!-- server-two avoids port conflicts by incrementing the ports in
                             the default socket-group declared in the server-group -->
                        <socket-bindings port-offset="150"/>
                    </server>
                    <server name="server-three" group="other-server-group" auto-start="false">
                        <!-- server-three avoids port conflicts by incrementing the ports in
                             the default socket-group declared in the server-group -->
                        <socket-bindings port-offset="250"/>
                    </server>
                </servers>
            </host>
            

             

            Thanks

            • 3. Re: Jboss 7.1 final clustering - destination address may not be null
              rhusar

              Honestly, I am not a big fan of <any-ipv4-address/> because if you just use one interface that (hopefully) is not changing so why expose the server to other interfaces if you dont use them?

               

              Otherwise whats the problem now? Can you setup clustering/mod_cluster and see whether it works?

              • 4. Re: Jboss 7.1 final clustering - destination address may not be null
                mpgong

                It is wierd in the sense that this one win7 machine i can't use the ip but on another i can.  So it could be an OS issue that is preventing me from using the an ip to bind the address.

                 

                Any thoughts as to what might cause this to happen on one win7 machine and not another?

                • 5. Re: Jboss 7.1 final clustering - destination address may not be null
                  rhusar

                  Yes, awkward win. What is the log, why you cannot? Try removing the interface and adding it again? Kill running processes that might use the ports? I have no idea, but really doesnt seem to be an AS issue.

                  • 6. Re: Jboss 7.1 final clustering - destination address may not be null
                    mpgong

                    well, it definitely an OS/hardware thing.  Switched to two other machines and now mod_cluster seems to be working better.

                     

                    Running into a new problem - after simulating a failed server case it worked at first but then it didn't - in the error.log i received this error

                     

                    [Tue Dec 04 13:08:17 2012] [error] proxy: CLUSTER: (balancer://other-server-group). All workers are in error state

                     

                    So now hitting the cluster-demo gives a 503 error code.

                     

                    Thoughts?

                     

                    Thanks

                    • 7. Re: Jboss 7.1 final clustering - destination address may not be null
                      rhusar

                      Looking at mod_cluster_manager page, all both nodes and all contexts registered?

                      • 8. Re: Jboss 7.1 final clustering - destination address may not be null
                        mpgong

                        in the mod_cluster_manager, i see the one node that is still running.  The other node i shutdown so i didn't expect to see it.

                         

                        This is what is looks like :

                         

                        mod_cluster/1.1.x

                        Auto Refresh show DUMP output show INFO output

                        Node f11c764c-fca5-3081-8f69-7285a93b18e4 (http://MTND2UA1480L5X:8330):

                        Enable Contexts Disable Contexts

                        Balancer: other-server-group,LBGroup: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 65,Ttl: 60000000,Status: OK,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 1

                        Virtual Host 1:

                        Contexts:

                        /cluster-demo, Status: ENABLED Request: 0 Disable /, Status: ENABLED Request: 0 Disable 

                        Aliases:

                        example.com localhost default-host 

                         

                         

                        So right now i would expect the request to go to this server but I still get the 503 error.  The only way things work again if i restart everything. Seems like a config issue but not sure where to look,.

                         

                        Thanks