11 Replies Latest reply on Apr 1, 2016 3:37 AM by Miroslav Novak

    AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup

    user 1561108 Newbie

      2016-02-22 02:45:14,778 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221012: Using AIO Journal

      2016-02-22 02:45:15,412 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final

      2016-02-22 02:45:15,412 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-2) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final

      2016-02-22 02:45:15,807 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]

      2016-02-22 02:45:16,392 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE

      2016-02-22 02:45:16,443 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP

      2016-02-22 02:45:16,529 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ

      2016-02-22 02:45:16,544 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP

      2016-02-22 02:45:18,496 INFO  [org.wildfly.extension.messaging-activemq] (MSC service thread 1-2) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor

      2016-02-22 02:45:18,495 INFO  [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor

      2016-02-22 02:45:18,526 INFO  [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor

      2016-02-22 02:45:18,513 INFO  [org.wildfly.extension.messaging-activemq] (MSC service thread 1-2) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor

      2016-02-22 02:45:19,497 ERROR [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ224000: Failure in initialisation: java.net.BindException: Address already in use

          at sun.nio.ch.Net.bind0(Native Method)

          at sun.nio.ch.Net.bind(Net.java:433)

          at sun.nio.ch.Net.bind(Net.java:425)

          at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)

          at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)

          at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125)

          at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485)

          at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089)

          at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430)

          at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415)

          at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903)

          at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198)

          at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348)

          at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)

          at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)

          at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)

          at java.lang.Thread.run(Thread.java:745)

      2016-02-22 02:45:19,513 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221001: Apache ActiveMQ Artemis Message Broker version 1.1.0.wildfly-011 [nodeID=fc0f80eb-d753-11e5-aceb-4fdaa0432112]

       

      my activemq subsystem inside standalone-full.xml looks like this:

       

      <subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">

          <server name="default">

              <security-setting name="#">

                  <role name="guest" delete-non-durable-queue="true" create-non-durable-queue="true" consume="true" send="true"/>

              </security-setting>

              <address-setting name="#" message-counter-history-day-limit="10" page-size-bytes="2097152" max-size-bytes="10485760" expiry-address="jms.queue.ExpiryQueue" dead-letter-address="jms.queue.DLQ"/>

              <http-connector name="http-connector" endpoint="http-acceptor" socket-binding="http"/>

              <http-connector name="http-connector-throughput" endpoint="http-acceptor-throughput" socket-binding="http">

                  <param name="batch-delay" value="50"/>

              </http-connector>

              <in-vm-connector name="in-vm" server-id="0"/>

              <http-acceptor name="http-acceptor" http-listener="default"/>

              <http-acceptor name="http-acceptor-throughput" http-listener="default">

                  <param name="batch-delay" value="50"/>

                  <param name="direct-deliver" value="false"/>

              </http-acceptor>

              <remote-acceptor name="http-connector" socket-binding="http"/>

              <in-vm-acceptor name="in-vm" server-id="0"/>

              <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>

              <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>

              <jms-queue name="myQueue" entries="java:/jms/queue/myQueue java:jboss/exported/jms/queue/myQueue"/>

              <connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>

              <connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>

              <pooled-connection-factory name="activemq-ra" transaction="xa" entries="java:/JmsXA java:jboss/DefaultJMSConnection$

          </server>

      </subsystem>

       

      Where is the dupe binding exception coming from?

       

      When I attempt a connection with my JMS client over http-remoting, I get a netty related error back indicating perhaps there is some legacy netty object bound to what should be my remote http acceptor?

        • 1. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
          Justin Bertram Master

          This looks like your problem:

           

            <remote-acceptor name="http-connector" socket-binding="http"/>

           

          You're trying to bind an acceptor to the "http" port which I imagine is already bound by that point by the HTTP server (i.e. Undertow).

          • 2. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
            Jay SenSharma Master

            Can you try removing the following line from your subsystem and see if it works:

             

            <remote-acceptor name="http-connector" socket-binding="http"/>

             

             

            Regards

            Jay SenSharma

            • 3. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
              user 1561108 Newbie

              This removed the startup error. Can I ask why that setting comes in the default version of standalone-full.xml on Wildfly 10 if it doesn't actually work? At least I don't think I put that in myself.

               

              But it hasn't prevented my remote client (with default remote http protocol) from failing to connect with the server. From the log it is looking to connect via the alternative netty method. As you can see from my standalone-full.xml, I have not set netty up nor do I particularly want to. I like the idea of just using the http port to connect.

               

              Anyway here is the logs from client attempt to connect before failing on attempt to create JMS2.0 JMSContext from the connection factory:

              Feb 22, 2016 6:38:35 PM org.xnio.Xnio <clinit>

              INFO: XNIO version 3.3.4.Final

              Feb 22, 2016 6:38:36 PM org.xnio.nio.NioXnio <clinit>

              INFO: XNIO NIO Implementation Version 3.3.4.Final

              Feb 22, 2016 6:38:36 PM org.jboss.remoting3.EndpointImpl <clinit>

              INFO: JBoss Remoting version (unknown)

              Exception in thread "naming-client-message-receiver-1-thread-1" java.lang.NoClassDefFoundError: io/netty/util/Version

                at org.apache.activemq.artemis.core.remoting.impl.netty.TransportConstants.<clinit>(TransportConstants.java:276)

                at org.apache.activemq.artemis.uri.TCPServerLocatorSchema.internalNewObject(TCPServerLocatorSchema.java:42)

                at org.apache.activemq.artemis.uri.TCPServerLocatorSchema.internalNewObject(TCPServerLocatorSchema.java:32)

                at org.apache.activemq.artemis.utils.uri.URISchema.newObject(URISchema.java:94)

                at org.apache.activemq.artemis.utils.uri.URISchema.newObject(URISchema.java:37)

                at org.apache.activemq.artemis.utils.uri.URIFactory.newObject(URIFactory.java:59)

                at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.newLocator(ServerLocatorImpl.java:402)

                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.<init>(ActiveMQConnectionFactory.java:176)

                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.<init>(ActiveMQConnectionFactory.java:169)

                at org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory.<init>(ActiveMQJMSConnectionFactory.java:34)

                at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

                at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

                at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

                at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

                at org.jboss.marshalling.reflect.SerializableClass.invokeConstructor(SerializableClass.java:493)

                at org.jboss.marshalling.reflect.SerializableClass.callNoArgConstructor(SerializableClass.java:450)

                at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1290)

                at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)

                at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213)

                at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:45)

                at org.jboss.naming.remote.protocol.v1.Protocol$1$3.read(Protocol.java:156)

                at org.jboss.naming.remote.protocol.v1.Protocol$1$3.read(Protocol.java:149)

                at org.jboss.naming.remote.protocol.v1.BaseProtocolCommand.readResult(BaseProtocolCommand.java:59)

                at org.jboss.naming.remote.protocol.v1.Protocol$1.handleClientMessage(Protocol.java:149)

                at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1$MessageReceiver$1.run(RemoteNamingStoreV1.java:232)

                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

                at java.lang.Thread.run(Thread.java:745)

              Caused by: java.lang.ClassNotFoundException: io.netty.util.Version

                at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

                at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

                at java.security.AccessController.doPrivileged(Native Method)

                at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

                at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

                at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

                at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

                ... 28 more

              javax.naming.NamingException: Unable to invoke lookup, status=WAITING

                at org.jboss.naming.remote.protocol.v1.Protocol$1.execute(Protocol.java:98)

                at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.lookup(RemoteNamingStoreV1.java:95)

                at org.jboss.naming.remote.client.HaRemoteNamingStore$1.operation(HaRemoteNamingStore.java:276)

                at org.jboss.naming.remote.client.HaRemoteNamingStore.namingOperation(HaRemoteNamingStore.java:132)

                at org.jboss.naming.remote.client.HaRemoteNamingStore.lookup(HaRemoteNamingStore.java:272)

                at org.jboss.naming.remote.client.RemoteContext.lookupInternal(RemoteContext.java:104)

                at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:93)

                at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:146)

                at javax.naming.InitialContext.lookup(InitialContext.java:411)

                at my.package.App.run(App.java:54)

              • 4. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                Justin Bertram Master

                This removed the startup error. Can I ask why that setting comes in the default version of standalone-full.xml on Wildfly 10 if it doesn't actually work? At least I don't think I put that in myself.

                That remote-acceptor does not appear in any of the default Wildfly 10 configuration files.  It must have been added by you or someone else.

                 

                But it hasn't prevented my remote client (with default remote http protocol) from failing to connect with the server. From the log it is looking to connect via the alternative netty method. As you can see from my standalone-full.xml, I have not set netty up nor do I particularly want to. I like the idea of just using the http port to connect.

                The JMS implementation shipped with Wildfly 10 (i.e. Apache ActiveMQ Artemis) uses Netty for its remote transport.  The initial HTTP connection uses a simple upgrade mechanism to move from HTTP to a TCP connection implemented by Netty.  This upgrade mechanism allows the server to listen on just a single port (i.e. the HTTP port) yet support lots of different services (e.g. Netty based messaging transport).

                 

                Anyway here is the logs from client attempt to connect before failing on attempt to create JMS2.0 JMSContext from the connection factory...

                This looks like a basic classpath issue.  You should only need <JBOSS_HOME>/bin/client/jboss-client.jar on your classpath.  What's on your classpath when your client receives this error?

                • 5. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                  user 1561108 Newbie

                  Ok so netty is still used under the hood then? And the associated libraries would still be required?

                   

                  This is my maven pom dependency list on the client app:

                   

                  <dependency>
                  <groupId>javax</groupId>
                  <artifactId>javaee-api</artifactId>
                  <version>7.0</version>
                  </dependency>

                   

                   

                  <dependency>
                  <groupId>jboss</groupId>
                  <artifactId>jboss-client</artifactId>
                  <version>4.0.2</version>
                  </dependency>

                     

                  <dependency>
                  <groupId>org.apache.activemq</groupId>
                  <artifactId>artemis-jms-client</artifactId>
                  <version>1.1.0.wildfly-011</version>
                  </dependency>

                   

                   

                  <dependency>
                  <groupId>org.apache.activemq</groupId>
                  <artifactId>artemis-core-client</artifactId>
                  <version>1.1.0.wildfly-011</version>
                  </dependency>

                   

                   

                  <dependency>
                  <groupId>org.apache.activemq</groupId>
                  <artifactId>artemis-commons</artifactId>
                  <version>1.1.0.wildfly-011</version>
                  </dependency>
                  <dependency>
                  <groupId>commons-beanutils</groupId>
                  <artifactId>commons-beanutils</artifactId>
                  <version>1.9.2</version>
                  </dependency>
                  <dependency>
                  <groupId>org.jboss</groupId>
                  <artifactId>jboss-remote-naming</artifactId>
                  <version>2.0.4.Final</version>
                  </dependency>
                  <dependency>
                  <groupId>org.jboss.xnio</groupId>
                  <artifactId>xnio-nio</artifactId>
                  <version>3.3.4.Final</version>
                  </dependency>
                  <dependency>
                  <groupId>org.jboss.xnio</groupId>
                  <artifactId>xnio-api</artifactId>
                  <version>3.3.4.Final</version>
                  </dependency>
                  • 6. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                    Justin Bertram Master

                    Ok so netty is still used under the hood then?

                    That's correct.

                     

                    And the associated libraries would still be required?

                    That's correct.

                     

                    However, rather than specifying a bunch of individual dependencies I would recommend you use this:

                     

                            <dependency>
                                <groupId>org.wildfly</groupId>
                                <artifactId>wildfly-jms-client-bom</artifactId>
                                <version>10.0.0.Final</version>
                                <type>pom</type>
                            </dependency>
                    

                     

                    That should be much simpler.  You can see a simple of example of this in action in the Wildfly JMS quickstart.

                    • 7. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                      user 1561108 Newbie

                      I am afraid even with the bom (and a manually cleaned out target directory to be sure) I am still getting NoClassDefFoundError on io.netty.util.Version on client startup. The bom did not download any additional dependencies either from what the eclipse console tells me.

                      • 8. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                        user 1561108 Newbie

                        I enabled debug logging and I see the following strange error just before the no class found error:

                         

                        2016-02-23 10:48:18 DEBUG org.jboss.remoting.remote.client     - Client authentication failed for mechanism JBOSS-LOCAL-USER: javax.security.sasl.SaslException: Failed to read server challenge [Caused by java.io.FileNotFoundException: /opt/wildfly-10.0.0.Final/standalone/tmp/auth/local2434344343753975699.challenge (No such file or directory)]

                        Exception in thread "naming-client-message-receiver-1-thread-1" java.lang.NoClassDefFoundError: io/netty/util/Version

                        The remoting error seems a strange one, I'm not sure what 'JBOSS-LOCAL-USER' is doing - I am running the client on a separate box.

                        • 9. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                          Justin Bertram Master

                          I'm not 100% sure what's going on in your client, but it looks like your JNDI lookup is somehow trying to go to a local server (and hence use local auth mechanism).  Do you have an instance of Wildfly running locally?  How have you configured your JNDI lookup?  It may be best to have close look at the JMS quickstart I linked previously to have a reference point for your own client.

                          • 10. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                            user 1561108 Newbie

                            To anyone else with the later problem - the supplied wildfly client bom is missing 3 netty jars:

                             

                            <dependency>
                            <groupId>io.netty</groupId>
                            <artifactId>netty-common</artifactId>
                            <version>4.0.34.Final</version>
                            </dependency>
                                   
                            <dependency>
                            <groupId>io.netty</groupId>
                            <artifactId>netty-handler</artifactId>
                            <version>4.0.34.Final</version>
                            </dependency>

                             

                             

                            <dependency>
                            <groupId>io.netty</groupId>
                            <artifactId>netty-codec-http</artifactId>
                            <version>4.0.34.Final</version>
                            </dependency>

                             

                            Once these are added the client starts up correctly.

                            • 11. Re: AMQ224000: Failure in initialisation: java.net.BindException: Address already in use on Wildfly 10 startup
                              Miroslav Novak Master

                              This is dependency tree for wildfly-jms-client-bom for dependency mentioned above by Justin, Netty is there:

                              [INFO] ------------------------------------------------------------------------
                              [INFO] Building client 1.0-SNAPSHOT
                              [INFO] ------------------------------------------------------------------------
                              [INFO] 
                              [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ client ---
                              [INFO] org.jboss.jms.client:client:jar:1.0-SNAPSHOT
                              [INFO] \- org.wildfly:wildfly-jms-client-bom:pom:10.0.0.Final:compile
                              [INFO]    +- commons-beanutils:commons-beanutils:jar:1.9.2:compile
                              [INFO]    |  \- commons-collections:commons-collections:jar:3.2.1:compile
                              [INFO]    +- org.apache.activemq:artemis-commons:jar:1.1.0.wildfly-011:compile
                              [INFO]    |  +- io.netty:netty-all:jar:4.0.30.Final:compile
                              [INFO]    |  \- com.google.guava:guava:jar:18.0:compile
                              [INFO]    +- org.apache.activemq:artemis-core-client:jar:1.1.0.wildfly-011:compile
                              [INFO]    |  +- org.jgroups:jgroups:jar:3.6.0.Final:compile
                              [INFO]    |  +- org.apache.activemq:artemis-selector:jar:1.1.0.wildfly-011:compile
                              [INFO]    |  \- org.apache.activemq:artemis-journal:jar:1.1.0.wildfly-011:compile
                              [INFO]    |     \- org.apache.activemq:artemis-native:jar:1.1.0.wildfly-011:compile
                              [INFO]    +- org.apache.activemq:artemis-hqclient-protocol:jar:1.1.0.wildfly-011:compile
                              [INFO]    +- org.apache.activemq:artemis-jms-client:jar:1.1.0.wildfly-011:compile
                              [INFO]    |  \- javax.inject:javax.inject:jar:1:compile
                              [INFO]    +- org.jboss:jboss-remote-naming:jar:2.0.4.Final:compile
                              [INFO]    |  \- org.jboss:jboss-ejb-client:jar:2.0.0.Final:compile
                              [INFO]    +- org.jboss.logging:jboss-logging:jar:3.3.0.Final:compile
                              [INFO]    +- org.jboss.marshalling:jboss-marshalling:jar:1.4.10.Final:compile
                              [INFO]    +- org.jboss.marshalling:jboss-marshalling-river:jar:1.4.10.Final:compile
                              [INFO]    +- org.jboss.remoting:jboss-remoting:jar:4.0.18.Final:compile
                              [INFO]    +- org.jboss.sasl:jboss-sasl:jar:1.0.5.Final:compile
                              [INFO]    +- org.jboss.spec.javax.jms:jboss-jms-api_2.0_spec:jar:1.0.0.Final:compile
                              [INFO]    +- org.jboss.xnio:xnio-api:jar:3.3.4.Final:compile
                              [INFO]    +- org.jboss.xnio:xnio-nio:jar:3.3.4.Final:compile
                              [INFO]    +- org.slf4j:jcl-over-slf4j:jar:1.7.7.jbossorg-1:compile
                              [INFO]    |  \- org.slf4j:slf4j-api:jar:1.7.7.jbossorg-1:compile
                              [INFO]    \- org.wildfly.checkstyle:wildfly-checkstyle-config:jar:1.0.4.Final:compile
                              

                               

                              Issue you're seeing is more likely to be related to security.