13 Replies Latest reply on Mar 13, 2017 9:25 AM by Justin Bertram

    WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall

    George Turner Newbie

      I was hoping version 9 would fix this problem, but alas, not.

       

      Caused by: HornetQNotConnectedException[errorType=NOT_CONNECTED message=HQ119007: Cannot connect to server(s). Tried with all available servers

          at org.hornetq.core.clinet.impl.ServerLocatorImpl.createSessionFactory(ServletLocatorImpl.java:905)

       

      We are using standard http-remoting connection code.  We can lookup and call remote EJB methods fine.  We have used a workaround in the code that involves adding hornet to the plugin manifest and adding this line in the code:

       

           final ConnectionFactory cf = (ConnectionFactory) ctx.lookup("jms/RemoteConnectionFactory");

          if (HornetQJMSConnectionFactory.class.isAssignableFrom(cf.getClass())) {

              ((HornetQJMSConnectionFactory)cf).getServletLocator().getStaticTransportConfigurations()[0].getParams().put("host", ip);

          }

       

      Nothing else seems to work, any help would be great.

        • 1. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
          jaikiran pai Master

          Could you please add more context to this, including complete exception stacktraces and more details on what you are doing? It's hard to understand what this is about without that context? Was there any bug report that was filed against 8.x and was expected to be fixed in 9.x?

          • 2. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
            George Turner Newbie

            This is actually pretty straight forward and I am shocked that this fails, as it has been a standard capability in JMS for years, but I suspect that the http-remoting has something to do with a mis-connect with HornetQ.

             

            Simple client call:

            Properties topicProperties = new Properties();

            topicProperties.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");

            topicProperties.put(Context.PROVIDER_URL, "http-remoting://" + host + ":" + port);

            InitialContext ctx = new InitialContext(topicProperties);

            ConnectionFactory tmp = (ConnectionFactory) topicCtx.lookup("jms/RemoteConnectionFactory");

            connection = tmp.createConnection();

             

            Results in:

             

            Jun 11, 2015 8:26:07 AM org.xnio.Xnio <clinit>

            INFO: XNIO version 3.3.1.Final

            Jun 11, 2015 8:26:07 AM org.xnio.nio.NioXnio <clinit>

            INFO: XNIO NIO Implementation Version 3.3.1.Final

            Jun 11, 2015 8:26:07 AM org.jboss.remoting3.EndpointImpl <clinit>

            INFO: JBoss Remoting version 4.0.9.Final

            javax.jms.JMSException: Failed to create session factory

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:673)

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:112)

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:107)

              at com.lmco.spacefence.netcentric.test.client.TestMessageConsumer.<init>(TestMessageConsumer.java:36)

              at com.lmco.spacefence.netcentric.test.client.TestMessageConsumer.main(TestMessageConsumer.java:24)

            Caused by: HornetQNotConnectedException[errorType=NOT_CONNECTED message=HQ119007: Cannot connect to server(s). Tried with all available servers.]

              at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:905)

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:669)

              ... 4 more

            Disconnected from the target VM, address: '127.0.0.1:54275', transport: 'socket'

             

            The server is behind a NAT firewall

            • 3. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
              George Turner Newbie

              By the way, the ConnectionFactory instance returned shows the following in a debugger.  The params host value is of the IP of the server itself behind the NAT firewall instead of the IP address that we connect to.

               

              initialConnectors = {org.hornetq.api.core.TransportConfiguration[1]@2183}

              0 = {org.hornetq.api.core.TransportConfiguration@2196} "TransportConfiguration(name=http-connector, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=8080&host=10-10-20-77&http-upgrade-enabled=true&http-upgrade-endpoint=http-acceptor"

                name = {java.lang.String@2198} "http-connector"

                factoryClassName = {java.lang.String@2199} "org.hornetq.core.remoting.impl.netty.NettyConnectorFactory"

                params = {java.util.HashMap@2200}  size = 4

                 0 = {java.util.HashMap$Node@2203} "port" -> "8080"

                 1 = {java.util.HashMap$Node@2204} "host" -> "10.10.20.77"

                 2 = {java.util.HashMap$Node@2205} "http-upgrade-enabled" -> "true"

                 3 = {java.util.HashMap$Node@2206} "http-upgrade-endpoint" -> "http-acceptor"

              • 4. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                jaikiran pai Master

                This has been reported a few times and I thought this was solved but it looks like not. Here's one such thread discussing this AS 7 Messaging ( HORNETQ ) Client access remote:// Problem with Server listening on 0.0.0.0 Interface (it starts off on a slightly different note), but the underlying problem is the same. Read through that thread.

                 

                The workaround appears to be Re: AS 7 Messaging ( HORNETQ ) Client access remote:// Problem with Server listening on 0.0.0.0 Interface (it was being discussed in the context of AS7, so you'll have to check if that works for WildFly 8/9). Here's an explanation of that workaround https://developer.jboss.org/message/810113#810113

                • 5. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                  George Turner Newbie

                  Would you please provide a concrete solution that works for WildFly 8/9?  The references that you have provided show multiple solutions, going back to version 7, and the configuration is different.  I cannot determine if adding the netty connector (where I have no "messaging" connector) is the solution or by using the client-mapping, which I believe is probably what I need to do, but again, I have no socket-binding or messaging or remoting, as they were removed.

                  • 7. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                    George Turner Newbie

                    BTW, I would like to enter this as a current bug, can you help with that?

                    • 8. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                      jaikiran pai Master

                      You can file bug reports (JIRA) here WildFly - JBoss Issue Tracker. Post relevant details including the sample code/application if any and point to this thread.

                      • 9. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                        jukka sirviö Newbie

                        Did you find workaround to this? I have quite exactly similar problem!

                        Normal JMS messaging have (of course) worked fine for years...

                        • 10. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                          jukka sirviö Newbie

                          Ok, it was quite easy to solve in JMS client, below is related code how to do it:

                           

                          ......

                               final Properties env = new Properties();

                            env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");

                            env.put(Context.PROVIDER_URL, "http-remoting://xxx.xxx.xxx.xxx:8080"); //<-- put here the natted ip address or dns name & port

                            env.put(Context.SECURITY_PRINCIPAL, "user");

                            env.put(Context.SECURITY_CREDENTIALS, "pass");

                               Context context;

                            try {

                            context = new InitialContext(env);

                                      //<-- begin of essential part

                            Map<String, Object> connectionParams = new HashMap<String, Object>();

                                      connectionParams.put(org.hornetq.core.remoting.impl.netty.TransportConstants.HOST_PROP_NAME, "xxx.xxx.xxx.xxx"); // <-- put here the natted ip address!

                                      connectionParams.put(org.hornetq.core.remoting.impl.netty.TransportConstants.PORT_PROP_NAME, "8080");

                                      connectionParams.put(org.hornetq.core.remoting.impl.netty.TransportConstants.HTTP_UPGRADE_ENABLED_PROP_NAME, "true");

                                      connectionParams.put(org.hornetq.core.remoting.impl.netty.TransportConstants.HTTP_UPGRADE_ENDPOINT_PROP_NAME, "http-acceptor");

                                      TransportConfiguration transportConfiguration =

                                          new TransportConfiguration(

                                          "org.hornetq.core.remoting.impl.netty.NettyConnectorFactory",

                                          connectionParams);

                                      //<-- end of essential part

                           

                                      connectionFactory = HornetQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.TOPIC_CF, transportConfiguration);

                            /// continue as usual, destination, connection, session, producer/consumer etc...

                          ......

                          • 11. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                            jukka sirviö Newbie

                            Dear George Turner, this is response to your inquiry on WFLY-4769 Jira, here is the code for wILDFLY 10.0 ArtemisMQ, Please mark this as answer if you find this useful !!

                             

                            String serverName = "xxxyyyzzz";

                            env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");

                            env.put(Context.PROVIDER_URL, serverName);

                            env.put(Context.SECURITY_PRINCIPAL, "user");

                            env.put(Context.SECURITY_CREDENTIALS, "pass");

                            Context context;

                            try {

                            context = new InitialContext(env);

                            Map<String, Object> connectionParams = new HashMap<String, Object>();

                            connectionParams.put(org.apache.activemq.artemis.core.remoting.impl.netty.TransportConstants.HOST_PROP_NAME, "127.0.0.1"); // <-- PUT THE NATTED IP HERE!

                            connectionParams.put(org.apache.activemq.artemis.core.remoting.impl.netty.TransportConstants.PORT_PROP_NAME, "80");

                            connectionParams.put(org.apache.activemq.artemis.core.remoting.impl.netty.TransportConstants.HTTP_UPGRADE_ENABLED_PROP_NAME, "true");

                            connectionParams.put(org.apache.activemq.artemis.core.remoting.impl.netty.TransportConstants.HTTP_UPGRADE_ENDPOINT_PROP_NAME, "http-acceptor");

                             

                            TransportConfiguration transportConfiguration =

                               new org.apache.activemq.artemis.api.core.TransportConfiguration(

                                 "org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory",

                                 connectionParams);

                             

                            connectionFactory = ActiveMQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.TOPIC_CF, transportConfiguration);

                            • 12. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                              janaka buddhika Newbie

                              Dear jukka sirviö,

                              Does this require special configuration in jboss. Im using above JMS client code with below standalong-full.xml. But still ended up with below messages from log

                               

                              [Remoting "config-based-naming-client-endpoint" task-2] DEBUG org.jboss.remoting.remote.client - Client authentication failed for mechanism JBOSS-LOCAL-USER: javax.security.sasl.SaslException: Failed to read server challenge [Caused by java.io.FileNotFoundException: \home\janaka\jboss-eap-7.0\standalone\tmp\auth\local4810015721206975876.challenge (The system cannot find the path specified)]

                              .......

                              .......

                              ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ119007: Cannot connect to server(s). Tried with all available servers.

                               

                               

                              Im using  Jboss EAP 7.0 which is based on wildfly 10.0.

                               

                              Please note that my JMS queus can be accessed without any issue from same server or any other server from the same network but cant be accessed from out side through NAT gateway.  Any kind of clue help would be highly appreciated.

                               

                               

                              Standalone.xml

                               

                                <subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">
                                <server name="default">
                                <security-setting name="#">
                                <role name="guest" delete-non-durable-queue="true" create-non-durable-queue="true" consume="true" send="true"/>
                                </security-setting>
                                <address-setting name="#" message-counter-history-day-limit="10" page-size-bytes="2097152" max-size-bytes="10485760" expiry-address="jms.queue.ExpiryQueue" dead-letter-address="jms.queue.DLQ"/>
                                <http-connector name="http-connector" endpoint="http-acceptor" socket-binding="http"/>
                                <http-connector name="http-connector-throughput" endpoint="http-acceptor-throughput" socket-binding="http">
                                <param name="batch-delay" value="50"/>
                                </http-connector>
                                <in-vm-connector name="in-vm" server-id="0"/>
                                <http-acceptor name="http-acceptor" http-listener="default"/>
                                <http-acceptor name="http-acceptor-throughput" http-listener="default">
                                <param name="batch-delay" value="50"/>
                                <param name="direct-deliver" value="false"/>
                                </http-acceptor>
                                <in-vm-acceptor name="in-vm" server-id="0"/>
                                <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
                                <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
                                <jms-queue name="fromNGSQueue" entries="java:/jboss/exported/jms/queue/fromNGS"/>
                                <jms-queue name="toNGSQueue" entries="java:/jboss/exported/jms/queue/toNGS"/>
                                <connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
                                <connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>
                                <pooled-connection-factory name="activemq-ra" transaction="xa" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm"/>
                                </server>
                                </subsystem>

                               

                              Console output.

                               

                              14:13:55.301 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider

                              14:13:55.326 [main] DEBUG org.jboss.naming.remote.client.InitialContextFactory - Looking for jboss-naming-client.properties using classloader sun.misc.Launcher$AppClassLoader@4e0e2f2a

                              14:13:55.327 [main] DEBUG org.jboss.naming.remote.client.InitialContextFactory - jboss.naming.client.endpoint.create.options. has the following options {}

                              14:13:55.328 [main] DEBUG org.jboss.naming.remote.client.InitialContextFactory - jboss.naming.client.remote.connectionprovider.create.options. has the following options {}

                              14:13:55.349 [main] INFO org.xnio - XNIO version 3.3.6.Final

                              14:13:55.367 [main] INFO org.xnio.nio - XNIO NIO Implementation Version 3.3.6.Final

                              14:13:55.559 [Remoting "config-based-naming-client-endpoint" Accept] DEBUG org.xnio.nio - Started channel thread 'Remoting "config-based-naming-client-endpoint" Accept', selector sun.nio.ch.WindowsSelectorImpl@714ebb1c

                              14:13:55.559 [Remoting "config-based-naming-client-endpoint" I/O-1] DEBUG org.xnio.nio - Started channel thread 'Remoting "config-based-naming-client-endpoint" I/O-1', selector sun.nio.ch.WindowsSelectorImpl@6e6846c0

                              14:13:55.603 [main] INFO org.jboss.remoting - JBoss Remoting version 4.0.0.Final

                              14:13:55.628 [main] DEBUG org.jboss.naming.remote.client.InitialContextFactory - jboss.naming.client.connect.options. has the following options {}

                              14:13:55.756 [main] DEBUG org.apache.activemq.artemis.logs - using dummy address ffffffa1:46:3f:fffffff1:78:76

                              14:13:58.205 [Remoting "config-based-naming-client-endpoint" task-2] DEBUG org.jboss.remoting.remote.client - Client authentication failed for mechanism JBOSS-LOCAL-USER: javax.security.sasl.SaslException: Failed to read server challenge [Caused by java.io.FileNotFoundException: \home\janaka\jboss-eap-7.0\standalone\tmp\auth\local4810015721206975876.challenge (The system cannot find the path specified)]

                              14:13:59.878 [main] DEBUG org.apache.activemq.artemis.core.client - Trying reconnection attempt 0/1

                              14:13:59.878 [main] DEBUG org.apache.activemq.artemis.core.client - Trying to connect with connector = org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@491666ad, parameters = {httpUpgradeEnabled=true, port=8080, httpPpgradeEndpoint=http-acceptor, host=192.168.1.154} connector = null

                              14:13:59.888 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework

                              14:13:59.892 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available

                              14:13:59.892 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available

                              14:13:59.892 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available

                              14:13:59.892 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true

                              14:13:59.893 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: Windows

                              14:13:59.894 [main] DEBUG io.netty.util.internal.PlatformDependent - Java version: 8

                              14:13:59.894 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false

                              14:13:59.894 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available

                              14:13:59.894 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noJavassist: false

                              14:13:59.895 [main] DEBUG io.netty.util.internal.PlatformDependent - Javassist: unavailable

                              14:13:59.895 [main] DEBUG io.netty.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.

                              14:13:59.896 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\user\AppData\Local\Temp (java.io.tmpdir)

                              14:13:59.896 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)

                              14:13:59.896 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false

                              14:13:59.899 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple

                              14:13:59.918 [main] DEBUG io.netty.channel.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8

                              14:13:59.935 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false

                              14:13:59.936 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512

                              14:13:59.961 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 8

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 8

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize: 512

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768

                              14:13:59.962 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192

                              14:13:59.975 [main] DEBUG org.apache.activemq.artemis.core.client - Started Netty Connector version 4.0.30.Final

                              14:13:59.975 [main] DEBUG org.apache.activemq.artemis.core.client - Remote destination: /192.168.1.154:8080

                              14:13:59.982 [main] DEBUG io.netty.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x2f1ff72a552357d5 (took 1 ms)

                              14:14:00.014 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled

                              14:14:00.014 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536

                              14:14:21.053 [main] DEBUG org.apache.activemq.artemis.core.client - Connector towards NettyConnector [host=192.168.1.154, port=8080, httpEnabled=false, httpUpgradeEnabled=true, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] failed

                              Exception in thread "main" javax.jms.JMSRuntimeException: Failed to create session factory

                                at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:88)

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:262)

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:248)

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:238)

                                at com.vishuo.icmdb.ngs.jms.HelloWorldJMSConsumer2.main(HelloWorldJMSConsumer2.java:63)

                              Caused by: javax.jms.JMSException: Failed to create session factory

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:727)

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:255)

                                ... 3 more

                              Caused by: ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ119007: Cannot connect to server(s). Tried with all available servers.]

                                at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:775)

                                at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:724)

                                ... 4 more

                              14:14:21.124 [Remoting "config-based-naming-client-endpoint" task-9] DEBUG org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1 - Channel end notification received, closing channel Channel ID b6654f8f (outbound) of Remoting connection 274dfe6d to vishuo.ddns.net/203.116.141.46:1800

                              • 13. Re: WildFly 8 and 9. Connecting to topic using http-remoting fails when server is behind NAT firewall
                                Justin Bertram Master

                                The question related to this forum thread has already been answered.  Please start a new thread with a full explanation of your use-case and problem.