Configuring multi-homed with firewall address translation NAT

[hughbragg]

I've been searching through these forums and documentation for days and I still haven't been able to understand how to configure this setup.

There are lots of documents (mostly old JBoss 4.2), but nothing that puts it all together and makes it easy to get a demo up and running.

I really hope someone can put me straight, my head is hurting.

I want to use HornetQ to send data outside a private network. It's protected by a firewall which forwards all traffic from the consumer to the MOM over a public ip address <publicAddress>, but the MOM only has an internal NAT address <privateAddress> and it can't connect directly to the internet or bind to it's own <publicAddress>. The producer also lives inside the private network without access to the <publicAddress> and uses the MOM's <privateAddress> to communicate. It's an existing application that uses JMS. There is a very high throughput at times so we want to stay away from http tunneling.

I have this working using JBoss 4.2.3 jbossmq. JBoss binds to its <privateAddress> and the producer has no problems sending. The consumer connects to the <publicAddress> and ignores the JNDI connectAddress by using the -Dorg.jboss.mq.il.uil2.serverAddress=<publicAddress> startup parameter.

This solution obviously prevents the consumer connecting to any other server, but it works for this single purpose.

Ideally I'd like to use HornetQ as a standalone server with JMS. I can't see how to configure it. It looks like I can add 2 adaptor configurations to netty but this isn't clear. I would like the external and internal connections to use the same default port, just different addresses. Is this possible? How can I configure it?

My next alternative is to use JBossEE. There are a few references to how to setup multi-homed servers and NAT but I can't find a worked example and it seems there are 2 alternatives both involving more than 1 configuration file, but no clear instructions what to do. I am hoping someone could help to clear things up. Is there a worked example, where? How can I configure this?

Would anyone please help me?

Regards,

Hugh

[clebert.suconic]

I'm not sure what's the issue.. it's pretty clear what are the options at the documentation.

You can see what are the connectors... and you could even use the servlet transport:

http://hornetq.sourceforge.net/docs/hornetq-2.1.1.Final/user-manual/en/html/index.html

I will try to help you If you ask something more specific. I would be designing your architecture if I guessed any more than this and I don't have a clear picture.

[hughbragg]

Thanks Clebert,

I want to configure 2 acceptors both on the same port but with different ip addresses. I don't know if this is possible. I'm guessing that because the server has no access to that address this can't be done. No NIC is ever attached to this address. The network infrastructure guys configured it this way because it makes their life easy. All they need to do is configure 1 rule on their firewall to forward all traffic coming from the external client destined for the <publicAddress> to the <privateAddress>

Can you advise me. This is not strictly multi homed is it? Could it work?

The trouble is that the server resolves it's own address first before it publishes the service in JNDI and as such the client is able to connect.

Plan b. Create a different connector for each client. The publisher would have the <privateAddress> and the consumer would have the <publicAddress>. The server would have an acceptor to the <privateAddress>

Is this the right approach? Should this work?

Hugh

[clebert.suconic]

"I want to configure 2 acceptors both on the same port but with different ip addresses "

I don't think this should be a problem...

However, you need to be able to accept on the address.. it's a simple configuration issue that goes beyond hornetQ. more on your network infra-structure / config.

[hughbragg]

JMS message consumer client has normal access to any public address using its <consumerIP>

JMS message producer client has no public access using its NAT <producerIP>

MOM message broker has no public access using its NAT <momIP> (JBoss must bind to <momIP>)

Firewall

     1.accepts all traffic routed to a public address range which includes a specific <publicIP>

     2. forwards all traffic arriving from <consumerIP> addressed to <publicIP> to <momIP>

---

JMS message consumer client connects to the JMS service via <publicIP>

JMS message producer client connects to the JMS service via <momIP>

[clebert.suconic]

So, you're basically asking how to configure a firewal?

I'm not a firewal specialist..

From HornetQ point of view, all you have to do is to configure the acceptors at the IPs as any other service.

I will let somebody else to answer this thread though. Maybe I'm missing the point.. but so far you didn't ask a HornetQ question as far as I understand.

[hughbragg]

It's not about configuring a firewall. The firewall is already configured. I can't do anything about it.

I'm asking how to make HornetQ work through this firewall.

The acceptors don't seem to be helpful as the public address is not accessible by the HornetQ machine. There is no way for it to bind to this address.

The address is translated by the firewall before it's seen there. The difficulty is with the JNDI connect address. This is published to the JNDI used by HornetQ or JBoss with its NAT address which is unreachable by the JMS message consumer client. The firewall can't translate this so messaging fails.

I'm not a firewall specialist either nor a JBoss specialist or even a HornetQ specialist. I'm just a java developer who's thrown in the deep end.

Please, if anyone can help I'd be eternally grateful.

Hugh

[clebert.suconic]

"The acceptors don't seem to be helpful as the public address is not accessible by the HornetQ machine."

So, that makes it a firewal issue.

You just have to make the right question at the right place. I'm not sure we would be able to help you.

That would be the same as having an apache listening to the same port. It's just a socket accepting connections.. nothing fancy!

[hughbragg]

This works for jbossmq.

Are you saying that HornetQ isn't able to do this?

I think my problem is that I'm just a newbie and I haven't figured out how to configure this properly.

Are connectors declared on the client or the host?

Perhaps I just need to declare 1 acceptor and 2 different connectors, 1 for each incoming interface.

[clebert.suconic]

jboss-mq is also the same man.. it's just an acceptor on a given port.

It's as simple as creating an socket acceptor in Java, C++, C or even assembler if you like.

jboss-mq, jboss-messaging (through remoting), HornetQ (through netty), apache, tomcat.. all do that.

It's not our job to redirect any ports and IPs.. that's a firewal's responsibility. (you just need to configure routes.. etc). i.e. you just need to configure your firewal.

If you're looking for a feature existent on jboss-mq, you can provide me a link or something. Case I'm missing the point and you're asking something different I didn't understand.

[clebert.suconic]

BTW I just realized you asked something specific I didn't ask before.

You can define multiple Acceptors anyway you like.

You can define connectors at the server for JNDI distribution as we are required to do that per JMS spec.

For some reference: http://hornetq.sourceforge.net/docs/hornetq-2.1.2.Final/user-manual/en/html_single/index.html#configuring-transports.connectors

And this nice blog written by Jeff Mesnil some time ago:

http://hornetq.blogspot.com/2009/10/understanding-connectors-acceptors.html

[hughbragg]

I've been reading this article http://community.jboss.org/thread/48925 amongst others.

The org.jboss.mq.il.uil.serverAddress allowed me to connect to the server through the firewall.

Another solution I tried was from  http://community.jboss.org/wiki/ConfigUIL2 but once I used the  ConnectAddress the internal message producer wasn't able to connect any  longer.

I can't test if my original solution will work for  hornetq right now, but I suspect that because it's a jbossmq specific  parameter, it won't work.

I'm working on the assumption that hornetq is at least as configurable as jbossmq.

Thanks for the blog article. It still isn't clear to  me (maybe becuse I'm a newbie) how to configure the system to do the  same or a similar thing here.

Specifically, do I put a hornetq-configuration.xml  file on the client with connectors to tell it to try to connect to the  jms using it's public ip or do i put a hornetq-configuration.xml file on  the server with connectors to tell the client to use the public  address?

Generally, is there any way to tell the client to use a specific ip to connect to the jms server?

Right now I'm trying this on my server:

default-with-hornetq/deploy/hornetq.sar/hornetq-jms.xml:

<connection-factory name="NettyConnectionFactory">
      <connectors>
         <connector-ref connector-name="netty"/>
      </connectors>
      <entries>
         <entry name="/ConnectionFactory"/>
         <entry name="/XAConnectionFactory"/>
      </entries>
   </connection-factory>
  
   <connection-factory name="NettyPublicConnectionFactory">
      <connectors>
         <connector-ref connector-name="netty-public"/>
      </connectors>
      <entries>
         <entry name="/PublicConnectionFactory"/>
         <entry name="/XAPublicConnectionFactory"/>
      </entries>
   </connection-factory>

default-with-hornetq/deploy/hornetq.sar/hornetq-configuration.xml:

<connectors>
      <connector name="netty">
         <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
         <param key="host"  value="${jboss.bind.address:localhost}"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      </connector>
     
      <connector name="netty-public">
         <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
         <param key="host"  value="${public.firewall.address:localhost}"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      </connector>

</connectors>

   <acceptors>  
      <acceptor name="netty">
         <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
         <param key="host"  value="${jboss.bind.address:localhost}"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      </acceptor>
     
      <acceptor name="netty-public">
         <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
         <param key="host"  value="${public.firewall.address:localhost}"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      </acceptor>

I start the JBoss with -Djboss.bind.address=  parameter

Does this look right?

[andreas_back]

Hello Hugh,

you are asking:

> Generally, is there any way to tell the client to use a specific ip to connect to the jms server?

You could try something like

(1)     <connector name="netty">
              <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
              <param key="host"  value="external-address"/>
              <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
          </connector>

where "external-address" is an address which the client can reach the server by. It is just the address you use with a successful

(2)       telnet external-address 5445

to reach the server from the client/consumer. To make (2) running is a simply a matter of the network configuration - you only have to start the HornetQ server. You can check the port by issuing

(3)      telnet localhost 5445

on  the server. In (3) you can also substitute localhost with one of the servers addresses.

If you have (2) running, then you can try (1).

With best regards,

Andreas

[clebert.suconic]

It would have been much easier if you had asked the right question ;-)

- How to bind HornetQ to a specific address?

Instead of adding a firewal, bindings.. routes.... and etc to the question :-)

You can just change the acceptors accordingly as Andreas answered (Thanks Andreas)

[hughbragg]

So I finally sorted this out if anyone's interested.

I already tried Andreas' suggestion but the bind failed.

HornetQ documentation doesn't mention where to keep configuration files. I thought perhaps the client used the connectors configuration files. It might be obvious to anyone with experience setting it up, but not to a newbie.

The trouble turned out to be with the way the machine was setup. JBoss wasn't able to bind to the public address because it didn't exist. This wasn't a problem for the JBossMQ because you could tell the client to ignore the JNDI address published by the server.

The solution was to add the public address to the servers NIC so JBoss could bind to it. Being in promiscuous mode enabled the tcp/ip stack to sort out the internal routing. Retrying the original configuration with 2 connectors and 2 acceptors worked.