HornetQ with JGroups

[realkobe]

Hi,

I want to configure HornetQ to use JGroups and use it in my Java Client. Unfortunetaly I don't find a working example for my client.

I took the jgroups configuration from Chapter 38. Clusters and adjusted

<FILE_PING location="/tmp/file.ping.dir"/>

The server starts and opens some ports, probably port 7500 is for Jgroups

~> netstat -tulpen 2> /dev/null | grep java
tcp6       0      0 :::62500                :::*                    LISTEN      1000       2081291    9255/java          
tcp6       0      0 :::62501                :::*                    LISTEN      1000       2081273    9255/java          
tcp6       0      0 127.0.0.1:43909         :::*                    LISTEN      1000       945698     2488/java          
tcp6       0      0 :::62502                :::*                    LISTEN      1000       2081205    9255/java          
tcp6       0      0 :::62503                :::*                    LISTEN      1000       2081204    9255/java          
tcp6       0      0 :::46029                :::*                    LISTEN      1000       2081185    9255/java          
tcp6       0      0 fe80::4538:3b62:c:52464 :::*                    LISTEN      1000       2081292    9255/java          
tcp6       0      0 fe80::4538:3b62:c:57600 :::*                    LISTEN      1000       2082278    9255/java          
udp6       0      0 :::7500                 :::*                                1000       2081293    9255/java          

But how to connect?

Create a DiscoveryGroup (no clue about the parameters)

private DiscoveryGroupConfiguration getGroupConfiguration() {
    String name = "name";
    String localBindAddress = "localhost";
    String groupAddress = "localhost";
    int groupPort = 7500;
    long refreshTimeout = 1000;
    long discoveryInitialWaitTimeout = 1000;
    return new DiscoveryGroupConfiguration(name, localBindAddress, groupAddress, groupPort,
            refreshTimeout, discoveryInitialWaitTimeout);
}

Create a connection factory

private HornetQConnectionFactory createDiscoveryGroupFactory() {
     HornetQConnectionFactory factory = HornetQJMSClient.createConnectionFactoryWithHA
          getGroupConfiguration(), JMSFactoryType.CF);
     return factory;
}

Move into JMS world by creating a JMS connection

public Connection createConnection() throws JMSException {
    return createFactory().createConnection();
}

This code does *not* work, it produces

Failed to create discovery group socket

java.net.SocketException: Not a multicast address

    at java.net.MulticastSocket.joinGroup(MulticastSocket.java:310)

    at org.hornetq.core.cluster.impl.DiscoveryGroupImpl.start(DiscoveryGroupImpl.java:126)

    at org.hornetq.core.client.impl.ServerLocatorImpl.initialise(ServerLocatorImpl.java:338)

    at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:553)

    at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:611)

    at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:121)

    at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:116)

I also tried with JGroupsBroadcastGroupConfiguration (from hornetq-core-client 2.3.2.Final)

JGroupsBroadcastGroupConfiguration jgroupsConfig = new JGroupsBroadcastGroupConfiguration(
        "jgroups.xml", "hornetq_broadcast_channel");
String name = "name";
String localBindAddress = jgroupsConfig.getLocalBindAddress();
String groupAddress = jgroupsConfig.getGroupAddress();
int groupPort = jgroupsConfig.getGroupPort();
long refreshTimeout = 1000;
long discoveryInitialWaitTimeout = 1000;

But this code produces null, null and -1 for localBindAddress, groupAddress and groupPort.

I have no clue how to setup my client. Might someone help me, please?

By the way: Finally I want to setup in jgroups.xml a set of servers as seed servers. I aim to 1 live and 2 backup servers using unicast.

Regards

Stephan

[gaohoward]

Can you explain a bit more about what are 'seed servers'? and what is unicast? It looks to me that if you want 'unicast' you shouldn't use broadcast like jgroups.

Anyway it seems that you didn't create a discovery group with jgroups correctly. I suggest you try using UDP first and if UDP works you can change to jgroups based on your UDP client.

[realkobe]

Hi,

you are right. I don't want "unicast" but a configurable discovery group via Jgroups. Jgroups is the requirement given to me. Now I need to build a client that can talk with Jgroups-configured HornetQ Server(s).

"seed server" are a subset of a set of Servers that can tell the client about the topology beyond them. E.g. there are node1 up to node9. These servers know about each other and know how to synchronize. The client knows only about node1, node2 and node3 ( == seed servers): although a client knows only a subset it can talk to each server.

How does a Java Client for JMS but without JNDI for HornetQ, configured with JGroups, look like? Without JGroups it looks like

DiscoveryGroupConfiguration discoveryGroup = new DiscoveryGroupConfiguration("231.7.7.7", 9876);
HornetQConnectionFactory factory = HornetQJMSClient.createConnectionFactoryWithHA(discoveryGroup, JMSFactoryType.CF);
javax.jms.Connection connection = factory.createConnection();

// edit

I also tried a "static" client configuration by using TransportConfiguration

private TransportConfiguration createTransportConfiguration(HostName host) {
     HashMap<String, Object> map = new HashMap<String, Object>();
     map.put(TransportConstants.HOST_PROP_NAME, host.getValue());
     map.put(TransportConstants.PORT_PROP_NAME, "62500");
     TransportConfiguration tc = new TransportConfiguration(NettyConnectorFactory.class
          .getName(), map);
     return tc;
}

private TransportConfiguration[] createTransportConfiguration() {
     return new TransportConfiguration[] {
          createTransportConfiguration(HostName.of("broker-node1")),
          createTransportConfiguration(HostName.of("broker-node2")),
          createTransportConfiguration(HostName.of("broker-node3")) };
}

private HornetQConnectionFactory createStaticFactory() {
     TransportConfiguration[] tc = createTransportConfiguration();
     HornetQConnectionFactory factory = HornetQJMSClient.createConnectionFactoryWithHA(tc, JMSFactoryType.CF);
     return factory;
}

public Connection createConnection() throws JMSException {
     return createStaticFactory().createConnection();
}

Unfortunately that does not work for a composition of 1 live and 2 backup servers (node1 = live, node2+3 = backup).

Initially: It successfully connects to node1 (live) but tries to connect to these backups, too. After the maximum of retries it gives up.

After live went down and a backup came up: There is no connection possible.

If this worked this would be quite enough.

PS: Your WYSIWYG editor is terrible!

[realkobe]

Hey,

did I misunderstand JGroups?

JGroups is for server to server communication so that they can synchronize.

Is JGroups for client to server communication, too?

In general I just asked how to failover with static configuration (Silent HornetQ Failover with 1 Live and 2 Backup).

[jbertram]

HornetQ uses JGroups simply as a server discovery mechanism.  Cluster nodes use it to discover other cluster nodes.  Clients use it to discover servers.  That's it.

JGroups can use a number of different mechanisms for this purpose and one of the main reasons HornetQ integrated JGroups was so that we could use the S3_ping mechanism for cloud-based clusters that can't use UDP and for which static TCP configuration is impractical.

Furthermore, JGroups is not the only server discovery mechanism HornetQ servers and clients can use.  Servers and clients can be configured to use a very simple UDP mechanism which in general is preferred to JGroups because it is simpler to configure.

I'm surprised someone would give a requirement to use JGroups (unless of course JGroups is providing something that the other clustering mechanisms don't).  Typically requirements are stated in functional terms, not technological ones (e.g. use clustering without any reference to how specifically the clustering is done).

[realkobe]

The point is, those HornetQ instances are running in a network (x.y.150.z) and the client is running in a different network (x.y.40.z) thus a udp multicast needs to pass a "network border". I was told to not increase a TTL to (eventually) make this work but to use JGroups because it is configurable. I don't know much more and I'm exploring how to configure / use it - unfortunately not very succesfully.

I need to configure a failover cluster and a client that can re-connect. Maybe JGroups is not the best way if there is another way to do so. Maybe we should pause this JGroups problem and try to finde a (good) solution in Silent HornetQ Failover with 1 Live and 2 Backup ?

[jbertram]

If your clients need to use discovery and they aren't on the same network as the servers then UDP likely won't work without some special configuration to allow UDP to traverse the two networks.

I don't really understand what you're saying about TTL.  Can you clarify this point?

[realkobe]

If I understood correct then the TTL gets decremented when a multicast passes a network border. Thus TTL+1 would work for this multicast if it passed only one border.

But nevertheless this is NOT advised.