How to use the JGroups transport defined in the JGroups subsystem in standalone-ha.xml

[mbabacek]

Dear Infinispan fellows,

I would like to define my own caches programatically in my deployment at runtime of my EE application.

i.e. instead of:

.transport().addProperty("configurationFile", "my-jgroups-tcp.xml")

I would like to, somehow, use the transport defined in JGroups Wildfly 10 subsystem in standalone-ha.xml. I imagine I could reference it probably by a particular channel name, e.g.

.transport().addProperty("channelProvidedByWildFlyJGroups", "myChannel")

The reason for this setup is that I need both the caches defined by default in Wildfly application server within its own Infinispan subsystem and at the same time, I need my own caches, specific to my deployment where I would like to play with Search and indexing.

I don't want to define another, new, JGroups transport in my deployment -- it would be configuration duplication and I would need twice the network properties so as to allow two independent JGroups transports to coexist,

Note that it is not possible to enable indexing and search on caches defined in the default WIldfly 10 Infinispan subsystem -- and it is by design: https://issues.jboss.org/browse/WFLY-5539

Could you recommend me how to proceed? Is it possible to fetch the WildFly JGroups transport and inject it into my default cache manager in my deployment?

THX

-K-

[wdfink]

Infinispan for EAP (the subsystem) is not meant for customized caches beside the one used implicit, this ispn bits are older and not tested for other use-cases.

The Infinispan (or even JDG) bits are a different version and contain more features. Also the JGroups configuration is often different and has to be.

You should keep that JGroups different! as the newer ISPN might require a newer JGroups version.

If you want to share the properties I would recommend to use ${my.jgroups.....} epxressions, you can use that inside of the subsystem and inside of the jgroups-*.xml for ISPN to prevent from different settings.

Does that makes sense to you?

[mbabacek]

It does and it is the way I originally approached, i.e. separate Infinispan for my deployment, separate Jgroups transport.

I used the same network properties for both transports - bind addresses, ports, etc. The result of this two JGroups transports coexistence was a weird suite of Timeout exceptions - WFLY-5158. I tried to mitigate these timeouts even by radical .awaitInitialTransfer(false) which only postponed the problem up until the test traffic arrived.

At the moment when I had configured caches in the Infinispan subsystem with JGroups transport defined within JGroups subsystem -- clustering started to work as expected, but for indexing and search of course.

I will try to come back to using two JGroups transports and I'll try to define completely separate ports for both of them. Unfortunately, given my test environment -- Docker Weave network -- I cannot choose two completely different network interfaces, both JGroups transports must share the same network interface.

I'll share the outcome.

[pferraro]

The simplest way is to inject an existing cache container into your application and register a new configuration programmatically.  That way the cache container is already wired with its associated channel.

e.g.

@Resource(lookup = "java:jboss/infinispan/container/mycontainer")
private EmbeddedCacheManager manager;
private Cache<?, ?> cache;

@PostConstruct
public void init() {
   ConfigurationBuilder builder = new ConfigurationBuilder()
   // custom config here
   this.manager.defineConfiguration("mycache", builder.build());
   this.cache = this.manager.getCache("mycache");
   this.cache.start();
}

@PreDestroy
public void destroy() {
   this.cache.stop();
   this.manager.undefineConfiguration("mycache");
}

[nadirx]

Alternatively, you could construct the GlobalConfiguration by passing in a transport which wraps the channel created by the JGroups subsystem. Look at the one used by the Infinispan subsystem:

https://github.com/wildfly/wildfly/blob/master/clustering/infinispan/extension/src/main/java/org/jboss/as/clustering/infinispan/ChannelTransport.java

You would then use it from something like:

builder.transport().transport(new ChannelTransport(transport.getChannel(), transport.getChannelFactory()))

[mbabacek]

Paul, thank you for this ample example. It does work for me. I'll clean up my code and publish an example application, linked on this thread, so as others could easily use it for reference.

[mbabacek]

Dear Tristan, I'll try this approach as well and post my results. Thank you.