modeshape-clustering quickstart not working in Wildfly 8 Final with ModeShape 4.0.0.Alpha2

[ma6rl]

I am attempting to run the the latest version of of the mode shape-clustering quick start in Wildfly 8.0 with the ModeShape 4.0.0.Alpha2 kit installed.

I cloned the latest version of the quick start and copied:

- standalone-modeshape-master.xml

- standalone-modeshape-slave.xml

into <JBOSS_HOME>/standalone/configuration

I get the following error when I run:

./standalone.sh -c standalone-modeshape-master.xml

=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /apps/wildfly

  JAVA: /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/java

  JAVA_OPTS:  -server -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

15:35:38,682 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Final

15:35:38,903 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.0.Final

15:35:38,977 INFO  [org.jboss.as] (MSC service thread 1-6) JBAS015899: WildFly 8.0.0.Final "WildFly" starting

15:35:39,592 ERROR [org.jboss.as.server] (Controller Boot Thread) JBAS015956: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: JBAS014676: Failed to parse configuration

  at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:112) [wildfly-controller-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.server.ServerService.boot(ServerService.java:331) [wildfly-server-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:256) [wildfly-controller-8.0.0.Final.jar:8.0.0.Final]

  at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_21]

Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[171,13]

Message: JBAS014789: Unexpected element '{urn:jboss:domain:ee:2.0}ejb-annotation-property-replacement' encountered

  at org.jboss.as.controller.parsing.ParseUtils.unexpectedElement(ParseUtils.java:85) [wildfly-controller-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.ee.subsystem.EESubsystemParser20.readElement(EESubsystemParser20.java:110)

  at org.jboss.as.ee.subsystem.EESubsystemParser20.readElement(EESubsystemParser20.java:45)

  at org.jboss.staxmapper.XMLMapperImpl.processNested(XMLMapperImpl.java:110) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

  at org.jboss.staxmapper.XMLExtendedStreamReaderImpl.handleAny(XMLExtendedStreamReaderImpl.java:69) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

  at org.jboss.as.server.parsing.StandaloneXml.parseServerProfile(StandaloneXml.java:1131) [wildfly-server-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.server.parsing.StandaloneXml.readServerElement_1_4(StandaloneXml.java:458) [wildfly-server-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:145) [wildfly-server-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:107) [wildfly-server-8.0.0.Final.jar:8.0.0.Final]

  at org.jboss.staxmapper.XMLMapperImpl.processNested(XMLMapperImpl.java:110) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

  at org.jboss.staxmapper.XMLMapperImpl.parseDocument(XMLMapperImpl.java:69) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

  at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:104) [wildfly-controller-8.0.0.Final.jar:8.0.0.Final]

  ... 3 more

15:35:39,599 FATAL [org.jboss.as.server] (Controller Boot Thread) JBAS015957: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.

15:35:39,614 INFO  [org.jboss.as] (MSC service thread 1-1) JBAS015950: WildFly 8.0.0.Final "WildFly" stopped in 7ms

[hchiorean]

I've opened [MODE-2198] Fix clustering quickstart - JBoss Issue Tracker.

Last time we updated the quickstart it was against Wildfly 8.0.0.CR1, so maybe something changed in the configuration format between CR1 and Final. The Infinispan, JGroups and ModeShape relevant parts of the configuration should still be relevant though.

Also, given the current indexing redesign, the *-jms.xml configuration files are not relevant anymore so they will be removed.

[ma6rl]

Thanks.

[ma6rl]

I am attempting to fix the issues with the existing mode shape-clustering quickstart that are discussed in https://issues.jboss.org/browse/MODE-2198

I have modified the existing standalone-modeshape-master.xml and standalone-modeshape-slave.xml files and renamed them to standalone-modeshape-node1.xml and standalone-modeshape-node1.xml in order to avoid confusion. I have removed the JMS files.

I have updated the README.md to reflect these changes. I also updated the pom.xml to use the wildfly plugin instead of the jboss-as plugin, these changes are also reflected in the README.md.

With the above changes I am now able to start two Wildfly instances and access the Web applications running on them. I also see changes made on one instance reflected in the other.

Where I am currently running into issues is with the Arquillian tests. If I start and stop the node1 Wildfly instance the tests run successfully the first time, subsequent attempt to run the test result in a failure with the following stack trace (see below). Restarting the instance resolves this but only for the first execution of the test.

One thing I did notice while investigating this is that the modeshape-bom-jbossas referenced in the pom.xml is still using:

<jboss.server.version>8.0.0.CR1</jboss.server.version>

instead on

<jboss.server.version>8.0.0.Final</jboss.server.version>

I did change this locally but it did not seem to make a difference.

I have committed all my changes to https://github.com/lucasweb78/quickstart the plan was to issue a pull request once I got everything working but I want to hold of on this until I can resolve the test issues.

I was wondering if anyone has encountered this issue with previous versions of the quick start or with other Arquillian tests?

Update 1:

The same problem occurs if you deploy the mode shape-clustering.war, undeploy the war and then run the tests. It seems that undeploying the war leaves the server in a state that Arquillian does not like. I do not see any problems with the Web Application if I deploy and undeploy the war multiple times.

I did update all of the Arquillian artifacts to 1.1.4.Final locally to see if it solved the problem but it did not seem to make a difference.

End Update 1.

-------- STACK TRACE -----------

Apr 29, 2014 7:15:35 PM org.jboss.as.arquillian.container.ManagementClient checkSuccessful

ERROR: Operation {

    "operation" => "read-resource",

    "recursive" => "true",

    "address" => undefined

} did not succeed. Result was {

    "outcome" => "failed",

    "rolled-back" => true

}

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.358 sec <<< FAILURE!

org.modeshape.quickstart.clustering.ClusteringControllerTest  Time elapsed: 2358 sec  <<< ERROR!

java.lang.RuntimeException: org.jboss.as.arquillian.container.ManagementClient$UnSuccessfulOperationException: undefined

     at org.jboss.as.arquillian.container.ManagementClient.getWebUri(ManagementClient.java:140)

     at org.jboss.as.arquillian.container.ManagementClient.getProtocolMetaData(ManagementClient.java:158)

     at org.jboss.as.arquillian.container.CommonDeployableContainer.deploy(CommonDeployableContainer.java:147)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$3.call(ContainerDeployController.java:161)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$3.call(ContainerDeployController.java:128)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.executeOperation(ContainerDeployController.java:271)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.deploy(ContainerDeployController.java:127)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)

     at org.jboss.arquillian.container.impl.client.ContainerDeploymentContextHandler.createDeploymentContext(ContainerDeploymentContextHandler.java:78)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)

     at org.jboss.arquillian.container.impl.client.ContainerDeploymentContextHandler.createContainerContext(ContainerDeploymentContextHandler.java:57)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)

     at org.jboss.arquillian.container.impl.client.container.DeploymentExceptionHandler.verifyExpectedExceptionDuringDeploy(DeploymentExceptionHandler.java:50)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:135)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:115)

     at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$1.perform(ContainerDeployController.java:95)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$1.perform(ContainerDeployController.java:80)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.forEachDeployment(ContainerDeployController.java:263)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.forEachManagedDeployment(ContainerDeployController.java:239)

     at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.deployManaged(ContainerDeployController.java:79)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:135)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:115)

     at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)

     at org.jboss.arquillian.container.test.impl.client.ContainerEventController.execute(ContainerEventController.java:101)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)

     at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:60)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)

     at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:75)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

     at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:135)

     at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:115)

     at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.beforeClass(EventTestRunnerAdaptor.java:80)

     at org.jboss.arquillian.junit.Arquillian$2.evaluate(Arquillian.java:182)

     at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:314)

     at org.jboss.arquillian.junit.Arquillian.access$100(Arquillian.java:46)

     at org.jboss.arquillian.junit.Arquillian$3.evaluate(Arquillian.java:199)

     at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

     at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:147)

     at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)

     at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)

     at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)

     at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)

     at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)

     at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)

     at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Caused by: org.jboss.as.arquillian.container.ManagementClient$UnSuccessfulOperationException: undefined

     at org.jboss.as.arquillian.container.ManagementClient.checkSuccessful(ManagementClient.java:338)

     at org.jboss.as.arquillian.container.ManagementClient.executeForResult(ManagementClient.java:330)

     at org.jboss.as.arquillian.container.ManagementClient.readResource(ManagementClient.java:325)

     at org.jboss.as.arquillian.container.ManagementClient.readRootNode(ManagementClient.java:215)

     at org.jboss.as.arquillian.container.ManagementClient.getWebUri(ManagementClient.java:137)

     ... 95 more

[hchiorean]

I personally have never come across this issue, but from other issues with Arquillian I can confirm that it's extremely difficult to solve/figure out any such problem. You can always try the Arquillian forums and ask the devs.

From a quickstart perspective, the test was never meant to be a clustered test. Its only purpose is to make sure that the web application is deployed and can be accessed in the default node (the node that doesn't have custom ports).

As long as the 2 nodes (via the webapps) can communicate with each other, that's fine with me.

One thing you can try is using a random name for the test war, making sure that on each run Arquillian will never try to deploy the same application. It may happen that the undeploy is the problem and the web application resource is not properly cleaned from the container.

[ma6rl]

I tried using a random name as you suggested but it did not make any difference. I also re-ran the test but using a local-cache instead of a replicated-cache and did not see the issue. Given this it looks like the problem is related to using a replicated-cache or one of the lower sub-systems such as JGroups or ModCluster.

I am also running into a larger issue (which may or may not be related to this) with clustering as a whole.

If start two Wildfly instances (node1 and node2) using the configuration XML I committed to my forked project (lucasweb78/quickstart · GitHub) and deploy the modeshape-clustered.war everything works as expected. I can bring up the web application on both nodes and changes made on one node are replicated and visible on the second node.

The problem occurs when I shut down the two Wildfly instances and restart them. At this point one of two things happens:

1. One of the nodes does not correctly deploy the modeshape-clustered.war on restart, the only issue shown in the logs is:

13:58:01,419 INFO  [org.jboss.weld.Bootstrap] (weld-worker-1) WELD-000119: Not generating any bean definitions from org.modeshape.quickstart.clustering.ClusteringController because of underlying class loading error: Type org.modeshape.quickstart.clustering.ClusteringController from [Module "deployment.modeshape-clustering.war:main" from Service Module Loader] not found.  If this is unexpected, enable DEBUG logging to see the full error.

2. One of the nodes gets stuck trying to register MBeans (org.infinispan.jmx.CacheJmxRegistration), the last log message you see is:

13:57:07,865 INFO  [org.jboss.as.clustering.infinispan] (default task-1) JBAS010281: Started binary-fs cache from modeshape-binary-cache-container container

The server then sits in this state until the process times out.

At this point the only way to get both nodes running again correctly is to:

1. Undeploy the application from each node
2. Stop the nodes
3. Delete the contents of wildfly/standalone/data/modeshape
4. Start the nodes
5. Deploy the application

At this point you can access the web application on both nodes and everything works as expected, until you restart the wildfly instances again.

I do see the following errors in the logs when I stop the Wildfly instances:

14:01:17,680 ERROR [org.jgroups.jmx.JmxConfigurator] (ServerService Thread Pool -- 66) unregister MBean failed jgroups:type=protocol,cluster="modeshape",protocol=FORK

14:01:17,680 WARN  [org.jgroups.jmx.JmxConfigurator] (ServerService Thread Pool -- 66) MBean unregistration failed javax.management.MBeanRegistrationException

I did a search for this error message and the only reference I could find to it is in some comments from yourself and Randall related to [MODE-2188] Manage custom indexes - JBoss Issue Tracker. At this stage I do not know if this is related to the issue but the fact that I am seeing issues with registering MBeans after receiving this error makes me suspicious.

If you want to try any of this out you can use my forked project lucasweb78/quickstart · GitHub, I can also issue a pull request if you want as it does fix the issues with the current modeshape-clustering not working at all.

UPDATE

The above behavior only happens if you shutdown and restart both instances. If you just restart one instance and navigate to the web application (e.g. http://localhost:8081/modeshape-clustering/main.jsf) you get the following errors in the logs:

14:44:54,988 INFO  [org.jboss.as.clustering.infinispan] (default task-1) JBAS010281: Started binary-fs cache from modeshape-binary-cache-container container

14:44:55,207 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node2/modeshape: dropping unicast message to wrong destination a5d70f3c-f79b-0dd6-c673-07744dbde99e

14:44:55,208 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node2/modeshape-binary-cache-container: dropping unicast message to wrong destination 3cfde9e7-ccef-dc4d-23de-ecb610bd3a46

14:44:55,208 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node2/modeshape-binary-cache-container: dropping unicast message to wrong destination 21d17d01-b5b5-087c-77a9-77483fcf1d97

.......

This continues until you kill the server.

[ma6rl]

I have just installed the new ModeShape 4.0 Alpha 3 Wildfly subsystem and grabbed the latest changes to the mode shape-clustering-quikstart.

The good news is the problems reported above have now been fixed and I can stop and restart a wildfly instance and it will reconnect to the cluster. There is unfortunately a side effect to doing this. Once I restart a Wildfly instance and it reconnects the following gets logged repeatedly by the restarted wildfly instance:

11:27:46,982 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:46,983 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

(full stack trace is below)

Steps to recreate:

1. Start two Wildfly instances using the standalone-modeshape-node1.xml and standalone-modeshape-node2.xml included in the modehape-clustering quickstary

2. Navigate to http://localhost:8080/modeshape-clustering/ and http://localhost:8081/modeshape-clustering/ in two separate brewer windows/tabs

3. Check that changes made on one instance are visible in the other instance

4. Stop the Wildfly instance (node1)

5. Start the Wildfly instance (node1)

Expected:

It should reconnect to the cluster and changes made to one instance are visible in the other instance

Actual:

It does reconnect to the cluster and works as expected but outputs the above log message continually. (See stack trace below)

11:27:44,523 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (default task-2) ISPN000078: Starting JGroups Channel

11:27:44,534 WARN  [org.jgroups.protocols.UDP] (default task-2) JGRP000015: the receive buffer of socket DatagramSocket was set to 20MB, but the OS only allocated 5.03MB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)

11:27:44,534 WARN  [org.jgroups.protocols.UDP] (default task-2) JGRP000015: the receive buffer of socket MulticastSocket was set to 25MB, but the OS only allocated 5.03MB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)

11:27:44,537 INFO  [stdout] (default task-2)

11:27:44,537 INFO  [stdout] (default task-2) -------------------------------------------------------------------

11:27:44,537 INFO  [stdout] (default task-2) GMS: address=node1/modeshape, cluster=modeshape, physical address=127.0.0.1:55200

11:27:44,537 INFO  [stdout] (default task-2) -------------------------------------------------------------------

11:27:44,700 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (default task-2) ISPN000094: Received new cluster view: [node2/modeshape|5] (2) [node2/modeshape, node1/modeshape]

11:27:44,702 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (default task-2) ISPN000079: Cache local address is node1/modeshape, physical addresses are [127.0.0.1:55200]

11:27:44,706 INFO  [org.infinispan.factories.GlobalComponentRegistry] (default task-2) ISPN000128: Infinispan version: Infinispan 'Infinium' 6.0.1.Final

11:27:45,096 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:45,098 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:45,296 INFO  [org.infinispan.jmx.CacheJmxRegistration] (default task-2) ISPN000031: MBeans were successfully registered to the platform MBean server.

11:27:45,532 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:45,533 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:45,944 INFO  [org.jboss.as.clustering.infinispan] (default task-2) JBAS010281: Started clustered-repo cache from modeshape container

11:27:45,981 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:45,982 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:46,364 INFO  [org.infinispan.jmx.CacheJmxRegistration] (default task-2) ISPN000031: MBeans were successfully registered to the platform MBean server.

11:27:46,364 INFO  [org.jboss.as.clustering.infinispan] (default task-2) JBAS010281: Started clustered-repo/system cache from modeshape container

11:27:46,482 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:46,483 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:46,851 INFO  [org.infinispan.jmx.CacheJmxRegistration] (default task-2) ISPN000031: MBeans were successfully registered to the platform MBean server.

11:27:46,851 INFO  [org.jboss.as.clustering.infinispan] (default task-2) JBAS010281: Started clustered-repo/default cache from modeshape container

11:27:46,982 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:46,983 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:47,483 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:47,484 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:47,984 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:47,985 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:48,484 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:48,485 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:48,985 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

11:27:48,985 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination 79a00b02-141b-68cd-33f5-fe9200a58e51

11:27:49,486 WARN  [org.jgroups.protocols.TP$ProtocolAdapter] (INT-1,shared=udp) JGRP000031: node1/modeshape: dropping unicast message to wrong destination d65a6fc5-acf4-6fb8-8035-39ffb2a88bc1

UPDATE:

It looks like this is a known issue with JGroups, see:

[WFLY-2632] JGroups drops unicast messages after shutdown/restart - JBoss Issue Tracker

[JGRP-1755] TP: dropping message to wrong destination in a shared transport - JBoss Issue Tracker

[WFLY-3003] Dropping unicast message to wrong destination warn after cluster node rejoin - JBoss Issue Tracker

At this stage it looks like the fix isn't schedule until Wildfly 9. It looks like it was fixed in JGroups 3.4.2 and 3.5. It also looks like the warnings stop being logged after 60 to 120 seconds due to a timeout and that they can also be safely ignored.