Clustering/Infinispan Issue.

[kevgo]

I am new to JBoss as such I am posting to this thread. 

Here's the scoop;

- I have 3 separate JBoss instances running on 3 separate servers

- No problems deploying non-distributable web apps

- have secured JBoss admin console and apps via https.

- have created cluster for 3 nodes via mod_cluster, each node registering with the cluster, jgroups using UDP

Issue:  I receive the following error when deploying a distributable web application within the cluster.  I believe I am missing something obvious with my infinispan configuration.  Admittedly params have been changed many times to the point I may have lost myself.. lol.  Can someone shed some light on the following and help me decipher what's happening?  As mentioned new to JBoss.. be gentle.

13:25:48,612 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 52) MSC000001: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.transaction.xa.recovery.RecoveryManagerImpl.init(org.infinispan.remoting.rpc.RpcManager,org.infinispan.commands.CommandsFactory,org.infinispan.transaction.TransactionTable,org.infinispan.transaction.TransactionCoordinator,org.infinispan.transaction.xa.TransactionFactory) on object of type RecoveryManagerImpl with parameters [org.infinispan.remoting.rpc.RpcManagerImpl@5e610b99, org.infinispan.commands.CommandsFactoryImpl@35409a37, org.infinispan.transaction.TransactionTable@244cac31, org.infinispan.transaction.TransactionCoordinator@53a3a76f, org.infinispan.transaction.xa.TransactionFactory@3ad48086]

  at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:91) [jboss-as-clustering-common-7.3.0.Final-redhat-10.jar:7.3.0.Final-redhat-10]

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]

  at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]

  at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Final-redhat-1]

Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.transaction.xa.recovery.RecoveryManagerImpl.init(org.infinispan.remoting.rpc.RpcManager,org.infinispan.commands.CommandsFactory,org.infinispan.transaction.TransactionTable,org.infinispan.transaction.TransactionCoordinator,org.infinispan.transaction.xa.TransactionFactory) on object of type RecoveryManagerImpl with parameters [org.infinispan.remoting.rpc.RpcManagerImpl@5e610b99, org.infinispan.commands.CommandsFactoryImpl@35409a37, org.infinispan.transaction.TransactionTable@244cac31, org.infinispan.transaction.TransactionCoordinator@53a3a76f, org.infinispan.transaction.xa.TransactionFactory@3ad48086]

  at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:248)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:167)

  at org.infinispan.factories.InterceptorChainFactory.register(InterceptorChainFactory.java:84)

  at org.infinispan.factories.InterceptorChainFactory.createInterceptor(InterceptorChainFactory.java:73)

  at org.infinispan.factories.InterceptorChainFactory.buildInterceptorChain(InterceptorChainFactory.java:135)

  at org.infinispan.factories.InterceptorChainFactory.construct(InterceptorChainFactory.java:289)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:296)

  at org.infinispan.factories.ComponentRegistry.getOrCreateComponent(ComponentRegistry.java:161)

  at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)

  at org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:86)

  at org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:814)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)

  at org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:188)

  at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)

  at org.infinispan.factories.InternalCacheFactory.bootstrap(InternalCacheFactory.java:101)

  at org.infinispan.factories.InternalCacheFactory.createAndWire(InternalCacheFactory.java:80)

  at org.infinispan.factories.InternalCacheFactory.createCache(InternalCacheFactory.java:64)

  at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:682)

  at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)

  at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)

  at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:559)

  at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:113)

  at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:104)

  at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)

  at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86) [jboss-as-clustering-common-7.3.0.Final-redhat-10.jar:7.3.0.Final-redhat-10]

  ... 4 more

Caused by: java.lang.ClassCastException: org.infinispan.transaction.TransactionTable cannot be cast to org.infinispan.transaction.xa.recovery.RecoveryAwareTransactionTable

  at org.infinispan.transaction.xa.recovery.RecoveryManagerImpl.init(RecoveryManagerImpl.java:101)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45]

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45]

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]

  at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]

  at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)

  ... 114 more

[wdfink]

No idea at the moment

Could you describe a bit what you've installed and configured.

If you use the standalone*-ha.xml configuration it should work.

Where/when do you see the error?

[kevgo]

Bit more info:

JBoss EAP 6.2
java version "1.7.0_45"

Reverted to default infinispan values, errors messages changed a bit.

Using standalone-ha.xml;

<?xml version='1.0' encoding='UTF-8'?>

<server xmlns="urn:jboss:domain:1.5">

    <extensions>
        <extension module="org.jboss.as.clustering.infinispan"/>
        <extension module="org.jboss.as.clustering.jgroups"/>
        <extension module="org.jboss.as.connector"/>
        <extension module="org.jboss.as.deployment-scanner"/>
        <extension module="org.jboss.as.ee"/>
        <extension module="org.jboss.as.ejb3"/>
        <extension module="org.jboss.as.jaxrs"/>
        <extension module="org.jboss.as.jdr"/>
        <extension module="org.jboss.as.jmx"/>
        <extension module="org.jboss.as.jpa"/>
        <extension module="org.jboss.as.jsf"/>
        <extension module="org.jboss.as.logging"/>
        <extension module="org.jboss.as.mail"/>
        <extension module="org.jboss.as.modcluster"/>
        <extension module="org.jboss.as.naming"/>
        <extension module="org.jboss.as.pojo"/>
        <extension module="org.jboss.as.remoting"/>
        <extension module="org.jboss.as.sar"/>
        <extension module="org.jboss.as.security"/>
        <extension module="org.jboss.as.threads"/>
        <extension module="org.jboss.as.transactions"/>
        <extension module="org.jboss.as.web"/>
        <extension module="org.jboss.as.webservices"/>
        <extension module="org.jboss.as.weld"/>
    </extensions>

    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <server-identities>
                    <ssl>
                        <keystore path="/usr/share/jboss-as/jboss-eap-6.2/jboss.keystore" keystore-password="xxxxxxx"/>
                    </ssl>
                    <secret value="amJvc3NhZG1pbjEh"/>
                </server-identities>
                <authentication>
                    <local default-user="$local"/>
                    <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
                <authorization map-groups-to-roles="false">
                    <properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/>
                </authorization>
            </security-realm>
            <security-realm name="ApplicationRealm">
                <authentication>
                    <local default-user="$local" allowed-users="*"/>
                    <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/>
                </authorization>
            </security-realm>
        </security-realms>
        <audit-log>
            <formatters>
                <json-formatter name="json-formatter"/>
            </formatters>
            <handlers>
                <file-handler name="file" formatter="json-formatter" path="audit-log.log" relative-to="jboss.server.data.dir"/>
            </handlers>
            <logger log-boot="true" log-read-only="false" enabled="false">
                <handlers>
                    <handler name="file"/>
                </handlers>
            </logger>
        </audit-log>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket-binding native="management-native"/>
            </native-interface>
            <http-interface security-realm="ManagementRealm">
                <socket-binding https="management-https"/>
            </http-interface>
        </management-interfaces>
        <access-control provider="simple">
            <role-mapping>
                <role name="SuperUser">
                    <include>
                        <user name="$local"/>
                    </include>
                </role>
            </role-mapping>
        </access-control>
    </management>

    <profile>
        <subsystem xmlns="urn:jboss:domain:logging:1.3">
            <console-handler name="CONSOLE">
                <level name="WARN"/>
                <formatter>
                    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] %s%E%n"/>
                </formatter>
            </console-handler>
            <periodic-rotating-file-handler name="FILE" autoflush="true">
                <formatter>
                    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
                </formatter>
                <file relative-to="jboss.server.log.dir" path="server.log"/>
                <suffix value=".yyyy-MM-dd"/>
                <append value="true"/>
            </periodic-rotating-file-handler>
            <logger category="com.arjuna">
                <level name="WARN"/>
            </logger>
            <logger category="org.apache.tomcat.util.modeler">
                <level name="WARN"/>
            </logger>
            <logger category="org.jboss.as.config">
                <level name="DEBUG"/>
            </logger>
            <logger category="sun.rmi">
                <level name="WARN"/>
            </logger>
            <logger category="jacorb">
                <level name="WARN"/>
            </logger>
            <logger category="jacorb.config">
                <level name="ERROR"/>
            </logger>
            <root-logger>
                <level name="INFO"/>
                <handlers>
                    <handler name="CONSOLE"/>
                    <handler name="FILE"/>
                </handlers>
            </root-logger>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:datasources:1.1">
            <datasources>
                <datasource jndi-name="java:jboss/datasources/ExampleDS" pool-name="ExampleDS" enabled="true" use-java-context="true">
                    <connection-url>jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</connection-url>
                    <driver>h2</driver>
                    <security>
                        <user-name>sa</user-name>
                        <password>sa</password>
                    </security>
                </datasource>
                <drivers>
                    <driver name="h2" module="com.h2database.h2">
                        <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
                    </driver>
                </drivers>
            </datasources>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:deployment-scanner:1.1">
            <deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:ee:1.1">
            <ear-subdeployments-isolated>false</ear-subdeployments-isolated>
            <spec-descriptor-property-replacement>false</spec-descriptor-property-replacement>
            <jboss-descriptor-property-replacement>true</jboss-descriptor-property-replacement>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:ejb3:1.4">
            <session-bean>
                <stateless>
                    <bean-instance-pool-ref pool-name="slsb-strict-max-pool"/>
                </stateless>
                <stateful default-access-timeout="5000" cache-ref="simple" clustered-cache-ref="clustered"/>
                <singleton default-access-timeout="5000"/>
            </session-bean>
            <pools>
                <bean-instance-pools>
                    <strict-max-pool name="slsb-strict-max-pool" max-pool-size="20" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
                    <strict-max-pool name="mdb-strict-max-pool" max-pool-size="20" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
                </bean-instance-pools>
            </pools>
            <caches>
                <cache name="simple" aliases="NoPassivationCache"/>
                <cache name="passivating" passivation-store-ref="file" aliases="SimpleStatefulCache"/>
                <cache name="clustered" passivation-store-ref="infinispan" aliases="StatefulTreeCache"/>
            </caches>
            <passivation-stores>
                <file-passivation-store name="file"/>
                <cluster-passivation-store name="infinispan" cache-container="ejb"/>
            </passivation-stores>
            <async thread-pool-name="default"/>
            <timer-service thread-pool-name="default">
                <data-store path="timer-service-data" relative-to="jboss.server.data.dir"/>
            </timer-service>
            <remote connector-ref="remoting-connector" thread-pool-name="default"/>
            <thread-pools>
                <thread-pool name="default">
                    <max-threads count="10"/>
                    <keepalive-time time="100" unit="milliseconds"/>
                </thread-pool>
            </thread-pools>
            <default-security-domain value="other"/>
            <default-missing-method-permissions-deny-access value="true"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:infinispan:1.4">
            <cache-container name="singleton" aliases="cluster ha-partition" default-cache="default">
                <transport lock-timeout="60000"/>
                <replicated-cache name="default" mode="SYNC" batching="true">
                    <locking isolation="REPEATABLE_READ"/>
                </replicated-cache>
            </cache-container>
            <cache-container name="web" aliases="standard-session-cache" default-cache="repl" module="org.jboss.as.clustering.web.infinispan
">
                <transport lock-timeout="60000"/>
                <replicated-cache name="repl" mode="ASYNC" batching="true">
                    <file-store/>
                </replicated-cache>
                <replicated-cache name="sso" mode="SYNC" batching="true"/>
                <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0">
                    <file-store/>
                </distributed-cache>
            </cache-container>
            <cache-container name="ejb" aliases="sfsb sfsb-cache" default-cache="repl" module="org.jboss.as.clustering.ejb3.infinispan">
                <transport lock-timeout="60000"/>
                <replicated-cache name="repl" mode="ASYNC" batching="true">
                    <eviction strategy="LRU" max-entries="10000"/>
                    <file-store/>
                </replicated-cache>
                <!--
                  ~  Clustered cache used internally by EJB subsytem for managing the client-mapping(s) of
                  ~                 the socketbinding referenced by the EJB remoting connector
                  -->
                <replicated-cache name="remote-connector-client-mappings" mode="SYNC" batching="true"/>
                <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0">
                    <eviction strategy="LRU" max-entries="10000"/>
                    <file-store/>
                </distributed-cache>
            </cache-container>
            <cache-container name="hibernate" default-cache="local-query" module="org.jboss.as.jpa.hibernate:4">
                <transport lock-timeout="60000"/>
                <local-cache name="local-query">
                    <transaction mode="NONE"/>
                    <eviction strategy="LRU" max-entries="10000"/>
                    <expiration max-idle="100000"/>
                </local-cache>
                <invalidation-cache name="entity" mode="SYNC">
                    <transaction mode="NON_XA"/>
                    <eviction strategy="LRU" max-entries="10000"/>
                    <expiration max-idle="100000"/>
                </invalidation-cache>
                <replicated-cache name="timestamps" mode="ASYNC">
                    <transaction mode="NONE"/>
                    <eviction strategy="NONE"/>
                </replicated-cache>
            </cache-container>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:jaxrs:1.0"/>
        <subsystem xmlns="urn:jboss:domain:jca:1.1">
            <archive-validation enabled="true" fail-on-error="true" fail-on-warn="false"/>
            <bean-validation enabled="true"/>
            <default-workmanager>
                <short-running-threads>
                    <core-threads count="50"/>
                    <queue-length count="50"/>
                    <max-threads count="50"/>
                    <keepalive-time time="10" unit="seconds"/>
                </short-running-threads>
                <long-running-threads>
                    <core-threads count="50"/>
                    <queue-length count="50"/>
                    <max-threads count="50"/>
                    <keepalive-time time="10" unit="seconds"/>
                </long-running-threads>
            </default-workmanager>
            <cached-connection-manager/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:jdr:1.0"/>
        <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="udp">
            <stack name="udp">
                <transport type="UDP" socket-binding="jgroups-udp"/>
                <protocol type="PING"/>
                <protocol type="MERGE3"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
                <protocol type="FD"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="pbcast.NAKACK"/>
                <protocol type="UNICAST2"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="UFC"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
                <protocol type="RSVP"/>
            </stack>
            <stack name="tcp">
                <transport type="TCP" socket-binding="jgroups-tcp"/>
                <protocol type="TCPPING">
                    <property name="initial_hosts">
                        xxx.xxx.xxx.xx[7600],xxx.xxx.xxx.xx[7600]
                    </property>
                    <property name="num_initial_members">
                        2
                    </property>
                    <property name="port_range">
                        0
                    </property>
                    <property name="timeout">
                        10000
                    </property>
                </protocol>
                <protocol type="MPING" socket-binding="jgroups-mping"/>
                <protocol type="MERGE2"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                <protocol type="FD"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="pbcast.NAKACK"/>
                <protocol type="UNICAST2"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="UFC"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
                <protocol type="RSVP"/>
            </stack>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:jmx:1.3">
            <expose-resolved-model/>
            <expose-expression-model/>
            <remoting-connector/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:jpa:1.1">
            <jpa default-datasource="" default-extended-persistence-inheritance="DEEP"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:jsf:1.0"/>
        <subsystem xmlns="urn:jboss:domain:mail:1.1">
            <mail-session jndi-name="java:jboss/mail/Default">
                <smtp-server outbound-socket-binding-ref="mail-smtp"/>
            </mail-session>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:modcluster:1.1">
            <mod-cluster-config advertise-socket="modcluster" balancer="mycluster" advertise="true" socket-timeout="5" sticky-session="true" worker-timeout="3" node-timeout="3" connector="ajp">
                <dynamic-load-provider>
                    <load-metric type="busyness"/>
                </dynamic-load-provider>
            </mod-cluster-config>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:naming:1.4">
            <remote-naming/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:pojo:1.0"/>
        <subsystem xmlns="urn:jboss:domain:remoting:1.1">
            <connector name="remoting-connector" socket-binding="remoting" security-realm="ApplicationRealm"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:resource-adapters:1.1"/>
        <subsystem xmlns="urn:jboss:domain:sar:1.0"/>
        <subsystem xmlns="urn:jboss:domain:security:1.2">
            <security-domains>
                <security-domain name="spnego-connect" cache-type="default">
                    <authentication>
                        <login-module code="com.sun.security.auth.module.Krb5LoginModule" flag="required"/>
                    </authentication>
                </security-domain>
                <security-domain name="spnego-server" cache-type="default">
                    <authentication>
                        <login-module code="com.sun.security.auth.module.Krb5LoginModule" flag="required">
                            <module-option name="storeKey" value="true"/>
                        </login-module>
                    </authentication>
                </security-domain>
                <security-domain name="other" cache-type="default">
                    <authentication>
                        <login-module code="Remoting" flag="optional">
                            <module-option name="password-stacking" value="useFirstPass"/>
                        </login-module>
                        <login-module code="RealmDirect" flag="required">
                            <module-option name="password-stacking" value="useFirstPass"/>
                        </login-module>
                    </authentication>
                </security-domain>
                <security-domain name="jboss-web-policy" cache-type="default">
                    <authorization>
                        <policy-module code="Delegating" flag="required"/>
                    </authorization>
                </security-domain>
                <security-domain name="jboss-ejb-policy" cache-type="default">
                    <authorization>
                        <policy-module code="Delegating" flag="required"/>
                    </authorization>
                </security-domain>
            </security-domains>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:threads:1.1">
            <thread-factory name="http-connector-factory" group-name="uq-thread-pool" thread-name-pattern="HTTP-%t" priority="9"/>
            <unbounded-queue-thread-pool name="uq-thread-pool">
                <max-threads count="100"/>
                <keepalive-time time="50" unit="seconds"/>
                <thread-factory name="http-connector-factory"/>
            </unbounded-queue-thread-pool>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:transactions:1.4">
            <core-environment>
                <process-id>
                    <uuid/>
                </process-id>
            </core-environment>
            <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>
            <coordinator-environment enable-statistics="false" enable-tsm-status="false" default-timeout="300"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:web:1.5" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">
            <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
            <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>
            <connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true" executor="uq-thread-pool">
                <ssl name="https" password="xxxxx" certificate-key-file="/usr/share/jboss-as/jboss-eap-6.2/jboss.keystore"/>
            </connector>
            <virtual-server name="default-host" enable-welcome-root="false">
                <alias name="localhost"/>
                <alias name="example.com"/>
            </virtual-server>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:webservices:1.2">
            <modify-wsdl-address>true</modify-wsdl-address>
            <wsdl-host>${jboss.bind.address:127.0.0.1}</wsdl-host>
            <endpoint-config name="Standard-Endpoint-Config"/>
            <endpoint-config name="Recording-Endpoint-Config">
                <pre-handler-chain name="recording-handlers" protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP ##SOAP12_HTTP_MTOM">
                    <handler name="RecordingHandler" class="org.jboss.ws.common.invocation.RecordingServerHandler"/>
                </pre-handler-chain>
            </endpoint-config>
            <client-config name="Standard-Client-Config"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:weld:1.0"/>
    </profile>

    <interfaces>
        <interface name="management">
            <inet-address value="${jboss.bind.address.management:xxx.xxx.xxx.xx}"/>
        </interface>
        <interface name="public">
            <inet-address value="${jboss.bind.address:xxx.xxx.xxx.xx}"/>
        </interface>
        <interface name="unsecure">
            <inet-address value="${jboss.bind.address.unsecure:xxx.xxx.xxx.xx}"/>
        </interface>
    </interfaces>

    <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
        <socket-binding name="management-native" interface="management" port="${jboss.management.native.port:9999}"/>
        <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
        <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9443}"/>
        <socket-binding name="ajp" port="8009"/>
        <socket-binding name="http" port="8080"/>
        <socket-binding name="https" port="8443"/>
        <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
        <socket-binding name="jgroups-tcp" port="7600"/>
        <socket-binding name="jgroups-tcp-fd" port="57600"/>
        <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
        <socket-binding name="jgroups-udp-fd" port="54200"/>
        <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>
        <socket-binding name="remoting" port="4447"/>
        <socket-binding name="txn-recovery-environment" port="4712"/>
        <socket-binding name="txn-status-manager" port="4713"/>
        <outbound-socket-binding name="mail-smtp">
            <remote-destination host="mailserv.gnb.ca" port="25"/>
        </outbound-socket-binding>
    </socket-binding-group>

</server>

While deploying the app NBF_DEV.ear getting the following from NBF_DEV.ear.failed;

{"JBAS014653: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-2" => {"JBAS014671: Failed services" =

> {"jboss.infinispan.web.default-host/NBF_DEV" => "org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/NBF_DEV

: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialSta

teTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl

    Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.wait

ForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl

    Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache default-host/NBF_DEV on WebP03/web"}}}}

Server.log:

16:08:56,820 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Final-redhat-2
16:08:57,309 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
16:08:57,515 INFO  [org.jboss.as] (MSC service thread 1-2) JBAS015899: JBoss EAP 6.2.0.GA (AS 7.3.0.Final-redhat-10) starting
16:08:57,525 DEBUG [org.jboss.as.config] (MSC service thread 1-2) Configured system properties:
[Standalone] =
awt.toolkit = sun.awt.X11.XToolkit
file.encoding = UTF-8
file.encoding.pkg = sun.io
file.separator = /
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.awt.headless = true
java.awt.printerjob = sun.print.PSPrinterJob
java.class.path = /usr/share/jboss-as/jboss-eap-6.2/jboss-modules.jar
java.class.version = 51.0
java.endorsed.dirs = /usr/java/jdk1.7.0_45/jre/lib/endorsed
java.ext.dirs = /usr/java/jdk1.7.0_45/jre/lib/ext:/usr/java/packages/lib/ext
java.home = /usr/java/jdk1.7.0_45/jre
java.io.tmpdir = /tmp
java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
java.net.preferIPv4Stack = true
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.7.0_45-b18
java.specification.name = Java Platform API Specification
java.specification.vendor = Oracle Corporation
java.specification.version = 1.7
java.util.logging.manager = org.jboss.logmanager.LogManager
java.vendor = Oracle Corporation
java.vendor.url = http://java.oracle.com/
java.vendor.url.bug = http://bugreport.sun.com/bugreport/
java.version = 1.7.0_45
java.vm.info = mixed mode
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.name = Java Virtual Machine Specification
java.vm.specification.vendor = Oracle Corporation
java.vm.specification.version = 1.7
java.vm.vendor = Oracle Corporation
java.vm.version = 24.45-b08
javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder
javax.xml.datatype.DatatypeFactory = __redirected.__DatatypeFactory
javax.xml.parsers.DocumentBuilderFactory = __redirected.__DocumentBuilderFactory
javax.xml.parsers.SAXParserFactory = __redirected.__SAXParserFactory
javax.xml.stream.XMLEventFactory = __redirected.__XMLEventFactory
javax.xml.stream.XMLInputFactory = __redirected.__XMLInputFactory
javax.xml.stream.XMLOutputFactory = __redirected.__XMLOutputFactory
javax.xml.transform.TransformerFactory = __redirected.__TransformerFactory
javax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema = __redirected.__SchemaFactory
javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom = __redirected.__XPathFactory
jboss.bind.address = xxx.xxx.xxx.xx
jboss.home.dir = /usr/share/jboss-as/jboss-eap-6.2
jboss.host.name = slv76nbfwebp03
jboss.modules.dir = /usr/share/jboss-as/jboss-eap-6.2/modules
jboss.modules.system.pkgs = org.jboss.byteman
jboss.node.name = WebP03
jboss.qualified.host.name = slv76nbfwebp03
jboss.server.base.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone
jboss.server.config.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone/configuration
jboss.server.data.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone/data
jboss.server.deploy.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone/data/content
jboss.server.log.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone/log
jboss.server.name = slv76nbfwebp03
jboss.server.persist.config = true
jboss.server.temp.dir = /usr/share/jboss-as/jboss-eap-6.2/standalone/tmp
line.separator =

logging.configuration = file:/usr/share/jboss-as/jboss-eap-6.2/standalone/configuration/logging.properties
module.path = /usr/share/jboss-as/jboss-eap-6.2/modules
org.jboss.boot.log.file = /usr/share/jboss-as/jboss-eap-6.2/standalone/log/server.log
org.jboss.resolver.warning = true
org.xml.sax.driver = __redirected.__XMLReaderFactory
os.arch = amd64
os.name = Linux
os.version = 2.6.32-431.1.2.el6.x86_64
path.separator = :
sun.arch.data.model = 64
sun.boot.class.path = /usr/java/jdk1.7.0_45/jre/lib/resources.jar:/usr/java/jdk1.7.0_45/jre/lib/rt.jar:/usr/java/jdk1.7.0_45/jre/lib/sunrsasign.jar:/usr/java/jdk1.7.0_45/jre/lib/jsse.jar:/usr/java/jdk1.7.0_45/jre/lib/jce.jar:/usr/java/jdk1.7.0_45/jre/lib/charsets.jar:/usr/java/jdk1.7.0_45/jre/lib/jfr.jar:/usr/java/jdk1.7.0_45/jre/classes
sun.boot.library.path = /usr/java/jdk1.7.0_45/jre/lib/amd64
sun.cpu.endian = little
sun.cpu.isalist =
sun.io.unicode.encoding = UnicodeLittle
sun.java.command = /usr/share/jboss-as/jboss-eap-6.2/jboss-modules.jar -mp /usr/share/jboss-as/jboss-eap-6.2/modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/usr/share/jboss-as/jboss-eap-6.2 -Djboss.server.base.dir=/usr/share/jboss-as/jboss-eap-6.2/standalone -c standalone-ha.xml
sun.java.launcher = SUN_STANDARD
sun.jnu.encoding = UTF-8
sun.management.compiler = HotSpot 64-Bit Tiered Compilers
sun.os.patch.level = unknown
user.country = US
user.dir = /
user.home = /home/jboss
user.language = en
user.name = jboss
user.timezone = America/Moncton
16:08:57,530 DEBUG [org.jboss.as.config] (MSC service thread 1-2) VM Arguments: -D[Standalone] -XX:+UseCompressedOops -Xms4303m -Xmx4303m -Xss256k -XX:MaxPermSize=512m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.node.name=WebP03 -Djboss.bind.address=142.139.xxx.xx -Dorg.jboss.boot.log.file=/usr/share/jboss-as/jboss-eap-6.2/standalone/log/server.log -Dlogging.configuration=file:/usr/share/jboss-as/jboss-eap-6.2/standalone/configuration/logging.properties
16:08:59,495 INFO  [org.xnio] (MSC service thread 1-1) XNIO Version 3.0.7.GA-redhat-1
16:08:59,519 INFO  [org.jboss.as.server] (Controller Boot Thread) JBAS015889: Creating http management service using secure-socket-binding (management-https)
16:08:59,522 INFO  [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
16:08:59,537 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 3.2.17.GA-redhat-1
16:08:59,632 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 31) JBAS010280: Activating Infinispan subsystem.
16:08:59,651 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 35) JBAS010260: Activating JGroups subsystem.
16:08:59,694 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 38) JBAS012605: Activated the following JSF Implementations: [main, 1.2]
16:08:59,720 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (IronJacamar 1.0.21.Final-redhat-1)
16:08:59,731 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 41) JBAS011800: Activating Naming Subsystem
16:08:59,797 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 46) JBAS013171: Activating Security Subsystem
16:08:59,890 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 27) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
16:08:59,930 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 50) JBAS015537: Activating WebServices Extension
16:09:00,008 INFO  [org.jboss.as.security] (MSC service thread 1-2) JBAS013170: Current PicketBox version=4.0.19.SP2-redhat-1
16:09:00,162 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: Starting Naming Service
16:09:00,262 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-2) JBAS015400: Bound mail session [java:jboss/mail/Default]
16:09:00,655 INFO  [org.apache.coyote.http11] (MSC service thread 1-2) JBWEB003001: Coyote HTTP/1.1 initializing on : http-/xxx.xxx.xxx.xx:8080
16:09:00,685 INFO  [org.apache.coyote.http11] (MSC service thread 1-2) JBWEB003000: Coyote HTTP/1.1 starting on: http-/xxx.xxx.xxx.xx:8080
16:09:00,748 INFO  [org.apache.coyote.ajp] (MSC service thread 1-2) JBWEB003046: Starting Coyote AJP/1.3 on ajp-/xxx.xxx.xxx.xx:8009
16:09:01,219 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 31) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:01,222 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 52) MODCLUSTER000001: Initializing mod_cluster ${project.version}
16:09:01,285 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 31) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:01,304 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 52) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
16:09:01,662 INFO  [org.jboss.ws.common.management] (MSC service thread 1-2) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.2.2.Final-redhat-2
16:09:02,474 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-2) JBAS015012: Started FileSystemDeploymentService for directory /usr/share/jboss-as/jboss-eap-6.2/standalone/deployments
16:09:02,512 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "ClusterWebApp.war" (runtime-name: "ClusterWebApp.war")
16:09:02,518 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
16:09:02,929 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003001: Coyote HTTP/1.1 initializing on : http-/142.xxx.xxx.xx:8443
16:09:02,936 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003000: Coyote HTTP/1.1 starting on: http-/142.xxx.xxx.xx:8443
16:09:02,977 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on xxx.xxx.xxx.xx:4447
16:09:03,029 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on xxx.xxx.xxx.xx:9999
16:09:04,188 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,196 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,202 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,202 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,202 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,203 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:09:04,248 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups Channel
16:09:04,262 INFO  [stdout] (MSC service thread 1-1)
16:09:04,267 INFO  [stdout] (MSC service thread 1-1) -------------------------------------------------------------------
16:09:04,267 INFO  [stdout] (MSC service thread 1-1) GMS: address=WebP03/web, cluster=web, physical address=142.xxx.xxx.xx:55200
16:09:04,267 INFO  [stdout] (MSC service thread 1-1) -------------------------------------------------------------------
16:09:04,505 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000094: Received new cluster view: [WebP01/web|97] [WebP01/web, WebP03/web]
16:09:04,616 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Cache local address is WebP03/web, physical addresses are [xxx.xxx.xxx.xx:55200]
16:09:04,627 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Delirium' 5.2.7.Final
16:09:04,654 INFO  [org.jboss.as.clustering] (MSC service thread 1-1) JBAS010238: Number of cluster members: 2
16:09:04,703 INFO  [org.infinispan.factories.TransactionManagerFactory] (ServerService Thread Pool -- 53) ISPN000161: Using a batchMode transaction manager
16:09:04,713 INFO  [org.infinispan.factories.TransactionManagerFactory] (ServerService Thread Pool -- 52) ISPN000161: Using a batchMode transaction manager
16:09:04,885 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 52) ISPN000031: MBeans were successfully registered to the platform MBean server.
16:09:04,884 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 53) ISPN000031: MBeans were successfully registered to the platform MBean server.
16:09:05,002 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 52) JBAS010281: Started default-host/ClusterWebApp cache from web container
16:09:05,011 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 53) JBAS010281: Started repl cache from web container
16:09:05,035 INFO  [org.jboss.web] (ServerService Thread Pool -- 52) JBAS018210: Register web context: /ClusterWebApp
16:09:05,214 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 28) JBAS018559: Deployed "ClusterWebApp.war" (runtime-name : "ClusterWebApp.war")
16:09:05,249 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015962: Http management interface listening on https://142.xxx.xxx.xxhttps://142.xxx.xxx.xx:9443/management:9443/management
16:09:05,250 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015952: Admin console listening on https://142.xxx.xxx.xx:9443
16:09:05,250 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.GA (AS 7.3.0.Final-redhat-10) started in 8960ms - Started 179 of 306 services (126 services are passive or on-demand)
16:12:30,715 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015876: Starting deployment of "NBF_DEV.ear" (runtime-name: "NBF_DEV.ear")
16:12:34,867 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "null" (runtime-name: "NBF_DEV.war")
16:12:34,867 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "null" (runtime-name: "NBF_DEVF.war")
16:12:39,724 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015960: Class Path entry ./ in /content/NBF_DEV.ear/NBF_DEVF.war  does not point to a valid jar for a Class-Path reference.
16:12:39,725 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015960: Class Path entry genrt.80.jar in /content/NBF_DEV.ear/NBF_DEVF.war  does not point to a valid jar for a Class-Path reference.
16:12:39,833 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015960: Class Path entry ./ in /content/NBF_DEV.ear/NBF_DEV.war  does not point to a valid jar for a Class-Path reference.
16:12:39,833 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015960: Class Path entry genrt.80.jar in /content/NBF_DEV.ear/NBF_DEV.war  does not point to a valid jar for a Class-Path reference.
16:12:43,433 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:12:43,433 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:12:43,433 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:12:43,434 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
16:12:43,531 INFO  [org.infinispan.factories.TransactionManagerFactory] (ServerService Thread Pool -- 70) ISPN000161: Using a batchMode transaction manager
16:12:43,583 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 70) ISPN000031: MBeans were successfully registered to the platform MBean server.
16:12:43,584 INFO  [org.infinispan.factories.TransactionManagerFactory] (ServerService Thread Pool -- 72) ISPN000161: Using a batchMode transaction manager
16:12:43,596 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 72) ISPN000031: MBeans were successfully registered to the platform MBean server.
16:12:43,628 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 70) JBAS010281: Started default-host/NBF_DEVF cache from web container
16:12:43,629 INFO  [org.jboss.web] (ServerService Thread Pool -- 70) JBAS018210: Register web context: /NBF_DEVF
16:13:43,610 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 72) MSC000001: Failed to start service jboss.infinispan.web.default-host/NBF_DEV: org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/NBF_DEV: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:91)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:886)
at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:657)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:646)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:549)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:217)
at org.infinispan.CacheImpl.start(CacheImpl.java:582)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:686)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:559)
at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:113)
at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:104)
at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86)
... 4 more
Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache default-host/NBF_DEV on WebP03/web
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
... 18 more

16:13:43,845 ERROR [org.jboss.as.server] (DeploymentScanner-threads - 2) JBAS015870: Deploy of deployment "NBF_DEV.ear" was rolled back with the following failure message:
{"JBAS014671: Failed services" => {"jboss.infinispan.web.default-host/NBF_DEV" => "org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/NBF_DEV: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
    Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
    Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache default-host/NBF_DEV on WebP03/web"}}
16:13:43,935 INFO  [org.jboss.web] (ServerService Thread Pool -- 362) JBAS018224: Unregister web context: /NBF_DEVF
16:13:43,968 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 362) MODCLUSTER000021: All pending requests drained from default-host:/NBF_DEVF in 0.0 seconds
16:13:44,182 INFO  [org.infinispan.eviction.PassivationManagerImpl] (ServerService Thread Pool -- 315) ISPN000029: Passivating all entries to disk
16:13:44,182 INFO  [org.infinispan.eviction.PassivationManagerImpl] (ServerService Thread Pool -- 315) ISPN000030: Passivated 0 entries in 0 milliseconds
16:13:44,197 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 315) JBAS010282: Stopped default-host/NBF_DEVF cache from web container
16:13:44,975 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015877: Stopped deployment null (runtime-name: NBF_DEV.war) in 1141ms
16:13:44,999 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015877: Stopped deployment null (runtime-name: NBF_DEVF.war) in 1164ms
16:13:45,025 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015877: Stopped deployment NBF_DEV.ear (runtime-name: NBF_DEV.ear) in 1191ms

Any assistance deciphering the above would be appreciated.  I wouldn't wouldn't be surprised if it's related to the app (CA Gen generated code - also still in Dev) in some way.  I'm also not a programmer.. lol.. making deciphering logs a little more challenging.

Thanks.

Kevin.

[neethal]

Hi.. were you able to figure out the issue. even i am facing the same error in the logs

ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 69) MSC000001: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type JGroupsTransport

        at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:87)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_75]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_75]

        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_75]

        at org.jboss.threads.JBossThread.run(JBossThread.java:122)

Caused by: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type JGroupsTransport

        at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:247)

        at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:678)

        at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)

        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)

        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:559)

        at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:109)

        at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:100)

        at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)

        at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:82)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_75]

Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type JGroupsTransport

[wdfink]

Do you see timeouts within the root-cause?

If not you should open a different discussion and provide the configuration and logfile.

Please attach it instead of posting it as comment (you need to edit the comment for attachments)