1 Reply Latest reply on Sep 15, 2014 10:11 AM by anastkakolyris

Wildfly 8.1.0.Final - Problem while removing/replacing one of two web apps

anastkakolyris Sep 12, 2014 10:05 AM

Hello everyone. This is my first post here so excuse me if i do not provide all the required information.

I am currently working on a project which involves two web applications, a web portal and the core web application with which the portal communicates. I have setup Wildfly 8.1.0.Final in domain mode and enabled clustering using the following guide::

https://docs.jboss.org/author/display/WFLY8/WildFly+8+Cluster+Howto#WildFly8ClusterHowto-ClusterConfiguration

We are using two nodes (master/slave) belonging to main-server-group. The profile we are using is "full-ha". Both web apps have a web.xml (for session replication):

<?xml version="1.0" encoding="UTF-8"?>

<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"

xsi:schemaLocation="http://java.sun.com/xml/ns/javaee

http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd">

</web-app>

The load balancing, session replication and session failover are working fine. The problem occurs when we remove/replace the portal from the web administration console. From the logs we can see the below:

Master: 2014-09-11 14:25:15,675 DEBUG [org.infinispan.affinity.KeyAffinityServiceImpl] (KeyAffinityService Thread Pool -- 1) Shutting down KeyAffinity service for key set: [master:server-one/web]

Slave: 2014-09-11 14:25:15,675 DEBUG [org.infinispan.affinity.KeyAffinityServiceImpl] (KeyAffinityService Thread Pool -- 1) Shutting down KeyAffinity service for key set: [slave:server-one/web]

We are also getting the below warning:

[Server:server-one] 15:36:40,150 WARN [org.springframework.security.web.context.HttpSessionSecurityContextRepository] (default task-12) Failed to create a session, as response has been committed. Unable to store SecurityContext.

every time we try to access the core web application from a mobile client (Login fails). In order to get things working again we have to remove the core web application and re-deploy it.

After some searching, we think that this has something to do with infinispan and the cache container and distributed cache. We tried creating a new cache container and distributed cache and use that for our core web application and let the portal use the default cache container (web). In our core web application we added a jboss-web.xml :

<?xml version="1.0" encoding="UTF-8"?>

<jboss-web>

<replication-config>

<cache-name>funzytalkweb.funzytalkdist</cache-name>

</replication-config>

</jboss-web>

and also added inside jboss-deployment-structure.xml the following:

<?xml version="1.0" encoding="UTF-8"?>

<jboss-deployment-structure>

<!--

Check why below module is needed:

http://stackoverflow.com/questions/13943110/aspectj-aspect-under-jboss-as-7-throws-xlintinvalidabsolutetypename

-->

<module name="org.infinispan"/>

</dependencies>

</deployment>

</jboss-deployment-structure>

The core web application is deployed fine but whenever we try to access it we get the below exceptions:

2014-09-12 13:34:07,094 DEBUG [org.infinispan.marshall.core.VersionAwareMarshaller] (default task-1) Object is not serializable: java.io.NotSerializableException: org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:892)

at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)

at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)

at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:57) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:42) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:30) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:395) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:146)

at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)

at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)

at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:57) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.marshallParameters(CacheRpcCommandExternalizer.java:116) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:100) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:59) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:395) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:146)

at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)

at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)

at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectToObjectStream(AbstractJBossMarshaller.java:74)

at org.infinispan.marshall.core.VersionAwareMarshaller.objectToBuffer(VersionAwareMarshaller.java:77) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commons.marshall.AbstractMarshaller.objectToBuffer(AbstractMarshaller.java:41)

at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectToBuffer(AbstractDelegatingMarshaller.java:85)

at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectToBuffer(MarshallerAdapter.java:23) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:331) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processCalls(CommandAwareRpcDispatcher.java:407) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommands(CommandAwareRpcDispatcher.java:132) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:524) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:281) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.distribution.TxDistributionInterceptor.prepareOnAffectedNodes(TxDistributionInterceptor.java:219) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.distribution.TxDistributionInterceptor.visitPrepareCommand(TxDistributionInterceptor.java:203) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:96) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:42) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:263) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:194) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:94) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:101) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:83) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:64) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.CacheImpl.endBatch(CacheImpl.java:777) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:53) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

at org.wildfly.clustering.web.infinispan.InfinispanBatcher$1.end(InfinispanBatcher.java:56)

at org.wildfly.clustering.web.infinispan.InfinispanBatcher$1.close(InfinispanBatcher.java:46)

at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:72)

at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:719) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:522) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:287) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:73) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:146) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.server.Connectors.executeRootHandler(Connectors.java:177) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_11]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_11]

at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_11]

Caused by: an exception which occurred:

in object org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry@3e55509

-> toString = org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry@3e55509

in object org.infinispan.commands.write.PutKeyValueCommand@8926b235

-> toString = PutKeyValueCommand{key=PH4Rrf3ZywuheYcOB1iKVH5c, value=org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry@3e55509, flags=null, putIfAbsent=true, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}

in object org.infinispan.commands.tx.PrepareCommand@68397063

-> toString = PrepareCommand {modifications=[PutKeyValueCommand{key=PH4Rrf3ZywuheYcOB1iKVH5c, value=org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry@3e55509, flags=null, putIfAbsent=true, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}, PutKeyValueCommand{key=PH4Rrf3ZywuheYcOB1iKVH5c, value={SPRING_SECURITY_CONTEXT=org.springframework.security.core.context.SecurityContextImpl@ffc9c9a9: Authentication: org.springframework.security.authentication.UsernamePasswordAuthenticationToken@ffc9c9a9: Principal: com.msensis.funzy.rest.security.RestUser@36366d: Username: tazz; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Not granted any authorities; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 192.168.1.60; SessionId: null; Not granted any authorities}, flags=[SKIP_LOCKING], putIfAbsent=true, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=true, gtx=GlobalTransaction:<slave:server-one/funzytalkweb>:3:local, cacheName='default-host/funzy', topologyId=2}

2014-09-12 13:34:07,103 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-1) ISPN000136: Execution error: org.infinispan.commons.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)

at org.infinispan.commons.util.Util.rewrapAsCacheException(Util.java:581)

at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommands(CommandAwareRpcDispatcher.java:141) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]