1 Reply Latest reply on Dec 14, 2017 11:54 AM by galder.zamarreno

    HQ122001: Unhandled exception thrown from onMessage: org.infinispan.commons.CacheException: Could not commit implicit transaction

    prateekjainaa

      We have 2 jboss instances with replicated/sync infinispan caches. It is intermittently giving OOM error when they are restarted one by one. Here is the exception trace:

       

      2017-12-01 23:37:24,739 WARN  [org.infinispan.cache.impl.CacheImpl] (Thread-34 (HornetQ-client-global-threads-1783982683)) ISPN000160: Could not complete injected transaction.: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.

      at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1243)

      at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)

      at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)

      at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1628) [infinispan-core-7.1.1.Final.jar:7.1.1.Final]

      at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1585) [infinispan-core-7.1.1.Final.jar:7.1.1.Final]

      at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1056) [infinispan-core-7.1.1.Final.jar:7.1.1.Final]

      at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1048) [infinispan-core-7.1.1.Final.jar:7.1.1.Final]

      at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:432) [infinispan-core-7.1.1.Final.jar:7.1.1.Final]

      at com.blah.cache.infinispan.producer.CacheWrapper.put(CacheWrapper.java:179) [sdk-cache-infinispan-non-cdi-3.48.1.jar:3.48.1]

       

      From heap dump:

       

      Old I/O client worker ([id: 0x1d4e1890, /10.247.246.151:48195 => /10.247.244.26:5445]), runningThread=Thread-48 (HornetQ-client-global-threads-1783982683)

        at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)

        at org.jboss.netty.buffer.HeapChannelBuffer.<init>(I)V (HeapChannelBuffer.java:42)

        at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(I)V (BigEndianHeapChannelBuffer.java:34)

        at org.jboss.netty.buffer.ChannelBuffers.buffer(Ljava/nio/ByteOrder;I)Lorg/jboss/netty/buffer/ChannelBuffer; (ChannelBuffers.java:134)

        at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(Ljava/nio/ByteOrder;I)Lorg/jboss/netty/buffer/ChannelBuffer; (HeapChannelBufferFactory.java:68)

        at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(Ljava/nio/ByteOrder;ILorg/jboss/netty/buffer/ChannelBufferFactory;)V (DynamicChannelBuffer.java:58)

        at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(Ljava/nio/ByteOrder;I)V (DynamicChannelBuffer.java:43)

        at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(Ljava/nio/ByteOrder;I)Lorg/jboss/netty/buffer/ChannelBuffer; (ChannelBuffers.java:207)

        at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(I)Lorg/jboss/netty/buffer/ChannelBuffer; (ChannelBuffers.java:197)

        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.append(Lorg/jboss/netty/buffer/ChannelBuffer;I)V (HornetQFrameDecoder2.java:184)

        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(Lorg/jboss/netty/channel/ChannelHandlerContext;Lorg/jboss/netty/channel/MessageEvent;)V (HornetQFrameDecoder2.java:94)

        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(Lorg/jboss/netty/channel/ChannelHandlerContext;Lorg/jboss/netty/channel/ChannelEvent;)V (SimpleChannelUpstreamHandler.java:70)

        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(Lorg/jboss/netty/channel/DefaultChannelPipeline$DefaultChannelHandlerContext;Lorg/jboss/netty/channel/ChannelEvent;)V (DefaultChannelPipeline.java:564)

        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(Lorg/jboss/netty/channel/ChannelEvent;)V (DefaultChannelPipeline.java:559)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Lorg/jboss/netty/channel/Channel;Ljava/lang/Object;Ljava/net/SocketAddress;)V (Channels.java:268)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Lorg/jboss/netty/channel/Channel;Ljava/lang/Object;)V (Channels.java:255)

        at org.jboss.netty.channel.socket.oio.OioWorker.process()Z (OioWorker.java:71)

        at org.jboss.netty.channel.socket.oio.AbstractOioWorker.run()V (AbstractOioWorker.java:73)

        at org.jboss.netty.channel.socket.oio.OioWorker.run()V (OioWorker.java:51)

        at org.jboss.netty.util.ThreadRenamingRunnable.run()V (ThreadRenamingRunnable.java:108)

        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run()V (DeadLockProofWorker.java:42)

        at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run()V (VirtualExecutorService.java:175)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1145)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:615)

        at java.lang.Thread.run()V (Thread.java:745)

       

      Any clues what could cause this? We are using Jboss EAP 6.4.0 Feel free to ask in case you need more information.