1 2 Previous Next 24 Replies Latest reply on May 19, 2017 9:17 AM by cosmin cosmin

    Full gc is constantly being performed

    cosmin cosmin Newbie

      We have an infinispan 7.0.3 instance running that performs unnecessary garbage collections and the infinispan client terminates with this exception:

      ERROR [RetryOnFailureOperation] (EJB default - 29) ISPN004007: Exception encountered. Retry 10 out of 10: org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException

       

      On the infinispan server.log we have this exception:

      2017-05-03 09:53:07,288 WARNING [io.netty.channel.AbstractChannel] (HotRodServerMaster-6-1) Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xebdfa00d, /127.0.0.1:16094 => /127.0.0.2:11222]: java.lang.OutOfMemoryError: unable to create new native thread
               at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_51]
               at java.lang.Thread.start(Thread.java:713) [rt.jar:1.7.0_51]
               at io.netty.util.concurrent.SingleThreadEventExecutor.startThread(SingleThreadEventExecutor.java:843)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:717)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:416)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:60)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:48)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:64)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.bootstrap.ServerBootstrap$ServerBootstrapAcceptor.channelRead(ServerBootstrap.java:252)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:92)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350) [netty-all-4.0.20.Final.jar:4.0.20.Final]
               at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)[netty-all-4.0.20.Final.jar:4.0.20.Final]
               at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]

       

      The infinispan is running on a SunOS 5.11 11.3 sun4v sparc sun4v machine.

      Our java version is :

      java version "1.8.0_121"

      Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

      Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

       

      I've also attached the detailed gc log.

       

      Any hints are appreciated.

        1 2 Previous Next