Infinispan questions

[rcd]

I've got a project with a bunch of Arquillian tests and a Jenkins job to build/test after each commit. Recently, after adding some new tests, I started getting the infamous "out of PermGen space" error that typically indicates a classloader leak. After investigating, I've found and fixed some problems in my own code and some other libraries, but there's a classloader leak related to Infinispan that I can't figure out how to fix.

Specifically, I have the same problem as in WFLY-2605: a TxCleanupService thread is holding a reference to the deployment classloader. The third comment on that issue says that the problem is that the cache container is configured with start="EAGER", and the fourth comment confirms that removing eager start fixes the problem. Indeed, my cache container is configured with eager start. But if I remove eager start from the cache container configuration, I run into the problem described here (looking up the cache container in JNDI fails because it's not started) and the application doesn't even deploy. So it's a catch-22: if I have eager start, then I have a classloader leak, but if I don't have eager start, then the application doesn't work.

Question #1: how can I get WildFly to inject the cache container without eager start? Is that even possible?

Question #2: what happens to the cache container and its caches when a deployment that has been using them gets undeployed? Comment #3 on WFLY-2605 seems to imply that the cache container's lifecycle is the same as the deployment's, but I don't know if that's true in general or just some magic in the Hibernate/Infinispan integration.

[pferraro]

1.  start="EAGER" is only meant for caches and caches that are accessed remotely (e.g. via hotrod or similar) - where there is otherwise no mechanism to start the cache.  If you only need to access the cache-container or cache from an application, you can use EE resource injection.  This will ensure that the cache-container or cache is started when your application is deployed, and stopped when the last dependent application is undeployed.

To inject a cache-container named foo, use the following in your code:

@Resource(lookup = "java:jboss/infinispan/container/foo")
private EmbeddedCacheManager container;

Or, to inject a cache named bar:

@Resource(lookup = "java:jboss/infinispan/cache/foo/bar")
private Cache<K, V> cache;

2. The code above will establish a service dependency on the cache-container or cache JNDI binding to your deployment.  The JNDI bindings themselves depend on the corresponding cache-container or cache service.  Since these services are on-demand, they will start only when some other service demands them to start, and will stop when that dependency is no longer required.

[pferraro]

I should also note that JPA applications will (by default) automatically depend on the hibernate cache-container.  Likewise, distributable web applications will (by default) automatically depend on the web cache-container - and EJB deployments will (by default) automatically depend on the ejb cache-container.

[rcd]

Thanks for the clarification about start="EAGER". I'm also happy to hear that the caches will stop automatically when they're no longer needed.

But I still don't know why I have a problem without eager start. I'm injecting the cache container using @Resource, exactly like you said, but I get the below exception when I try to deploy the application. The root cause is that the container service isn't started, but according to what you said, it should have been. Any idea why this might happen? It sounds like a bug to me.

2014-05-05 12:32:43,720 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./song-gaming-portal: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./song-gaming-portal: Failed to start service

  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.0.Final.jar:1.2.0.Final]

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]

  at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]

Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: JBAS016076: Error injecting resource into CDI managed bean. Can't find a resource named java:jboss/infinispan/container/GamingPortal

  at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:216)

  at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:86)

  at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.start(UndertowDeploymentService.java:71)

  at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.0.Final.jar:1.2.0.Final]

  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.0.Final.jar:1.2.0.Final]

  ... 3 more

Caused by: java.lang.IllegalArgumentException: JBAS016076: Error injecting resource into CDI managed bean. Can't find a resource named java:jboss/infinispan/container/GamingPortal

  at org.jboss.as.weld.services.bootstrap.WeldResourceInjectionServices.resolveResource(WeldResourceInjectionServices.java:187)

  at org.jboss.as.weld.services.bootstrap.WeldResourceInjectionServices$1.createResource(WeldResourceInjectionServices.java:135)

  at org.jboss.weld.injection.AbstractResourceInjection.getResourceReference(AbstractResourceInjection.java:44)

  at org.jboss.weld.injection.AbstractResourceInjection.injectResourceReference(AbstractResourceInjection.java:53)

  at org.jboss.weld.util.Beans.injectEEFields(Beans.java:331)

  at org.jboss.weld.injection.producer.ResourceInjector.inject(ResourceInjector.java:59)

  at org.jboss.weld.injection.producer.DefaultInjector$1.proceed(DefaultInjector.java:66)

  at org.jboss.weld.injection.InjectionContextImpl.run(InjectionContextImpl.java:48)

  at org.jboss.weld.injection.producer.DefaultInjector.inject(DefaultInjector.java:64)

  at org.jboss.weld.injection.producer.BasicInjectionTarget.inject(BasicInjectionTarget.java:90)

  at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:150)

  at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)

  at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:98)

  at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:78)

  at com.sgi.song.gp.service.SONGv1.event.EventServiceCacheProducer$Proxy$_$$_WeldClientProxy.toString(Unknown Source)

  at com.sgi.song.gp.cdi.EagerInitExtension.containerInit(EagerInitExtension.java:54)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45]

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45]

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]

  at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]

  at org.jboss.weld.injection.MethodInjectionPoint.invokeOnInstanceWithSpecialValue(MethodInjectionPoint.java:93)

  at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:266)

  at org.jboss.weld.event.ExtensionObserverMethodImpl.sendEvent(ExtensionObserverMethodImpl.java:125)

  at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:253)

  at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:232)

  at org.jboss.weld.event.FastEvent.fire(FastEvent.java:125)

  at org.jboss.weld.servlet.HttpContextLifecycle.contextInitialized(HttpContextLifecycle.java:113)

  at org.jboss.weld.servlet.WeldInitialListener.contextInitialized(WeldInitialListener.java:95)

  at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:173)

  at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:187)

  ... 7 more

Caused by: javax.naming.NameNotFoundException: Error looking up infinispan/container/GamingPortal, service service jboss.naming.context.java.jboss.infinispan.container.GamingPortal is not started

  at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:133)

  at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

  at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

  at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

  at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

  at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

  at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

  at javax.naming.InitialContext.lookup(InitialContext.java:411) [rt.jar:1.7.0_45]

  at javax.naming.InitialContext.lookup(InitialContext.java:411) [rt.jar:1.7.0_45]

  at org.jboss.as.weld.services.bootstrap.WeldResourceInjectionServices.resolveResource(WeldResourceInjectionServices.java:185)

  ... 36 more

[pferraro]

Can you paste your <cache-container name="GamingPortal"/> configuration?

[rcd]

<cache-container name="GamingPortal" start="EAGER" module="org.infinispan.query">

                <transport lock-timeout="60000"/>

                <local-cache name="ES-subscriptions">

                    <transaction mode="NON_XA"/>

                </local-cache>

                <invalidation-cache name="SONGv1-localRouting" mode="ASYNC">

                    <transaction mode="NON_XA"/>

                </invalidation-cache>

                <replicated-cache name="SONGv1-epaInfo" mode="ASYNC">

                    <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                    <indexing index="ALL">

                        <property name="hibernate.search.default.directory_provider">

                            ram

                        </property>

                        <property name="hibernate.search.model_mapping">

                            com.sgi.song.gp.protocol.SONGv1.cluster.SearchMappingFactory

                        </property>

                    </indexing>

                </replicated-cache>

                <replicated-cache name="SONGv1-aliasToEPA" mode="ASYNC">

                    <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                </replicated-cache>

            </cache-container>

[pferraro]

OK - your configuration looks fine.

Is the object containing the @Resource injection a POJO?  If so, make sure you annotate it with @ManagedBean - otherwise the requisite dependency is not established.

[rcd]

Thanks, that was the piece I was missing! Having said that, the classes in question are @ApplicationScoped CDI beans, and I would have expected that to work. Is that a bug, or is there a reason that @ManagedBean is required even though it's already a CDI bean?