4 Replies Latest reply on May 18, 2016 12:46 PM by xxd

    Wildfly 10 - Security Domain Not Using Infinispan Configuration

    justinlinz

      I am running Wildfly 10.0.0.Final and have a security domain with DatabaseServerLoginModule setup to use infinispan for caching.  I configure the cache to have expiration lifespan and max-idle values, however it seems they are not being set.  The relevant configuration I have in standalone.xml is:

       

      <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
         <cache-container name="security" default-cache="auth-cache">
            <local-cache name="auth-cache">
               <expiration lifespan="10" max-idle="10"/>
            </local-cache>
         </cache-container>
      </subsystem>
      
      <security-domain name="mySecurityDomain" cache-type="infinispan">
         <authentication>
            <login-module code="org.jboss.security.auth.spi.DatabaseServerLoginModule" flag="required">
                ...
                ...
            </login-module>
         </authentication>
      </security-domain>
      
      

       

      It does create the cache and uses it as I would expect, except is not expiring entries.  Looking at the configuration for the cache object created in jconsole, I also notice it appears to have default values for the configurationAsProperties attribute:

       

      expiration.attributes=[wakeUpInterval=60000, maxIdle=-1, reaperEnabled=true, lifespan=-1]

       

      Also debugging during startup, I stepped through org.jboss.as.security.plugins.InfinispanAuthenticationCacheFactory.getCache, and its getting a null value returned from the cachemanager call here:

       

      Configuration baseCfg = cacheManager.getCacheConfiguration("auth-cache");
      
      

       

      Am I missing something in my configuration?  From what I could tell, this should work.

       

      Thanks,

      Justin

        • 1. Re: Wildfly 10 - Security Domain Not Using Infinispan Configuration
          barmazu

          Hi Justin,

           

          I can confirm the same happening on my Wildfly 10 instance. I tried to see if any different on Wildfly 9 but it is exactly the same case:

          expiration.lifespan is set to default value (-1 or infinity) regardless of the setting in the config.

          • 2. Re: Wildfly 10 - Security Domain Not Using Infinispan Configuration
            andrei.serea

            Hi!

             

            I am facing the same problem and after digging deep and hooking some byteman rules, I narrowed it down to a race condition between the org.wildfly.clustering.infinispan.spi module, responsible for loading the configurations and the security domain service, responsible for loading the ... security domains.

            Here are the logs that show exactly how, sometimes, the call from the security domain service startup thread that wants the cache is executed BEFORE the call from org.wildfly.clustering.infinispan.spi that loads the configuratiokn:

             

            2016-04-04 18:41:22,802 INFO  [stdout] (MSC service thread 1-4) () getCacheConfiguration returned for auth-cache null

            2016-04-04 18:41:22,803 INFO  [stdout] (ServerService Thread Pool -- 58) () trace DefaultCacheManager defineConfiguration execute

            2016-04-04 18:41:22,804 INFO  [stdout] (MSC service thread 1-4) () Stack trace for thread MSC service thread 1-4

            2016-04-04 18:41:22,805 INFO  [stdout] (MSC service thread 1-4) () org.infinispan.manager.DefaultCacheManager.getCacheConfiguration(DefaultCacheManager.java:734)

            2016-04-04 18:41:22,805 INFO  [stdout] (MSC service thread 1-4) () org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCacheConfiguration(DefaultCacheContainer.java:91)

            2016-04-04 18:41:22,806 INFO  [stdout] (MSC service thread 1-4) () org.jboss.as.security.plugins.InfinispanAuthenticationCacheFactory.getCache(InfinispanAuthenticationCacheFactory.java:61)

            2016-04-04 18:41:22,806 INFO  [stdout] (MSC service thread 1-4) () org.jboss.as.security.plugins.JNDIBasedSecurityManagement.createSecurityDomainContext(JNDIBasedSecurityManagement.java:279)

            2016-04-04 18:41:22,806 INFO  [stdout] (MSC service thread 1-4) () org.jboss.as.security.service.SecurityDomainService.start(SecurityDomainService.java:98)

            2016-04-04 18:41:22,806 INFO  [stdout] (MSC service thread 1-4) () org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)

            2016-04-04 18:41:22,807 INFO  [stdout] (MSC service thread 1-4) () org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)

            2016-04-04 18:41:22,807 INFO  [stdout] (MSC service thread 1-4) () java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

            2016-04-04 18:41:22,807 INFO  [stdout] (MSC service thread 1-4) () java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

            2016-04-04 18:41:22,807 INFO  [stdout] (MSC service thread 1-4) () java.lang.Thread.run(Thread.java:745)

            2016-04-04 18:41:22,808 INFO  [stdout] (ServerService Thread Pool -- 58) () defineConfiguration called with param cacheName = auth-cache, eviction: lifespan=10000

            2016-04-04 18:41:22,809 INFO  [stdout] (ServerService Thread Pool -- 58) () Stack trace for thread ServerService Thread Pool -- 58

            2016-04-04 18:41:22,809 INFO  [stdout] (ServerService Thread Pool -- 58) () org.infinispan.manager.DefaultCacheManager.defineConfiguration(DefaultCacheManager.java:-1)

            2016-04-04 18:41:22,809 INFO  [stdout] (ServerService Thread Pool -- 58) () org.infinispan.manager.DefaultCacheManager.defineConfiguration(DefaultCacheManager.java:344)

            2016-04-04 18:41:22,810 INFO  [stdout] (ServerService Thread Pool -- 58) () org.jboss.as.clustering.infinispan.DefaultCacheContainer.defineConfiguration(DefaultCacheContainer.java:76)

            2016-04-04 18:41:22,810 INFO  [stdout] (ServerService Thread Pool -- 58) () org.wildfly.clustering.infinispan.spi.service.ConfigurationBuilder.start(ConfigurationBuilder.java:75)

            2016-04-04 18:41:22,810 INFO  [stdout] (ServerService Thread Pool -- 58) () org.wildfly.clustering.service.AsynchronousServiceBuilder$1.run(AsynchronousServiceBuilder.java:102)

            2016-04-04 18:41:22,811 INFO  [stdout] (ServerService Thread Pool -- 58) () java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

            2016-04-04 18:41:22,817 INFO  [stdout] (ServerService Thread Pool -- 58) () java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

            2016-04-04 18:41:22,818 INFO  [stdout] (ServerService Thread Pool -- 58) () java.lang.Thread.run(Thread.java:745)

            2016-04-04 18:41:22,818 INFO  [stdout] (ServerService Thread Pool -- 58) () org.jboss.threads.JBossThread.run(JBossThread.java:320)

             

            This bug has been filed on Wildfly 8 a long time ago...

            https://issues.jboss.org/browse/WFLY-3858

             

            If, maybe, we all vote for it, it will get fixed.

             

            Also, there is a very easy way to verify it is a race problem: just put a breakpoint at the beginning of InfinispanAuthenticationCacheFactory.getCache and you will see that, because of the delay introduced by this breakpoint in the thread, the cache config will get loaded

            • 3. Re: Wildfly 10 - Security Domain Not Using Infinispan Configuration
              justinlinz

              Short of an actual fix, but I did implement a workaround that gets the job done.  The code below is for an EJB that will execute on startup and apply configuration changes to the security cache.  Pretty basic, but it gets the expiration settings, which is what I was really concerned about.  You can change other configurations in a similar manner as needed.

               

               

              import javax.annotation.PostConstruct;
              import javax.annotation.Resource;
              import javax.ejb.Singleton;
              import javax.ejb.Startup;
              import org.infinispan.manager.EmbeddedCacheManager;
              
              
              @Startup
              @Singleton
              public class SecurityCacheConfigurationBandage {
              
                public static final long LIFESPAN = 86400000;
                public static final long MAX_IDLE = 86400000;
                public static final String SECURITY_DOMAIN_NAME = "mySecurityDomain";
              
              
                @Resource(lookup = "java:jboss/infinispan/container/security")
                private EmbeddedCacheManager cacheContainer;
              
              
                @PostConstruct
                public void configureSecurityCache() {
                cacheContainer.getCache(SECURITY_DOMAIN_NAME).stop();
              
                cacheContainer.getCache(SECURITY_DOMAIN_NAME).getCacheConfiguration().expiration().attributes().attribute("lifespan").set(LIFESPAN);
                cacheContainer.getCache(SECURITY_DOMAIN_NAME).getCacheConfiguration().expiration().attributes().attribute("maxIdle").set(MAX_IDLE);
              
                cacheContainer.getCache(SECURITY_DOMAIN_NAME).start();
                }
              }
              
              
              1 of 1 people found this helpful
              • 4. Re: Wildfly 10 - Security Domain Not Using Infinispan Configuration
                xxd

                Hello, Justin,

                 

                Thanks for the workaround. But it seems although the configuration was set, the cache entry still fails to get expired on my machine... Have you verified the expiration?

                 

                The workaround I found (and I am using Wildfly-10), was to use this method when interacting with the cache: 'V put(K key, V value, long lifespan, TimeUnit lifespanUnit, long maxIdleTime, TimeUnit maxIdleTimeUnit);'. And I verified that this works.

                 

                What's missing is that, we cannot change the eviction strategy from NONE to anything else, as that setting is immutable... And unfortunately, based on my verification, the configuration change with "cacheConfiguration.eviction().size(XXX);" is not respected either... So we cannot set a cache size limit...

                 

                Hope this helps...