6 Replies Latest reply on Dec 5, 2017 4:30 AM by Gustavo Fernandes

    Remote script execution fails with marshaling error on 9.2.1 but succeeds on 8.2.8.Final

    Vikrant Singh Newbie

      Hi,

       

      I am upgrading keycloak from 3.2.1 to 3.4.0 with external Infinispan 9.2.1. Keycloak communicates to external Infinispan on hotrod protocol(using remote store).

       

      As part of initial cache loading from external Infinispan, keycloak executes below sccript.

       

      function loadSessions() {  var flagClazz = cache.getClass().getClassLoader().loadClass("org.infinispan.context.Flag"); 
        var localFlag = java.lang.Enum.valueOf(flagClazz, "CACHE_MODE_LOCAL"); 
        var cacheStream = cache.getAdvancedCache().withFlags([ localFlag ]).entrySet().stream();
        var result = cacheStream.skip(first).limit(max).collect(java.util.stream.Collectors.toMap(
          new java.util.function.Function() {
            apply: function(entry) {
              return entry.getKey();
            }
          },
          new java.util.function.Function() {
            apply: function(entry) {
              return entry.getValue();
            }
          }
        ));
      
      
        cacheStream.close();
        return result;
      };
      
      loadSessions();
      
      
      loadSessions();
      
      
      
      
      loadSelolossions();
      looo
      

       

      Above script fails on external Infinispan 9.1.2.Final server with below exception but same script executes successfully on Infinispan 8.2.8.Final. Can someone please help on this. Any suggestions are highly appreciated.

       

      2017-11-28 16:51:37,357 DEBUG [org.infinispan.server.hotrod.HotRodExceptionHandler] (HotRod-ServerWorker-9-3) Exception caught: org.infinispan.commons.CacheException: java.lang.ClassNotFoundException: org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper from [Module "org.infinispan.commons:main" from local module loader @1c2c22f3 (finder: local module finder @18e8568 (roo
      ts: /opt/jboss/infinispan-server/modules,/opt/jboss/infinispan-server/modules/system/layers/base))]
      at org.infinispan.commons.dataconversion.MarshallerEncoder.fromStorage(MarshallerEncoder.java:36)
      at org.infinispan.cache.impl.EncoderEntryMapper.decode(EncoderEntryMapper.java:43)
      at org.infinispan.cache.impl.EncoderEntryMapper.apply(EncoderEntryMapper.java:57)
      at org.infinispan.cache.impl.EncoderEntryMapper.apply(EncoderEntryMapper.java:23)
      at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
      at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
      at org.infinispan.commons.util.Closeables$SpliteratorAsCloseableSpliterator.tryAdvance(Closeables.java:143)
      at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
      at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
      at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
      at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
      at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
      at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
      at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
      at org.infinispan.stream.impl.local.LocalCacheStream.collect(LocalCacheStream.java:258)
      at org.infinispan.util.AbstractDelegatingCacheStream.collect(AbstractDelegatingCacheStream.java:273)
      at jdk.nashorn.internal.scripts.Script$Recompilation$1$58$\^eval\_.loadSessions(:5)
      at jdk.nashorn.internal.scripts.Script$\^eval\_.:program(:22)
      at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
      at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
      at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
      at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:421)
      at jdk.nashorn.api.scripting.NashornScriptEngine.access$300(NashornScriptEngine.java:73)
      at jdk.nashorn.api.scripting.NashornScriptEngine$3.eval(NashornScriptEngine.java:514)
      at javax.script.CompiledScript.eval(CompiledScript.java:92)
      at org.infinispan.scripting.impl.ScriptingManagerImpl.execute(ScriptingManagerImpl.java:239)
      at org.infinispan.scripting.impl.LocalRunner.runScript(LocalRunner.java:19)
      at org.infinispan.scripting.impl.ScriptingManagerImpl.runScript(ScriptingManagerImpl.java:222)
      at org.infinispan.scripting.impl.ScriptingTaskEngine.runTask(ScriptingTaskEngine.java:44)
      at org.infinispan.tasks.impl.TaskManagerImpl.runTask(TaskManagerImpl.java:99)
      at org.infinispan.server.hotrod.ContextHandler.realRead(ContextHandler.java:120)
      at org.infinispan.server.hotrod.ContextHandler.lambda$channelRead0$0(ContextHandler.java:52)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.ClassNotFoundException: org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper from [Module "org.infinispan.commons:main" from local module loader @1c2c22f3 (finder: local module finder @18e8568 (roots: /opt/jboss/infinispan-server/modules,/opt/jboss/infinispan-server/modules/system/layers/base))]
      at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:198)
      at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:363)
      at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:351)
      at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:93)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
      at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1087)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1354)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:275)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:208)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
      at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:134)
      at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromByteBuffer(AbstractJBossMarshaller.java:112)
      at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
      at org.infinispan.commons.dataconversion.MarshallerEncoder.fromStorage(MarshallerEncoder.java:34)
      ... 35 more

       

      Thanks,

      Vikrant

        • 2. Re: Remote script execution fails with marshaling error on 9.2.1 but succeeds on 8.2.8.Final
          Vikrant Singh Newbie

          Hi, Attached is the configuration I am using currently and this is the keycloak class from where script it is being called keycloak/RemoteCacheSessionsLoader.java at 3.4.0.Final · keycloak/keycloak · GitHub .

          Below is remote store configuration at keycloak side if that may have any affect.

           

                    <cache-container name="keycloak" module="org.keycloak.keycloak-model-infinispan" jndi-name="infinispan/Keycloak">
                          <transport lock-timeout="60000"/>
                          <local-cache name="realms">
                              <eviction strategy="LRU" max-entries="10000"/>
                          </local-cache>
                          <local-cache name="users">
                              <eviction strategy="LRU" max-entries="10000"/>
                          </local-cache>
                          <local-cache name="authorization">
                              <eviction strategy="LRU" max-entries="10000"/>
                          </local-cache>
                          <local-cache name="keys">
                              <eviction strategy="LRU" max-entries="1000"/>
                              <expiration max-idle="3600000"/>
                          </local-cache>
                          <replicated-cache name="work" mode="SYNC">
                              <remote-store cache="work" remote-servers="remote-cache" fetch-state="false" passivation="false" preload="false" purge="false" shared="true">
                                  <property name="rawValues">
                                      true
                                  </property>
                                  <property name="marshaller">
                                      org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory
                                  </property>
                              </remote-store>
                          </replicated-cache>
                          <distributed-cache name="sessions" mode="SYNC" owners="2">
                              <remote-store cache="sessions" remote-servers="remote-cache" fetch-state="false" passivation="false" preload="false" purge="false" shared="true">
                                  <property name="rawValues">
                                      true
                                  </property>
                                  <property name="marshaller">
                                      org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory
                                  </property>
                              </remote-store>
                          </distributed-cache>
                          <distributed-cache name="authenticationSessions" mode="SYNC" owners="2"/>
                          <distributed-cache name="offlineSessions" mode="SYNC" owners="2">
                              <remote-store cache="offlineSessions" remote-servers="remote-cache" fetch-state="false" passivation="false" preload="false" purge="false" shared="true">
                                  <property name="rawValues">
                                      true
                                  </property>
                                  <property name="marshaller">
                                      org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory
                                  </property>
                              </remote-store>
                          </distributed-cache>
                          <distributed-cache name="loginFailures" mode="SYNC" owners="2">
                              <remote-store cache="loginFailures" remote-servers="remote-cache" fetch-state="false" passivation="false" preload="false" purge="false" shared="true">
                                  <property name="rawValues">
                                      true
                                  </property>
                                  <property name="marshaller">
                                      org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory
                                  </property>
                              </remote-store>
                          </distributed-cache>
                          <distributed-cache name="actionTokens" mode="SYNC" owners="2">
                              <eviction strategy="NONE" max-entries="-1"/>
                              <expiration interval="300000" max-idle="-1"/>
                              <remote-store cache="actionTokens" remote-servers="remote-cache" fetch-state="false" passivation="false" preload="true" purge="false" shared="true">
                                  <property name="rawValues">
                                      true
                                  </property>
                                  <property name="marshaller">
                                      org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory
                                  </property>
                              </remote-store>
                          </distributed-cache>
                      </cache-container>
          

           

          Thanks,

          Vikrant

          • 3. Re: Remote script execution fails with marshaling error on 9.2.1 but succeeds on 8.2.8.Final
            Gustavo Fernandes Apprentice

            I see you expect to retrieve the raw byte[ ] from the cache and unmarshal it in the client, could you try using:

             

            var EncodingClazz = Java.type("org.infinispan.commons.dataconversion.IdentityEncoder").class

            var cacheStream = cache.getAdvancedCache().withEncoding(EncodingClazz).withFlags([ localFlag ])

            • 5. Re: Remote script execution fails with marshaling error on 9.2.1 but succeeds on 8.2.8.Final
              Vikrant Singh Newbie

              That worked. I am not familiar with remote scripting, can you please tell why original script is failing?

               

              Thanks for the help.

              • 6. Re: Remote script execution fails with marshaling error on 9.2.1 but succeeds on 8.2.8.Final
                Gustavo Fernandes Apprentice

                It's failing due to a change in default behavior that happened between Infinispan 8.2.x and 9.1.x.

                 

                In version 8.2.x, the cache given to a script ('cache' variable) was not decorated and thus the script would interact with data as it was stored: byte[]  representing marshalled org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper objects written via Hot Rod.

                 

                In 9.1.x, the cache present in the script context by default tries to unmarshall the entities in the cache so that you can deal with  org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper directly rather than byte[], that's why it failed, since objects of  org.keycloak.models.sessions.infinispan.changes.SessionEntityWrapper were not present in the Infinispan server.

                 

                In any case, using the IdentityEncoder guarantees that no unmarshalling will happen and you will always get byte[] when reading the cache. Feel free to open a JIRA if that behaviour does not suit your needs.