13 Replies Latest reply on Nov 21, 2006 10:09 AM by brian.stansberry

    JbossCache with Oracle 10g cluster - exception=java.lang.Ill

    incivinci

      I am trying to configure JBossCache 1.2.3 with Oracle 10g Application Server in a clustered environment (2 instances managed by a loadbalancer)

      I was successful in configuring the above setup and wrote a simple web application to test the setup.
      The 2 instances are able to identify each other as is indicated by the following log output (which I did by writing a context listener which instantiates and configures a static instance of TreeCache)

      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache viewAccepted
      INFO: viewAccepted(): new members: [192.5.2.87:1323, 192.5.2.87:1326]

      The 2 instances are also able to replicate the instance of TreeCache on server startup as indicated by the following log output

      first instance

      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: received the state (size=192 bytes)
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: transient state: 140 bytes
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: setting transient state
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: locking the old tree
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: locking the old tree was successful
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: setting the transient state was successful
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor _setState
      INFO: forcing release of all locks in old tree
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache fetchStateOnStartup
      INFO: state was retrieved successfully (in 62 milliseconds)


      second instance

      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor getState
      INFO: locking the tree to obtain transient state
      Sep 17, 2005 12:22:20 PM org.jboss.cache.TreeCache$MessageListenerAdaptor getState
      INFO: returning the transient state (140 bytes)

      Now I try to attach an object to the TreeCache on the first instance but I get the following error message on the second instance

      Sep 17, 2005 12:22:52 PM org.jgroups.blocks.RpcDispatcher handle
      SEVERE: exception=java.lang.IllegalArgumentException: java.lang.ClassNotFoundException:
      com.incivinci.cachemanager.CacheManager
      Sep 17, 2005 12:22:52 PM org.jgroups.blocks.RpcDispatcher handle
      SEVERE: exception=java.lang.IllegalArgumentException: java.lang.ClassNotFoundException:
      com.incivinci.cachemanager.CacheManager

      when the second instance tries to synchronise the state of object.

      Even though the class is in the classpath.

      I checked it with simple objects like String and HashMap..no problem in caching and replicating updates with these...but any custom made class is leading to the same problem

        • 1. Re: JbossCache with Oracle 10g cluster - exception=java.lang
          belaban

          This is a classloader issue caused by your app running in its own scoped classloader. The incoming call doesn't know how to pick the correct classloader due to the fact that it doesn't know the webapp/classloader mapping.
          Scott, Ben and I are working on a solution. In the meantime, you can do the following:
          - Use MarshalledValue to wrap your user-defined classes as parameters
          - Put your user-defined classes in the lib directory

          • 2. Re: JbossCache with Oracle 10g cluster - exception=java.lang
            incivinci

            Thanks Bela.

            Will try out the suggested options and update you.

            • 3. Re: JbossCache with Oracle 10g cluster - exception=java.lang
              incivinci

              Bela, both the options give the desired results.

              Awaiting your update on a straight forward approach.

              Thanks.

              • 4. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                belaban

                Does this means it works, or not ?

                • 5. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                  incivinci

                  It works.

                  • 6. Re: JbossCache with Oracle 10g cluster - exception=java.lang

                    In 1.2.4beta, there is a class called TreeCacheMarshaller that can handle the scoped class loader for you. Idea is for user to register beforehand the application class loader as a region of (fqn, cl) pair.

                    Then, unmarshalling will be done accordingly.

                    -Ben

                    • 7. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                      incivinci

                      Will try it out.

                      Please can you elaborate more on the usage.

                      Thanks.

                      • 8. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                        incivinci

                        Hi Ben,

                        Reference - JBossCache 1.2.4 beta with Oracle 10g Application Server in a clustered environment (2 instances managed by a loadbalancer)

                        I have done the following -

                        I configure TreeCache as follows -

                        1. set "UseMarshalling" to true in the XML configuration file.

                        2. configure TreeCache as follows -

                        /*
                        * static block to instantiate the TreeCache with required properties
                        */
                        static {
                        if (treeCache == null){
                        try {

                        treeCache = new TreeCache();
                        PropertyConfigurator config = new PropertyConfigurator();
                        config.configure(treeCache, "objectcache.xml");
                        treeCache.createService();
                        treeCache.startService();
                        ClassLoader ccl = Thread.currentThread().getContextClassLoader();
                        treeCache.registerClassLoader("/incivinci", ccl);
                        } catch(Exception e){
                        e.printStackTrace();
                        }
                        }
                        }


                        Observation -

                        1. When both the instances behind the loadbalancer are up and running. The object is marshalled and unmarshalled across the instances without the ClassNotFoundException.

                        2. Now I down one of the instance to check the replication. This too works fine without the ClassNotFoundException.

                        3. Now I bring the down instance up and the when the TreeCache tries to sync with the other instance I get the ClassNotFoundException as follows -

                        05/09/26 14:24:14 Configuring the TreeCache - org.jboss.cache.TreeCache [0 nodes, 0 locks
                        ]
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.PropertyConfigurator
                        INFO: Found existing property editor for org.w3c.dom.Element: org.jboss.util.propertyedit
                        or.ElementEditor@124111a
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.PropertyConfigurator configure
                        INFO: configure(): attribute size: 16
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.TreeCache setClusterConfig
                        INFO: setting cluster properties from xml to: UDP(mcast_addr=228.1.2.3;mcast_port=45566;i
                        p_ttl=8;ip_mcast=true;mcast_send_buf_size=800000;mcast_recv_buf_size=150000;ucast_send_bu
                        f_size=800000;ucast_recv_buf_size=150000;loopback=true):PING(timeout=2000;num_initial_mem
                        bers=3;up_thread=true;down_thread=true):MERGE2(min_interval=10000;max_interval=20000):FD(
                        shun=true;up_thread=true;down_thread=true;timeout=2500;max_tries=5):VERIFY_SUSPECT(timeou
                        t=3000;num_msgs=3;up_thread=true;down_thread=true):pbcast.NAKACK(gc_lag=50;retransmit_tim
                        eout=300,600,1200,2400,4800;max_xmit_size=8192;up_thread=true;down_thread=true):UNICAST(t
                        imeout=300,600,1200,2400,4800;window_size=100;min_threshold=10;down_thread=true):pbcast.S
                        TABLE(desired_avg_gossip=20000;up_thread=true;down_thread=true):FRAG(frag_size=8192;down_
                        thread=true;up_thread=true):pbcast.GMS(join_timeout=5000;join_retry_timeout=2000;shun=tru
                        e;print_local_addr=true):pbcast.STATE_TRANSFER(up_thread=true;down_thread=true)
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.TreeCache _createService
                        WARNING: No transaction manager lookup class has been defined. Transactions cannot be used
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.TreeCache createPessimisticInterceptorChain
                        INFO: interceptor chain is:
                        class org.jboss.cache.interceptors.CallInterceptor
                        class org.jboss.cache.interceptors.PessimisticLockInterceptor
                        class org.jboss.cache.interceptors.UnlockInterceptor
                        class org.jboss.cache.interceptors.ReplicationInterceptor
                        Sep 26, 2005 2:24:14 PM org.jboss.cache.TreeCache _createService
                        INFO: cache mode is REPL_ASYNC
                        Sep 26, 2005 2:24:15 PM org.jboss.cache.TreeCache _createService
                        INFO: USE_MARSHALLING is true. We will marshall/unmarshall the value.
                        Sep 26, 2005 2:24:15 PM org.jgroups.protocols.UDP createSockets
                        INFO: sockets will use interface 192.5.2.87
                        Sep 26, 2005 2:24:15 PM org.jgroups.protocols.UDP createSockets
                        INFO: socket information:
                        local_addr=192.5.2.87:1438, mcast_addr=228.1.2.3:45566, bind_addr=/192.5.2.87, ttl=8
                        sock: bound to 192.5.2.87:1438, receive buffer size=150000, send buffer size=800000
                        mcast_recv_sock: bound to 192.5.2.87:45566, send buffer size=800000, receive buffer size=
                        150000
                        mcast_send_sock: bound to 192.5.2.87:1439, send buffer size=800000, receive buffer size=1
                        50000
                        05/09/26 14:24:15
                        -------------------------------------------------------
                        GMS: address is 192.5.2.87:1438
                        -------------------------------------------------------
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache startService
                        INFO: my local address is 192.5.2.87:1438
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache viewAccepted
                        INFO: viewAccepted(): [192.5.2.87:1426|3] [192.5.2.87:1426, 192.5.2.87:1438]
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache _setState
                        INFO: received the state (size=704 bytes)
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache _setState
                        INFO: transient state: 652 bytes
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache _setState
                        INFO: setting transient state for /
                        Sep 26, 2005 2:24:17 PM org.jboss.cache.TreeCache _setState
                        SEVERE: failed setting transient state
                        java.lang.ClassNotFoundException: com.incivinci.beantocache.UserInfoBean

                        at com.evermind.naming.ContextClassLoader.findClass(ContextClassLoader.java:314)
                        at java.lang.ClassLoader.loadClass(ClassLoader.java:299)
                        at com.evermind.naming.ContextClassLoader.loadClass(ContextClassLoader.java:139)
                        at java.lang.ClassLoader.loadClass(ClassLoader.java:255)
                        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315)
                        at java.lang.Class.forName0(Native Method)
                        at java.lang.Class.forName(Class.java:217)
                        at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:558)
                        at org.jboss.invocation.MarshalledValueInputStream.resolveClass(MarshalledValueIn
                        putStream.java:95)
                        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1513)
                        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1435)
                        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1626)
                        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1274)
                        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:324)
                        at java.util.HashMap.readObject(HashMap.java:986)
                        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j
                        ava:25)
                        at java.lang.reflect.Method.invoke(Method.java:324)
                        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:824)
                        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1746)
                        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1646)
                        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1274)
                        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:324)
                        at org.jboss.cache.TreeNode.readExternal(TreeNode.java:543)
                        at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1686)
                        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1644)
                        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1274)
                        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:324)
                        at EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap.readObject(Concurrent
                        ReaderHashMap.java:1391)
                        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j
                        ava:25)
                        at java.lang.reflect.Method.invoke(Method.java:324)
                        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:824)
                        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1746)


                        Have I missed out something.

                        • 9. Re: JbossCache with Oracle 10g cluster - exception=java.lang

                          This problem arises because the initial state transfer is set to true. And during the initial state transfer, there is no way for JBossCache to know which classloader to use.

                          Solution is to:

                          1. Turn off initial state transfer (fetchStateTransfer to false)

                          2. You will need to specifically get the partial state from the other node. We have a new API for this:
                          _getState(Fqn fqn);

                          We will document this in the final 1.2.4 release (coming up in 1.5 weeks). Meanwhile, please check out under JBossCache_1_2_4 branch now and you will see.

                          Please let me know if there is additional problem.

                          -Ben

                          • 10. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                            brian.stansberry

                            The API you want to use is activateRegion(String fqn), which 1) fetches state from another node on the cluster for the subtree rooted at fqn, and 2) enables marshalling for the subtree. _getState is the internal call one node makes to another when it needs a partial state transfer.

                            This call should come after your call to registerClassLoader.

                            Another call you may want to add is treeCache.setInactiveOnStartup(true), before your call to createService(). This will prevent the cache handling any replication messages for an fqn until an activateRegion() call has been made for the fqn or one of its parents. Without this, if another node tries to replicate some data in the /incivinci subtree before you can call registerClassLoader, you will get errors.

                            • 11. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                              incivinci

                              Thanks Brian and Ben. It works after those suggestions.

                              • 12. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                                yesucan3

                                Bela,
                                I am facing the same issue and want to reslove this by configuration.
                                I am trying this soln but its not working.
                                - Put your user-defined classes in the lib directory

                                which lib directory should i put my user defined classes?

                                • 13. Re: JbossCache with Oracle 10g cluster - exception=java.lang
                                  brian.stansberry

                                  The "lib" directory is more of a JBoss AS term. In Oracle AS you need to put the classes in a location where they will be loaded by the same classloader that loads the jboss-cache.jar and jgroups.jar classes, wherever that is. Or use the registerClassloader/activateRegion API discussed above.