1 2 Previous Next 16 Replies Latest reply on Feb 9, 2007 6:58 PM by Murthy

    when eviction is configured in the clustered environment, th

    Murthy Newbie

      when eviction is configured in the clustered environment, the tomcat server is not shutting down properly. And I have to kill the process manually. My configuration is as below

      <?xml version="1.0"?>


      READ_COMMITTED
      REPL_ASYNC
      JForumCacheStage



      <!-- UDP: if you have a multihomed machine,
      set the bind_addr attribute to the appropriate NIC IP address -->

      <!-- UDP: On Windows machines, because of the media sense feature
      being broken with multicast (even after disabling media sense)
      set the loopback attribute to true -->
      <UDP mcast_addr="228.1.2.3" mcast_port="48866"
      ip_ttl="64" ip_mcast="true" bind_addr="xxx.xx.xx.xx"
      mcast_send_buf_size="300000" mcast_recv_buf_size="640000"
      ucast_send_buf_size="300000" ucast_recv_buf_size="640000"
      loopback="false"/>

      <PING timeout="2000" num_initial_members="3" up_thread="false" down_thread="false"/>
      <MERGE2 min_interval="60000" max_interval="90000"/>
      <FD shun="true" up_thread="true" down_thread="true" timeout="2500" max_tries="5"/>
      <VERIFY_SUSPECT timeout="1500" up_thread="false" down_thread="false"/>

      <pbcast.NAKACK gc_lag="50" retransmit_timeout="100,200,300,600,1200,2400,4800"
      max_xmit_size="32768" up_thread="false" down_thread="false"/>

      <UNICAST timeout="300,600,1200,2400,3600" window_size="100" min_threshold="10" down_thread="false"/>
      <pbcast.STABLE desired_avg_gossip="40000" up_thread="false" down_thread="false" max_bytes="300000"/>

      <FRAG frag_size="16384" down_thread="false" up_thread="false"/>
      <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="true" print_local_addr="true"/>
      <pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>



      org.jboss.cache.eviction.LRUPolicy
      <!-- Specific eviction policy configurations. This is LRU -->


      <!-- Cache wide default -->
      3600

      <!-- Cache wide default -->

      2000
      3600


      200
      0
      0


      300
      0
      0


      200
      0
      0


      200
      0
      0


      <!--attribute name="maxNodes">300</attribute-->
      10
      3600
      0






        • 1. Re: when eviction is configured in the clustered environment
          Brian Stansberry Master

          This is a bug -- http://jira.jboss.com/jira/browse/JBCACHE-963 . Thanks for reporting it.

          If Tomcat undeploys webapps as part of normal shutdown (not sure if it does -- JBoss AS definitely does), the workaround is to ensure that your webapp calls cache.stopService() as part of undeploy. You should do that anyway.

          • 2. Re: when eviction is configured in the clustered environment
            Murthy Newbie

            Tkanks bstansberry. I will add cache.stopService while undeploying the webapp

            • 3. Re: when eviction is configured in the clustered environment
              Murthy Newbie

              After adding cache.stopService() the tomcat shuts down proeperly if the cache is not triggered(not added any data). If cache is triggered by adding nodes the tomcat doesn't shutdown properly.

              • 4. Re: when eviction is configured in the clustered environment
                Brian Stansberry Master

                Can you post your config again?

                Use the 'code' button to surround it with 'code' tags; this will allow it to display properly.

                I'm guessing there's another thread not getting cleaned up properly; your config will likely indicate where. Although it's odd that we don't see this with the AS.

                • 5. Re: when eviction is configured in the clustered environment
                  Murthy Newbie

                  Here is the configuration

                  <?xml version="1.0"?>
                  <server>
                   <mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TreeCache">
                   <attribute name="IsolationLevel">READ_COMMITTED</attribute>
                   <attribute name="CacheMode">REPL_ASYNC</attribute>
                   <attribute name="ClusterName">CacheStage</attribute>
                  
                   <attribute name="ClusterConfig">
                   <config>
                   <!-- UDP: if you have a multihomed machine,
                   set the bind_addr attribute to the appropriate NIC IP address -->
                  
                   <!-- UDP: On Windows machines, because of the media sense feature
                   being broken with multicast (even after disabling media sense)
                   set the loopback attribute to true -->
                   <UDP mcast_addr="228.1.2.3" mcast_port="48866"
                   ip_ttl="64" ip_mcast="true" bind_addr="xxx.xx.xx.xx"
                   mcast_send_buf_size="300000" mcast_recv_buf_size="640000"
                   ucast_send_buf_size="300000" ucast_recv_buf_size="640000"
                   loopback="false"/>
                  
                   <PING timeout="2000" num_initial_members="3" up_thread="false" down_thread="false"/>
                   <MERGE2 min_interval="60000" max_interval="90000"/>
                   <FD shun="true" up_thread="true" down_thread="true" timeout="2500" max_tries="5"/>
                   <VERIFY_SUSPECT timeout="1500" up_thread="false" down_thread="false"/>
                  
                   <pbcast.NAKACK gc_lag="50" retransmit_timeout="100,200,300,600,1200,2400,4800"
                   max_xmit_size="32768" up_thread="false" down_thread="false"/>
                  
                   <UNICAST timeout="300,600,1200,2400,3600" window_size="100" min_threshold="10" down_thread="false"/>
                   <pbcast.STABLE desired_avg_gossip="40000" up_thread="false" down_thread="false" max_bytes="300000"/>
                  
                   <FRAG frag_size="16384" down_thread="false" up_thread="false"/>
                   <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="true" print_local_addr="true"/>
                   <pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
                   </config>
                   </attribute>
                  
                   <attribute name="EvictionPolicyClass">org.jboss.cache.eviction.LRUPolicy</attribute>
                   <!-- Specific eviction policy configurations. This is LRU -->
                   <attribute name="EvictionPolicyConfig">
                   <config>
                   <!-- Cache wide default -->
                   <attribute name="wakeUpIntervalSeconds">3600</attribute>
                  
                   <!-- Cache wide default -->
                   <region name="/_default_">
                   <attribute name="maxNodes">2000</attribute>
                   <attribute name="timeToLiveSeconds">3600</attribute>
                   </region>
                   <region name="/modules/">
                   <attribute name="maxNodes">200</attribute>
                   <attribute name="timeToLiveSeconds">0</attribute>
                   <attribute name="maxAgeSeconds">0</attribute>
                   </region>
                   <region name="/temps/">
                   <attribute name="maxNodes">300</attribute>
                   <attribute name="timeToLiveSeconds">0</attribute>
                   <attribute name="maxAgeSeconds">0</attribute>
                   </region>
                   <region name="/images/">
                   <attribute name="maxNodes">200</attribute>
                   <attribute name="timeToLiveSeconds">0</attribute>
                   <attribute name="maxAgeSeconds">0</attribute>
                   </region>
                   <region name="/codes/">
                   <attribute name="maxNodes">200</attribute>
                   <attribute name="timeToLiveSeconds">0</attribute>
                   <attribute name="maxAgeSeconds">0</attribute>
                   </region>
                   <region name="/cats">
                   <!--attribute name="maxNodes">300</attribute-->
                   <attribute name="maxNodes">10</attribute>
                   <attribute name="timeToLiveSeconds">3600</attribute>
                   <attribute name="maxAgeSeconds">0</attribute>
                   </region>
                   </config>
                   </attribute>
                  
                   </mbean>
                  </server>
                  



                  From the log:
                  INFO: Illegal access: this web application instance has been stopped already. Could not load java.io.PrintWriter. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
                  java.lang.IllegalStateException
                  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1238)
                  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1198)
                  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
                  at org.jgroups.util.Util.printStackTrace(Util.java:162)
                  at org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1305)
                  at java.lang.Thread.run(Thread.java:595)
                  ERROR: [153.18.96.62:35325] exception=java.lang.NullPointerException, stack trace=java.lang.NullPointerException
                  at org.jgroups.Version.compareTo(Version.java:54)
                  at org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1280)
                  at java.lang.Thread.run(Thread.java:595)
                  (2007-01-31 16:01:01,959 UDP.UcastReceiverThread_org.jgroups.protocols.UDP)
                  Exception in thread "TimeScheduler.Thread" java.lang.NullPointerException
                  at org.jgroups.util.TimeScheduler$Loop.run(TimeScheduler.java:138)
                  at java.lang.Thread.run(Thread.java:595)
                  ERROR: exception=java.lang.NullPointerException, stack trace=java.lang.NullPointerException
                  at org.jgroups.Version.compareTo(Version.java:54)
                  at org.jgroups.protocols.UDP.run(UDP.java:227)
                  at java.lang.Thread.run(Thread.java:595)
                  (2007-01-31 16:01:02,882 UDP mcast receiver_org.jgroups.protocols.UDP)
                  ERROR: [153.18.96.62:35325] exception=java.lang.NullPointerException, stack trace=java.lang.NullPointerException
                  at org.jgroups.Version.compareTo(Version.java:54)
                  at org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1280)
                  at java.lang.Thread.run(Thread.java:595)
                  (2007-01-31 16:01:04,460 UDP.UcastReceiverThread_org.jgroups.protocols.UDP)
                  ERROR: [153.18.96.62:35325] exception=java.lang.NullPointerException, stack trace=java.lang.NullPointerException
                  at org.jgroups.Version.compareTo(Version.java:54)
                  at org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1280)
                  at java.lang.Thread.run(Thread.java:595)
                  (2007-01-31 16:01:06,962 UDP.UcastReceiverThread_org.jgroups.protocols.UDP)
                  ERROR: exception=java.lang.NullPointerException, stack trace=java.lang.NullPointerException
                  at org.jgroups.Version.compareTo(Version.java:54)
                  at org.jgroups.protocols.UDP.run(UDP.java:227)
                  at java.lang.Thread.run(Thread.java:595)
                  (2007-01-31 16:01:07,297 UDP mcast receiver_org.jgroups.protocols.UDP)

                  • 6. Re: when eviction is configured in the clustered environment
                    Brian Stansberry Master

                    How are you stopping the cache; i.e. what code invokes stopService() and where is it invoked?

                    Your log shows continuing operations of the JGroups channel, which should have stopped when stopService() returns.

                    • 7. Re: when eviction is configured in the clustered environment
                      Murthy Newbie

                      I used cache.stopService() in servlet destory and I can see in the log that code is executed

                      • 8. Re: when eviction is configured in the clustered environment
                        Brian Stansberry Master

                        What JBoss Cache release? If you're not using the jgroups.jar that comes with that release, please advise the JGroups version as well.

                        • 9. Re: when eviction is configured in the clustered environment
                          Murthy Newbie

                          we are using jboss-cache-1.3.0.SP2.jar and jgroups-all-2.2.7.jar

                          • 10. Re: when eviction is configured in the clustered environment
                            Brian Stansberry Master

                            It's not clear to me what's going on in the messages you log. Apparently JGroups is reading a packet(s) with null data; that's what's causing the NPEs mentioned in the error messages. Perhaps this is preventing the threads associated with the UDP protocol stopping properly. But what the root issue is I don't know.

                            Can you try a more recent JGroups release? Basically any subsequent release is compatible with JBC 1.3.0.SP2, so I'd say try 2.4.1. (See http://labs.jboss.com/portal/jbosscache/compatibility/index.html for compatibility listings.)

                            • 12. Re: when eviction is configured in the clustered environment
                              Murthy Newbie

                              I tried with 2.4.1, 2.2.8 and 2.2.9.1 But I was not succesfull. For time being I commented the eviction attribute.

                              • 13. Re: when eviction is configured in the clustered environment
                                Brian Stansberry Master

                                Commenting the eviction attribute fixed it?

                                • 14. Re: when eviction is configured in the clustered environment
                                  Murthy Newbie

                                  Here is the summary from my findings.
                                  If eviction attribute is commented the tomcat is shutting down properly. But when eviction is uncommented the tomcat is not shutting down properly no matter cache is loaded or not loaded. After adding cache.stopservice code the server was shutdown properly with eviction attribute uncommented but the cache is not loaded with data. When eviction attribute is uncommented and cache is loaded the server is not shutting down properly.

                                  1 2 Previous Next