High CPU usage in Weblogic 8.1

[ottuzzi]

Hi there,

I'm evaluating Treecache as cluster cache in our environment: we are running a weblogic8.1SP2 cluster deployed on Solaris 5.9 (aka Solaris 9) with SUN JDK 1.4.1_02.

Here is the configuration I'm using:

<?xml version="1.0" encoding="UTF-8"?>
<server>

 <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>


 <mbean code="org.jboss.cache.TreeCache"
 name="jboss.cache:service=TreeCache">

 <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
 <attribute name="CacheMode">REPL_ASYNC</attribute>
 <attribute name="UseReplQueue">false</attribute>
 <attribute name="ReplQueueInterval">0</attribute>
 <attribute name="ReplQueueMaxElements">0</attribute>
 <attribute name="ClusterName">TreeCache-RA-Cluster</attribute>
 <attribute name="ClusterConfig">
 <config>
 <UDP mcast_addr="228.1.2.3" mcast_port="48866"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false"/>
 <PING timeout="2000" num_initial_members="3"
 up_thread="false" down_thread="false"/>
 <MERGE2 min_interval="10000" max_interval="20000"/>
 <FD_SOCK/>
 <VERIFY_SUSPECT timeout="1500"
 up_thread="false" down_thread="false"/>
 <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
 max_xmit_size="8192" up_thread="false" down_thread="false"/>
 <UNICAST timeout="600,1200,2400" down_thread="false"/>
 <pbcast.STABLE desired_avg_gossip="20000"
 up_thread="false" down_thread="false"/>
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
 shun="true" print_local_addr="true"/>
 <FC max_credits="2000000" down_thread="false" up_thread="false"
 min_threshold="0.20"/>
 <FRAG frag_size="8192" down_thread="false" up_thread="true"/>
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
 </config>
 </attribute>

 <attribute name="FetchInMemoryState">true</attribute>
 <attribute name="InitialStateRetrievalTimeout">20000</attribute>
 <attribute name="SyncReplTimeout">20000</attribute>
 <attribute name="LockAcquisitionTimeout">15000</attribute>
 <attribute name="EvictionPolicyClass">org.jboss.cache.eviction.LRUPolicy</attribute>
 <attribute name="EvictionPolicyConfig">
 <config>
 <attribute name="wakeUpIntervalSeconds">30</attribute><!-- 30 SECONDS -->
 <!-- Cache wide default -->
 <region name="/_default_">
 <attribute name="maxNodes">10000</attribute>
 <attribute name="timeToLiveSeconds">1800</attribute><!-- 30 MINUTES -->
 <attribute name="maxAgeSeconds">1800</attribute><!-- 30 MINUTES -->
 </region>
 <region name="/USER/">
 <attribute name="maxNodes">1000</attribute>
 <attribute name="timeToLiveSeconds">900</attribute><!-- 15 MINUTES -->
 <attribute name="maxAgeSeconds">900</attribute><!-- 15 MINUTES -->
 </region>
 </config>
 </attribute>
 <attribute name="UseRegionBasedMarshalling">true</attribute>

 </mbean>

 </server>

During tests I see a very HIGH CPU usage caused by TreeCache: without cache I see a CPU usage of about 10%, with TreeCache I can see even 20% CPU usage.
I made more tests to better understand the problem and I found that the simple cache initialization (I commented out all put/get/remove methods accessing Treecache and I simply initialized cache with the above XML) is enough to see the abnormal CPU usage: what is happening there?

Any help would be greatly appreciated.
Thanks
Bye
Piero

[genman]

What if you reconfigure this as a local cache? What is displayed if you turn on debug logging?

I wonder if JGroups is eating your CPU...

[ottuzzi]

Hi,

well it looks like our environment is running with JVM hotspot CLIENT instead of SERVER.
As soon as it will switch to SERVER hotspot I'll let you know how much CPU is using really.

As a side note I already setted logging at DEBUG level and JGroups is logging as expected... some messages when it sweeps cache... nothing more to seem suspicious at my eyes.

Thanks
Bye
Piero