Skip navigation

Ben Wang's Blog

8 posts


When we planned the JBoss Cache 1.4 release weeks back, in addition to feature implementation, we have set out the goal to improve the overall cache performance. For PojoCache, that means performing the necessary benchmark first and making corresponding optimization. Now that 1.4beta is out, I am glad to report that we have achieved that objective!


In 1.4, first of all, the underlying replication layer has used more efficient marshalling to minimize the replication overhead. Second of all, much efforts have also been spent on refactoring the POJO attachment module such that it is more efficient in CPU usage and also reduction in replication message size.


To illustrate the performance of PojoCache, I have posted a Wiki article online to detail the comparison between TreeCache (a plain cache system) and PojoCache. Interested readers should refer it for more information.


Overall, it is expected that there would be a tradeoff between a plain cache system and a POJO cache one (that uses fine-grained replication) in terms of overall performance. The tradeff depends on the POJO object size, POJO lifetime, and also the update frequency and pattern. For example, if I have a small POJO with updates to most of the fields during a single request, we'd expect a plain cache system will be adequate. However, for other cases with medium to large size POJO and selected field updates, a POJO cache will not only have advantages in performance, it will also offer transparent, automatic replication! In one case that we studied in the article, e.g., PojoCache is 3.5 times faster than TreeCache. And if the POJO lifetime is longer, then we expect the speedup would be even greater!


Finally if you care not just performance but also want to maintain the object relationship during replication, PojoCache is then the cache that you should use!


Oh, in the upcoming JBossWorld in Vegas, I will have a session presenting more about PojoCache. Come join us there for the scoop and the fun!

-Ben Wang, Lead, PojoCache


Brian and I have done a web clustering webinar on this Wednesday (3/1). Besides the international phone connection problems that I've had (I made it only at the last minute), it went pretty well, so I thought. We have covered many details, including configuration options, performance tuning, etc. You should check it out if you are interested.


One topic that we also covered in there was benchmark numbers. JBoss has worked with the Japan OSS community in a later stage to produce these results. We have used JBento (also an open-soruce project), a J2EE micro benchmark framwork, to perform, measure, and collect the http session replication tests and data.


In there, we produced the latest numbers using JBoss AS 4.0.3SP1, while updating JBossCache to 1.2.4SP1 and JGroups to 2.2.9. (Note that couple of bug fixes have also been checked into the upcoming 4.0.4 release to improve reliability.) It is shown that with basic configuration, the latest JBoss web clustering can produce decent results as well!


This work is in no way complete since effort for a complete benchmark and full tuning is very extensive. But we are very appreciative of the Japan OSS community already! Currently work is under way to evaluate with different configuration options, payload and cluster sizes. There are still lots of work remained for further optimization, but stay tuned...

Ben Wang

Visiting Japan

Posted by Ben Wang Nov 23, 2005



I have been visiting Japan for almost two weeks now, first doing a public training, and secondly working with Japan OSS community here on JBoss web clustering benchmark. I will have more to say on the clustering benchmark effort in a later blog.


But last night, I have had a dinner with a group of people both from Japan JBoss mailing list folks (started by Fusayuki "Miki" Minamoto) and our partners in Japan. We had a total of 37 people crowded into a Japanese resturant to enjoy nabe (like seafood hotpot) and drinks. Food and atmosphere were great. But I was really touched to see the interest level there (not just for JBoss but also open source as a whole!)


Also as a result of last night's gathering, they have decided to start a JBoss User Group (JBUG) in Tokyo. That should provide an ideal platform for JBoss users to share and exchange information in Japan.


Lastly, Japan is probably not known to have big presence in OSS development, IMO. Projects that I am personally aware of are like Ruby and Javassist (and I am sure there are others that I have left out). But I was told in this trip that Seasar, a domestic DI/AOP container is very popular in Japan recently. I have to admit that this is my first time hearing about it.





We have just released JBoss Cache 1.2.4. In addition to the previous beta version, this final release contains couple additional bug fixes. Most importantly, Brian Stansberry has put in a lot of effort to complete the marshalling layer, including the initial state transfer (under marshalling.) Now a user can register its own class loader under a cache region.


The release announcement can be seen here. And I have also blogged about the beta release features previously. Check it out!


JBoss Cache 1.2.4 beta has been released. As usual, you can use it in two modes: plain cache (through TreeCache component) or so-called fine-grained POJO cache (through TreeCacheAop component). Both components are in-memory, replicated, transactional, and persistent as well. You can download it here, or check out the online doc. I thought I go over the key features here.


1.2.4 started out as mostly a POJO cache release since we have major enhancement for the TreeCacheAop component. First of all, I have totally revamped the "distributed" object-graph model. By "distributed", I meant automatic preservation of object graph relationship during either replication or persistency (through the cache loader).


Previous model has problems to handle object-graph relationship when there is complicated interaction between POJOs and Collection classes, especially when it is recursive (or cyclic). In the current model, pure object reference counting is used such that complex interactions can be handled directly without problem. The design discussion was carried out here


To illustrate that, there are two "real-life" JUnit examples under tests/functional/org/jboss/cache/aop/intergrated directory. I am also preparing an article to further describe the full functionality. So stay tuned!


In addition, I have re-written the TreeCacheAop documentation to cover the various topics in more details.


On the plain cache side, we have added overflowing (passivation/activation) feature. For example, with cache loader, eviction will persist the evicted data into a persistent store (can be Berkeley- or JDBC-DB). So next time the data is retrieved, it will be from the persistent store. This feature is acting simliar to ejb passivation/activation. Note that this feature will be basis for future http session clustering passivation, for example.


Finally, we have added a marshalling mechanism for JBoss Cache to handle replication under a scoped class loader. Class loading can be an irritating problem when an application is run under AS. This is especially true when JBoss Cache is deployed since the JGroups replication layer has no notion of an application class loader! What we have done is to provide a registry for applications to register their associated class loader to JBoss Cache so the data can be unmarshalled correctly.


For the full details, you can visit this Forum post.

I have blogged previously on the new JBoss Clustering roadmap and how it has been re-architected. To give more details, JBoss Clustering mainly consists of 3 main parts:

    EJB client-side cluster interceptor that has intelligence of cluster toplogy and load balance policy. One of the strength of JBoss cluster interceptor is its transparency to the client. That is, configuration is done on the server side and downloaded to the client at runtime. As a result, client is not even aware of it is using clustered service!


    For the existing EJB2.1, we have HA-RMI that runs top of the RMI layer. But for EJB3.0, Bill Burke has implemented HA-Remoting that runs on top of JBoss Remoting. It further provides flexibility on the selection of transport layer mechanism.

  • State replication. Previoulsy we have different state replication mechansim to deal with http session and sfsb, for example. Now, we will use JBossCache for everything that is involved with state replication. Modules that currently have been/are transitioning to using JBossCache are:
    • Http session (including SSO). This has been introduced since AS 3.2.6 that is bundled with the Tomcat module.
    • EJB3.0 SFSB. This has been introduced in EJB3 Beta release.
    • EJB3.0 Entity. This has been introduced in EJB3 Beta release.
    • DistributedState. This is an internal component that is currently implemented by Scott Marlow (from Novell) in AS 5.0
    • HA-JNDI. This is the JBoss clustered jndi server (that can be released separately in the future.) It is currently implemented by Jerry Gauthier (from Novell) in AS 5.0
  • Distributed RPC. There are times when a moudle needs to execute distributed RPC call. For example, in JBoss HA-JNDI, sometimes we will need to ask the other local jndi servers to see if they hold a specific EJBHome stub. Currently, HAPartition is the distributed RPC layer.


To sum up all these relationship, below is a figure illustrating the new layered component architecture of JBoss Clustering. You can see that the clustering stack from the bottom up that starts with JGroups, then with JBossCache and HAServices, and so on.



So what are the advantages of this layered approach, you ask? Well, for one, since stack modules can be independently released, it is much easier to maintain. For example, a bug fix in JBossCache only involves with a patch of jboss-cache.jar.


In the same token, when JBossCache implements buddy replication feature, that means JBoss Clustering will be able to support sub-paritioning of cluster group and thus promote large number scalability. In another example, Hany Mesha (also from Novell) has recently implemented the cache overflowing feature in the upcoming JBossCache 1.2.4 release. It will be used for http session and ejb clustering passivation/activation features next.


Finally, the figure below shows the mutual release dependency of JBoss Clustering components and their main features.



Here are highlights of some of the key features in different components:

  • JGroups 2.3
    • Channel multiplexing. This enables different users of JGroups to share a single JChannel. As a result, it can simplify and consolidate all the JGroups configuration used by various components.
    • Partial state transfer. Needed to guarantee state integrity.
  • JBossCache 1.2.5 & 1.3
    • Optimistic locking. Potentially can increase concurrency during short transaction span.
    • Buddy replication. Only replicated on a configurable sub-group.
  • AS 4.04 & 5.0
    • HA-JNDI. Clustered jndi server using JBossCache.
    • Fine-grained replication for SFSB and http session.

I am currently working on the JBossCache 1.2.4 release. The release will be mostly focused on JBossCacheAop and also the features needed to support fine-grained http session replication. One of the outstanding feature is for JBossCacheAop to handle scoped class loader (that can be a common requirement in a web app.) But I will defer it to another blog when it is completed.

Here, I would like to talk about annotation. One of the features in release 1.2.3 is the JDK1.4-style annotation support for JBossCacheAop. It is documented in the release doco. But since I have people asked me the usage, I thought I illustrate it here with an example of the fine-grained http session replication.

Since annotation (both JDK1.4 and 1.5) is supported by the JBossAop, there is really no ground-breaking development to apply it to JBossCacheAop per se. But without annotation, a user will always need to specify the Pojo (in this case, Address and Person ) instrumentation in a jboss-aop.xml file such as:


  <prepare expr="field(* org.jboss.test.cache.test.standAloneAop.Address->*)" />

  <prepare expr="field(* $instanceof{org.jboss.test.cache.test.standAloneAop.Person}->*)" />


Because we use dyanmic aop, we require every field in your Pojo to be "aspectized". It can become tedious for user to list every single one of them. Annotation in this case will work perfectly.

To facilitate the use of annotation, I have provided an AopMarker and InstanceOfAopMarker interfaces acting as 1.4 annotation holder. Then there is a system-wide jboss-aop.xml that is located under resources directory to associate the annotation as follows:


  <prepare expr="field(* org.jboss.cache.aop.AopMarker->*)" />

  <prepare expr="field(* $instanceof{org.jboss.cache.aop.InstanceOfAopMarker}->*)" />


Note that InstanceOfAopMarker denotes that any sub-class or interface implementation of the Pojo will be automatically instrumented as well.

With these, all a user needs to declare in the pojo becomes very simple. It is just one line of annotation code. For example,


 * @@org.jboss.cache.aop.AopMarker


public class Address {...}


 * @@org.jboss.cache.aop.InstanceOfAopMarker


public class Person {...}

Will declare both Address and Person Pojos to be "aspectized" under JBossCacheAop automatically (without using a jboss-aop.xml).

So let&apos;s say when the next release of JBossAS supports fine-grained http session replication, to use it: 1) you will specify in your jboss-web.xml (inside your web app) that the replication granularity is "FIELD", 2) then declaring your Pojo(s) like above, the http session replication will be fine-grained automatically.

For example, when you do:

Person ben = new Person(); 

session.setAttribtue("ben", ben); // This setup Pojo ben to be managed.


Only the name field in object ben will be replicated!

Of course because this is a 1.4 construct, you will still need an annotation compiler step to pre-compile your Pojos. But when we support 1.5 native annotation in the future release, there will be no need for pre-compiling and it will become mostly transparent for application developers!


I have been asked to come up with the clustering roadmap recently. In doing so, I thought I share with the readers here on the latest for the JBoss Clustering project.


JBoss Clustering as many of you know is easy to configure and is mostly client transparent. As a matter of fact, from the client side, all you need to do is having your jndi lookup pointing to that of HA JNDI (default port 1100 , for istance). If you are new to JBoss Clustering, then the wiki page is the starting point: Clustering wiki. We currently also in the process of updating clustering documentation.


For those of you who are interested in the details, I have created the clustering tasks under Clustering Jira. Detailed high level overview is still to come. But any feedback or contribution is welcome!


Note that the Jira tasks don't have the release schedule since clustering by itself is a cross-cutting concern spanning JGroups, JBossCache, AS, EJB3, Pojo server, etc. Therefore, we have used a timeline-based pseudo schedule to denote what's the latest.


Couple of notable points in the task list are:

  1. We plan to use JBossCache for any state replication in the clustering framework. Examples like http session replication, stateful session and entity beans in ejb3 (where the entity bean clustering is supported through Hibernate's second-level cache.) This makes a lot sense in term of modular development as JBossCache is getting popular (just like JBossCache sits on top that of JGroups). Features added in JBossCache can be readily applied to the "application layer" on top. For example, we are developing the overflowing (passivation/activation) feature in JBossCache that will benefit the http session replication and stateful session bean immediately!
  2. Since we are using JGroups and JBossCache stacks in various places, a central configuration management is needed. Details of this will come soon.
  3. New implementation of http session replication based on JBossCache has been completed in release 3.2.6 and later. Currently I am working toward completion of the fine-grained http session replication where we use JBossCacheAop to perform pojo level replication. This is an exciting development and has the potential to minimize network traffic and performance enhancment. I will describe this in more details once it is fully completed.


Finally, in addition to existing clustering contributors, we currently have 3 Novell's developers on board to work on JBoss Clustering in areas such as farming, HA JNDI, and JBossCache overflowing. The partnership has been working well so far. This can set up a new way of corporate sponsorship of JBoss open-source product development since companies can grow in-house expertise while making the JBoss product itself more robust!