JBossDeveloper

Ok, I get your point now.

Quoting JPA/Hibernate book, section 1.2.5 The problem of data navigation

And object persistence solution provides functionality for fetching the data of associated objects only when the object is first accessed.  However, this piecemeal style of data access is fundamentally inefficient in the context of a relational database, because it requires executing one statement for each node or collection of the object network that is accessed.  This is the dreaded n+1 selects problem.

This mismatch in the way you access objects in Java and in a relational database is perhaps the single most common source of performance problems in Java applications.  There is a natural tension between too many selects and too big selects, which retrieve unnecessary information into memory.

section 1.4.2 Generic ORM problems

How do we efficiently retrieve data with associations?  Efficient access to relational data is usually accomplished via table joins.  Object-oriented applications usually access data by navigating an object network.  Two data access patterns should be avoided when possible: the n+1 selects problem, and its complement, the Cartesian product problem (fetching too much data in a single select).

So if you use EAGER loading or join fetch queries, the danger is possibly loading the entire object graph of your db schema.

And if you use the default fetch plan (LAZY loading), you face the n+1 selects problem.

I have not seen or heard any good explanations on this from a best practices perspective in the context of using SMPC to deal with these two problems.

There is some treatment on optimizations for these problems in Chapter 13 of JPA/Hibernate (Optimizing Fetching and Caching).

For example:

Prefetching proxies and collections with a batch strategy is really a blind guess.  It's a smart optimization that can significantly reduce the number of SQL statements that are otherwise necessary to initialize all the objects you're working with.  The only downside of prefetching is, of course, that you may prefetch data you won't need in the end.  The trade-off is possibly higher memory consumption, with fewer SQL statements.  The latter is often much more important: Memory is cheap, but scaling database servers isn't.

If you switch from the default strategy to queries that eagerly fetch data with joins, you may run into another problem, the Cartesian product issue.  Instead of executing too many SQL statements, you may now (often as a side effect) create statements that retrieve too much data.
You need to find the middle ground between the two extremes: the correct fetching strategy for each procedure and use case in your application.

So I think the point really is determining how to handle the n+1 selects and Cartesian product problems for your particular domain model.  I'm not sure this is specific to SMPC as you will be navigating object graphs and using EAGER and LAZY fetch plans no matter what if you're using Hibernate.  And you will have potentially excessive selects if you design/implement your app improperly and it may be much worse if the app has large load...

So if you use EAGER loading or join fetch queries, the danger is possibly loading the entire object graph of your db schema.

This is not a real danger. Yes, over-eager loading occasionally happens for a short time during development, when attention is not paid to this aspect. But for any serious business application, during testing, the developer ought to set hibernate.show_sql to true to check what the queries do. The SQL query issued by Hibernate is always the same so there is no issue of having the production system use a different 'HQL execution plan'. Or employ more sophisticated approaches. In all cases, inadvertent eager loads are soon discovered and fixed. It is just a temporary problem and, like many other performance problems, is something to watch for and correct. If you have queries going against tables with millions of records you do that anyway and there are many other knobs to turn. In most cases in business apps, on certain pages you need data about Foos and its Bars, while on others only about Foos. The queries will be different, there is no single query for Foo that will serve all the use cases throughout the application. Just saying 'an extended SMPC will take care of that with lazy loading' is a bit of lazy designing :-)

I think that the emphasis on avoid LIEs (while doing very little to talk about the downsides of having multiple queries) hasn't really served Seam. It can be easily be dismissed as a red herring - as already mentioned by Tim, it's often a beginner's problem.

Jean Luc wrote on Aug 25, 2009 16:35:


I think that the emphasis on avoid LIEs (while doing very little to talk about the downsides of having multiple queries) hasn't really served Seam. It can be easily be dismissed as a red herring - as already mentioned by Tim, it's often a beginner's problem.

Well the topic discussion has officially digressed (but I guess the original topic discussion is complete anyways?)

This last point you made is a very good one.  It's like Seam SMPC solves the LIEs problem successfully in all use cases however it potentially introduces the n+1 selects problem if you're using the default LAZY loading fetch plan without proper optimizations.

Interestingly, as a side note, JPA uses EAGER fetch by default for @ManyToOne and @OneToOne which is different from Hibernate which is LAZY for all types by default - at least after a hbm2java revengr, but whatever, that is configurable per relationship and then overridable using fetch join syntax in your custom JPAQL query.

Here is a very good article entitled Hibernate: Understanding Lazy Fetching dated Aug 2005:

http://www.javalobby.org/java/forums/t20533.html

This is really worth the read and quite honestly, a little over my head.  I've noticed this guy writes some very detailed and helpful articles.

We are trying to balance/solve the following:

1) avoid extra db round-trips
2) avoid n+1 selects problem
3) avoid Cartesian products problem

all while using SMPC and Hibernate manual flush.  It's difficult to even write a wiki article on this because it really depends on the data model and use case, etc.  But it definitely needs to be addressed.

As far as LIEs are concerned, the article addresses the solution as follows:

There are really two common answers to this problem as seen by the community right now. The first is to explicitly initialize all associations and collections required before returning it. The second is to keep the session open until the view is done rendering (commonly referred to as 'Open Session In View').

And of course we have SMPC which solves LIEs pretty easily for the Seam developer I'd say, but may introduce the n+1 selects problem or replace it with Cartesian products problem depending on how the developers implements and configures the EAGER/LAZY loading and optimizations cited earlier in this thread (see JPA/Hibernate for more).

How bout using Seam with something like Spring's JdbcTemplate and four stored procs per table?  Sure, you lose a lot of things like conversational cache and 1st level cache, but would it overall be a better solution?

This is really a general ORM problem: avoiding db round trips, avoiding n+1 selects, avoiding Cartesian products, no?

Or how bout using OODBMS and something like db4o OODBMS or InterSystems Cache with Seam apps?

Hi everyone,

As Arbi, Tony etc. point out, we are all extremely busy getting JSR-299 (CDI), and the reference implementation ready for Java EE 6. To achieve this, with the staff Red Hat provides for Seam (5 full time engineers), we have had to really concentrate hard on that effort - and hence I have not had time to work on the forum. Trust me - I'm looking forward to getting back to Seam as much as you guys are (hopefully!).

Norman and Christian are still working hard on Seam 2 maintainance, so hopefully the bad bugs get fixed (but not as fast as you would like, I know) - but to let them get on with this, we do need the community to step and work the forums, help people debug their problems etc.

We should have Web Beans ready pretty soon now, and we've started to formalize the Seam 3 goals. I'll have more to share with you all after JBoss World :-) To give you a preview the key goals are:

• CDI-ification (of course)


• loose coupling of functionality (everything is a module, just take the bits you want)


• EE6 based (JSF 2 etc.)

And if you want more info on what Seam 3 will look like - just take a look at some of the webinars and podcasts I've recorded.

I'll blog about this more after JBoss World.

I started this wiki in knowledgebase:

http://seamframework.org/Documentation/PreventingDbRoundtripsLIEsN1SelectsAndCartesianProducts

It's not complete and I will update as I have the time.  I'm considering this an advanced topic and most likely some advanced Seam/Hibernate guys will need to proof-read it, etc.  It's covering a lot of ground.

Arbi,

Thanks and compliments (Tony to) for turning an initially 'yet another meaningless discussion' into a very constructive one.

I did not think it (the original topic) was another meaningless discussion.

Anyways, here's a tidbit from the ref doc that I was unaware of regarding SMPC:

Seam-managed persistence contexts are extremely efficient in a clustered environment. Seam is able to perform an optimization that EJB 3.0 specification does not allow containers to use for container-managed extended persistence contexts. Seam supports transparent failover of extended persisence contexts, without the need to replicate any persistence context state between nodes.

Arbi Sookazian wrote on Aug 26, 2009 07:30:


I did not think it (the original topic) was another meaningless discussion.

Sorry, no, now I re-read my post I see I missed something (it was late). I meant to say:

...for turning what initially could have turned into a 'yet another meaningless discussion' into a very constructive one...

And indeed the initial post was also not meaningless (sorry ;-)) But the topic subject could have attracted some people to start commenting on all kinds of negative things. So my compliments remain.

Great discussion.

This is a very interesting topic. Here are my two cents:
I started to use Seam as of 1.2.0 version more than 2 years ago. I have the chance to choose the technology for that project in my firm and I decided to give a try to that new and unknown technology called Seam. I convinced my leader and the number of projects in my firm that use seam have grown and grown ever since. Performance and bugs have been improved with newer versions and seam activity has been alive all this time in jboss as well as in the community.
When I started with seam this forum (then it was located in a different web) was really helpful and Pete Muir was a very active supporter. As has been stated, seam team is now focused on what is to come and its strict timeline. Pete´s previous response to this thread deserves to be thanked. I try to help whenever I have the chance as I see quite a few do.
But it´s a pity my firm is an exception, at least in Spain, and according to the posts I´ve read this happens all over the world. Howewer Spring and Struts are widely used.
It´s true that the learning curve it´s a drawback but I do think it´s worth the effort. I really hope Seam to grow and spread, and to become a really well known technology for firms. This would encourage firms to use it for their incoming projects.
Maybe Seam 3 and web beans would achieve that.

Brett Shelley wrote on Aug 26, 2009 10:39:


The initiating reason for my discussion was to gain further insight on the future of the product.  Should I recommend it to my Fortune 100 international clients here in Stuttgart, Germany as their core AJAX web development platform? is the heart of my interest. They just adopted Eclipse RCP over .NET as their long-term approach  - affecting thousands of engineers -- but the Web 2.0 approach is still up in the air. 

So, what I am hearing goes something along the lines of:

1) A small development team is maintaining and cultivating the current product

2) A number of neat features have arisen from the product and are making their way into core enterprise java

3) The current product release will undergo a significant refactoring with the arrival of J6EE.  Thus, adopting
it in its present state might be premature

4) The learning curve is significant...etc.

This has been really good stuff, thanks for the meaningful comments!

Ask yourself do I need a stateless framework (Spring) or stateful framework (Seam)?  I think Spring has the concept of conversation support now with Spring Web Flow but never tried it out.  Am I writing small apps or large, external-facing apps?  What are my non-functional requirements in terms of performance, scalability, etc. when evaluating these stacks.  For small apps, RoR is supposed to be very good and has a very strong installation base but supposedly has scalability problems (read about Scala and twitter.com, for example).  I am unaware of any very large user-base, external-facing Seam apps.

You should try to understand the QA process with Seam if you can (honestly, I'm not sure exactly how they QA their stack, I'm assuming unit and integration testing but not sure if they have a separate QA team specifically for Seam, etc.)

If I were you, I'd wait until Seam 3.x.  JBoss has a history of releasing GA bits that require patches.  The latest example IIRC is JBoss 5.x deployment problems of Seam 2.x apps.

But I've been using Seam 2.0.2-FP with JBoss 4.3 EAP for several months now without too many issues in terms of bugs.

Also understand that the learning curve with any technology stack that deals with Web 2.0 (AJAX) and distributed computing with clustering support, web services support, remoting, messaging, pluggable MVC, etc. is going to be high.  This is true with Spring (have you read about Spring AOP or AspectJ?) as well as .NET.  They always say the learning curve with Seam is high so the adoption rate has been low but I counter that the learning curve for any distributed web platform is going to be high.

One main area you should focus on in your POC while evaluating Seam, especially if it's a large userbase, external-facing 24x7 web app, is performance.  JSF and its life cycle has been known to be a performance bottleneck (read about the rendered attribute).  This may have been optimized in JSF 2.0 and Seam 3, I'm not sure.  The heavy use of interceptors (which is required for bijection, for example) can also be a bottleneck.  That is why the @BypassInterceptors annotation is available.  Also, remember that in Seam 2.x, there is no selective dynamic injection available.  This means that all the @In, @DataModelSelection and @DataModelSelectionIndex annotated member variables in your Seam components will always be injected prior to any business method call.  You may not always want or need that (i.e. that's my concept of selective dyanmic injection by adding an attribute to the @In annotation, specifying for which business methods to dynamically inject prior to calling them).  There's also the n+1 selects and Cartesian products ORM problems as well as implementing 2nd level cache and query cache as required to improve performance and minimize db hits.

Transaction and persistence with Seam/JPA/Hibernate is very difficult to understand once you dive into the details.  Global transactions, application transactions, atomic conversations, AUTO/COMMIT/MANUAL flush, distributed transactions, CMT, BMT, when to use EJB or plain JavaBeans (@Transactional), transaction propagation, etc.

Conversations and nested conversations and workspaces can be very complex as well.  How to begin them (using declarative or programmatic code), when to begin them, when to end them.

More complicated situations arise like when you need to access more than one RDBMS in the same app.  Then you need more than one SMPC.

What's the best way to integrate legacy (or new) stored procs?  Seam does not have the equivalent of Spring's JdbcTemplate, but perhaps you can integrate some Spring beans into your Seam app, thus making your solution even more complicated!

Nice things:  reverse engineer your db schema using hbm2java (via seam-gen), little XML coding, interceptors instead of overkill AOP, remoting, Excel, PDF support, Wicket/GWT/JSF support, solution for LIEs, conversation context and jBPM integration, etc.

i.e. lots of features!  But ask yourself, do I need all these features?  Do I need to write apps where the user can do comparison shopping using multiple tabs in the same browser, each being a separate, isolated conversation/workspace?  Do I need workflow support via jBPM integration?

And don't forget the tooling in JBoss Tools.

Bottom line:  be prepared to do a lot of reading, stepping thru the core codebase, looking at the distro examples and experimentation (POCs) when you start writing Seam apps.  I've been doing this myself non-stop for the last 2.5 years.  If you have no experience with RichFaces or ICEfaces or JSF or Hibernate/JPA, this will be a very frustrating experience.  The problem with Seam is you can't really take on one part of the stack at a time and learn it on its own (well, with JSF you could use managed beans and start from there I guess).  Seam is an integration layer that really touches all layers (presentation tier, business tier, db tier, etc.)

And yes, a lot of the concepts and features in Seam have found their way into JSF 2.0 and CDI (JSR 299) and possibly other JSRs in EE 6 I'm unaware of (like EJB 3.1 and JPA 2.0?)  But the manual flush is still not one of the FlushModeTypes in JPA 2.0!

For more info on Seam 3 and seam-gen (they should make some stickies in the Seam forum so ppl know about these valuable wikis in the knowledgebase area), see this link:

http://seamframework.org/Documentation/Design

That's enough for now and HTH!

Arbi Sookazian wrote on Aug 26, 2009 17:47:

You should try to understand the QA process with Seam if you can (honestly, I'm not sure exactly how they QA their stack, I'm assuming unit and integration testing but not sure if they have a separate QA team specifically for Seam, etc.)

This is what they at least have: http://hudson.jboss.org/hudson/view/Seam/

I reiterate my statement.

Hire a professional technical writer to work on chapters of the documentation.

Transactions, security, conversations, bpm.

They will not cloud the text by saying.... Well you do it this way , but of course you could also do it that way... And skip to some random section ahead if you don't use this etc.

The doco is Great! It just needs to be a lot better to lower the bar from masochist to 9-5 programmer.

The follow the examples chapters are great and then it's all too confusing!

This, I believe will push seam into orbit.