-
1. Re: Clustering strategy for jBPM5
garytse Jul 12, 2011 11:54 PM (in response to garytse)Any one please help? Has anyone able to share a single stateful knowledge session across 2 nodes with success or failures?
-
2. Re: Clustering strategy for jBPM5
arkper Sep 8, 2011 12:30 AM (in response to garytse)No. I have tried to do so, but it appears to be an issue when Hibernate tries to update SessionInfo shared by more than one node. It fails to commit the transaction if the entity has been updated by another node. The exception is as follows:
Hibernate: update SessionInfo set lastModificationDate=?, rulesByteArray=?, startDate=?, OPTLOCK=? where id=? and OPTLOCK=?
Sep 7, 2011 10:39:34 PM org.drools.persistence.jta.JtaTransactionManager commit
WARNING: Unable to commit transaction
javax.persistence.OptimisticLockException
: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.drools.persistence.info.SessionInfo#1
It should be noted that it's just a warning, but I'm not sure if the necessary updates are re-attempted in order to persist the state accuratly.
Something like Terracotta should help to cluster Knowledge Session as it has a potential to instrument the byte code and introduce the necessary locking mechanism to make the shared objects "thread"-safe even for threads running in different JVMs.
Has anybody tried and succeeded integrating jBPM5 with Terracotta?
-
3. Re: Clustering strategy for jBPM5
salaboy21 Sep 9, 2011 6:51 PM (in response to arkper)That's right and it's correct.
If two nodes wants to do operations on the same sessionInfo one of the two will fail and the other will success. that's the way how locking works.
Your application should add the logic to retry the operation if it fails and you don't want to miss the second modification (the one that fails).
Best Regards
-
4. Re: Clustering strategy for jBPM5
arkper Sep 9, 2011 10:18 PM (in response to salaboy21)Are you suggesting that this try-if-fail-retry approach should be used as a way to cluster a jBPM5 application?
-
5. Re: Clustering strategy for jBPM5
salaboy21 Sep 10, 2011 12:51 AM (in response to arkper)I'm not suggesting anything, but that's how transactional databases work. If you get two connections from different places that modifies the same row in the database one will success and the other will fail. The expcetion is clear about that:
javax.persistence.OptimisticLockException
It's not a jbpm5 problem, its how databases work. If you not ensure that two thread can keep everything consistent one needs to fail.
In which scenario are you getting that exception? what are you trying to achieve?
Best regards
-
6. Re: Clustering strategy for jBPM5
arkper Sep 10, 2011 10:56 AM (in response to salaboy21)I'm trying to achieve exactly what this thread is all about - have a cluster of several jBPM5 applications. They all areconnected to an enterprise service bus that can easily round-robin requests coming from the clustered web application. The exact use case is as follows:
1. jBPM5 Node 1 gets a request to create a process instance.There are several human tasks that need to be created - so Node 1 communicates to a cluster of Task Servers to have that done.
2. A user completes a human task which results in a Task server Node X communicating its completion to jBPM5 Node 2.
The point that I'm trying to make is that unless the jBPM5 nodes share the same knowledge session, Node 2 will not be able to pick up the task completion event and move the process instance alone because the process instance would only be available to the node that had created it. So, as Gary Tse had suggested in the opening message, I tried session sharing by executing the following on each node:
StatefulKnowledgeSession ksession = JPAKnowledgeService.loadStatefulKnowledgeSession(1, kbase, config, env);
I reported the predictable failure due to optimistic locking not being the right strategy to handle concurrent updates from multiple nodes. So this isn't an option to build a clustered jBPM5 application. So I'm going to ask you again (I've asked you the same question in a different thread dedictaed to clustering) - what is your approach to building a clustered jBPM5 application for scalability and high availability? Every high volume mission critical enterprise application requires this as a matter of very high priority.
-
7. Re: Clustering strategy for jBPM5
salaboy21 Sep 10, 2011 3:15 PM (in response to arkper)I will say something similar to my previous answer, probably I can help you with your problem:
You can create a simple mechanism that do the load for you depending on the processes that you know that are already pending for execution.
As you mention you need to use: StatefulKnowledgeSession ksession = JPAKnowledgeService.loadStatefulKnowledgeSession(1, kbase, config, env);
I'm not sure why you hard code the 1 for the session ID, but if you have multiple nodes doing that in a clustered environment one of the nodes will sucess on the process continuation. The other nodes will fail and you can probably discard that expcetions because you know that at least one node continue the process. Can you please name state the problems that you find in this approach? probably I'm missing something.. I'm just trying to help.
Best Regards
-
8. Re: Clustering strategy for jBPM5
arkper Sep 10, 2011 6:26 PM (in response to salaboy21)Mauricio:
First of all, thank you very much for your time and desire to help. That's truly appreciated.
What you seem to be missing though is that in order for a cluster totry and solve the scalability problem, only ONE, but ANY one, node should be able to process ANY given valid request. In your approach, ALL nodes would try and only one would succeed to process a request. Clearly, this wouldn't address the scalability concern since ALL nodes would be trying to do the job with only one succeeding and all the other ones wasting their resources while they could have been working on other concurrent requests which would have increased the overall cluster capacity. Your approach would probably address the fail-over concern somewhat, but not at all the scalability one...
As a community, we should push the jBPM5 team very hard to introduce the clustering support into the product. They seem to take it for granted, but in my humble opinion, it's a huge void as it makes the product completely unfit for the enterprise-grade applications.
-
9. Re: Clustering strategy for jBPM5
salaboy21 Sep 10, 2011 6:43 PM (in response to arkper)If I don't understand you wrong, if you change the strategy to pesimistic locking only one will start and succeed right? that can be easily done with hibernate overridings, Am I wrong?
Cheers
-
10. Re: Clustering strategy for jBPM5
salaboy21 Sep 10, 2011 6:45 PM (in response to arkper)My opinion about the community is that we are the community and we don't need to push anyone we can introduce new mechanisms if we agree on the implementation and we make those improvements as community. I'm a community member and I tried to spend time fixing bugs and adding features, but sometimes is not enough Any help is appreciated and more than welcome.
Best regards
-
11. Re: Clustering strategy for jBPM5
newbird Jul 25, 2012 8:39 AM (in response to garytse)Hi,
Did you get any work around or solution for this?
Please let me know, i am also having the same problem....
-
12. Re: Clustering strategy for jBPM5
arkper Sep 10, 2012 6:46 PM (in response to garytse)We ended up resorting to process partitioning with sticky processes. All requests are round-robined across N jBPM5 nodes. Once a process instance is created on partition X it is tagged with that partition and it'll always be served by partition X. Each partition has an active and stand-by nodes for fail-over. It's not real clustering, but it does help to scale up... Evidently, jBPM 5.1 is not designed for clustering.
-
13. Re: Clustering strategy for jBPM5
massimogentilini Sep 11, 2012 4:30 AM (in response to salaboy21)My two cents:
1) The availability of a simple clustering strategy for upcoming version of JBPM is a mandatory asset to gain acceptance of JBPM in the enterprise world. Is not possible to have to resort to hack or something when in the JBoss products lineup there are at least a couple of distributed cache solution that can be adopted to allow clustering
2) The name itself "loadStatefulKnowledgeSession" has the implicit meaning that the session cannot be easily shared, if the session is "stateful" then it goes against the "stateless" requirement that are behind a "pure" cluster solution. There should be a way to have a different provider or to configure the JPAKnowledgeService in a way that enables an easy clustering solution and manage concurrency without resorting to database locking and retries, like having a shared cache mechanism to share the knowledge session across nodes in a way that is fault tolerant, highly available and has good performance
I do not know it there is an easy way to push for a solution or to create a Jira request so that it can be voted but, from my point of view, this other enterprisey requirements are the ones that need to be pushed to make JBPM suitable for a larger adoption.
Regards
Massimo