-
1. Re: Stateless Process Engine executor threading issues
kukeltje Feb 28, 2007 3:00 PM (in response to jorgemoralespou_2)my first impression is that you do not use jbpm as it was inteded to be used. Holding on to a context blocks the engine for other processinstances to run. Getting a new context each time is not expensive, so get one, signal it, keep the token/processinstance id somewhere and go on with other instances. Once needed, get a context again, use the token/process id to retrieve the instance and signal it.
-
2. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Feb 28, 2007 3:18 PM (in response to jorgemoralespou_2)Thanks Ronald,
But actually I pasted an old version of the code.try { // Fetch JbpmContext ctx = getJbpmConfiguration().createJbpmContext(); // Get process definition ProcessDefinition processDefiniton = null; processDefiniton = ctx.getGraphSession() .findLatestProcessDefinition(processName); if (processDefiniton == null) { throw new ProcessDefinitionNotFoundException(processName); } // Create instace, add input params as context variables ProcessInstance instance = new ProcessInstance(processDefiniton); instance.getContextInstance().setTransientVariable( "REQUEST", req); // Execute process do{ instance.getRootToken().signal(); } while (!EndState.class.isAssignableFrom(instance.getRootToken().getNode().getClass())); // Get output parameters Map<String, Serializable> response = (Map<String, Serializable>) instance.getContextInstance().getTransientVariable("RESPONSE"); return response; } catch (Exception e) { throw new JbpmEngineException("Unexpected error", e); } finally { if (ctx != null) { ctx.close(); } } }
This code, as I think obtains a context, and signals every token in the execution path right through the end. I don get hold to a context.
I'm not sure if your answer is still aplicable. I'll try what you suggest though.
Also, is there any way of not using services like Messaging, Authentication and Logging? I've read that commenting the code in jbpm.cfg.xml should do the work, but it? actually doing all the work but persisting to database. -
3. Re: Stateless Process Engine executor threading issues
kukeltje Feb 28, 2007 3:37 PM (in response to jorgemoralespou_2)you do complete one instance of a process in one loop, continuously holding on to a context. That is (afaik, but never tried it) blocking.
-
4. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Feb 28, 2007 4:08 PM (in response to jorgemoralespou_2)You might be right. And that probably explains the behaviour I'm observing, but how can I actually signal a Token if I'm not hold to a Context.
Execution of nodes should occur within a Context, as long as I know, and since I signaling every node in the execution path, the execution get's done right.
For what I understand from the sources, JBPMContext is bound to the execution thread, and for that reason, seems multi-threadable. As long as I instantiate one JBPMContext in a new thread, the one dispatching the http request from my ws faccade, it should scale properly.
Probably there is an issue with such a load for Hibernate, or something else that escapes my eyes.
Also, I don't want to persist results from executions, or at least only the minimal required, the process instance result, but I don't see any way of disabling this, or any other superflual services for my needs. -
5. Re: Stateless Process Engine executor threading issues
kukeltje Feb 28, 2007 4:28 PM (in response to jorgemoralespou_2)Yes, execution of nodes should occur in a context. For that a jBPM context is used. It does not mean it has to be the identical, same context. That is why I said you have to hold on to a tokenid or processid to use that to get to the token/process when opening a new context. Look at the examples, testcases etc...
And yes, openening a new context in a new thread should scale (at least I hope so).... disabeling real persistency is simple by using an in-memory hsqld database and deleting the processinstance once it completes. If you do use a database, tuning of indexes etc... should indeed be done. jBPM only delivers minimal tuning.. -
6. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Mar 1, 2007 10:41 AM (in response to jorgemoralespou_2)Hi Ronald,
I still don't get the point. Our executions are in such a way, for now, that once a token is signaled, it must continue through the end, that's why we get hold on to the context. There is no state saving till the end of the execution, because there isn't any waiting state. I don't see any clue in the examples or in any other document you pointed me.
Any change we made so far, leaves us at the same point, double client load, double response time, and plenty of free cpu.
Is there any concurrency issue or synchronicity issue regarding JBPM's context? -
7. Re: Stateless Process Engine executor threading issues
kukeltje Mar 1, 2007 10:52 AM (in response to jorgemoralespou_2)Is there any concurrency issue or synchronicity issue regarding JBPM's context?
Not that I know of if it is in another thread, but I don't know everything :-)....
Can you create a minimal testcase (no web, just standalone unittest) with real multithreading, that demonstrates the problem.
btw, do you call any external services synchronously? if so, how long do they take to process? -
8. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Mar 1, 2007 11:25 AM (in response to jorgemoralespou_2)Hi Ronald,
We're trying to isolate the problem and work a testcase that runs outside the AS. It 's no easy job, due to high integration with the AS., but I'll try to post it here when we have it.
btw, do you call any external services synchronously? if so, how long do they take to process?
Sure we call external services. In fact our system is acting as a small orchestration software, but that is the point on running it inside a multithreaded AS. I expect it to scale on load properly, not double. The external systems are also low on load, so the problem is internal to our system. -
9. Re: Stateless Process Engine executor threading issues
kukeltje Mar 1, 2007 11:39 AM (in response to jorgemoralespou_2)I also tried contacting one of the core developers to get a statement on this... I also already told him you use it in a kind of orchestration way.
The reason I asked is if you use forks, and want a kind of parallel processing, you have to use async actions in the nodes, otherwise all 'legs' of a fork will be processed sequentially.
And that an AS is multithreadded does not mean everything is.... -
10. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Mar 1, 2007 11:48 AM (in response to jorgemoralespou_2)"kukeltje" wrote:
I also tried contacting one of the core developers to get a statement on this... I also already told him you use it in a kind of orchestration way.
Thank you very much for your interest."kukeltje" wrote:
The reason I asked is if you use forks, and want a kind of parallel processing, you have to use async actions in the nodes, otherwise all 'legs' of a fork will be processed sequentially.
And that an AS is multithreadded does not mean everything is....
We don't use forks. Right now, only nodes, decisions and exception handlers.
I have already read about the forking issue, but that is not our case.
As soon as we have a real testcase, I'll post it here. -
11. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Mar 1, 2007 1:16 PM (in response to jorgemoralespou_2)Hi Ronald, I'm posting here the code.
Web service@WebService(name = "ProcessTestService", targetNamespace = "http://test/ws", serviceName = "ProcessTestService") @SOAPBinding(style = SOAPBinding.Style.RPC) public class ProcessTestService { public String testInTransaction(String param1) throws Exception { Map request = new HashMap(); StatelessEngine engine = new StatelessEngine(); Object o = engine.executeProcessInstance("test-som-process", request); return "result"; } @WebMethod public String test(String param1) throws Exception { String retValue = null; UserTransaction tx = getUserTransaction(); tx.begin(); try { retValue = testInTransaction(param1); tx.commit(); } catch (Exception e) { tx.rollback(); throw e; } return retValue; } private UserTransaction getUserTransaction() { UserTransaction ut = null; try { InitialContext ctx = new InitialContext(); ut = (UserTransaction) ctx.lookup("UserTransaction"); } catch (NamingException e) { appLog.error("Error obtaining the UserTransaction", e); } return ut; } }
The engineimport java.util.Map; import org.jbpm.JbpmContext; import org.jbpm.graph.def.ProcessDefinition; import org.jbpm.graph.exe.ProcessInstance; import org.jbpm.graph.node.EndState; import com.hp.som.bpm.SomJbpmEngineConfigurationMBean; import javax.naming.Context; import javax.naming.InitialContext; import org.jbpm.JbpmConfiguration; public class StatelessEngine{ public Object executeProcessInstance(String processName, Map request) throws Exception{ JbpmContext ctx = null; try{ ctx = getJbpmConfiguration().createJbpmContext(); // Get process definition ProcessDefinition processDefiniton = ctx.getGraphSession().findLatestProcessDefinition(processName); if (processDefiniton == null) { throw new Exception("Process definition not found"); } ProcessInstance instance = new ProcessInstance(processDefiniton); instance.getContextInstance().setTransientVariable("REQUEST", request); // Execute process do{ instance.getRootToken().signal(); }while (!instance.getRootToken().hasEnded()); // Check that execution is on a end state. if not, throw exception if (!EndState.class.isAssignableFrom(instance.getRootToken() .getNode().getClass())) { throw new Exception("Process finished not in an End Node"); } Object response = instance.getContextInstance().getTransientVariable("RESPONSE"); return response; } finally { if (ctx != null) ctx.close(); } } private static JbpmConfiguration conf = null; protected JbpmConfiguration getJbpmConfiguration() throws Exception { if (this.conf == null){ JbpmConfiguration config = null; String jndiName = null; Context ctx = new InitialContext(); // Get jndi name for jbpm configuration jndiName = "java:/jbpm/JbpmConfiguration"; config = (JbpmConfiguration)ctx.lookup(jndiName); this.conf = config; } return this.conf; } }
The test process<process-definition xmlns="" name="test-som-process"> <start-state name="start"> <transition name="" to="node1"></transition> </start-state> <end-state name="end"> </end-state> <node name="node1"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="node3"></transition> </node> <node name="node3"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="node4"></transition> </node> <node name="node4"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="node5"></transition> </node> <node name="node5"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="node6"></transition> </node> <node name="node6"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="node8"></transition> </node> <node name="node8"> <action class="com.hp.som.test.TestActionHandler"></action> <transition name="" to="end"></transition> </node> </process-definition>
And finally test handlerpublic class TestActionHandler implements ActionHandler { public void execute(ExecutionContext context) throws Exception { try { // Create a process logger whith process name customExecute(context); // The action have been executed correctly context.getToken().signal(); } catch (Exception e) { e.printStackTrace(); throw e; } } protected void customExecute(ExecutionContext context) throws Exception { String param = "Do nothing important"; } }
When you load test the web service, you can see how response time doubles as client doubles, but cpu stays low. -
12. Re: Stateless Process Engine executor threading issues
kukeltje Mar 1, 2007 2:04 PM (in response to jorgemoralespou_2)Thanks, but... this is not something I can run out of the box. I need a webcontainer (that I have) , webservice loadtestclient (strange word, but I don't have it) etc...etc...etc... so unfortunately it's not usable for me this way (it's not a unit test). Do you think you can still create a unit test? (actionhandler as an inner class etc..) process as string internally etc.. that way I just have one class and run it...
-
13. Re: Stateless Process Engine executor threading issues
jorgemoralespou_2 Mar 1, 2007 2:12 PM (in response to jorgemoralespou_2)I'll try to post it tomorrow morning. Thanks.
-
14. Re: Stateless Process Engine executor threading issues
tom.baeyens Mar 2, 2007 8:14 AM (in response to jorgemoralespou_2)jorgemoralespou_2,
you can use jBPM persistent or transiently. so you have to decide if you want persistence first. you always need a jbpmContext, but in case you don't need persistence, you can just parse the process from a file and create instances with the ProcessInstance constructor (all inside one or more jbpm context blocks)
apart from that, you should not require an engine that drives executions through the process. if you need to feed an external trigger, it means that your process behaves as a wait state. so it's a matter of not using wait states to avoid the external signals. if you create a new process instance and signal it, it will run until the end in one go if there are no waitstates inside the process.