1 2 Previous Next 15 Replies Latest reply on Apr 20, 2010 6:25 AM by prachi.mukhija

    Solution: JBPM3 and Spring Integration

    mr_magoo

      I thought I would pass on some of the hard one lessons here to help people avoid some of the pain I have had to go through. While I think my scenario is perhaps not one that jbpm has in mind specifically, I fear that it is quite a typical scenario in the wild.
      Also I think someone else might have a suggestion or ten. I am not claiming to be a jbpm guru.


      Required Reading:
      http://www.jorambarrez.be/blog/2009/04/17/integrating-jbpm3-with-spring-part1/
      http://www.jorambarrez.be/blog/2008/08/26/jbpm-job-executor-for-spring/

      Problem
      JBPM3 architecture assumes and relies heavily on an xml configuration file and dependencies are spread throughout several classes. (sometimes in surprising places and ways)
      Penetration of this is rather difficult and a few versions ago nearly impossible due to private/final members/methods in key classes. This was most evident in the job executor and scheduler.
      Previously spring-modules was sufficient to integrate with spring, although quite clunky due to the above restrictions. Current wisdom suggests that using the latest version of jbpm with spring-modules can introduce subtle bugs and it appears that the jbpm part of spring modules is now dead.

      The reference links above outline an alternative solution but it is somewhat flawed unfortunately when it comes to async processing. It also does not go into specifics on the spring config side of things and how to remove the dependency on the jbpm configuration xml file and have a fully spring injected environment.

      I will assume that you have read the references above and have downloaded the source code they reference.

        • 1. Re: Solution: JBPM3 and Spring Integration
          mr_magoo

          The problem to point out is the job executor thread. The overrided example given does not synchronise the acquire jobs properly. This means that there will be a race condition in the acquire jobs thread between the acquireJobs synchronized block in JobExecutorThread and the unsynched one in SpringExecutorThread.

          The correct implementation is:

          public class SpringExecutorThread extends JobExecutorThread {
          
           private TransactionTemplate transactionTemplate;
           private JobExecutor jobExecutor;
          
           public GfgSpringExecutorThread(
           String name,
           JobExecutor jobExecutor,
           JbpmConfiguration jbpmConfiguration,
           TransactionTemplate transactionTemplate,
           int idleInterval,
           int maxIdleInterval,
           long maxLockTime,
           int maxHistory
           ) {
           super(name, jobExecutor, jbpmConfiguration, idleInterval, maxIdleInterval, maxLockTime, maxHistory);
           this.transactionTemplate = transactionTemplate;
           this.jobExecutor=jobExecutor;
           }
           @Override
           protected Collection acquireJobs() {
           synchronized(jobExecutor) {
           return (Collection) transactionTemplate.execute(new TransactionCallback() {
           public Object doInTransaction(TransactionStatus transactionStatus) {
           return GfgSpringExecutorThread.super.acquireJobs();
           }
           });
           }
           }
           @Override
           protected void executeJob(final Job job) {
           transactionTemplate.execute(new TransactionCallback() {
           public Object doInTransaction(TransactionStatus transactionStatus) {
           GfgSpringExecutorThread.super.executeJob(job);
           return null;
           }
           });
           }
           @Override
           protected Date getNextDueDate() {
           return (Date) transactionTemplate.execute(new TransactionCallback() {
           public Object doInTransaction(TransactionStatus transactionStatus) {
           return GfgSpringExecutorThread.super.getNextDueDate();
           }
           });
           }
          }


          • 2. Re: Solution: JBPM3 and Spring Integration
            mr_magoo

            Note that because of the use of the transaction template, hibernate will assume that any staleobject exception coming out of the commit is an error. It will log and throw this exception.
            Personally I think this is very bad form in a library! :) In a multi-instance environment you will have to manually silence it as these optimistic lock failures are perfectly ok. They are caught and silenced in the core jbpm executor code but of course will not occur because of the delayed commit.

            e.g.

            org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.jbpm.job.ExecuteNode
            Job#140392]
            at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1782)
            [...]
            at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) <== CULPRIT!
            [...]
            at org.jbpm.SpringExecutorThread.acquireJobs(SpringExecutorThread.java:33)
            


            Please note that the above code does not deal with this problem at all. You will end up with 2 logged errors for every optimistic lock failure in a multi instance environment. The errors should be benign otherwise.







            • 3. Re: Solution: JBPM3 and Spring Integration
              mr_magoo

              The next issue is the extension of the jbpm configuration object.

              In this example I would like my configuration to autodeploy my process definitions at startup and fail with warnings if something is not right. I would also like the option of safely starting the jobexecutor AFTER everything is deployed and safe for processing jobs that may have already been queued up.
              Notes:
              The object factory being injected is our own custom one that I will talk about next.

              The line:
              JbpmConfiguration.Configs.setDefaultObjectFactory(objectFactory);
              Is necessary because I noticed jbpm attempting to use the default factory if it was not set. This step was the one that was preventing me from completely removing all dependency on an xml file and caused some subtle transaction bugs.
              A bit random to be honest and I have not looked deeply into why it is trying to undermine my config behind my back. Nasty little troll...

              JbpmSpringDelegationNode.beanFactory = applicationContext;
              This is the bit of code I am most unhappy with. Basically there is no way currently (unlike in jbpm4) to hook the process def lookup to the spring beanfactory. So we have to inject it manually.
              The new Spring 2.5/3.0 configuration annotations could be used here to force injection of properties, but we do not use them. Until then...yuck.

              public class JbpmConfigurationFactoryBeanImpl implements ApplicationContextAware, FactoryBean, InitializingBean, ApplicationListener {
               private static org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger(JbpmConfigurationFactoryBeanImpl.class);
              
               private JbpmConfiguration jbpmConfiguration;
               private TransactionTemplate transactionTemplate;
               private ObjectFactory objectFactory;
               private boolean startJobExecutor;
               private SessionFactory sessionFactory;
              
               private Resource[] processDefinitionsResources;
              
               @Override
               public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
               JbpmSpringDelegationNode.beanFactory = applicationContext;
               }
              
               public void afterPropertiesSet() throws Exception {
               JbpmConfiguration.Configs.setDefaultObjectFactory(objectFactory);
               jbpmConfiguration = new JbpmConfiguration(objectFactory);
               JbpmContext ctx = null;
               try {
               ctx = jbpmConfiguration.createJbpmContext();
               ctx.setSessionFactory(sessionFactory);
               } finally {
               if (ctx != null) {
               ctx.close();
               }
               }
               if (getProcessDefinitionsResources() != null) {
               InputStream inputStream = null;
               for (int i = 0; i < getProcessDefinitionsResources().length; i++) {
               try {
               ctx = jbpmConfiguration.createJbpmContext();
               Resource definitionLocation = getProcessDefinitionsResources();
               inputStream = definitionLocation.getInputStream();
               final ProcessDefinition processDefinition = ProcessDefinition.parseXmlInputStream(inputStream);
               final JbpmContext finalContext = ctx;
              
               getTransactionTemplate().execute(new TransactionCallback() {
               @Override
               public Object doInTransaction(TransactionStatus status) {
               finalContext.deployProcessDefinition(processDefinition);
               return null;
               }
               });
               } finally {
               if (inputStream != null) {
               inputStream.close();
               }
               if (ctx != null) {
               ctx.close();
               }
               }
               }
               }
               StaleObjectLogConfigurer.hideStaleObjectExceptions();
               }
               public Object getObject() throws Exception {
               return jbpmConfiguration;
               }
               @Override
               public Class getObjectType() {
               return JbpmConfiguration.class;
               }
               @Override
               public boolean isSingleton() {
               return true;
               }
               @Override
               public void onApplicationEvent(ApplicationEvent applicationEvent) {
               if (applicationEvent instanceof ContextClosedEvent) {
               jbpmConfiguration.getJobExecutor().stop();
               }
               if (applicationEvent instanceof ContextStartedEvent) {
               if (startJobExecutor) {
               log.info("Starting job executor ...");
               jbpmConfiguration.startJobExecutor();
               log.info("Job executor started.");
               }
               }
               }
               public void setStartJobExecutor(boolean startJobExecutor) {
               this.startJobExecutor = startJobExecutor;
               }
               public void setObjectFactory(ObjectFactory objectFactory) {
               this.objectFactory = objectFactory;
               }
               public void setSessionFactory(SessionFactory sessionFactory) {
               this.sessionFactory = sessionFactory;
               }
               public Resource[] getProcessDefinitionsResources() {
               return processDefinitionsResources;
               }
               public void setProcessDefinitionsResources(Resource[] processDefinitionsResources) {
               this.processDefinitionsResources = processDefinitionsResources;
               }
               public TransactionTemplate getTransactionTemplate() {
               return transactionTemplate;
               }
               public void setTransactionTemplate(TransactionTemplate transactionTemplate) {
               this.transactionTemplate = transactionTemplate;
               }
               }


              • 4. Re: Solution: JBPM3 and Spring Integration
                mr_magoo

                The next is the object factory. We use a map object to store the various nebulous properties that are required as well as getting those core, spring injected service factories in there.
                The map makes specifying boolean and string properties a little cleaner and allows us to remove some ugly naming in our core spring config file that would otherwise be needed.

                public class SpringObjectFactory implements ObjectFactory, ApplicationContextAware {
                
                 private ApplicationContext applicationContext;
                 private Map<String, Object> jbpmProperties;
                
                 @Override
                 public Object createObject(String objectName) {
                 if (getJbpmProperties().containsKey(objectName)) {
                 return getJbpmProperties().get(objectName);
                 }
                 else if (getApplicationContext().containsBean(objectName)) {
                 return getApplicationContext().getBean(objectName);
                 }
                 else {
                 return null;
                 }
                 }
                 @Override
                 public boolean hasObject(String objectName) {
                 return
                 getJbpmProperties().containsKey(objectName)||
                 getApplicationContext().containsBean(objectName);
                 }
                 public void setJbpmProperties(Map<String, Object> jbpmProperties) {
                 this.jbpmProperties = jbpmProperties;
                 }
                 public Map<String, Object> getJbpmProperties() {
                 return jbpmProperties;
                 }
                 @Override
                 public void setApplicationContext(ApplicationContext applicationContext) {
                 this.applicationContext = applicationContext;
                 }
                 public ApplicationContext getApplicationContext() {
                 return applicationContext;
                 }
                }
                


                • 5. Re: Solution: JBPM3 and Spring Integration
                  mr_magoo

                  Other java classes we use that are almost the same or identical as the ones in the articles. Just for completeness.
                  Lock monitor thread:

                  public class SpringLockMonitorThread extends LockMonitorThread {
                   private TransactionTemplate transactionTemplate;
                  
                   public SpringLockMonitorThread(JbpmConfiguration jbpmConfiguration,
                   int lockMonitorInterval, int maxLockTime, int lockBufferTime, TransactionTemplate transactionTemplate) {
                   super(jbpmConfiguration, lockMonitorInterval, maxLockTime,lockBufferTime);
                   this.transactionTemplate = transactionTemplate;
                   }
                   @Override
                   protected void unlockOverdueJobs() {
                   transactionTemplate.execute(new TransactionCallback() {
                   public Object doInTransaction(TransactionStatus transactionStatus) {
                   SpringLockMonitorThread.super.unlockOverdueJobs();
                   return null;
                   }
                   });
                   }
                   public void setTransactionTemplate(TransactionTemplate transactionTemplate) {
                   this.transactionTemplate = transactionTemplate;
                   }
                  }
                  


                  Job executor:
                  public class SpringJobExecutor extends JobExecutor {
                   private TransactionTemplate transactionTemplate;
                  
                   @Override
                   public synchronized void start() {
                   if (!isStarted) {
                   for (int i = 0; i < nbrOfThreads; i++) {
                   startThread();
                   }
                   lockMonitorThread = new SpringLockMonitorThread(jbpmConfiguration, lockMonitorInterval, maxLockTime, lockBufferTime, transactionTemplate);
                   isStarted = true;
                   }
                   }
                   @Override
                   protected Thread createThread(String threadName) {
                   log.info("Creating JobExecutor thread " + threadName);
                   return new GfgSpringExecutorThread(threadName, this, jbpmConfiguration, transactionTemplate, idleInterval, maxIdleInterval, maxLockTime, historyMaxSize);
                   }
                   public void setTransactionTemplate(TransactionTemplate transactionTemplate) {
                   this.transactionTemplate = transactionTemplate;
                   }
                  }


                  • 6. Re: Solution: JBPM3 and Spring Integration
                    mr_magoo

                    The horribly ugly delegation node...

                    public class JbpmSpringDelegationNode implements DecisionHandler, ActionHandler, AssignmentHandler {
                     private static final long serialVersionUID = 1L;
                    
                     private String beanName;
                     protected static BeanFactory beanFactory;
                     private Object bean;
                    
                     public String decide(ExecutionContext executionContext) throws Exception {
                     DecisionHandler dh = getBean();
                     return dh.decide(executionContext);
                     }
                     public void execute(ExecutionContext executionContext) throws Exception {
                     ActionHandler ah = getBean();
                     ah.execute(executionContext);
                     }
                     public void assign(Assignable assignable, ExecutionContext executionContext) throws Exception {
                     AssignmentHandler ah = getBean();
                     ah.assign(assignable, executionContext);
                     }
                     @SuppressWarnings("unchecked")
                     private <T> T getBean() {
                     if (bean == null) {
                     bean = beanFactory.getBean(beanName);
                     }
                     return (T) bean;
                     }
                     public String getBeanName() {
                     return beanName;
                     }
                     public void setBeanName(String beanName) {
                     this.beanName = beanName;
                     }
                    
                    }




                    • 7. Re: Solution: JBPM3 and Spring Integration
                      mr_magoo

                      Finally, the spring config to tie it all together which has hopefully survived sanitation intact. :) You will not the class names of the above classes have been fudged to protect the innocent.


                      Notes:
                      I also do not claim this is the perfect solution. There are probably further optimisations that could be made to reduce the size/complexity.

                      We create our own object factory and define various properties in spring.

                      The default jbpm context has the services injected manually to catch code that ignores the object factory. They are all named THE SAME as they would be in the jbpm config file. This is so our custom object factory will pick them up. You could rename them and put them into the objectfactory's map also.


                      <?xml version="1.0" encoding="UTF-8"?>
                      <beans>
                       <bean id="jbpmConfiguration" class="JbpmConfigurationFactoryBeanImpl">
                       <property name="transactionTemplate" ref="transactionTemplate" />
                       <property name="sessionFactory" ref="jBPMSessionFactory" />
                       <property name="startJobExecutor" value="true" />
                       <property name="objectFactory" ref="springJbpmObjectFactory" />
                       <property name="processDefinitionsResources">
                       <list>
                       <value>classpath:where is my xml???</value>
                       </list>
                       </property>
                       </bean>
                       <bean id="springJbpmObjectFactory" class="SpringObjectFactory">
                       <property name="jbpmProperties">
                       <map>
                       <entry key="jbpm.hide.stale.object.exceptions">
                       <bean class="java.lang.Boolean"><constructor-arg value="true" /></bean>
                       </entry>
                       <entry key="resource.business.calendar" value="org/jbpm/calendar/jbpm.business.calendar.properties" />
                       <entry key="resource.default.modules" value="org/jbpm/graph/def/jbpm.default.modules.properties" />
                       <entry key="resource.converter" value="org/jbpm/db/hibernate/jbpm.converter.properties" />
                       <entry key="resource.action.types" value="org/jbpm/graph/action/action.types.xml" />
                       <entry key="resource.node.types" value="org/jbpm/graph/node/node.types.xml" />
                       <entry key="resource.parsers" value="org/jbpm/jpdl/par/jbpm.parsers.xml" />
                       <entry key="resource.varmapping" value="org/jbpm/context/exe/jbpm.varmapping.xml" />
                       <entry key="resource.mail.templates" value="jbpm.mail.templates.xml" />
                       <entry key="jbpm.mail.smtp.host" value="localhost" />
                       <entry key="jbpm.mail.from.address" value="jbpm@noreply" />
                       <entry key="jbpm.byte.block.size">
                       <bean class="java.lang.Integer"><constructor-arg value="1024" /></bean>
                       </entry>
                       <entry key="jbpm.mail.address.resolver">
                       <bean class="org.jbpm.identity.mail.IdentityAddressResolver"></bean>
                       </entry>
                       <entry key="jbpm.variable.resolver">
                       <bean class="org.jbpm.jpdl.el.impl.JbpmVariableResolver"></bean>
                       </entry>
                       <entry key="jbpm.task.instance.factory">
                       <bean class="org.jbpm.taskmgmt.impl.DefaultTaskInstanceFactoryImpl"></bean>
                       </entry>
                       </map>
                       </property>
                       </bean>
                       <bean id="default.jbpm.context" class="org.jbpm.JbpmContext" scope="prototype">
                       <constructor-arg>
                       <bean class="org.jbpm.svc.Services">
                       <constructor-arg>
                       <map>
                       <entry key="message" value-ref="message" />
                       <entry key="authentication" value-ref="authentication" />
                       <entry key="scheduler" value-ref="scheduler" />
                       <entry key="tx" value-ref="tx" />
                       <entry key="persistence" value-ref="persistence" />
                       </map>
                       </constructor-arg>
                       </bean>
                       </constructor-arg>
                       <constructor-arg ref="springJbpmObjectFactory" />
                       </bean>
                       <bean id="jbpmCommandService" class="org.jbpm.command.impl.CommandServiceImpl">
                       <constructor-arg ref="jbpmConfiguration" />
                       </bean>
                       <bean id="jbpm.job.executor" class="SpringJobExecutor">
                       <property name="transactionTemplate" ref="transactionTemplate" />
                       <property name="jbpmConfiguration" ref="jbpmConfiguration" />
                       <property name="name" value="SpringJbpmJobExecutor" />
                       <property name="nbrOfThreads" value="40" />
                       <property name="idleInterval" value="5000" />
                       <property name="maxIdleInterval" value="1700000" />
                       <property name="historyMaxSize" value="20" />
                       <property name="maxLockTime" value="300000" />
                       <property name="lockMonitorInterval" value="60000" />
                       <property name="lockBufferTime" value="5000" />
                       </bean>
                       <!-- services -->
                       <bean id="persistence" class="org.jbpm.persistence.db.DbPersistenceServiceFactory">
                       <property name="sessionFactory" ref="jBPMSessionFactory" />
                       <property name="transactionEnabled" value="false" />
                       <property name="currentSessionEnabled" value="true" />
                       </bean>
                       <bean id="authentication" class="org.jbpm.security.authentication.DefaultAuthenticationServiceFactory" />
                       <bean id="message" class="org.jbpm.msg.db.DbMessageServiceFactory" />
                       <bean name="tx" class="org.jbpm.tx.TxServiceFactory" />
                       <bean name="scheduler" class="org.jbpm.scheduler.db.DbSchedulerServiceFactory" />
                      </beans>


                      • 8. Re: Solution: JBPM3 and Spring Integration
                        mr_magoo

                        And that is it. Hopefully that helps someone avoid the painful experience I had.

                        Of course everyone is rushing off to jbpm4 now and I wish I could post something similar on that. (or not have to at all!) Unfortunately after a week of frustration it just would not work with JTA/Spring and async processing. I tried every configuration suggestion on the net and nothing worked. (mainly due to the job executor)

                        A shame as jbpm4 is a step in the right direction on all fronts IMHO. I eagerly await when I can make the move myself.

                        • 9. Re: Solution: JBPM3 and Spring Integration
                          jbarrez

                          Thanks for the posts!

                          I added a link to this thread on my blog.

                          • 10. Re: Solution: JBPM3 and Spring Integration
                            jbarrez

                            Btw for reference, your jBPM4 issues are described here: http://www.jorambarrez.be/blog/2009/04/17/integrating-jbpm3-with-spring-part1/

                            It makes me wonder, since the GWT console uses JTA transactions. Can you post your configs here?

                            • 11. Re: Solution: JBPM3 and Spring Integration
                              mmusaji

                               

                              "mr_magoo" wrote:
                              Note that because of the use of the transaction template, hibernate will assume that any staleobject exception coming out of the commit is an error. It will log and throw this exception.
                              Personally I think this is very bad form in a library! :) In a multi-instance environment you will have to manually silence it as these optimistic lock failures are perfectly ok. They are caught and silenced in the core jbpm executor code but of course will not occur because of the delayed commit.

                              e.g.
                              org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.jbpm.job.ExecuteNode
                              Job#140392]
                              at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1782)
                              [...]
                              at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) <== CULPRIT!
                              [...]
                              at org.jbpm.SpringExecutorThread.acquireJobs(SpringExecutorThread.java:33)
                              


                              Please note that the above code does not deal with this problem at all. You will end up with 2 logged errors for every optimistic lock failure in a multi instance environment. The errors should be benign otherwise.


                              This is good to know. I'm not using Spring but I am getting this error when I'm executing multiple jobs in parallel using JBPM 4 and MySQL. I've been seeing all sorts of posts/blogs etc skimming this issue but none explaining the cause or if there is a solution.

                              The problem I am having is my workflow doesn't always complete because of the errors. It seems like it's a little hit and miss and sometimes will carry on through to completion and sometimes not.

                              http://www.jboss.org/index.html?module=bb&op=viewtopic&t=160253

                              Maybe should post in another thread.




                              • 12. Re: Solution: JBPM3 and Spring Integration
                                kukeltje

                                PLEASE post in topics that are ON TOPIC this one is about Spring and jBPM 3, you do not use EITHER

                                • 13. Re: Solution: JBPM3 and Spring Integration
                                  mr_magoo

                                  Hey there,

                                  I wish I could, but I dumped them a few weeks ago once I had JBPM3 all working. Regretting that now.

                                  Also there were about 50 variants. That in itself is wierd as the spring integration docs are only a few paragraphs.

                                  You guys tempted me with promises of a painless spring integration....now I feel so cheated and cheap. ;)

                                  I will give it another hack when I have some time and be sure to keep a log and post then.






                                  • 14. Re: Solution: JBPM3 and Spring Integration
                                    jbarrez

                                    @MrMagoo: there are a lot of people who are using the Spring integration without problems, so don't feel cheap ;-)

                                    Do keep us posted on your progress, I'm interested in seeing in what area we can improve things.

                                    1 2 Previous Next