handling bulk amounts of processinstances; transaction probl
macd Feb 5, 2008 10:41 AMCan anybody give some advice?
I am trying to figure out if and how jBPM can handle large amounts of processinstances at once.
The scenario is this: a collection of domain-objects (possible 100.000) has to be processed.
All elements may need the same update, but it may also be possible to deal with each element individually;
The 1st approach i tried was to create 100.000 processinstances individually. This costs lots of time: almost 4 hours, which is not acceptible.
So i tried some different approaches:
1. use batches when closing a jbpmContext to create and signall instances
2. let a process create 10.000 subprocesses (foreachfork)
3. use foreachfork for each element of the collection
4. tried those last 2 with an asynchronous processstep
This last method ensured that each fork was scheduled as a job
One problem occurred every time: when upsizing the collection from 100 to 10.000 instances, forks or subprocesses, i recieve an errormessage concerning a transaction:
- transaction not active, cannot open connection
- transaction closed, cannot commit
i searched the forums an found that this is probably caused by a transaction-timeout, which you happily can change.
http://wiki.jboss.org/wiki/Wiki.jsp?page=TransactionTimeout
i think this essentially doesn't fix my problems, cause i can't be upsizing my timeout to eternity. Besides it puzzles me that even the job-executor gets these exceptions.
So my guess is that i should change the config. I know there are some options left, but i don't know what is wise. Can anybody give me some advice?
thanx, Marc
my current config
JBOSS 4.2.2 GA
JBPM 3.2.2
Oracle 9
<jbpm-context> <service name="persistence"> <factory> <bean class="org.jbpm.persistence.db.DbPersistenceServiceFactory"> <field name="isTransactionEnabled"><false /></field> <field name="isCurrentSessionEnabled"><true /></field> </bean> </factory> </service> <service name="tx" factory="org.jbpm.tx.TxServiceFactory" /> <service name="message" factory="org.jbpm.msg.db.DbMessageServiceFactory" /> <service name="scheduler" factory="org.jbpm.scheduler.db.DbSchedulerServiceFactory" /> <!--service name="logging" factory="org.jbpm.logging.db.DbLoggingServiceFactory" /--> <service name="authentication" factory="org.jbpm.security.authentication.DefaultAuthenticationServiceFactory" /> ... <bean name="jbpm.job.executor" class="org.jbpm.job.executor.JobExecutor"> <field name="jbpmConfiguration"><ref bean="jbpmConfiguration" /></field> <field name="name"><string value="JbpmJobExector" /></field> <field name="nbrOfThreads"><int value="50" /></field> <field name="idleInterval"><int value="5000" /></field> <field name="maxIdleInterval"><int value="3600000" /></field> <!-- 1 hour --> <field name="historyMaxSize"><int value="20" /></field> <field name="maxLockTime"><int value="600000" /></field> <!-- 10 minutes --> <field name="lockMonitorInterval"><int value="60000" /></field> <!-- 1 minute --> <field name="lockBufferTime"><int value="5000" /></field> <!-- 5 seconds --> </bean> </jbpm-context> <hibernate-configuration> <session-factory> <property name="hibernate.session_factory_name">JbpmHibernateSessionFactory</property> <property name="hibernate.connection.autocommit">false</property> <property name="hibernate.jndi.class">org.jnp.interfaces.NamingContextFactory</property> <property name="hibernate.dialect">org.hibernate.dialect.Oracle9iDialect</property> <property name="hibernate.show_sql">false</property> <property name="hibernate.format_sql">false</property> <property name="hibernate.use_sql_comments">false</property> <property name="hibernate.cache.provider_class">org.hibernate.cache.HashtableCacheProvider</property> <!-- org.hibernate.cache.EhCacheProvider --> <property name="hibernate.connection.datasource">java:/JbpmDS</property> <property name="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</property> <property name="hibernate.transaction.manager_lookup_class">org.hibernate.transaction.JBossTransactionManagerLookup</property> <property name="jta.UserTransaction">java:comp/UserTransaction</property> ... ... </hibernate-configuration>