collecting and processing of JBPM5 BAM data
jbride Aug 1, 2012 4:40 PMHi,
we've recently had a customer request for, among other things, capturing events from a jbpm5 process engine executing in SY to support business activity monitoring.
the purpose of this post is to discuss what this might look like in SY
JPAWorkingMemoryDbLogger
jbpm5 currently ships with the following class to capture and persist audit trail events from the process engine : https://github.com/droolsjbpm/jbpm/blob/master/jbpm-bam/src/main/java/org/jbpm/process/audit/JPAWorkingMemoryDbLogger.java
JPAWorkingMemoryDbLogger persists using the same transaction and JPA persistence-unit that the jbpm5 process engine is operating in.
Although JPAWorkingMemoryDbLogger has been demonstrated to slow down the process engine in performance tests .... the implementation is straight-forward and there are use-cases where it is may be important.
recommend that JPAWorkingMemoryDbLogger be supported in SY and optionally enabled in the switchyard.xml (ie: <bam type="JPA" /> )
AsyncBamProducer / Consumer
in load tests, we have found the jbpm5 process engine to be fast. in environments where the engine is experiencing more load .... it's important to avoid anything that could potentially slow it down.
we have observed that JPAWorkingMemoryDbLogger reduces the speed of the engine by a factor of 2-3.
Subsequently, another possibility for capturing and processing of audit trail events is an async approach where events are sent to a JMS broker optimized for speed (ie: hornetq or MRG-M)
on a different thead (most likely operating on a different jvm on a different OS), a JMS consumer receives the events and persists in batch to a data warehouse
this architecture has the dual benifit of relieving the single greatest bottleneck on the jbpm5 process engine as well as persisting those events in a data warehouse specifically tuned for that purpose.
an implementation of the AsyncBamProducer can be found here: https://github.com/jboss-sso/processFlowProvision/blob/master/common/src/main/java/org/jboss/processFlow/bam/AsyncBAMProducer.java
recommend that AsyncBAMProducer (or something similar) be supported in SY and optionally enabled in switchyard.xml (ie: <bam type="ASYNC" /> )
KnowledgeRuntimeLoggerFactory
somewhat related to business activity monitoring is the KnowledgeRuntimeLogger functionality that drools provides to capture low-level rules engine events.
this is already supported in SY and exposed as a configurable option in switchyard.xml (ie: <audit type="CONSOLE" /> )
this drools knowledgesession logging is useful.
recommend that this functionality remain in SY and that BAM processing options (as mentioned in the above two) be in addition to the existing <audit .... /> config .
thoughts ?
thanks! jeff