Reporting Engine Integration

[fady.matar]

the jBPM team is currently implementing a BAM module in the jBPM console. I'm currently trying to get few opinions on what reporting engine to integrate. Currently I'm thinking of three options:
1. BIRT
2. Jasper Reports
3. Pentaho BI

I would like to get the feedback of the community so that we investigate the time in the enhancements instead of comparing frameworks.

Waiting on feedback...

[jeffdelong]

For the BAM work I did in the past I looked at Jasper Reports, but rejected it early on. In the end, we did all reporting by dumping data to Excel.

We should think through the requirements for reports before settling on a solution. Are the reports on historical data or current data? Are there canned reports provided by the product? Is there a requirement for the user to be able to create custom reports? What type of user would specify the custom reports? An Eclipse based report design tool may not be appropriate environment for a business user to specify a report. What type of user would consume the reports? How will the reports be delivered? (screen, spreadsheet, pdf, hardcopy?).

Another criteria for the technology selection is the overall architecture. Is the database access all via Hibernate? If so, is the product providing a set of named queries (that users could augment), or using the Hibernate criteria API? Or will the reporting tool go directly to the database? is the database being designed to support arbitrary queries? Will the reporting UI be integrated into the BAM dashboard UI, or is it totally separate?

[kukeltje]

Pentaho can use both Jasper Reports and BIRT (http://wiki.pentaho.org/display/PentahoDoc/Integrating+BIRT+and+Jasper) , so if we go for integrating Pentaho, customers can choose either solution.

otoh, it is a more complex product compared to 'just' using BIRT or JasperReports embedded. Still I think pentaho people will be interested in helping out, e.g. see http://forums.pentaho.org/showthread.php?t=49927
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3939875

And (a little nitpicking), aren't you talking about BI if you use these tools and not specifically BAM?

[camunda]

I agree, that first the requirements should be somehow collected. Reporting can be a pain in the ass in my experiences...

In the simulation engine I added reports to show simulation results, there I use JasperReports (basiclly due to the fact, that I knew that much better myseld, so main evaluation criteria was Team Know-How ;-)), so this is already in the list of jBPM dependencies (if you want to use simulation reports).

Pentaho is a very cool tool. I like expecially the OLAP stuff (Mondrian and the JPivot browser), you give the business analyst powerful easy to use tools. But the main problem here is to design the underlaying database layout / OLAP Cubes.

If that is the goal of BAM (which I would say IS a part of BI) in jBPM, I would also start with these stuff first, and then select a reporting engine...

About one year ago, we chose Jasper for a project, because support for MDX connections and Cross-Tab-Support was much better in Jasper...

[tom.baeyens]

"camunda" wrote:
I use JasperReports (basiclly due to the fact, that I knew that much better myseld, so main evaluation criteria was Team Know-How ;-)), so this is already in the list of jBPM dependencies (if you want to use simulation reports).

it's in jbpm.3. for the first simple queries we do this might be an argument. when we port it to pvm, all options are open again.

[fady.matar]

Apologies for the late reply and thanks for the feedback. I'll highlight things in here pretty quickly.

BAM Vs BI:
The ultimate question of Business Intelligence Vs Business Activity Monitoring narrows down to this differentiation: BAM addresses real-time data while BI addresses historical data. BAM & BI are focusing on two different business problems; BAM is focusing on a quick response to immediate changing condition affecting the work environment while BI focuses on the analysis of historical data that can help enhancing the future operations. I hope all of us agree on this so that we proceed further with the discussions.

Reporting Framework:
I'm absolutely agnostic to the reporting tool, our selection must make sure that the reporting framework can perform or can be extended to perform the following:
1.Provide an easy for business analysts to design their reports.
2.Provide all business graphs (Pivot tables, clean charts (2D-3D) that can help visualizing trends and differences of values.
3.Provide an approach of easily querying the underlying jBPM database to extract those values.

Reports:
The solution must provide a set of basic reports that are embedded by default. However, we must ensure that the solution provides deployable report-templates into the system. Those report templates must be easily designed and fine-grained at the UI level. (Must be able to decorate / enhance the UI) and should be generated in a number of formats, HTML, CSV, PDF, and Office compatible files.

Dashboards:
the BAM solution is based on dashboards and auto-refreshing AJAX panel components that would auto-update to reflect the real-time data.

The BAM Database
The jBPM database is not optimized for querying and reporting. One alternative would be to create an OLAP-enabled database out of the jBPM database to enable end-users to perform ad hoc analysis of data in multiple dimensions, thereby providing the insight and understanding they need for better decision making. Ideally I would like to use Hibernate to build the BAM queries however I think this must be integrated with the reporting framework, a factor we need to take into account while evaluating the reporting frameworks.

What do you guys say?

[jeffdelong]

Fady,

I agree with almost all of what you said except the BAM vs BI. I think BAM is also interested in historical reporting, and would expect the BAM solution to support this. e.g., What was the average dollar volume of loan origination processes in November 2006 and how does that compare to November 2007 is a reasonable BAM request. This is just BI applied to data that is generated from business processes.

The BAM database tables should be different than the current jBPM tables, for the reasons you noted, as well as to keep historical data (i.e. data about completed process instances). Process instances should be deleted from the jBPM tables once the process is completed to keep the jBPM database performance optimal.

Jeff

[kukeltje]

Jeff,

Your example is (at leat to me) one of BI. A
BAM example is(again, to me):

How many instances of a process are currently running and how big is the chance that with the current number of resources the SLA will not be met.

So I agree with Fady's explanation.

[jeffdelong]

Your example is perfectly valid as well. My point is with the example I gave is that users would like to query about completed process instances. If you say this is not part of a BAM product and our BAM product won't support it, then users will look to other solutions to provide this type of information. So the real question is whether historical data (data about completed process instances) is included in the BAM requirements and will our BAM solution address these requirements?

[fady.matar]

BAM Vs BI, Jasper Vs BIRT. This is all irrelevant for now.
The following has to happen, we need to provide a dynamic reporting engine that can connect to any reporting tool to attract a larger audience. The end-users, business analysts or developers have the opportunity to use their favorite framework.

On another topic, while BAM deals with real-time data and BI with historical data, both of them are provided in the proposed solution. Whatever is real-time now is history in few seconds, so bottom line, the ability to pull out historical data is there.