Status of web console enhancements

[dmlloyd]

Right now I'm working on the new search mechanism for the web application. There are some items in the requirements (all of: the original, Tom's reworked one, and my reworked one on top of that) that I'm not 100% sure about, even now, so I'm still kinda thinking about those things. But the main recurring factor is that we need a good search/navigation system, and that's what I have in progress right now. A search mechanism is specified not only by the console requirements, but also the BAM application (which seems to overlap in functionality with the web console to a degree that remains to be seen).

The idea is to have a search mechanism that is very reusable. In this way, I hope to solve the problem with having multiple, similar searches by reducing the amount of effort required to create and maintain the different searches.

As far as implementation, I've come to the conclusion that the only real practical alternative is to use criteria queries to specify first the search results, and then add on any specified filter criteria. To attempt to maintain named queries for every kind of search would be hugely complicated.

Next will be to start integrating the search page into the pages of the web application, which should be the easy part, if I've done my job right.

[tom.baeyens]

But the main recurring factor is that we need a good search/navigation system, and that's what I have in progress right now.

once you have initial ideas, post them here for review.

A search mechanism is specified not only by the console requirements, but also the BAM application (which seems to overlap in functionality with the web console to a degree that remains to be seen).

as you say onward, hibernate's criteria api is the way to go. so the thing that i would like to see sketched out is the navigation between search pages, results and other functionality.

I've come to the conclusion that the only real practical alternative is to use criteria queries to specify first the search results, and then add on any specified filter criteria.

i absolutely agree


I get the impression that you are thinking too generic in this first iteration. I think the best approach is just hack 2 or 3 queries together ad hoc and get them in the web console, just to get a feel of the web console in general. No reusability in mind. Consider this a prototype.

Then we should design the BAM DB (shouldn't we call this a Business Intelligence DB instead ?) and build the generic query framework on top of that.

For the design of the BAM DB, i want to work this out in the jbpm 4 prototyping module: lightning. Once you're ready i can help you on your way in there.

Thoughts ?

[kukeltje]

BI ('offline')is more an ofline datawarehouse kind of database. Key Performance Indicators high, low levels that result from this are to be used by BAM ('real time') to show a kind of dashboard with a high, low and current level. So the BAM and BI database are (imo) two different ones.

[kukeltje]

meaning BAM could (should?) be done on the normal database and a BI DB should be a separate one (for load reasons).

[dmlloyd]

But the main recurring factor is that we need a good search/navigation system, and that's what I have in progress right now.

once you have initial ideas, post them here for review.

There's not a lot to it: there are filter fields, and a results pane. Entering data into the filter fields will modify the backing query. You can switch what you're searching for with a dropdown menu. The results are paginated with 10, 25, or 50 results per page.

Depending on what you search for, there's a drilldown link (or links) to navigate to e.g. process definitions or tasks.

I get the impression that you are thinking too generic in this first iteration. I think the best approach is just hack 2 or 3 queries together ad hoc and get them in the web console, just to get a feel of the web console in general. No reusability in mind. Consider this a prototype.

I'd say that this is what I've already done with the "average time per node" query. And as far as being too generic... well, I have to implement something for real at some point, right? In any case, I am just focusing on one or two queries, and I don't have a final mechanism to make it truily "generic" just yet, so right now it is sort of halfway between final and prototype.

Then we should design the BAM DB (shouldn't we call this a Business Intelligence DB instead ?) and build the generic query framework on top of that.

We need a generic query framework even for normal navigation of the runtime state in my opinion. I think it would be great to use the same framework in both places. The problem with the existing screen is, if you have more than a few tasks in each list, and a large number of process definitions, the screen becomes a bit cumbersome. And, there's no way to view the status of anything really, which is a function that is needed not only for BAM, but also as more of an administrative function to see if things are actually working.

"kukeltje" wrote:
BAM could (should?) be done on the normal database and a BI DB should be a separate one (for load reasons).

I agree that BI is separate from BAM. Another important thing to keep in mind is that, by our current requirements, BAM really encompasses monitoring what is currently happening, as well as the historical information. So what we're calling the "BAM Database" is probably better referred to as the "Historical Database" or something like that.

[tom.baeyens]

kukeltje, my idea with BAM vs BI is the following:

after an execution tx that inserted records in the BAM db, those BAM records are immediately convered to the BI database. it should be possible to do this:

1) in the runtime transaction itself
2) asynchronously through messaging.

the transformation to the BI db will assemble all the different logs into relevant info.

then, the runtime log db will always remain small as the BAM records are max temprorary.

the runtime will have the BAM db available for e.g. undo and other features that require the BI db.

it should be very well documented/managed which features depend on the BI db, as it will also be configurable, which logs are being produced.

we're talking next generation here and i think this covers best of both worlds.

also, i'm thinking to generate a large part of the logs by leveraging hibernate's capabilities for dirty flushing. by using a hibernate interceptor, we can let hibernate calculate the diffs in one tx. that can be stored as a log record. basically each version of an object will be stored. apart from these, other logs will be generated as well.

does that make sense to you.

[tom.baeyens]

i had some kids pulling my sleeve while typing the previous message. just to stress the brainstorm nature and non-decicive character of the previous message, this is what i intended:

my current idea with BAM vs BI is the following:

does that make sense to you.

[tom.baeyens]

now i just messed up again :-( one last try:

i had some kids pulling my sleeve while typing the previous message. just to stress the brainstorm nature and non-decicive character of the previous message, this is what i intended:

my current idea with BAM vs BI is the following:
...
does that make sense to you ?

[cocampo]

Tom, Roland:

What happened with integrating JBoss jBPM and Pentaho's BI Platform???.

Are you guys planning to develop BAM functionality from scratch?.

Regards.

[tom.baeyens]

It turned out that pentaho was interested in developing a new process language on top of jbpm for specifying transformations in an ETL type of environment.

Not sure what their status is on that.

We are indeed going to develop a dedicated BAM functionality ourselves.