Pluggable Context project for PVM

[salaboy21]

The main idea is to build a new project that create a new Pluggable Context (something like jbpmContext but POJO oriented) for PVM, one of the things that can be plugged in this context may be a JPA session for persistence, a JobExecutor, etc.

The POJO oriented design will give the project the capabilities of easy configuration (Spring like) and flexibility (if we use JPA to persist the process context and definition in diferent ways, like Relational Database, XMLs , Text Files, or whatever that implements JPA spec)

I wanna know if somone have some ideas about that, or if someone are already doing something like this for PVM..

[tom.baeyens]

it's already in pvm.

it's called environment. there is a default implementation that can be configured like this:

<?xml version="1.0" encoding="UTF-8"?>

<contexts xmlns="http://jbpm.org/pvm/1.0/wire">

 <environment-factory>
 <pvm-service />

 <job-test-helper />

 <standard-command-service>
 <retry-interceptor />
 <environment-interceptor />
 <transaction-interceptor />
 </standard-command-service>

 <deployer-manager resource="pvm.language.deployers.xml">
 <language name="api">
 <verify-version />
 <save-process />
 </language>
 </deployer-manager>

 <hibernate-configuration>
 <properties resource="hibernate.properties" />
 <mappings resource="org/jbpm/pvm/pvm.hibernate.mappings.xml" />
 <cache-configuration resource="org/jbpm/pvm/pvm.cache.xml" usage="nonstrict-read-write" />
 </hibernate-configuration>
 <hibernate-session-factory />

 <variable-types resource="org/jbpm/pvm/pvm.types.xml" />

 <business-calendar>
 <monday hours="9:00-12:00 and 12:30-17:00"/>
 <tuesday hours="9:00-12:00 and 12:30-17:00"/>
 <wednesday hours="9:00-12:00 and 12:30-17:00"/>
 <thursday hours="9:00-12:00 and 12:30-17:00"/>
 <friday hours="9:00-12:00 and 12:30-17:00"/>
 <holiday period="01/07/2008 - 31/08/2008"/>
 </business-calendar>

 </environment-factory>

 <environment>
 <transaction />
 <hibernate-session />
 <pvm-db-session />
 <message-session target="test" />
 <timer-session target="test" />
 </environment>

</contexts>

There is a possibility to plug in other implementations like Spring. But I didn't yet get to implementing that. The skeleton class is present already, though. It's

org.jbpm.pvm.internal.env.SpringEnvironmentFactory

The main thing to do first, I think is to identify what most common ways are for Spring users to manage hibernate sessions and transactions.

Great to see you start the dicsussion here !

[camunda]

Seeing this reminds me of an additional proposal, not sure if it is already on the agenda or maybe even implemented already:

Would be nice if the BusinessCalender is also really pluggable (not only configurable) and gets more information in the interface than in jbpm 3, so it is easier to support shift calender, applications in different time zones and so on...

Cheers
Bernd

P.S: Sorry for putting that in this thread ;-)

[salaboy21]

I see hibernate mappings in the environment configuration.. JPA is out of discussion?
I think it would be nice use annotations for mappings and use JPA to make the persistence more flexible and extensible..

[tom.baeyens]

"salaboy21" wrote:
I see hibernate mappings in the environment configuration.. JPA is out of discussion?

at some point we might switch to JPA. depending on the need for hibernate features that are outside the scope of JPA.

* programmatic building of the configuration
* on load interceptors (no need for that atm)

i think there were a couple others that i don't remember by heart. at the time we made the decision, those were a showstopper for me. i don't see a big problem, but it will take some time and research to see how this can be done in a portable way.

the big problem is not getting it to work with JPA. But to test it on all the different environments, app servers and implementations.

By using hibernate, we circumvent that problem a bit. Cause we can ship the exact version of the ORM implementation with the PVM project. That way we limit the number of QA that needs to be done.

"salaboy21" wrote:

I think it would be nice use annotations for mappings and use JPA to make the persistence more flexible and extensible..

I think a resource file is more flexible then an annotation cause an annotation requires recompilation and a resource file doesn't.

[tom.baeyens]

"camunda" wrote:
Seeing this reminds me of an additional proposal, not sure if it is already on the agenda or maybe even implemented already:

Would be nice if the BusinessCalender is also really pluggable (not only configurable)

That's on the TODO. Makes a lot of sense IMO. I've just finalized the XML based configuration of the business calendar. And I had the same thought. We should extract the interface:

public interface BusinessCalendar {

 Date add(Date date, Duration duration);

}

then it should be possible to plug different implementations.

[camunda]

Basically the interface should maybe even be a bit richer. Sometimes you have some requirements like "2 business days before shipping date", so having the ExecutionContext available would be a great advantage in order to implement your own business calender.

[mvaldes]

Joining the JPA vs Hibernate discussion...

We had this discussion some month ago when we starting with persistence on the PVM and the conclusion as that the current version of JPA has a lot of limitations and constraints compared with hibernate so we decided to start with hibernate.

We are following the JPA discussions and progress and most of those limitations will be fixed in the next version of the specification... at that moment would be great to add JPA support.

regards,
Miguel Valdes

[salaboy21]

In this moment I am reading a paper written by Tom and Miguel in may (2007).. were you mention this need to use some persistence layer that give a nice flexibility to store the process where you want... and be a decoupled for the process virtual machine..

For this reason I think the environment of PVM a separate project and be more POJO oriented so you can configure it (without all the homebrew XML parsing) in a Spring like manner. That give us the great possibility to inject all the context information in a non intrusive way.

Tell me if I am wrong but i think this is the right time to clear the PVM responsibilities and make multiple project focused in this kind of topics..

[dlopezleon]

Hi all, I'll try to make my contribution to this thread.
I see a few lacks in the way that configuration is handled today.
I think that you have in mind that every configuration injection comes from a parser, and that's not totally true. I think that having a robust programmatic configuration is great and gives us a good target to follow. In this topic becames Spring. Spring helps us to think in this pojo-oriented way. Even if you want some 'proprietary' configuration syntax, there exists a really easy manner to achieve this, using the xsd extensible configuration that they provide[1]. At this moment I think that spending time parsing and/or having static methods to call "my default" implementation of parser is not a good approach.
Going on with the pluggability stuff, I know that you have discussed before about hibernate vs. jpa but I have to agree with salaboy when he quotes your original paper[2] where you talk about the totally decoupled persistence layer. I think that discussing this point is the big deal. After that making any connector should be just an specific solution. On this topic I found that almost all the persistence thing in java is just relational db oriented. Somewhere is some guy arguing about making Jackrabbit[3] implement JPA[4] but there are not more than good feelings. To put the things a little more clear, just having a JPA layer doesn't solves the big problem at all, and beyond the generic 'P' in the JPA acronym they have @Column or @Table in their specification which clearly are coming from DB-world.
One architecture that I've found supporting multiple persistence stores is in grafitto project[5] but they don't use relational dbs but JCR, WebDav, etc. I think that we have a great open discussion on this topic.

Maybe not for this thread, but there is still some mess in the pvm project... I know that you are in a great refactoring time, and probably you have this in mind, but packages like org.jbpm.hibernate (and other thing related to persistence i.e.) are going to be put in a separates project? And now going totally away from the scope of this this discussion... what about 'mavenize' the projects?

[1]: http://static.springframework.org/spring/docs/2.0.x/reference/extensible-xml.html
[2]: http://docs.jboss.com/jbpm/pvm/article/
[3]: http://jackrabbit.apache.org/
[4]: http://rollerweblogger.org/roller/entry/apachecon_us_2007_wrapup look at Roller the Java Content Repository (JCR) API
[5]: http://incubator.apache.org/graffito/architecture.html

[camunda]

As long as the persistence is pluggable, I think it is still a good approach to have a tested, working out-of-the-box persistence implementation. This is hibernate in case of jBPM, so I think it is perfectly OK to have a package like "org.jbpm.hibernate", splitting that into different projects make the build process more complicated and doesn't deliver a real value...

If you change to JPA or whatever, you don't need hibernate anyway...

[tom.baeyens]

"salaboy21" wrote:
In this moment I am reading a paper written by Tom and Miguel in may (2007).. were you mention this need to use some persistence layer that give a nice flexibility to store the process where you want... and be a decoupled for the process virtual machine..

For this reason I think the environment of PVM a separate project and be more POJO oriented so you can configure it (without all the homebrew XML parsing) in a Spring like manner. That give us the great possibility to inject all the context information in a non intrusive way.

Tell me if I am wrong but i think this is the right time to clear the PVM responsibilities and make multiple project focused in this kind of topics..

do you realise how complex it is to store a model as complex as pvm process definitions and executions in multiple data stores ?

the current model was designed with hibernate type of ORM in mind. it had a big impact on the codebase as i want the database schema to be maintainable.

we should target only 1 persistence solution and that should be included in the PVM. It's got to be hibernate or JPA. Apart from that I also want java serialization to be working and optionally we could think of XML serialization.

it's all nice to see that in theory persistence should be separated. in practice, it always has an implication and it is a complex task. hibernate was also chosen because it made that task simple and feasible with the team we have.

supporting multiple transactional persistence technologies is going to be too complex for our team. I don't think we can handle that atm. Unless we find 3 full time qualified people that want to spend their time on this.

these are just my thoughts... not necessarily the thruth. so keep the discussion going.

[tom.baeyens]

salaboy21,

could you also point out what the goal would be of adding multiple persistence implementations ?

[tom.baeyens]

"dlopezleon" wrote:
Hi all, I'll try to make my contribution to this thread.
I see a few lacks in the way that configuration is handled today.
I think that you have in mind that every configuration injection comes from a parser, and that's not totally true.

can you point out those lacks a bit more concrete ? afaict, i didn't start from the assumption that all configuration injection comes from a parser.

"dlopezleon" wrote:
I think that having a robust programmatic configuration is great and gives us a good target to follow.

i agree programmatic configuration is great. afaict, that is feasible, though not very practical. it's usually easier to specify in an xml file like spring or our own wire format.

"dlopezleon" wrote:

In this topic becames Spring. Spring helps us to think in this pojo-oriented way. Even if you want some 'proprietary' configuration syntax, there exists a really easy manner to achieve this, using the xsd extensible configuration that they provide[1]. At this moment I think that spending time parsing and/or having static methods to call "my default" implementation of parser is not a good approach.

configuration is not the only usage of wire descriptors. a process includes references to user code. those objects need to be instantiated and wired together when the process executes. the wire descriptors are then part of the process definition and need to be stored in the db with the process. that's one of the main reasons why i opted for my own wiring as the default. to have full control over how the descriptors get stored in the db. if the descriptors are not designed for persistence with an ORM solution (like in spring or the jboss MC) then this is almost impossible to achieve in a manageable database schema.

"dlopezleon" wrote:

Going on with the pluggability stuff, I know that you have discussed before about hibernate vs. jpa but I have to agree with salaboy when he quotes your original paper[2] where you talk about the totally decoupled persistence layer. I think that discussing this point is the big deal.

me too. making a mistake in that matter would have a huge impact. that's why i like to hear about other people's opinion and discussions like this one.

"dlopezleon" wrote:

After that making any connector should be just an specific solution.

realising that is not so easy. e.g. with hibernate or JPA, you get lazy loading. this is something that enables us to write the full inner workings of the engine without references to a persistence service. this way, the PVM can also be used without a persistence service.

i don't see us being capable of working out multiple persistence solutions and i certainly not someone from the community as they usually can't work full time on it.

the risk is that we end up with 3 half baked persistence solutions of which people don't know which one to choose. i prefer 1 solid persistence tech that has more QA.

"dlopezleon" wrote:
One architecture that I've found supporting multiple persistence stores is in grafitto project[5] but they don't use relational dbs but JCR, WebDav, etc. I think that we have a great open discussion on this topic.

With JCR i can't write the engine code without a reference to JCR. With hibernate i can just navigate the object model. With JCR, I would have to go through a session facade to get the related information.

"dlopezleon" wrote:
Maybe not for this thread, but there is still some mess in the pvm project... I know that you are in a great refactoring time, and probably you have this in mind, but packages like org.jbpm.hibernate (and other thing related to persistence i.e.) are going to be put in a separates project? And now going totally away from the scope of this this discussion... what about 'mavenize' the projects?

my guess is that pvm will be used 80% in a persistent context. providing two separate projects that the user has to glue together is not going to be user friendly. i want to ship 1 default, solid, QA-ed persistence out of the box.

if people want to start a separate project to persist pvm through other technologies, i don't have a problem with that. in fact, i would encourage it if I think the quality of the work is good enough. if the impact is not too big, i'll definitely support such work by making sure the necessary changes in the pvm model itself.

is there any other things that you consider a 'mess' ?

[mvaldes]

my guess is that pvm will be used 80% in a persistent context. providing two separate projects that the user has to glue together is not going to be user friendly. i want to ship 1 default, solid, QA-ed persistence out of the box.

I follow Tom on this topic. Configuration and pluggability is great but regarding the persistence service the main priority for us is to guarantees that the default implementation is stable and robust. A lot of QA effort will be dedicated on this topic around the hibernate based service.

if people want to start a separate project to persist pvm through other technologies, i don't have a problem with that. in fact, i would encourage it if I think the quality of the work is good enough. if the impact is not too big, i'll definitely support such work by making sure the necessary changes in the pvm model itself.

Adding support for JPA is feasible because the next version of the specification will be really close to Hibernate... however we should check what would be the impact on the relational database mapping and the PVM XML configuration.

For others persistence technologies such JCR this really requires involvement of the community. At Jboss and Bull we are confortable while using hibernate while we have a good skills on this framework. JCR is completely another world that requires key people working on that.

This is a scoop, as not yet official, but the eXo Platform guys (responsibles for eXo JCR implementation) will probably join to add JCR support. At that time some issues will probably raise and some improvements will be required in the PVM persistence service... as Tom said we are open for improvements.

regards,
Miguel Valdes