ESB Packaging

[slaboure]

Hello,

Mark and I discussed a few things around JBoss ESB in Berlin, and I wanted to make sure to have them written somewhere. I'll probably put each in a different thread to make it easier.

The first item is about the packaging of the various ESB JARs. AFAICS, the 4 main ESB JARs are packaged inside a EAR. Is there a good reason to do that i.e. are we really interested to use an EE-specific packaging? If not, I'd suggest to either drop each JAR in the lib folder directly (the deploy folder would work as well) or, better, package them inside a SAR. That way, once we migrate to the microcontainer, we could simply drop any EAR/WAR/EJB-JAR specific deployer and only keep the service/POJO deployer.

Inside the SAR, it is then possible to not only specify/load them in the jboss-esb.sar/META-INF/jboss-service.xml but also to include various configuration parameters. More on this in another post.

Now the question is: are these four JAR one standalone thing or not? If not, I'd suggest to separate them in as many SARs as needed. This means that it is then possible to deploy/undeploy a specific SAR as needed, depending on which functionality is needed.

Last point, if we are going to SHARE some classes between any of these SARs (like some basic ESB classes, such as the Message interface, etc.) I see several options. First, the simplest, we put that jboss-esb-common.jar (I am making up names on the fly in case you are wondering what these names come from :) ) in the /lib folder of the specific server configuration. That means that we won't be able to dynamically load/unload that code, but I imagine that these classes are very stable as they would represent the lowest common denominator of the ESB. The other option, is to deploy that JAR directly in the /deploy folder and put appropriate tags in the other SARs. Last option: put these classes in the SAR that represents the core set of services without which any other plugin is useless anyway.

Bottom line: I think it is very important to have a fine-grain granularity in our deployments and be able to express between those and not rely on any EE-specific behaviour. Thoughts?

I'll write another thread around the configuration of the ESB soon (have to go for now...)

Cheers,



Sacha

[kconner]

One of the things I would like to see is a move towards a model similar to OSGi, but this is really something for after the GA.

Each component/service should be a separate bundle that could be dynamically loaded/unloaded/configured.

I also think that the ESB should not be tied to the AS as it may be desirable to deploy parts of it in an environment which does not have the necessary resources.

Kev

[slaboure]

Hello Kevin,

For what's needed at this point, the current MK can probably provide you with everything that is necessary for clean redeploys.

Concerning the reliance on the AS, that is pretty obvious, yes. I can provide you with a minimal distro of the AS (or scripts to setup one for the ESB) if needed, that's pretty easy. The current setup is fine to test the ESB behaviour but not as a final distribution mechanism (it is way to overloaded). Same for the minimal/all distributions, we probably don't need them for the OS => the final ESB distrib will most probably be much smaller than the OS one.

Finally, I think it would be a mistake not to properly componentize the ESB in services with proper packaging /configuration for the GA: that would make it much more difficult to setup and understand. Also, many JBoss AS users would find it *trivial* to use the ESB if it had a similar look-n-feel-n-configure. However, if they face an AS distribution with static JARs inside an EAR and JVM-wide configuration files available as System Properties, I bet the experience will not be the same at all. That won't help MarkL's 30 minutes criteria ;)

Cheers,



Sacha

[kconner]

Hiya Sacha.

I agree that the MK can probably provide most of the required functionality but I believe we should be looking further out. We should be aiming the next release at the MC or OSGi frameworks (or even both). I am looking at both of these for a post GA task.

You will not find anyone on the ESB team who disagrees with you about componentising the ESB. This is definitely important, especially if we are to obtain the ability to dynamically configure as much of the functionality as possible.

As for configuration, ESB currently uses the TS property manager. This does not require the configuration file to be JVM-wide nor does it require the properties to be exposed via System properties. While doing this is possible with the TS property code it is usually not done unless some dependent jar requires it. I do not believe any of the ESB properties are exposed via System properties but I may be wrong. If they are then it should be a quick fix.

It should, therefore, be straight forward to repackage the main ear/configuration file into a single exploded sar. Whether these jars can be further componentised at this time is really a question that only Esteban, Daniel, Kurt and Tom can answer.

As far as the dependency on AS is concerned, that comment was really targetted at small form devices (such as mobile phones and PDAs) but also extends to other devices. I strongly believe we should be constructing the ESB using small, self contained plugins which remain as neutral as possible to the deployed environment. These plugins could then be combined as needed and, with an appropriate adapter layer, could then execute using the frameworks installed on the devices.

If we are able to achieve this then I believe we will see the ESB being used in various configurations and devices which we have yet to consider.

Kev

[slaboure]

Kevin,

two questions for you:

1) why do you want to wait for the MC (or an OSGi personality on top of the MC) in order to compenentize the ESB? Is that by lack of time or by lack of features from the MK?

2) from a cofiguration standpoint, why are you relying on a TS specific configuration mechanism rather than on the AS/MK/MC standard mechanism? Per lack of feature of the MK or for some other reason?

Also, while I might be wrong, I do think the current ESB properties are JVM-wide: I think they are being loaded by the System Properties MBean, but I might be wrong.

On the two points above, I think they are pretty important as they really define how new users will discover our ESB and learn how to use/configure it.

[daniel.brum]

To answer some of your questions earlier:

1 - EAR packaging - that will go away very shortly. We only have the EJB notification from the old codebase which will be removed by Tom shortly. I have already removed the old object store mechanism. We essentially only need standalone JARs right now, which can be dropped in the server/lib to make use of the ESB services inside of a J2EE app.

2 - properties - all properties were moved to use the TS property manager a few months ago (since the beta1).

[kconner]

Sacha.

Componetising the ESB codebase at this point is not something I can comment on, this is really a question for the others as they have been responsible for the work.

To date I have been involved very little with the ESB project except for the initial design work that Mark and I undertook at the beginning of the year. Unfortunately TS has kept me very busy this year. :-)

Since the release of TS4.2.2, and with Jonathan's recruitment, I now have time where I can ramp up my involvement with ESB.

The codebase, as it currently stands, is an evolution of the rosetta codebase but it is still evolving. The people involved, both internally and in the community, have done a fantastic job to get it to the position it is in today. We still have a lot of work to do so that we can fulfill our vision of the ESB and I see MC/OSGi as a very important part of that evolution.

Kev

[kurtstam]

As Daniel already said we will soon have 3 jars, and no longer an ear. This codebase does not necessarely need a AS at this point, however it does need an (external) JMS provider. In my opinion we have the following options:

1. deploy the jars to the AS, the ESB is a app on top of the AS
2. start the ESB by itself with a small bootstrapper, the ESB is an app by itself. After GA we will need the MK in there to support all we need on the roadmap.
3. support both ways of running the ESB.

We have been moving towards option 2, the thing left to do is the little bootstrapper and start scripts in my opinion. Historically this is the way rosetta has been running with listeners runnning as 'adapter' on the various systems, talking to one central host. These listeners reread their configuration periodically.

It would be nice if we can agree on a packaging that would get us to option 3 in my option.

--Kurt

[kconner]

Kurt.

The long term goal is to use either MC, OSGi or both (not MK).

In any case the ESB services need to be configurable/constructed by as many frameworks as possible and need to be isolated from whichever one is in control.

The intention is to have integration layers for these frameworks, where needed, in much the same way that TS does with AS.

As for componentising the ESB, I was talking about the possibility of componentising into finer grained deployments than the jars we already have :-).

Kev

[marklittle]

One of the things I would like to see is a move towards a model similar to OSGi, but this is really something for after the GA.

JBoss Microcontainer support is on the roadmap for the next major release.

Each component/service should be a separate bundle that could be dynamically loaded/unloaded/configured.

I also think that the ESB should not be tied to the AS as it may be desirable to deploy parts of it in an environment which does not have the necessary resources.
Kev

Yes, that's always been the plan. The original Alpha release didn't have a dependency on an application server. One came in because of Rosetta. We're moving back to the original approach with the 4.0 release.

[marklittle]

Hello Kevin,

For what's needed at this point, the current MK can probably provide you with everything that is necessary for clean redeploys.

Concerning the reliance on the AS, that is pretty obvious, yes. I can provide you with a minimal distro of the AS (or scripts to setup one for the ESB) if needed, that's pretty easy. The current setup is fine to test the ESB behaviour but not as a final distribution mechanism (it is way to overloaded). Same for the minimal/all distributions, we probably don't need them for the OS => the final ESB distrib will most probably be much smaller than the OS one.

As discussed at Berlin, we're moving away from reliance on any AS for the running of the core ESB. Obviously if you want to plug in an infrastructural component/service that requires an application server, then that's a deployment choice. However, at the core level, there'll be no dependency. So, for example, if I want to tie together my clients and services via UDP, I'll not need anything more than the jars we ship.

Kurt is going to produce some "wizards" to improve deployment of the 4.0 ESB. Last time he and I spoke about this, one of those wizards is supposed to be "how to deploy on JBossAS", so this discussion is definitely still useful.

Finally, I think it would be a mistake not to properly componentize the ESB in services with proper packaging /configuration for the GA: that would make it much more difficult to setup and understand.

+1

Part of the wizards' job.

Also, many JBoss AS users would find it *trivial* to use the ESB if it had a similar look-n-feel-n-configure. However, if they face an AS distribution with static JARs inside an EAR and JVM-wide configuration files available as System Properties, I bet the experience will not be the same at all. That won't help MarkL's 30 minutes criteria ;)

Cheers,

Sacha

I agree, which is why we're looking at a few wizards. Maybe Kurt could start a separate forum entry with his ideas on the initial wizards for the 4.0 release?

[marklittle]

As for configuration, ESB currently uses the TS property manager. This does not require the configuration file to be JVM-wide nor does it require the properties to be exposed via System properties. While doing this is possible with the TS property code it is usually not done unless some dependent jar requires it. I do not believe any of the ESB properties are exposed via System properties but I may be wrong. If they are then it should be a quick fix.

I hope we've nuked all of the system properties by now.

It should, therefore, be straight forward to repackage the main ear/configuration file into a single exploded sar. Whether these jars can be further componentised at this time is really a question that only Esteban, Daniel, Kurt and Tom can answer.

As far as the dependency on AS is concerned, that comment was really targetted at small form devices (such as mobile phones and PDAs) but also extends to other devices. I strongly believe we should be constructing the ESB using small, self contained plugins which remain as neutral as possible to the deployed environment. These plugins could then be combined as needed and, with an appropriate adapter layer, could then execute using the frameworks installed on the devices.

+1

[marklittle]

Kevin,

two questions for you:

1) why do you want to wait for the MC (or an OSGi personality on top of the MC) in order to compenentize the ESB? Is that by lack of time or by lack of features from the MK?

Time. Too much to do for the 4.0 release so we agreed to drop a couple of things until the next release.

2) from a cofiguration standpoint, why are you relying on a TS specific configuration mechanism rather than on the AS/MK/MC standard mechanism? Per lack of feature of the MK or for some other reason?

See above. How the current configuration plays in an MC environment is up for grabs in the next release. But at the moment, the configuration files and PropertyManager are far superior to what was there in the beta release IMO.

Also, while I might be wrong, I do think the current ESB properties are JVM-wide: I think they are being loaded by the System Properties MBean, but I might be wrong.

On the two points above, I think they are pretty important as they really define how new users will discover our ESB and learn how to use/configure it.

Sacha, you may be looking at the Beta release. That's different to the GA/trunk, which is what Kevin has been discussing.

[marklittle]

Kurt.

The long term goal is to use either MC, OSGi or both (not MK).

In any case the ESB services need to be configurable/constructed by as many frameworks as possible and need to be isolated from whichever one is in control.

The intention is to have integration layers for these frameworks, where needed, in much the same way that TS does with AS.

As for componentising the ESB, I was talking about the possibility of componentising into finer grained deployments than the jars we already have :-).

Kev

I can't remember who I was talking to in Berlin, but the image I have is of one of my 4 year old son's toys: it's a jigsaw but the pieces come held into place by a foam wrapper. When you pull the pieces out you're left with a square within which is the outline of where the pieces go (hey, it's for 4 year olds!) If you consider the wrapper as the core ESB and the individual pieces as services, then that's where I'd like us to be heading: the core is pretty thin but critical to binding everything together; the individual services are implementations the core (or applications) need and they can all be separately deployed (or not) depending on your requirements.

Maybe it loses something in translation ;-)

[marklittle]

As Daniel already said we will soon have 3 jars, and no longer an ear. This codebase does not necessarely need a AS at this point, however it does need an (external) JMS provider. In my opinion we have the following options:

1. deploy the jars to the AS, the ESB is a app on top of the AS
2. start the ESB by itself with a small bootstrapper, the ESB is an app by itself. After GA we will need the MK in there to support all we need on the roadmap.
3. support both ways of running the ESB.

We have been moving towards option 2, the thing left to do is the little bootstrapper and start scripts in my opinion. Historically this is the way rosetta has been running with listeners runnning as 'adapter' on the various systems, talking to one central host. These listeners reread their configuration periodically.

It would be nice if we can agree on a packaging that would get us to option 3 in my option.

--Kurt

There are two separate issues here:

(i) how to package and configure the 4.0 GA. We've agreed that we need some wizards to help with this. I think there are at least two requirements/wizards here: how to run the ESB within an application server and how to run it stand-alone. If we can simplify the out-of-the-box experience for both of these for the GA then I think we'll satisfy many/most users of the GA.

(ii) how to package and configure the 5.0 GA (next release). As this discussion as pointed out, things will change when we support MC. Furthermore, feedback from (i), both from implementing the wizards and eventually/hopefully from users, will influence this.

Although we can/should definitely continue to discuss (ii), it can't be at the detriment of (i).