ProfileService on AS6

[clebert.suconic]

This is about the work being done at https://svn.jboss.org/repos/jbossas/branches/Branch_Hornet_Temporary_2/

I have no clue what profile service is about...

and I have a few tests failing on the profileservice test target on my branch. I have currently removed the call to profiletests from the main "tests" target on testsuite/build.xml

Apparently the tests are still referring the old JBoss messaging, even though I removed all the references to JBM on default and all.

Since I don't know what profile-service is about (I have been away from AS codebase since AS4), I have no clue how to fix/attack this.

It's probably not a big issue and easy to fix... but if Alexey Loubyansky and Emanuel Muckenhuber could give us some light here.

thanks

Clebert

[emuckenhuber]

{quote}
 Since I don't know what profile-service is about (I have been away from AS codebase since AS4), I have no clue how to fix/attack this.
{quote}

ProfileService has basically two responsibilties. First it controls the deployment process, triggering deploy/undeploy (hotdeployment as well) where the actual deployment is obviously done by the VDF. The second part is that it provides a management API, where ManagedObjects are based on the deployment attachments (e.g. JMSConfiguration) - which is the part we need to look at. The profileservice tests you mentioned are testing this management interfaces and are most likely failing because this integration does not exist anymore.

Basically how exposing ManagedObjects works is, when you have a ParsingDeployer parsing attachments - ProfileService uses those attachments to build managed objects based on some (management) annotations. Additionally what we usually do is to merge 2 managed objects - the one based on the attachment (used for configuration) and a runtime one (used to expose metrics/statistics).

I've committed a prototype to illustrate that a bit - everything ending with *MO in the "org.jboss.as.integration.hornetq.management.jms" is a template which is used when creating the managed object. I also made HornetQJMSParserDeployer implement ManagedObjectCreator to override the default managed object processing.

So while doing this prototype i noticed a couple of things i might just mention:
- usually deployment descriptors are annotated, in this case i had to basically use a indirection layer to somehow overlay the annotations - so that processing and retreiving values works properly.
- the merging i mentioned above - should (as far as i could see) merge JMS*Configuration and JMS*Control. The problem with this is that the destinal control MBeans are registered directly using the MBeanServer - where i had to register a InvocationContext in MC, so that value dispatching works from the ProfileService side. This is currently done in the Deployment beans you install.

Most of the configuration properties for queues/topics/... seem to be in the central HornetQ configuration now. So it would make sense to look into the integration of this configuration as well. At least making this managable through ProfileService would be needed for AS7 - where we need to see what needs to be done and how much actually need to be (and can be) done for AS6.

Maybe you can briefly look at that and see if you have further questions or concerns?

Thanks,
Emanuel

[clebert.suconic]

Why do we need another set of Objects to manage the same thing?

We have already rewrote our deployers with dependency injection...  then Andy worked on the management console integration.. so he provided a set of POJOs that could be called directly from management.

And now there is this Profile manager?  I'm just confused why we need to provide so many management interfaces for different places and they would all do the same thing.

If you could come on our IRC channel and have a chat about this, please? (we could feed this thread with the chat logs right after, to keep it registered for future reference)

[clebert.suconic]

Why don't we start over?

@Emmanuel, @Alexy and @Jason Greene: lets forget everything you know about Messaging Services and lets start over.

Queues are not really services. A Queue or a Topic is something managed by a Service...  the messaging server (some people call it message broker).

Each Messaging Server will have its own ways of managing destinations. JBoss MQ and JBoss Messaging 1.4 used the Services mechanism from JBoss 4. Other providers will have different concepts.

Requiring us to have a service for each Destination would be the same as requiring Oracle, or MySQL to have a Service definition for each table in order for being able to work on the application server. The same way a table is managed by the Database Server, The queue is managed by the Messaging Server.

It's a paradigm we wanted to change... Queues and topics are not services. That's just confusing. It only makes sense to you guys because you guys have been used to this concept for the past 4 years. So, that's why I say lets start fresh.

What I really believe the JON integration needs is a SPI class such as:

interface MessagingProviderSPI

{

     List<MessagingDestinationSPI> getTopics();

     List<MessagingDestinationSPI> getQueues();

     List<ConnectionFactorySPI> getConnectionFactories();

     void createCF(...);

     void createTopic(MessagingDestinationSPI destinatino);

     void deleteTopic(MessagingdestinationSPI destination);

     ....

}

interface MessagingDestinationSPI

{

     java.util.Properties<String, String> getProperties();

     void setProperties(java.util.Properties<String, String> properties>

}

You can add maybe methods to supported meta-data on the MessagingSPI.

and the MessagingSPI could then be a managed Object, or something like that.

It would be possible then to provide a MessagingSPI on any messaging provider out there. (Including JBoss MQ and JBoss Messaging, allthough I don't think it should be done.. I'm saying just theorically).

It would be up to the MessagingProviderSPI how to support persisting this. This way you don't need to worry about XMLs or anything.

Also, I don't see a requirement to support @depends annotation through the Messaging Service. The AS deployers we wrote are currently supporting that, but that goes beyond the scope of the console manager integration.

That's what I believe it should be done.

BTW: don't take the signatures on MessagingProviderSPI here too seriously.. this is just a basic idea on what I think it should be done.

We should have a meeting about this asap.

[jason.greene]

[e-mail] wrote:

I don't see a requirement to support @depends annotation through the Messaging Service. The as deployers we wrote are currently supporting that, but that goes beyond the scope of the console manager integration.

The whole reason for Queues/Destinations to be a "service" is so that you can have dependencies. If an application writes to a queue when it is deployed, and that queue is not there yet (but soon will be) the AS can hold off starting it until the queue is available. This is definitely a very useful feature and it is expected once you start allowing queues/topics to be hot deployed.

The AS management of all core EE services needs to be consistent via all of its interfaces (as required by Andiamo, and especially with the domain model in AS7). The only way this is possible is if all interfaces go through the same management APIs. In this example the creation of the queue should have the same affect via the gui as it would as dropping down a deployment descriptor.

Now you could definitely introduce an SPI like the above that would be in between the JMS integration layer and Hornet, but it would really just be an extra wrapper around hornetq APIs. If we did this extra work we would actually want to make sure the SPI actually does work with other JMS providers, and they would actually have an interest in integrating.

[dmlloyd]

Also remember that having managed queues and topics doesn't necessarily preclude programmatic creation of queues and topics!  It just means that people can program declaratively if they wish to do so.

Also, don't overstate the difficulty of allowing the declarative approach at this more granular level.  It really is easy to do.  Even if there was an SPI exactly like you describe, one would only have to instruct the MC to use the SPI of the provider to create the queues and topics.  Don't invent difficulty that isn't there.

[clebert.suconic]

The whole reason for Queues/Destinations to be a "service" is so that you can have dependencies. If an application writes to a queue when it is deployed, and that queue is not there yet (but soon will be) the AS can hold off starting it until the queue is available. This is definitely a very useful feature and it is expected once you start allowing queues/topics to be hot deployed.

Is that really a real issue BTW? I know we have always supported that... but will users really use that?

For example, in order to have an EntityBean working, you need to have the Table created on Oracle. Would you have an EntityBeans with a @depends annotation on the table created?

The only reason I can think of is keeping the tradition.

I'm already doing that anyway through the deployer I wrote on hornetq-int. I will see if I can come up with an easy solution on every queue would have a Deployment POJO. (Can we find another name for it? I really don't like the term service for Queues. Lets call it Deployment POJO or anything you like).

Now you could definitely introduce an SPI like the above that would be in between the JMS integration layer and Hornet, but it would really just be an extra wrapper around hornetq APIs. If we did this extra work we would actually want to make sure the SPI actually does work with other JMS providers, and they would actually have an interest in integrating.

How would the console introspect what destinations are deployed otherwise?

Let me do some work tonight about the @depends. Maybe that will help on the introspection. I will come up with more information on this.

[jason.greene]

[e-mail] wrote:

The whole reason for Queues/Destinations to be a "service" is so that you can have dependencies. If an application writes to a queue when it is deployed, and that queue is not there yet (but soon will be) the AS can hold off starting it until the queue is available. This is definitely a very useful feature and it is expected once you start allowing queues/topics to be hot deployed.

Is that really a real issue BTW? I know we have always supported that... but will users really use that?

For example, in order to have an EntityBean working, you need to have the Table created on Oracle. Would you have an EntityBeans with a @depends annotation on the table created?

The only reason I can think of is keeping the tradition.

I'm already doing that anyway through the deployer I wrote on hornetq-int. I will see if I can come up with an easy solution on every queue would have a Deployment POJO. (Can we find another name for it? I really don't like the term service for Queues. Lets call it Deployment POJO or anything you like).

That's definitely a fair question. I think it largely depends on whether or not you make queues/topics a deployable thing. Once something is deployable it really should support some kind of dependency. If you made them a global static configuration that was only editable at runtime by the management APIs (but those changes were persisted back), then yeah not including it is reasonable. Quantifying a feature loss like this is tough since it was the only way to do it in the past.

Now you could definitely introduce an SPI like the above that would be in between the JMS integration layer and Hornet, but it would really just be an extra wrapper around hornetq APIs. If we did this extra work we would actually want to make sure the SPI actually does work with other JMS providers, and they would actually have an interest in integrating.

How would the console introspect what destinations are deployed otherwise?

Let me do some work tonight about the @depends. Maybe that will help on the introspection. I will come up with more information on this.

Well all I mean is that in both cases with and without an SPI, if we have a destination deployer, we still need to integrate that with the PS. Or actually Emanuel and/or Alexey would help with that portion. That work would be the same.

[brian.stansberry]

[e-mail] wrote:

The whole reason for Queues/Destinations to be a "service" is so that you can have dependencies. If an application writes to a queue when it is deployed, and that queue is not there yet (but soon will be) the AS can hold off starting it until the queue is available. This is definitely a very useful feature and it is expected once you start allowing queues/topics to be hot deployed.

Is that really a real issue BTW? I know we have always supported that... but will users really use that?

For example, in order to have an EntityBean working, you need to have the Table created on Oracle. Would you have an EntityBeans with a @depends annotation on the table created?

I don't really buy the Table analogy. Closer IMO is a datasource.

The main use case for this to me is packaging an entire application together in an EAR or SAR, where the destinations are deployed as part of the ear or sar. If the clients (i.e. servlets or ejbs) declare in their deployment descriptors refs to destinations, web or EJB deployers should be able to detect this and internally establish dependencies on the destinations. The end user shouldn't have to engage in configuration tricks to ensure the destination starts first; it should just work.

Same analysis applies for a datasource.

[e-mail] wrote:

I'm already doing that anyway through the deployer I wrote on hornetq-int. I will see if I can come up with an easy solution on every queue would have a Deployment POJO. (Can we find another name for it? I really don't like the term service for Queues. Lets call it Deployment POJO or anything you like).

What queues don't go through the deployer? (Apologies if this has already been discussed on IRC.) I certainly don't see any need to support dependencies on destinations that end user apps create programatically by directly invoking on HornetQ.

[brian.stansberry]

[e-mail] wrote:

That's definitely a fair question. I think it largely depends on whether or not you make queues/topics a deployable thing. Once something is deployable it really should support some kind of dependency. If you made them a global static configuration that was only editable at runtime by the management APIs (but those changes were persisted back), then yeah not including it is reasonable. Quantifying a feature loss like this is tough since it was the only way to do it in the past.

Allowing people to package their entire application in one deployable unit seems really fundamental to me. Forcing people to scatter bits and pieces around and keep track of where we expect to find the various bits is a huge step backward.

[clebert.suconic]

What queues don't go through the deployer? (Apologies if this has already been discussed on IRC.) I certainly don't see any need to support dependencies on destinations that end user apps create programatically by directly invoking on HornetQ.

You can call HornetQ's JMSManager directly. You won't have dependency injection on this case.

Queues deployed through the hornetq-jms.xml file will have @depends working. (That's already working)

I certainly don't see any need to support dependencies on destinations that end user apps create programatically by directly invoking on HornetQ.

That was my initial expectation but it doesn't seem everybody is agreeing with that.

If everybody is happy with that, then I don't need to do anything here.

[clebert.suconic]

Well all I mean is that in both cases with and without an SPI, if we have a destination deployer, we still need to integrate that with the PS. Or actually Emanuel and/or Alexey would help with that portion. That work would be the same.

I just think the SPI would be straighter. It would be up to the implementation to persist the destinations and CF creation anywhere it wants.

My co workers are all in Europe so they are sleeping now. So I want to hear tomorrow from Andy Taylor what he thinks about all this.

From what we talked on IRC it seems a SPI would be straighter and easier to understand and maintain here.

[clebert.suconic]

I have a fix for the @depends issue.

Instead of installing the deployment bean from the deployer, I have the JMS Manager doing it. I'm extending the JMS Manager on Hornetq-int, and the POJO is deployed at the integration layer.

(Deployment Bean is what you guys used to call QueueService).

At the moment I'm using an empty bean to serve the purpose of depends only.

I have it on my local box ATM. I will have it committed tomorrow on my Branch.

[timfox]

[e-mail] wrote:

[e-mail] wrote:

The whole reason for Queues/Destinations to be a "service" is so that you can have dependencies. If an application writes to a queue when it is deployed, and that queue is not there yet (but soon will be) the AS can hold off starting it until the queue is available. This is definitely a very useful feature and it is expected once you start allowing queues/topics to be hot deployed.

Is that really a real issue BTW? I know we have always supported that... but will users really use that?

For example, in order to have an EntityBean working, you need to have the Table created on Oracle. Would you have an EntityBeans with a @depends annotation on the table created?

I don't really buy the Table analogy. Closer IMO is a datasource.

The main use case for this to me is packaging an entire application together in an EAR or SAR, where the destinations are deployed as part of the ear or sar. If the clients (i.e. servlets or ejbs) declare in their deployment descriptors refs to destinations, web or EJB deployers should be able to detect this and internally establish dependencies on the destinations. The end user shouldn't have to engage in configuration tricks to ensure the destination starts first; it should just work.

I actually like the table analogy.

As Clebert points out, if your application requires database tables they don't get created automatically on deployment of your ear - they're setup in advance by your DBA.

The same goes for messaging queues/topics/connection factories - these are managed by your administrator who has the power to create them, assign resources and other attributes. We can't just let any ear create queues/topic wherever they like - in the same way you wouldn't let an application just create whatever tables it chooses - this needs to be controlled by the admin.

Also this @depends would only work with HornetQ, since each messaging system internally would have a different notion of queue/topic and how it's created/deployed - this would make any apps that use the @depends non portable if they changed messaging provider.

IMO I don't think we should consider queues/topics are "services" that can be used with @depends. If an .ear requires a certain queue they need to talk to their messaging admin and make sure it is configured, before referencing it in an .ear.

[ataylor]

Ive created a couple of (simplistic) diagrams that demonstrate how this works, really so the HQ team get a better understanding of th eproblem at hand.  They are simplistic to make them easie to understand.

The first diagram shows what we have now i think, which will give us @depends functionality and management objects.

The only issue with this is that any queues created via the API bypass this so in diagram 2 i have shown how JBM 1.4 does it, basically it creates the deployment descriptor and deploys it via the jboss deployers. I don't think this is the best solution as it would mean bypassing HornetQ management.

The third diagram shows how we could do it, although i am not sure whether this is feasible or actually possible. Basically it creates the Management Object via some sort of interceptor that the HornetQ management API calls. Ive also added to the diagram how the jopr plugin interacts.

Hope these help.

regarding what functionality we should have i think @depends is a must. This is typically how app developers work and i agree that losing this functionality is a step backwards. From a messaging domain point of view tho its a lot of integration just to get this. A simpler solution was if there was an API that we could call to just add the dependency, simplistically something like, kernel.addDependency(String dependencyName), which we could just call via some sort of plugin to our management API.