DataSource/ServiceXSLDeployer processing issue

[starksm64]

A problem I ran into while trying to implement the CRUD usecases for *-ds.xml deployments is that the metadata representation of a datasource deployment should be a ServiceDeployment with a xml dom representation of the *-ds.xml deployment. It currently is a ServiceDeployment with a xml dom representation of the transformed *-service.xml containing the jca mbeans.

The two problems that exist are:
1. the profile service only wants to rerun the deployment processing from the Deployer.CLASSLOADER_DEPLOYER on to avoid duplicate metadata parsing and overwriting of the overriden attachment metadata coming from the ManagedObject updates. However, for a datasource deployment, the xsl transform occurrs during the parsing phase. The datasource deployment therefore would need to be run through the parsing deployers.
2. the datasource deployment does not have a valid representation of the *-ds.xml metadata. There is no xml dom view of the original *-ds.xml that can be updated via ManagedObject/ManagedProperty bindings. The xsl transform always goes to the *-ds.xml VirtualFile and applies the transform to the dom that results from parsing it.

The general issues are deployers not having overridable metadata representations, and not knowing where to start the parsing. The latter issue is really more of not being able to simply rerun all deployers and only have those which have work to do function. I view this is an issue with not having the DeploymentUnit attachments api sufficiently well defined. As a review, here is the current state of the attachments api and its meaning.

Deployers deal with DeploymentUnits as the view on a deployable unit of work. There are two types of DeploymentUnits, containers which can top-level deployments (ears, ejb-jars, wars, sars, -beans.xml, ...), and components which belong to a container deployment (ejbs, mbeans, mc beans, servlets, ...). Components are associated with their container via DeploymentUnit.addComponent(String). Container type deployment can be nested structurally as well, but the DeploymentUnit does not currently expose an api for accessing this. Its a todo.

Metadata can be associated with a DeploymentUnit. This is done using the Attachments api. There are a few different levels of Attachments. A DeploymentUnit is an extension of the Attachments api. A DeploymentUnit also has an Attachments accessed via getTransientManagedObjects(). Adding an attachment via addAttachment adds a transient attachment. To add a transient managed object attachment, call getTransientManagedObjects().addAttachment.

The difference between a transient attachment and transient managed object attachment is that transient managed object attachments are overrides to transient attachments. When you call the DeploymentUnit.getAttachment(), the implementation first looks for a transient managed object attachment override. If that is not found, then the transient attachments are consulted. Deployers should only populate the transient attachments while admin interfaces like the profile service populate the transient managed object attachments.

One should be able to run a deployment trough the deployers, and then re-run the resulting DeploymentUnit through again and have this essentially be a noop in terms of the creation of metadata attachments as a Deployer should look for an existing attachment and avoid recreating it from the deployment content. One area which this is broken is when more than one parsing deployer populates the same metadata model from multiple sources. This happens for standard descriptors/annotations that can be augmented/overriden by jboss descriptors/annotations. With the addition of the DeploymentUnit Set getTypes(), the check for whether parsing should be done in the case of shared metadata attachments could be whether the attachments exist, and the DeploymentUnit has been marked as being of the curent Deployer.getType().

For the datasource deployment issue, the XSL deployers still need to support a dom attachment that represents the pre-transformed xml model so that it can be overriden.

[starksm64]

Two related issues in terms of the profile service being able to run a deployment through all deployers to avoid having to make assumptions about at what level the metadata that can be edited is available.

The first is that the kernel lifecycle actions need to be stubbed out to be noops to avoid any actual beans being created. When the profile service is in the server, how can this be done?

This also requires separate notions of transient attachments as already requested by Bill. There should be attachments that are candiates for overrides and saving/serialization and those that are not. I think we need a terminology reorg so that:

- transient attachments are metadata that are not serialized or candiates for overriding by admin ManagedObjects.
- attachments are metadata that can be serialized and overriden by admin ManagedObjects.
- managed objects are metadata overrides of the attachments metadata.

A Deployer that uses transient attachments would need to check for these and recreate these from the attachment and other input as needed.

[weston.price]

Yeah, so I guess you indirectly understand what I was trying to tell you about 2 months ago ;-)

[adrian.brock]

This seems to be getting very confused?

ATTACHMENTS

The original design was the following:

You have two types of attachment:
1) Transient - these are created by the parsing deployers from the raw xml
inside the deployments
2) Predetermined - these are what the profile service (or some other invoker of the
deployers) passes in to override what is in the deployment.

The Predetermined attachments are a sort of alt-dd but done programmatically.

I can see an argument for having a third type of attachment which is internal state
related to the deployment but it is stuff that we don't want the profile service/management
layer to override. It will always be constructed at runtime.

However, this can also be trivially dealt with by the ManagedObjectBuilder
NOT creating a ManagedObject for that attachment type.

[adrian.brock]

DEPLOYMENT PROTOCOL

This is something that is not a part of the original prototype but it is a part of the
original design.

The basic idea is that the deployers can be run in two modes.
1) Runtime - this is what is implemented, it is the full deployment
2) Profile Service - in this case, the "real deployers" create contexts that have
stubbed out actions for the MC and JMX layers.

That is when it runs in Profile Service mode, it still goes through the full
deployment process, creates all the contexts and tries to resolve dependencies,
but it doesn't create any objects or invoke any setters or lifecycle methods.

NOTE: This is only for the deployments. It does need to do the full work
for the deployers and classloaders otherwise they wouldn't be able to do anything. :-)

[adrian.brock]

RUNTIME MODE AND PREDETERMINED ATTACHMENTS

The parsing deployers are still invoked for these.
But they should be written in such a way that if there is a predetermined
attachment they don't do the parsing.

They still might do other work after the parse!

There are complications like the XSL deployer which needs to be changed
so you can pass in some form of DOM representation as the predetermined
attachment and still do the XSL transform on it, but this is an exceptional case
caused by badly written deployers that don't have a proper metadata model.

[adrian.brock]

COMPONENTS

This is very much a prototype. When I wrote this it was just to get something
working and express the idea. Like I said before, this needs revisiting and
doing properly.

But what I will say is that components are an implementation detail.
The profile service should not be interested in the components directly,
it is not going to touch their attachments.

The components that get constructed from a particular metadata model
can change from release to release.

The only part the profile service needs to know about these is the
context names. i.e. what are the MBean/POJO names
so it can map missing dependencies back to the original deployment
that caused them.

[bill.burke]

But what I will say is that components are an implementation detail.
The profile service should not be interested in the components directly,
it is not going to touch their attachments.

Don't know about this....I had an idea to be able to serialize an EJB Container so that redeployment is super fast.

[adrian.brock]

But what I will say is that components are an implementation detail.
The profile service should not be interested in the components directly,
it is not going to touch their attachments.

Don't know about this....I had an idea to be able to serialize an EJB Container so that redeployment is super fast.

That's not what I'm talking about.

I'm saying that the users should modify the -ds.xml,
they shouldn't be allowed to modifed the underlying MBean definitions
and expect the modified version to work across releases.

I don't have a problem with advanced features in the profile service
that cache the components (either metadata or resulting objects)
as long as the cache is flushed when a new version of JBoss is released
or some significant global config changes.

In fact, it would be a useful debugging feature in the admin console
to be able to click on an "Advanced" button and see the underlying
MC/JMX configurations generated.

But, I'd imagine the feature you are talking about is incredibly difficult to
implement anyway.

The basics are pretty simple, you could replace the "InstantiateAction"
in the MC/JMX with a "DeserializeAction" but then it has still got to go through
configure/create/start to get up-to-date references and do things
like binding its entry point into jndi.

"No man is and island" and references to things like the
TransactionManager singleton cannot be serialized, they must be
reinjected.

[bill.burke]

But, I'd imagine the feature you are talking about is incredibly difficult to
implement anyway.

Not hard, just a lot of work as refactorings of AOP would be required.

The basics are pretty simple, you could replace the "InstantiateAction"
in the MC/JMX with a "DeserializeAction" but then it has still got to go through
configure/create/start to get up-to-date references and do things
like binding its entry point into jndi.

The EJB Deployer can do what it already does: instantiate the container itself and install them manually into the kernel. No need for special MC states.

[adrian.brock]

The EJB Deployer can do what it already does: instantiate the container itself and install them manually into the kernel. No need for special MC states.

I'm talking about solving the problem generically rather 20 different
solutions (each with their own bugs) in 20 different services.

[bill.burke]

The EJB Deployer can do what it already does: instantiate the container itself and install them manually into the kernel. No need for special MC states.

I'm talking about solving the problem generically rather 20 different
solutions (each with their own bugs) in 20 different services.

I'd rather have each with their own bugs rather than a buggy generic solution that I have to keep backward compatibility with.

At least with the EJB container, what a non-generic solution would be doing anyways is pushing as much information and processing to metadata and the metadata phase of deployment. For instance, much of the processing for EJB container is pointcut bindings. Theres no reason this processing cant be pushed to metadata.

[adrian.brock]

I'd rather have each with their own bugs rather than a buggy generic solution that I have to keep backward compatibility with.

Well if you not interested in providing a generic solution then
you shouldn't be discussing in the POJO Server forum. :-)

And good luck on trying to persuade Scott to use the 20 different implementations
of this feature from inside the profile service. :-)

[starksm64]

COMPONENTS

This is very much a prototype. When I wrote this it was just to get something
working and express the idea. Like I said before, this needs revisiting and
doing properly.

But what I will say is that components are an implementation detail.
The profile service should not be interested in the components directly,
it is not going to touch their attachments.

The components that get constructed from a particular metadata model
can change from release to release.

The only part the profile service needs to know about these is the
context names. i.e. what are the MBean/POJO names
so it can map missing dependencies back to the original deployment
that caused them.

There is a notion of components in the profile service since the management interface wants to be able to query by functional detail rather than structural. A service deployment has mbeans which have managed objects. How this maps back onto the DeploymentUnit attachments depends on the ModelObjectBuilder. I don't see why it could not map to a component DeploymentUnit if desired.

[starksm64]

This seems to be getting very confused?

ATTACHMENTS

The original design was the following:

You have two types of attachment:
1) Transient - these are created by the parsing deployers from the raw xml
inside the deployments
2) Predetermined - these are what the profile service (or some other invoker of the
deployers) passes in to override what is in the deployment.

The Predetermined attachments are a sort of alt-dd but done programmatically.

I can see an argument for having a third type of attachment which is internal state
related to the deployment but it is stuff that we don't want the profile service/management
layer to override. It will always be constructed at runtime.

However, this can also be trivially dealt with by the ManagedObjectBuilder
NOT creating a ManagedObject for that attachment type.

I don't see the confusion as, other than the new third form of attachment, this is how I have described the attachments usage is it not? The notion of the third attachment being classified separately was really only to control what attachments have some serialization requirement.