Artifact metadata and dependency management

[objectiser]

Currently artifacts are 'typed' based on information directly added to the artifacts. This is necessary because the different artifact types do not have a common notion of 'behavioural type', and in some cases it may be necessary to indicate that an artifact behaviours like a particular type (similar to the relationship between a class and interface). For example, a choreography may be of type X, with participants P1 and P2. We may then want to say that a BPEL process, of type Z 'implements' the behaviour X@P2 - so we need to store that metadata somewhere.

Eclipse plugins currently maintain dependency information between the artifacts based on this 'type' information - using the project as an initial scope, with related projects (established in the project descriptors) also being included where defined. So this means that if two artifacts, with complementary type information, are contained within the same project (or projects that are related), then conformance checking (etc) can be performed between them.

I wanted to explore this mechanism to highlight some potential future issues.

Currently (as mentioned) the type information used to compare different artifacts is obtained from within the artifact, whether it is a CDL, ESB or BPEL description. The problem with this approach is that it assumes any tool editing the artifact will preserve this additional information, which is currently the case with the tools being used, but cannot be guaranteed with future artifacts, or if different tools are used on these existing artifacts.

The options are:

a) continue with the current approach, and deal with any problematic situations as they arise on an artifact by artifact basis (not ideal)

b) provide some form of 'descriptor' file that holds the additional information associated with the artifact

c) use annotations associated with the artifact's resource

Obviously (a) is not ideal, so for now we should presume this is not an option.

Option (b) is ok as it externalises the additional artifact information. One problem I see with this approach is how it may be stored, along with the associated artifact, in a repository (e.g. SOA repository).

Option (c) may also be fine, although care would need to be taken when storing the project in a source code control system (svn, cvs, etc), to ensure that the relevant metadata is also committed. However this approach also would have a problem when the artifacts are stored in an SOA repository.

SOA repositories generally have a mechanism for defining explicit dependencies between artifacts. The issue is that they are not standard, and we need the ability to annotate these dependencies.

Unlike with the Eclipse environment, where dependency between artifacts could be inferred based on the 'type' information associated with each artifact, this may not be appropriate in an SOA repository where the scope is not controlled. Therefore having explicit dependencies between the artifacts would be preferrable. However, we also need to make sure that this dependency information is transferred into the Eclipse environment, in a manner that can be used regardless of the structure of the project into which those artifacts have been placed.

So the question is, is there a common mechanism that can manage the required metadata and dependencies, that can work both in Eclipse and across a SOA Repository?

[jeffdelong]

Gary,

Sorry for taking so long to respond but I am still trying to figure this out as it has impacts on how I had been currently organizing these artifacts in Guvnor. which has used the Guvnor category features to manage relationships in support of a SOA repository. So in your example:

If we can assume Participants are Services, then I already have categories for Services, types of services, and the ability to add a category for each individual service. I also have defined a category for Choreography model and BPEL Process. What I have not specified to date is to put choreography models in their own Gunvor package and create explicit categories for them, which is what I think I would recommend to support your example.

In my PolicyQuote example, I would first create a PolicyQuoteModels package as part of Business Analysis / Architectural Modeling (this is different than what I have done to date so I need to test it out a bit). I would then place the scenarios and the PolicyQuote.cdm in the PolicyQuoteModels package. I would then create a category for PolicyQuoteChoreography under the general category Choreography Models, and assign the PolicyQuote.cdm categories Choreography Models/PolicyQuoteChoreography and Choreography Models/PolicyQuoteChoreography/Implementations. I would then create the packages for each service: PolicyQuoteProcessService, PolicyQuoteCalculationService and PolicyQuoteEntityService along with the categories Services/Orchestrated Services/PolicyQuoteProcessService, Services/Task Services/PolicyQuoteCalculationService and Services/Entity Services/PolicyQuoteEntityService.

I would then assign the category Choreography Models / PolicyQuoteChoreography to PolicyQuote.cdm, along with the categories representing any other services that were participants in this choreography (e.g. Services/Task Services/PolicyQuoteCalculationService and Services/Entity Services/PolicyQuoteEntityService.)

I would add the PolicyQuoteProcess.bpel to the PolicyQuoteProcessService package, then give it the categories Services/Orchestrated Services/PolicyQuoteProcessService and BPEL Process Definition. I could also give it the category Choreography Models/PolicyQuoteChoreography/Implementations.

Having done all this, if I viewed the assets by category under Choreography Models / PolicyQuoteChoreography I would see

PolicyQuote.cdm

Implementations

     PolicyQuoteProcess.bpel

and if I looked under BPEL Process Definitions I would see

PolicyQuoteProcess.bpel

and if I looked under Services/Orchestrated Services/PolicyQuoteProcessService I would see

PolicyQuoteProcess.bpel

I.e. the BPEL Process PolicyQuoteProcess.bpel implements the behaviour described by the Choreography PolicyQuote.cdm as part of the PolicyQuoteProcessService.

[objectiser]

Hi Jeff

Looks complicated - if we wanted to use the repository in this way, hopefully the tools would be helpful in setting up all these categories.

I am assuming that the categories are equivalent to tagging a resource to enable it to be located based on dynamic hierarchies.

Therefore if we took option (b) in the original post, to have a (Testable Integration Architecture - .tia) project descriptor file, this could be stored as another artifact in the repository, having repository resource URIs to locate the resources, along side any other URI (Eclipse workspace URI) that might locate the resource in another environment.

The problem with the categories approach is how we can ensure the URI uniquely identifies the resource we want. Three possible solutions:

1) All resources of a particular type will be uniquely named - i.e. will only ever be a single CDM called policyQuote.cdm.

2) Have one hierarchy that represents a physical (unique) URI - all other category hierarchies are logical.

3) Locate resources by their JCR unique identifier.

Solution (1) might be possible, but is it a constraint that we should rely upon. What happens in Guvnor now if two resources in the same category have the same name. I assume the name is not the primary in the JCR db, so I guess multiple resources with the same name could be listed.

Solution (2) again might be possible, but could also be viewed as a constraint.

Solution (3) seems the best option, for a JCR based repository anyway.

To overcome the different URIs depending on environment, we could have a .tia schema along the lines:

<tia>

    <requirements>

        <resource id="1" name="SuccessfulQuote.scn" >

             <uri context="eclipse:PolicyQuoteModels"

                      locator="scenarios/SuccessfulQuote.scn" />

             <uri context="guvnor:??" locator="#12345678" />

        </resource>

    </requirements>

    <architecture>

        <resource id="2" name="PolicyQuote.cdm" >

             <dependency ref="1" />

             <dependency ref="3" />

             <uri context="eclipse:PolicyQuoteModels"

                      locator="models/PolicyQuote.cdm" />

             <uri context="guvnor:??" locator="#4393896983" />

        </resource>

    </architecture>

    ...

    <serviceContracts>

        <resource id="3" name="PolicyQuoteService.wsdl" >

             <dependency ref="2" />

             <uri context="eclipse:PolicyQuoteModels"

                      locator="contracts/wsdl/PolicyQuoteService.wsdl" />

             <uri context="guvnor:??" locator="#7689484784" />

        </resource>

    </serviceContracts>

</tia>

Sorry the post has become abit detailed, but thought it best to illustrate the purpose of the project descriptor in an Eclipse and Guvnor context. So hopefully we would have tools in both environments that could help manage the resources and their dependencies, and also trigger validation (automatically or manually) based on this information.

Regards

Gary

[jeffdelong]

Well it does require several manual step but each one is reasonably straightforward if you understand the relationships. And yes, have the integration tooling understand this and doing it as part of the upload process would be very nice.

Yes, the categories are a tagging mechanism and the Guvnor GUI allows the user to view the hierarchy of categories.

Asset names in Guvnor must be unique per package; but this is not an unreasonable restriction since it would be very confusing otherwise. So I could have another PolicyQuote.cdm but it would have to reside in another package. However it could have the same category, e.g., Choreography Model.

Each asset in Guvnor does have an asset Id. http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/Guvnor.html#asset=df62d65f-922a-4b98-a56c-a341ae49d15b is the URL for the GUI that displays the PolicyQuote.cdm). However I don't think this is the proper way to access the asset. Instead, what is visible in the Eclipse properties under Guvnor is the

Repository: http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav

and the

Path: /packages/PolicyQuoteModels/PolicyQuote.cdm

If you type http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav/packages/PolicyQuoteModels/PolicyQuote.cdm

into a browser, you will get an XML file with the choreography.

So perhaps the tia file should look like:

<tia>

    <requirements>

        <resource id="1" name="SuccessfulQuote.scn" >

             <uri context="eclipse:PolicyQuoteModels"

                      locator="scenarios/SuccessfulQuote.scn" />

             <uri context="http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav"

                      locator="/packages/PolicyQuoteModels/SuccessfulQuote.scn" />

        </resource>

    </requirements>

    <architecture>

        <resource id="2" name="PolicyQuote.cdm" >

             <dependency ref="1" />

             <dependency ref="3" />

             <uri context="eclipse:PolicyQuoteModels"

                      locator="models/PolicyQuote.cdm" />

             <uri context="http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav"

                      locator="/packages/PolicyQuoteModels/PolicyQuote.cdm" />

        </resource>

    </architecture>

    ...

    <serviceContracts>

        <resource id="3" name="PolicyQuoteService.wsdl" >

             <dependency ref="2" />

             <uri context="eclipse:PolicyQuoteModels"

                      locator="contracts/wsdl/PolicyQuoteService.wsdl" />

             <uri context="http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav"

                      locator="/packages/PolicyQuoteModels/PolicyService.wsdl" />

        </resource>

    </serviceContracts>

</tia>

[objectiser]

Asset names in Guvnor must be unique per package; but this is not an unreasonable restriction since it would be very confusing otherwise. So I could have another PolicyQuote.cdm but it would have to reside in another package. However it could have the same category, e.g., Choreography Model.

So the package structure is effectively the fully qualified location of any asset?

So in this example, the Choreography Model category would have two PolicyQuote.cdm entries? but they would reference assets in different package structures?

Your suggestions for the content/locator fields for Guvnor would be better - although in general I think these fields need to be free format to allow any of form of storage to be used. Might need to explicitly add a 'type' attribute to the uri element, to separate out the 'eclipse' or 'guvnor' value from the context - so then the context can be the specific instance (repository), and the locator being the location within that repository. Having the separate type would then make it easier to have a plugin that understands how to interpret the URI.

<tia>

    <requirements>

        <resource id="1" name="SuccessfulQuote.scn" >

             <uri type="eclipse" context="PolicyQuoteModels"

                      locator="scenarios/SuccessfulQuote.scn" />

             <uri type="guvnor" context="http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/webdav"

                      locator="/packages/PolicyQuoteModels/SuccessfulQuote.scn" />

        </resource>

    </requirements>

    ....

</tia>

[objectiser]

One other proposed change to the schema - instead of having a 'dependency' element, we should have 'relationship' element with a 'type' attribute that could be 'dependency' or other values, such as 'derivedFrom' or 'uses'. Allows other forms of relationship to be expressed, that won't necessarily imply validation is necessary - or it may mean a different type of validation is required.

For example, we may have one choreography that 'extends' another - in which case we may want to validate that one is a subset of the other.

This relationship may also help in terms of modelling dependencies between services (for example). Although this information should be recorded in a SOA repository, it may be useful to have a static declaration of what was intended as part of the project design?

[objectiser]

An initial schema representing the resource information and dependencies has been attached to: https://jira.jboss.org/browse/SAVARA-78