Rich metadata attached to S-RAMP artifacts

[lkrejci]

My use case is the following:

1. S-RAMP stores artifacts
2. RHQ deploys the artifacts
3. RHQ updates metadata on the artifacts in S-RAMP so that they're available for analysis

The problem with 3. is that the metadata RHQ can/should provide is quite rich and structured.

As an example of the metadata RHQ can provide, consider the following:

An artifact is deployed to a group of RHQ resources (meaning essentially that it is deployed to a number of servers). It is deployed with a certain deploy-time configuration and on some of the servers, the deployment failed with some error message, while on others it succeeded. All the deployments regardless of their success or failure produced a number of log messages captured by RHQ that are potentially useful.

What is the best way to model this in S-RAMP? Is S-RAMP even a good place to store such kind of data? I'm leaning towards saying yes it is, because the knowledge about where artifacts are currently deployed might come useful for some business processes when considering for restructuring the load balancing setup, etc.

Of course, one of course has to consider the auditing of this information and keeping the historical entries for all such metadata.

My main question about modelling the above outlined metadata is the fact that to store it "sensibly", I think the best way would be to store at least some of it (i.e. the logs of the deployment per resource) as new artifacts inside S-RAMP, linked together with the main artifact by relationships. But I am uncertain if such kind of artifacts even belongs to S-RAMP, because they're not exactly "artifacts" but rather "artefacts" (in archaeological, i.e. historical, sense) They just document what has happened to the artifact but are otherwise meaningless.

[eric.wittmann]

I think there are a couple of options to address this issue.  First, as you have mentioned, you can model the information in S-RAMP as artifacts and use relationships to tie stuff together.  This is a very appropriate approach in many cases.  It's definitely something s-ramp is intended for. 

The other option is non-compliant (not part of the s-ramp specification) but may be appropriate depending on the use-case.  The Overlord s-ramp implementation has an auditing capability.  We automatically audit changes to artifacts so that later we can show the user a history of what happened to it (what happened, when, and by whom).  There is an API to this auditing facility that allows clients to add additional, custom audit entries for an artifact. 

So I think it's a matter of how the information is going to be used.  If we think that users will want to use/leverage the meta-data in an S-RAMP query to find specific artifacts, then that meta-data needs to be modeled as artifacts.  If the data is purely logging or audit related information, then using the audit API may make sense.  The limitation there is that the data will be read-only to the user and cannot be used when performing a query.

[lkrejci]

Is it possible to have "content-less" artifacts? I.e. artifacts that would only group a set of metadata together.

Let's say I'd want to have a "Deployment" artifact that would describe a single deployment of an artifact. I don't want to model it merely as a set of custom properties on an artifact, because a single artifact can be deployed several times (to different servers, for example). Such deployment artifact would not have any data as such though. It would only contain properties describing the deployment (like what was the configuration of the deployment (if the deployment was configurable), whether it succeeded, what were the RHQ resources it went to, etc) - none of them would be applicable as the "content" of the artifact.

Currently, the CLI requires there to be a file to upload when creating an artifact. Are content-less artifacts just a corner case that is not yet handled or are they not allowed by the spec?

[eric.wittmann]

Content-less artifacts (aka non-document artifacts) are allowed by the spec, and in fact there are many of them in the built-in s-ramp data models (see the SOA and Service Implementation models in the spec - all of the artifact types in both of those models are non-document artifacts).

A non-document artifact in s-ramp is created by POSTing an Atom Entry to the appropriate REST endpoint (rather than POSTing a file).

The server impl. currently supports this for extension types.  The Extension model has two artifact types, ExtendedDocument and ExtendedArtifactType.  The former is used when file content is POSTed to:

/s-ramp/ext/{CustomTypeHere}

The latter is used when an Atom Entry is POSTed to the same REST endpoint.

You are right that the CLI currently has no "Add Artifact" command.  This is simply something that has not yet been implemented.  It's something that I could implement quickly if needed.

Also, to the point, your use-case for non-document artifacts is precisely why they exist!  So it's a great idea.