"Time-boxed" relationships in S-RAMP?

[lkrejci]

I am trying to model "time-boxed" relationships. I.e. a relationship between two documents that is valid only during a certain time-frame.

My use-case will use this information heavily and will frequently ask questions like: give me all the related documents as they were yesterday.

I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored and b) the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

How best to model a relationship that only existed for a certain period of time? I'm a little bit reluctant to just delete the relationship and try to use audit to figure out the time range it existed because of performance reasons.

Maybe instead of a direct relationship between source and target documents, I will have to have an intermediary to hold all the "from" and "to" properties?

Source --relatedBy--> Relationship(name=..., from=.., to=..) --targettedTo--> Target

How do you think I should approach this? Is S-RAMP even a good candidate for storing such data?

Thanks, Lukas

[brmeyer]

Hey Lukas -- interesting thought!

> I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored

If that's actually the case, can you open a JIRA w/ details?

> the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

I'd be more than happy to help come up with a solution.  Can you give me a little more detail on the specific use case?

[lkrejci]

> > I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored

> If that's actually the case, can you open a JIRA w/ details?

https://issues.jboss.org/browse/SRAMP-622

> > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

> Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

>

> I'd be more than happy to help come up with a solution.  Can you give me a little more detail on the specific use case?

This is for the new RHQ. We want to model and track the evolution of

relationships between resources in our inventory, i.e. "What production

machine was this app deployed to last week?", "What metrics could we collect

on this resource two years ago when it was version 1.0?", "What machines were

in this cluster last month?", etc.

So in a completely abstract model, the relationships between resources would

be annotated by the time they were in effect, the currently active

relationship having the "to" set to a kind of END_OF_TIME value. Graph

databases usually allow for this and they also usually allow for multiple

edges of the same name between 2 resources, so this abstract model is directly

applicable to them.

In a more specialized storage backends that we explore, like S-RAMP or even

RDF, this might not be so easy...

On the other hand both RDF and S-RAMP have a standardized query language which

is appealing.

[brmeyer]

> > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

> Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

Actually, just realized I completely mis-spoke about this.  What's restricted is custom property duplication, not relationships.  Apologies -- long day.  So, defining 1...n relationships between 2 nodes, with identical relationship names, is entirely possible.  And with the correction of SRAMP-622 (and ensuring the querying works), the time-boxed relationships wouldn't be difficult to define at all.

I wouldn't be against building these types of capabilities into the clients themselves, although it would probably just make use of the arbitrary attributes (I'm hesitant to start extending the actual spec schemas/bindings).

[lkrejci]

Definitely. This should not be codified on the S-RAMP side in any manner. Arbitrary attributes with semantics defined on the client-side is all I'm asking for

[lkrejci]

I'm a bit confused as to how to query for relationships with attributes.

The spec, chapter 4.2 in example 5 says the following:

/s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

o

Only WsdlDocument artifact instances corresponding to WSDL documents which have at

least one includedXsds relationship instance, that has a property name of

“someProperty”, whose value is “true” will be returned.

o

If the relationship in the predicate is not defined on a given artifact whose

scope is within the path expression, then the predicate evaluates to false and

that artifact is not returned.

So, given the following documents and the relationship with custom attribute between them:

wsdl/WsdlDocument/wsdl1 --includedXsds[someProperty = true]--> xsd/XsdDocument/xsd1

the query from the example 5 of the spec should return wsdl1. Am I right?

Unfortunately, that's not what I'm seeing when playing with the S-RAMP server.

The server seems to match the "someProperty" against the target of the relationship, NOT the attributes of the relationship itself (i.e. it would look for "someProperty" in xsd/XsdDocument/xsd1 in the above example), which is actually consistent with the S-RAMP server docs:

https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Virtualization/6/html/Development_Guide_Governance/chap-S-RAMP-1.html#S-RAMP_Query_Language

where there's this example:

/s-ramp/ext/ServletFilter[relatedDocument[@uuid="12345"]]  All servlet filter artifacts derived from (i.e. contain a relatedDocument relationship to) an artifact with UUID 12345.

Do I misunderstand the spec?

[lkrejci]

Ok, I see now that "includedXsds" on an WsdlDocument is not really persisted or modelled as a true relationship but rather is modelled as a "quality" of the document itself which can be "viewed" as a relationship, e.g. s-ramp/RelationshipArtifactVisitor.java at master · Governance/s-ramp · GitHub.

So in a generic sense, if modelled as above, the relationships are not supposed to have attributes of their own, AFAIUI. But then the XSD and the API allows for it - https://github.com/Governance/s-ramp/blob/master/s-ramp-api/src/main/java/org/oasis_open/docs/s_ramp/ns/s_ramp_v1/Relationship.java#L145

So if something is a "true" relationship, i.e. not an intrinsic one (as includedXsds on an WsdlDocument), it should be able to have attributes of its own. But then at least the modeshape schema doesn't seem to reflect that: https://github.com/Governance/s-ramp/blob/master/s-ramp-repository-jcr/modeshape/src/main/resources/org/overlord/sramp/repository/jcr/modeshape/sramp.cnd#L85

And then there's the question of how to even query for such custom attributes on the relationships, because the spec seems to be a bit confused about it (or more probably I am ).

So I guess modelling an attributed relationship using an "intermediary" document as mentioned in my original post is the only way forward?

[brmeyer]

Hey Lukas.

> Definitely. This should not be codified on the S-RAMP side in any manner. Arbitrary attributes with semantics defined on the client-side is all I'm asking for

Sorry, I meant the S-RAMP provided clients: Java client, shell, UI, etc.  I'd be up for adding these types of capabilities there, if they prove to be useful.  Behind the scenes, they'd simply wrap what you're already trying to do: using the extension attributes in meaningful ways.

> /s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

> Only WsdlDocument artifact instances corresponding to WSDL documents which have at least one includedXsds relationship instance, that has a property name of “someProperty”, whose value is “true” will be returned.

> If the relationship in the predicate is not defined on a given artifact whose scope is within the path expression, then the predicate evaluates to false and that artifact is not returned.

Definitely agree that the requirement is super vague.  That was originally interpreted as someProperty existing on the targeted artifact.  The last statement is the most telling: "...on a given artifact whose scope is within the path expression...".

> Ok, I see now that "includedXsds" on an WsdlDocument is not really persisted or modelled as a true relationship but rather is modelled as a "quality" of the document itself which can be "viewed" as a relationship

We do actually store, as an example, includedXsds as a real relationship.  However, the spec distinguishes between derived (cannot be created/updated/deleted), modeled (can be edited), and generic (ad-hoc) relationships.  But all of them are true relationships, behind the scenes.

> So in a generic sense, if modelled as above, the relationships are not supposed to have attributes of their own, AFAIUI. But then the XSD and the API allows for it

I believe that's true.  I don't remember actual mention of the custom relationship attributes, even though it's included in the spec.  But I'll go through again and verify.

> And then there's the question of how to even query for such custom attributes on the relationships

Regardless of what the spec does and doesn't require, big +1 from me on adding this capability.  Storing the relationship attributes will be easy.  Querying needs a bit of thought.  Do we add a disjunction to an existing query (although not sure if that would break spec compliance).  Example:

/s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

That would return artifacts who target an artifact with someProperty defined, OR artifacts whose relationships define someProperty.  It sounds ok in theory, but I'm not sure how I feel about setting that precedent.

Instead, I'd advocate starting to extend the query language with new XPath functions.  Eric Wittmann had this idea: /s-ramp/wsdl/WsdlDocument[includedXsds[s-ramp:getRelationshipAttribute(., 'someAttribute') = 'true]].  We'll discuss that in SRAMP-622.

Thanks for this!

[lkrejci]

Brett Meyer wrote:

> > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

> Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

Actually, just realized I completely mis-spoke about this.  What's restricted is custom property duplication, not relationships.  Apologies -- long day.  So, defining 1...n relationships between 2 nodes, with identical relationship names, is entirely possible.  And with the correction of SRAMP-622 (and ensuring the querying works), the time-boxed relationships wouldn't be difficult to define at all.

I wouldn't be against building these types of capabilities into the clients themselves, although it would probably just make use of the arbitrary attributes (I'm hesitant to start extending the actual spec schemas/bindings).

I finally got back to try things out with S-RAMP and I stumbled again

Thanks for adding the ability to store the relationship attributes in S-RAMP 0.7.0, that works like charm, but I fear that's actually slightly NOT what we wanted here While the model conceptually allows for multiple relationships with the same name originating on the same node, the way it is modelled is that there is always a single relationship of given type/name on a document and this relationship can have multiple targets.

So to model relationships like  "A -(depends; from: 1, to: 2)> B" and "A -(depends; from: 1, to: 2)> C", the attributes on the relationship itself are not enough. We'd need the attributes on the targets to be able to model that.

[brmeyer]

Lukaz, not persisting Target#otherAttributes (which does exist), in addition to Relationship#otherAttributes, was simply an oversight on my part.  Easy fix -- will do under [ARTIF-635] Target.getOtherAttributes() should be persisted and queryable - JBoss Issue Tracker.  Thanks!

[brmeyer]

Actually Lukaz, I completely forgot that this is already documented in [ARTIF-625] Support Target#otherAttributes - JBoss Issue Tracker.  I'm still catching back up after the holiday   I'll get that addressed ASAP.

[brmeyer]

Lukas, sorry for the spam...

What type of query would be most useful, here?  For the relationship otherAttributes, we did something like:

/s-ramp/xsd/XsdDocument[someRelationship[s-ramp:getRelationshipAttribute(., 'someName') = 'someValue']] (All XSD artifacts that contain a generic relationship named 'someRelationship', and that relationship has an 'otherAttribute' named 'someName' with a 'someValue' value)

If the otherAttribute is on the target, this could look like:

/s-ramp/xsd/XsdDocument[someRelationship[s-ramp:getTargetAttribute(., 'someName')]]

Say you have something like:

A -> someRelationship -> Target (otherAttribute: someName) -> B

In which direction would you want to query?  Initially, I was thinking getTargetAttribute would work similarly.  Find me all As that own a relationship that has a Target with the attribute.  But is the reverse more useful?  Find me all Bs that are the destinations of the relationships with the target attribute?

Hopefully that makes sense

[lkrejci]

It definitely does make sense. What I'm trying to do is to implement the

Blueprints API (https://github.com/tinkerpop/blueprints/wiki) on top of S-RAMP

so that we can compare it (to a degree) with graph databases.

To correctly implement that I actually need to be able to go both directions.

So I guess, selecting the A's in the above example would be the way forward,

if following queries were possible:

Situation:

A -(r; a = 1)> B

A -(r; a = 1)> C

Querying all vertices that an edge called "r", that has an attribute called a

equal to 1, points to from A:

/s-ramp/ext/Vertex[@uuid=A]/r[s-ramp:getTargetAttribute(.,"a")=1]

(Am I right thinking that this should work? While trying this (with

getRelationshipAttribute) I got a JCR-SQL2 exception).

To go "backwards", i.e. find all nodes that point to B using an edge that has

an attribute "a" equal to 1, I think the following query could work:

/s-ramp/ext/Vertex[r[s-ramp:getTargetAttribute(.,"a")=1 and @uuid="B"]]

(again, this does not currently work in S-RAMP as you point out in ARTIF-622)

What do you think?

Thanks for looking into this so quickly..

Btw. while the need for both directions stems currently from the Blueprints

API, I can easily see we are going to need this in Hawkular (the new name for

RHQ.next) when for example asking questions like "what are the machines this

application ran on last week?" (backwards direction) or "what applications run

on this machine now?" (forwards direction).

[brmeyer]

Lukas, query parsing, getTargetAttribute, and getRelationshipAttribute are all fixed up.  Until the next Keycloak is released, you won't be able to pull down the SNAPSHOT from Nexus.  If you want to re-try it prior to that, pull down both Keycloak and Artificer masters and build.