13 Replies Latest reply on Jan 15, 2015 1:52 PM by brmeyer

    "Time-boxed" relationships in S-RAMP?

    lkrejci

      I am trying to model "time-boxed" relationships. I.e. a relationship between two documents that is valid only during a certain time-frame.

       

      My use-case will use this information heavily and will frequently ask questions like: give me all the related documents as they were yesterday.

       

      I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored and b) the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

       

      How best to model a relationship that only existed for a certain period of time? I'm a little bit reluctant to just delete the relationship and try to use audit to figure out the time range it existed because of performance reasons.

       

      Maybe instead of a direct relationship between source and target documents, I will have to have an intermediary to hold all the "from" and "to" properties?

       

      Source --relatedBy--> Relationship(name=..., from=.., to=..) --targettedTo--> Target

       

      How do you think I should approach this? Is S-RAMP even a good candidate for storing such data?

       

      Thanks, Lukas

        • 1. Re: "Time-boxed" relationships in S-RAMP?
          brmeyer

          Hey Lukas -- interesting thought!

           

          > I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored

          If that's actually the case, can you open a JIRA w/ details?

           

          > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

          Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

           

          I'd be more than happy to help come up with a solution.  Can you give me a little more detail on the specific use case?

          • 2. Re: "Time-boxed" relationships in S-RAMP?
            lkrejci

            > > I thought I might use the arbitrary attributes the XSD seems to allow on relationships but they a) don't seem to be stored

            > If that's actually the case, can you open a JIRA w/ details?

             

            https://issues.jboss.org/browse/SRAMP-622

            > > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

            > Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

            >

            > I'd be more than happy to help come up with a solution.  Can you give me a little more detail on the specific use case?

             

            This is for the new RHQ. We want to model and track the evolution of

            relationships between resources in our inventory, i.e. "What production

            machine was this app deployed to last week?", "What metrics could we collect

            on this resource two years ago when it was version 1.0?", "What machines were

            in this cluster last month?", etc.

             

            So in a completely abstract model, the relationships between resources would

            be annotated by the time they were in effect, the currently active

            relationship having the "to" set to a kind of END_OF_TIME value. Graph

            databases usually allow for this and they also usually allow for multiple

            edges of the same name between 2 resources, so this abstract model is directly

            applicable to them.

             

            In a more specialized storage backends that we explore, like S-RAMP or even

            RDF, this might not be so easy...

             

            On the other hand both RDF and S-RAMP have a standardized query language which

            is appealing.

            • 3. Re: "Time-boxed" relationships in S-RAMP?
              brmeyer

              > > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

              > Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

               

              Actually, just realized I completely mis-spoke about this.  What's restricted is custom property duplication, not relationships.  Apologies -- long day.  So, defining 1...n relationships between 2 nodes, with identical relationship names, is entirely possible.  And with the correction of SRAMP-622 (and ensuring the querying works), the time-boxed relationships wouldn't be difficult to define at all.

               

              I wouldn't be against building these types of capabilities into the clients themselves, although it would probably just make use of the arbitrary attributes (I'm hesitant to start extending the actual spec schemas/bindings).

              1 of 1 people found this helpful
              • 4. Re: "Time-boxed" relationships in S-RAMP?
                lkrejci

                Definitely. This should not be codified on the S-RAMP side in any manner. Arbitrary attributes with semantics defined on the client-side is all I'm asking for

                • 5. Re: "Time-boxed" relationships in S-RAMP?
                  lkrejci

                  I'm a bit confused as to how to query for relationships with attributes.

                   

                  The spec, chapter 4.2 in example 5 says the following:

                   

                  /s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

                  o

                  Only WsdlDocument artifact instances corresponding to WSDL documents which have at

                  least one includedXsds relationship instance, that has a property name of

                  “someProperty”, whose value is “true” will be returned.

                  o

                  If the relationship in the predicate is not defined on a given artifact whose

                  scope is within the path expression, then the predicate evaluates to false and

                  that artifact is not returned.

                   

                  So, given the following documents and the relationship with custom attribute between them:

                   

                  wsdl/WsdlDocument/wsdl1 --includedXsds[someProperty = true]--> xsd/XsdDocument/xsd1

                   

                  the query from the example 5 of the spec should return wsdl1. Am I right?

                   

                  Unfortunately, that's not what I'm seeing when playing with the S-RAMP server.

                  The server seems to match the "someProperty" against the target of the relationship, NOT the attributes of the relationship itself (i.e. it would look for "someProperty" in xsd/XsdDocument/xsd1 in the above example), which is actually consistent with the S-RAMP server docs:

                  https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Virtualization/6/html/Development_Guide_Governance/chap-S-RAMP-1.html#S-RAMP_Query_Language

                  where there's this example:

                  /s-ramp/ext/ServletFilter[relatedDocument[@uuid="12345"]]  All servlet filter artifacts derived from (i.e. contain a relatedDocument relationship to) an artifact with UUID 12345.


                  Do I misunderstand the spec?

                  • 6. Re: "Time-boxed" relationships in S-RAMP?
                    lkrejci

                    Ok, I see now that "includedXsds" on an WsdlDocument is not really persisted or modelled as a true relationship but rather is modelled as a "quality" of the document itself which can be "viewed" as a relationship, e.g. s-ramp/RelationshipArtifactVisitor.java at master · Governance/s-ramp · GitHub.

                     

                    So in a generic sense, if modelled as above, the relationships are not supposed to have attributes of their own, AFAIUI. But then the XSD and the API allows for it - https://github.com/Governance/s-ramp/blob/master/s-ramp-api/src/main/java/org/oasis_open/docs/s_ramp/ns/s_ramp_v1/Relationship.java#L145

                     

                    So if something is a "true" relationship, i.e. not an intrinsic one (as includedXsds on an WsdlDocument), it should be able to have attributes of its own. But then at least the modeshape schema doesn't seem to reflect that: https://github.com/Governance/s-ramp/blob/master/s-ramp-repository-jcr/modeshape/src/main/resources/org/overlord/sramp/repository/jcr/modeshape/sramp.cnd#L85

                     

                    And then there's the question of how to even query for such custom attributes on the relationships, because the spec seems to be a bit confused about it (or more probably I am ).

                     

                    So I guess modelling an attributed relationship using an "intermediary" document as mentioned in my original post is the only way forward?

                    • 7. Re: "Time-boxed" relationships in S-RAMP?
                      brmeyer

                      Hey Lukas.

                       

                      > Definitely. This should not be codified on the S-RAMP side in any manner. Arbitrary attributes with semantics defined on the client-side is all I'm asking for

                       

                      Sorry, I meant the S-RAMP provided clients: Java client, shell, UI, etc.  I'd be up for adding these types of capabilities there, if they prove to be useful.  Behind the scenes, they'd simply wrap what you're already trying to do: using the extension attributes in meaningful ways.

                       

                      > /s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

                      > Only WsdlDocument artifact instances corresponding to WSDL documents which have at least one includedXsds relationship instance, that has a property name of “someProperty”, whose value is “true” will be returned.

                      > If the relationship in the predicate is not defined on a given artifact whose scope is within the path expression, then the predicate evaluates to false and that artifact is not returned.

                       

                      Definitely agree that the requirement is super vague.  That was originally interpreted as someProperty existing on the targeted artifact.  The last statement is the most telling: "...on a given artifact whose scope is within the path expression...".

                       

                      > Ok, I see now that "includedXsds" on an WsdlDocument is not really persisted or modelled as a true relationship but rather is modelled as a "quality" of the document itself which can be "viewed" as a relationship

                       

                      We do actually store, as an example, includedXsds as a real relationship.  However, the spec distinguishes between derived (cannot be created/updated/deleted), modeled (can be edited), and generic (ad-hoc) relationships.  But all of them are true relationships, behind the scenes.

                       

                      > So in a generic sense, if modelled as above, the relationships are not supposed to have attributes of their own, AFAIUI. But then the XSD and the API allows for it

                       

                      I believe that's true.  I don't remember actual mention of the custom relationship attributes, even though it's included in the spec.  But I'll go through again and verify.

                       

                      > And then there's the question of how to even query for such custom attributes on the relationships

                       

                      Regardless of what the spec does and doesn't require, big +1 from me on adding this capability.  Storing the relationship attributes will be easy.  Querying needs a bit of thought.  Do we add a disjunction to an existing query (although not sure if that would break spec compliance).  Example:

                       

                      /s-ramp/wsdl/WsdlDocument[includedXsds[@someProperty=’true’]]

                       

                      That would return artifacts who target an artifact with someProperty defined, OR artifacts whose relationships define someProperty.  It sounds ok in theory, but I'm not sure how I feel about setting that precedent.

                       

                      Instead, I'd advocate starting to extend the query language with new XPath functions.  Eric Wittmann had this idea: /s-ramp/wsdl/WsdlDocument[includedXsds[s-ramp:getRelationshipAttribute(., 'someAttribute') = 'true]].  We'll discuss that in SRAMP-622.

                       

                      Thanks for this!

                      • 8. Re: "Time-boxed" relationships in S-RAMP?
                        lkrejci

                        Brett Meyer wrote:

                         

                        > > the spec only allows 1 relationship of a certain name between 2 nodes. Am I right?

                        > Correct.  In fact, as of 0.7.0, we throw an exception if a node has a duplicated relationship (required by the spec).

                         

                        Actually, just realized I completely mis-spoke about this.  What's restricted is custom property duplication, not relationships.  Apologies -- long day.  So, defining 1...n relationships between 2 nodes, with identical relationship names, is entirely possible.  And with the correction of SRAMP-622 (and ensuring the querying works), the time-boxed relationships wouldn't be difficult to define at all.

                         

                        I wouldn't be against building these types of capabilities into the clients themselves, although it would probably just make use of the arbitrary attributes (I'm hesitant to start extending the actual spec schemas/bindings).

                        I finally got back to try things out with S-RAMP and I stumbled again

                        Thanks for adding the ability to store the relationship attributes in S-RAMP 0.7.0, that works like charm, but I fear that's actually slightly NOT what we wanted here While the model conceptually allows for multiple relationships with the same name originating on the same node, the way it is modelled is that there is always a single relationship of given type/name on a document and this relationship can have multiple targets.

                         

                        So to model relationships like  "A -(depends; from: 1, to: 2)> B" and "A -(depends; from: 1, to: 2)> C", the attributes on the relationship itself are not enough. We'd need the attributes on the targets to be able to model that.

                        • 9. Re: "Time-boxed" relationships in S-RAMP?
                          brmeyer

                          Lukaz, not persisting Target#otherAttributes (which does exist), in addition to Relationship#otherAttributes, was simply an oversight on my part.  Easy fix -- will do under [ARTIF-635] Target.getOtherAttributes() should be persisted and queryable - JBoss Issue Tracker.  Thanks!

                          • 10. Re: "Time-boxed" relationships in S-RAMP?
                            brmeyer

                            Actually Lukaz, I completely forgot that this is already documented in [ARTIF-625] Support Target#otherAttributes - JBoss Issue Tracker.  I'm still catching back up after the holiday   I'll get that addressed ASAP.

                            • 11. Re: "Time-boxed" relationships in S-RAMP?
                              brmeyer

                              Lukas, sorry for the spam...

                               

                              What type of query would be most useful, here?  For the relationship otherAttributes, we did something like:

                               

                              /s-ramp/xsd/XsdDocument[someRelationship[s-ramp:getRelationshipAttribute(., 'someName') = 'someValue']] (All XSD artifacts that contain a generic relationship named 'someRelationship', and that relationship has an 'otherAttribute' named 'someName' with a 'someValue' value)

                               

                              If the otherAttribute is on the target, this could look like:

                              /s-ramp/xsd/XsdDocument[someRelationship[s-ramp:getTargetAttribute(., 'someName')]]

                               

                              Say you have something like:

                              A -> someRelationship -> Target (otherAttribute: someName) -> B

                               

                              In which direction would you want to query?  Initially, I was thinking getTargetAttribute would work similarly.  Find me all As that own a relationship that has a Target with the attribute.  But is the reverse more useful?  Find me all Bs that are the destinations of the relationships with the target attribute?

                               

                              Hopefully that makes sense

                              • 12. Re: "Time-boxed" relationships in S-RAMP?
                                lkrejci

                                It definitely does make sense. What I'm trying to do is to implement the

                                Blueprints API (https://github.com/tinkerpop/blueprints/wiki) on top of S-RAMP

                                so that we can compare it (to a degree) with graph databases.

                                 

                                To correctly implement that I actually need to be able to go both directions.

                                So I guess, selecting the A's in the above example would be the way forward,

                                if following queries were possible:

                                 

                                Situation:

                                A -(r; a = 1)> B

                                A -(r; a = 1)> C

                                 

                                Querying all vertices that an edge called "r", that has an attribute called a

                                equal to 1, points to from A:

                                 

                                /s-ramp/ext/Vertex[@uuid=A]/r[s-ramp:getTargetAttribute(.,"a")=1]

                                 

                                (Am I right thinking that this should work? While trying this (with

                                getRelationshipAttribute) I got a JCR-SQL2 exception).

                                 

                                To go "backwards", i.e. find all nodes that point to B using an edge that has

                                an attribute "a" equal to 1, I think the following query could work:

                                 

                                /s-ramp/ext/Vertex[r[s-ramp:getTargetAttribute(.,"a")=1 and @uuid="B"]]

                                 

                                (again, this does not currently work in S-RAMP as you point out in ARTIF-622)

                                 

                                What do you think?

                                 

                                Thanks for looking into this so quickly..

                                 

                                Btw. while the need for both directions stems currently from the Blueprints

                                API, I can easily see we are going to need this in Hawkular (the new name for

                                RHQ.next) when for example asking questions like "what are the machines this

                                application ran on last week?" (backwards direction) or "what applications run

                                on this machine now?" (forwards direction).

                                • 13. Re: "Time-boxed" relationships in S-RAMP?
                                  brmeyer

                                  Lukas, query parsing, getTargetAttribute, and getRelationshipAttribute are all fixed up.  Until the next Keycloak is released, you won't be able to pull down the SNAPSHOT from Nexus.  If you want to re-try it prior to that, pull down both Keycloak and Artificer masters and build.