Core Concepts - s-ramp + modeshape (division of labor)

[eric.wittmann]

Overview

S-RAMP is using ModeShape as the backing store for all the artifact stored in the S-RAMP repository (I'm psyched that both S-RAMP and JCR refer to it as a 'repository' - it's not at all confusing when discussing them together).

At the moment, the latest version of ModeShape has some S-RAMP smarts built into it.  Specifically, ModeShape includes the following (in no particular order):

• An sramp.cnd file: defines the sramp data model as JCR mixin node types
• Sequencer base classes in the modeshape-sequencer-sramp module
• XSD and WSDL sequencers that extend the sramp sequencer
• XSD sequencer impl that adds a couple sramp properties (content encoding, content length)

Question

Should ModeShape be S-RAMP aware?  Or should ModeShape be responsible for understanding XML, Schema, WSDL, etc file types but without applying any sort of S-RAMP context?

Analysis

I think we have two choices - we can either fully implement the S-RAMP smarts in ModeShape itself, or we pull out the S-RAMP specific logic from ModeShape and move it into the S-RAMP project.

I'm sure there are pros and cons of both approaches.  I'll list the ones I can think of (pros in black, cons in red, neutral in grey):

S-RAMP Fully Implemented in ModeShape

• No need to refactor or remove existing ModeShape code - just add to it
• Improvements/bug fixes to the S-RAMP JCR model must follow the ModeShape release cycle
• Cannot use XML, XSD, WSDL, etc sequencers without dragging along S-RAMP properties

S-RAMP Smarts Moved from ModeShape to S-RAMP

• All Domain logic lives with the domain project
• May need to refactor ModeShape XML, XSD, WSDL, etc sequencers so they can be extended for s-ramp
• S-RAMP code is responsible for managing the S-RAMP data model (node type mixins)
• Improvements/bug fixes to S-RAMP smarts follow S-RAMP release schedule

Hm...I thought I'd be able to come up with more than that.  Well, I'll leave it there for now.

Thoughts?

[objectiser]

Hi Eric

I think as part of the initial prototyping work, to help get things up and running, having the s-ramp aspects in ModeShape may have been for expediency - especially since the s-ramp project has only recently been started as a standalone project.

However longer term I think it makes sense that all s-ramp related capabilities should be in the separate project, so that ModeShape is domain independent.

Regards

Gary

[kurtstam]

Hi Eric,

Randall had already implemented the JCR model for S-RAMP, and I just started using it. I don't think there is an issue with us deploying an updated S-RAMP jcr model to Modeshape if we need to. I think the sticking point is 'extending the sequencers'. They are a bit like stored procedures in a database. It has some advantages to have modeshape do the sequencing (faster and you can't bypass it), but it makes us dependent on the modeshape release cycle in a way unless we can extend or maintain. I think Randall has been thinking about making them more pluggable. Hopefully he can shine some light on this.

--Kurt

[eric.wittmann]

I agree that the sticking point is 'extending the sequencers'.    If we want to pull the s-ramp stuff out of ModeShape, then the easiest thing to do would probably be to refactor the existing sequencers a bit so they can be extended by s-ramp.  So there would be an SrampXsdSequencer (for example) that extended the XsdSequencer class provided by ModeShape.  The alternative seems harder:  change ModeShape to support a sequencer chain to give multiple sequencers a crack at the content.

[rhauch]

First, a little background.

ModeShape has had the notion of sequencers since the project's inception, and we wanted XSD and WSDL sequencers (among others) to extract the useful bits for data service functionality. However, before we implemented these two sequencers, I became aware of the S-RAMP effort, which conveniently defined an "object model" for both XSD and WSDL. We simply incorporated these object models into JCR node types. Other than this, there is nothing "S-RAMP-specific or -aware" in ModeShape.

If these sequencers are not compatible with the S-RAMP implementations needs, please let us know so that we can either change them. Of course, it's always possible for the S-RAMP implementation to own its own XSD and WSDL sequencers, though I think that just kicks the can down the road, since ModeShape users that want the S-RAMP sequencers would then have to wait for an S-RAMP release. :-)

And anyone can implement their own sequencers and plug them into ModeShape. With 3, we've changed the sequencer API to make it far easier to write one - it largely is just implementing an interface and using the JCR API.

Secondly, ModeShape's sequencers are already pluggable. The S-RAMP implementation can provide its own sequencers. And with ModeShape 3, we've tried to make it far easier to implement your own custom sequencers (implement an interface, use custom logic to read the content, and use the JCR API to generate the node structure). Plus, with ModeShape 3, you can manually call the sequencers from client code - if this is important, please try it and give us feedback as soon as possible so that we can make alterations to the API before 3.0.0.Final.

If we want to pull the s-ramp stuff out of ModeShape, then the easiest thing to do would probably be to refactor the existing sequencers a bit so they can be extended by s-ramp.  So there would be an SrampXsdSequencer (for example) that extended the XsdSequencer class provided by ModeShape.

This should be easy to do. Once again, we're happy to make changes to these sequencer to make them more reusable by S-RAMP, so please let us know what they need to be. Feel free to fork the ModeShape Git repository and submit pull-requests, too.

... but it makes us dependent on the modeshape release cycle in a way unless we can extend or maintain.

This is true of any library that you want to reuse. As always, there are pros/cons for reuse vs. reimplement. And I know what it's like to wait for a release that has fixes/changes you need, and it does make things very difficult. We've been waiting for an AS7.2 release for months.

I will say this, tho: we're currently on 3-week release cycles for ModeShape 3 betas, and plan to move to 4-6 week timebox (minor) releases after ModeShape 3.0.0.Final is out. If you have any concern about our schedule, please let us know.

[eric.wittmann]

Thanks for the reply.  I'll respond and then propose:

We simply incorporated these object models into JCR node types. Other than this, there is nothing "S-RAMP-specific or -aware" in ModeShape.

The modeshape-sequencer-sramp module has some base classes and the sramp.cnd file.  The sequencers themselves (XSD, WSDL, etc) extend the classes in the sramp module.  In practice, the sequencers add a couple of sramp-specific properties (sramp:contentType and sramp:contentSize) but they don't really do anything else sramp-y.  For example, the sequencers don't add any of the nodetypes.

For the record, I think what it's doing now is great.  I propose we do this:

• Get rid of the sramp module in ModeShape and move any of the common base classes found in there to some other module (perhaps the XML sequencer project becomes the base for the XSD and WSDL ones)
• Keep the XSD and WSDL sequencers in ModeShape
• Move the sramp.cnd out of ModeShape and into S-RAMP
• Refactor the ModeShape XSD and WSDL sequencers a bit so that S-RAMP can extend them
• Implement XSD and WSDL sequencers in S-RAMP that extend the XSD and WSDL sequencers that come with ModeShape (the S-RAMP ones would add S-RAMP node types and properties)

Basically I think we want to subclass the sequencers in S-RAMP just so we can add the S-RAMP specific properties and node types to the derived nodes.  We can make it a separate project in the S-RAMP codebase in case users want S-RAMP sequencers without actually using S-RAMP (not sure what the use case for that would be).

I'm happy to do this work if you agree it's the right approach.  I'll fork ModeShape and submit a pull request when it's ready.

This is true of any library that you want to reuse. As always, there are pros/cons for reuse vs. reimplement. And I know what it's like to wait for a release that has fixes/changes you need, and it does make things very difficult. We've been waiting for an AS7.2 release for months.

I will say this, tho: we're currently on 3-week release cycles for ModeShape 3 betas, and plan to move to 4-6 week timebox (minor) releases after ModeShape 3.0.0.Final is out. If you have any concern about our schedule, please let us know.

You're right, certainly.  And I don't think waiting on ModeShape releases would, in practice, be a problem.

[rhauch]

For example, the sequencers don't add any of the nodetypes.

They do, actually. See the XsdSequencer.initialize(...) and XmlSequencer.initialize(...) methods for two examples.

• Get rid of the sramp module in ModeShape and move any of the common base classes found in there to some other module (perhaps the XML sequencer project becomes the base for the XSD and WSDL ones)

Or rename the current 'modeshape-sequencer-sramp' module and packages? Is the use of "SRAMP" the problem, cause that's easily remedied. Please open an issue and we'll address. Note that the XSD and WSDL sequencers do not currently extend the XML sequencer (there's no point, since they all use their own parser mechanism).

• Keep the XSD and WSDL sequencers in ModeShape
• Move the sramp.cnd out of ModeShape and into S-RAMP

Removing the 'sramp.cnd' from ModeShape would be difficult, since it's required (and used) by the XSD and WSDL sequencers during initialization. Note the XML sequencer doesn't use the 'sramp.cnd' node types. Note that the "sramp.cnd" file is in the "org.modeshape.sequencer.sramp" package/namespace, so it's only used to register node types if the XSD or WSDL sequencers are configured.

• Refactor the ModeShape XSD and WSDL sequencers a bit so that S-RAMP can extend them
• Implement XSD and WSDL sequencers in S-RAMP that extend the XSD and WSDL sequencers that come with ModeShape (the S-RAMP ones would add S-RAMP node types and properties)

We certainly can refactor these sequencers so that you can extend them. But we can't remove the node types or make these generic without completely sacrificing the functionality. If you don't like the node types in the "sramp.cnd", then we can change them (obviously my preference) or you can choose to implement your own sequencers with your own node types and simply not use the ModeShape sequencers.

Basically I think we want to subclass the sequencers in S-RAMP just so we can add the S-RAMP specific properties and node types to the derived nodes.  We can make it a separate project in the S-RAMP codebase in case users want S-RAMP sequencers without actually using S-RAMP (not sure what the use case for that would be).

I'm happy to do this work if you agree it's the right approach.  I'll fork ModeShape and submit a pull request when it's ready.

That sounds great, actually! Please create a JIRA issue first, with a summary of the intended changes (re my concerns above).

[eric.wittmann]

You're right - I had to dive in a little deeper to get a beter understanding of what's going on.  What I didn't realize is that the xsd.cnd basically extends the sramp.cnd.  I guess what I was proposing was to invert (logically) that relationship, so that sramp.cnd sat on top of xsd.cnd.  However, I see why that's difficult now.

I really am trying to figure out what's the Right Thing. 

I think the choices are:

1) The ModeShape sequencers for XML, XSD, WSDL, Policy, etc are all fully S-RAMP aware.

2) The XML, XSD, WSDL, etc sequencers know nothing about S-RAMP but are extended to add these features

If we stick with #1 then I think S-RAMP developers would help out on the impl and contribute that code directly to ModeShape.

If we do #2 then I think there's a small amount of refactoring in the sequencers and then they are extended in an s-ramp project.

I think, at this point, I'm happy to do either of these.  But I'm still not sure which one is best...

[rhauch]

I really am trying to figure out what's the Right Thing. 

Understood. Me, too.

I think the choices are:

1) The ModeShape sequencers for XML, XSD, WSDL, Policy, etc are all fully S-RAMP aware.

2) The XML, XSD, WSDL, etc sequencers know nothing about S-RAMP but are extended to add these features

If we stick with #1 then I think S-RAMP developers would help out on the impl and contribute that code directly to ModeShape.

If we do #2 then I think there's a small amount of refactoring in the sequencers and then they are extended in an s-ramp project.

I think, at this point, I'm happy to do either of these.  But I'm still not sure which one is best...

Remember that the S-RAMP implementation will not be the only thing that relies upon the XML, XSD, and WSDL sequencers. The advantage of #1 is that the derived output of the sequencers is always the same, whereas with #2 it will be different in ModeShape/EDS versus S-RAMP.

As for #1, we absolutely would welcome any help, and are very open to change the sequencers (and the structure of their derived output) so that they work correctly for S-RAMP's needs.

And if #1 is chosen, then let's start by creating issues in ModeShape's JIRA -- perhaps one each for enhancing/improving/correcting the XSD, XML and WSDL sequencers (or a single one for all three; not sure it matters to me). Then on those issues we can start identifying and educating us about what needs to change, and you can link any pull-requests with proposed changes.

[kurtstam]

If someone wants to use their JCR implementation rather then ModeShape, we said before you could do this, but you'd still have to run modeshape on the side so the sequencers can do their job. Does choice 2 make it easier to run with your own JCR implementation?

[rhauch]

If someone wants to use their JCR implementation rather then ModeShape, we said before you could do this, but you'd still have to run modeshape on the side so the sequencers can do their job. Does choice 2 make it easier to run with your own JCR implementation?

Honestly, I don't think it matters one way or the other. Running sequencers with other JCR implementations will require a bit of work on ModeShape's part. (Right now, the sequencers are passed ModeShape-specific interfaces, but that's fixable. Want to log an issue?).

Option 2 implies that the S-RAMP custom sequencers extend/reuse ModeShape's sequencers, so S-RAMP will continue to have a dependency on ModeShape libraries (but no need to rely upon the JCR feature). And S-RAMP will have access to all of it's custom sequencers plus those in the ModeShape sequencer JARs.