Understanding and helping

[fbrier]

Just like romney, I am trying to understand what DNA does and how it can help manage information. My immediate needs are for some basic features and then expand on them. I am willing to write connectors (or sequencers? - still not understanding architecture) and UI components.

Specifically, I want to be able to pull in JDBC metadata into a repository and generate a web site of tables for each database. URL naming should be consistent so other HTML can link to it such as wikis or JavaDoc. I would like to be able to annotate that data from a web UI. I also want to pull in data from Hibernate hbm.xml or ERWin's UML/XMI format. I am not clear how the versioning would occur.

The XSD connector/sequencer is next, again with a nice web UI to expose it, similar to XMLSpy, but nicer.

Next I want to create links between columns in the database and elements and attributes in the XML schemas. I also want to create links from different levels in a hierarchy elements or attributes to specific requirements.

All of this is to evolve an enterprise view of my customer's data. They have lots of databases with different views and copies of the same data. The goal is to create a set of schemas to represent one unified view of the data with links to where it is persisted. Is this what DNA is for? Thank you.

Fred

[rhauch]

Fred:

Thanks for the great summary of your needs. DNA is indeed targeted to do what you're looking for. I'm currently working on a Getting Started document that goes into more detail about the architecture, what the various components do, and how to create your own sequencers.

But in the mean time, connectors and sequencers are not the same thing. Here's a brief description.

The federation engine will use connectors to talk to different sources to get at the content in those systems. Examples of connectors include JDBC (to access database schema), UDDI (to access registry information), JCR (to access other repositories), DBMS (to access/persist info in the database), SCM (to access files in source control), etc. The federation engine will implement the JCR API, so it really will be used as federated repositories.

Sequencers, on the other hand, process data (typically files) that is stored in JCR to extract structured content and store that also in JCR. The sequencing system can sit on top of existing JCR repositories (including federated repositories) - it basically extracts more useful information from what's already stored in the repository. And it uses the existing JCR versioning system.

So, in your post, pulling in JDBC metadata would probably be done with a connector, since the database is an external "service" that we can directly talk to and that can change its content.

Many of the other things you talked about would be best implemented by placing files into a JCR repository and letting the sequencers pull out all the useful information and also store that in the repository. Load DDL files into the repository, and let the DDL sequencers extract the structure and metadata for the database schema. Load Hibernate configuration files into the repository, and let the Hibernate sequencers extract the schema and mapping information. Load Java source into the repository, and let the Java sequencer extract the class structure, JavaDoc, annotations. Load a PNG, JPEG, or other image into the repository, and let the image sequencer extract the metadata from the image and save it in the repository. The same with XSDs, WSDL, WS policies, UML, MetaMatrix models, etc.

Roadmap: We'll start on the federation engine as soon as 0.1 is out. The 0.1 release will contain the sequencing system, and while there will be one sequencer, we hope the community will help build the ones they need. Serge has started on a Java sequencer, and MetaMatrix is starting on a sequencer for MetaMatrix models. Check out JIRA for the list of the ones we've thought of.

Your need for a web UI is very typical, which is why we also want to create a web interface (and RESTful service) that presents data using "domain-specific" views - that is, views that are specific to the type of data and user role. For example, if a user is viewing database information, the views should be structured to show all the information for a table and its columns, keys, and indexes. (This is in contrast with a "generic" node-based view where there is one page that shows the table and only links to the other columns, keys, etc. See http://www.jcr-explorer.org/screenshots.html for an example of a "generic" web UI.)

I hope this has been helpful. Again, take a look at JIRA (thanks for the comments already!), and if there's anything you want to tackle, please do so! If you are interested in working on a sequencer, start designing the JCR node types for the information your sequencer will produce. The upcoming Getting Started document will go into detail on how to implement and use your sequencer.

Randall

[virtualdatabase]

Hello Fred... Although this post is quite old I am wondering if you made any progress in sequencing Erwin XML files?  I was hoping somenoe has taken the XML sequencer and hybridized it to be specific to Erwin ( or other modeling tools that can output to XML)

Thank you..