Skip navigation

Artificer

10 posts

Finally!  It's...final!

 

Extensive testing of 1.0.0.Beta2 found only a handful of minor issues, so we're comfortable pressing on without another Beta release.


Obviously, not a whole lot new went into this.  Check out the full list of bugfixes: Release Notes - JBoss Issue TrackerWe did, however, sneak in a couple of enhancements:

 

 

A big "thank you" to everyone who contributed to Overlord S-RAMP 0.x and Artificer 1.0, as well as all the alpha/beta testers!  We're excited to see the Artificer community and project continue to grow and innovate.  Looking forward to the next steps on the roadmap -- onward!

 

Download: http://downloads.jboss.org/artificer/artificer-1.0.0.Final.zip

Docs: Index of /artificer/1.0.0.Final

Souce: Release artificer-1.0.0.Final · ArtificerRepo/artificer · GitHub

Docker: jboss/artificer-wildfly:1.0.0.Final


Software Artifact, Metadata, and Information Repository (Open Source) · Artificer

Welp, quite a bit happened after Beta1.  Thanks to a helpful amount of beta testers and several improvements, another beta release was merited.  Note that we're hoping to have Final out late next week, pending additional issues that come up.

 

A lot of attention was given to performance improvements in this release.  If you haven't already, check out the previous blog post: Huge Performance Increases in Artificer 1.0.0.Beta2

 

Highlights:

 

 

Numerous other fixes and improvements went into the release: Release Notes - JBoss Issue Tracker

 

Download: http://downloads.jboss.org/artificer/artificer-1.0.0.Beta2.zip

Docs: Index of /artificer/1.0.0.Beta2

Souce: Release artificer-1.0.0.Beta2 · ArtificerRepo/artificer · GitHub

Docker: jboss/artificer-wildfly:1.0.0.Beta2


Software Artifact, Metadata, and Information Repository (Open Source) · Artificer

Update: After enabling, by default, the query cache and Infinispan 2nd level cache, the write times decreased dramatically for this specific example (also 10-20x over JCR).  This is expected, due to several types of repeated queries used to initialize relationship targets, etc. that are now optimized by the cache.  On the other hand, you'll notice a slight increase in the initial query times.  This is also expected, as the first instance of each requires a "put" into the caches.  But, subsequent, repeated query executions will "hit" the cache and skip the DB entirely, the results get a lot more exciting...

 

When we announced the release of Artificer 1.0.0.Beta1, we described our switch to a new persistent storage setup (Artificer 1.0.0.Beta1 Released!).  While the initial performance results were promising, I was a bit disappointed.  However, Beta2 will change that.  After using a CQRS pattern, tweaking our use of Hibernate Search, and spending a lot of time with JProfiler, we were surprised by the magnitude of the improvements.  In our example, full-text queries improved over 10x, when compared to the old JCR setup.  Metadata queries improved a little under 20x!  Plus, neither of those used pagination, so real-world use cases will be exponentially faster.

 

2000 primary artifacts may not seem like much, even after derivation.  And that's fair enough.  However, this at least gives an idea as to why we made the decisions we did.  Even in this out-of-the-box example that skips clustering, in-memory caching, and other scaling techniques, the results are extremely promising.  Artificer should now be positioned to take on extremely large quantities of large documents, typical in enterprise development shops.

 

Note that this new setup heavily emphasizes fast reads over writes.  You'll notice that the writes have actually slowed a bit.  But, that's entirely expected, since there's a lot more to writing to RDBMS (as opposed to a JCR node), updating Hibernate Search indexes, etc.  But in the end, Artificer use cases have always leaned towards reads being the vital priority.  Although any solution has pros and cons, we feel this approach best fits where Artificer is headed.

 

A PDF of the results is attached!

First off, apologies for the delay between Alpha2 and Beta1.  But...

 

...some big things happened and the delay was 1.) necessary to finish and test them, and 2.) completely worth it!  The deets:

 

 

As always, the release details are here (including a few other minor JIRAs that I didn't mention): Release Notes - JBoss Issue Tracker

 

Note that this should be considered the feature-freeze for 1.0.  However, we plan to immediately launch into some new awesomeness for 1.1.

 

One other thing to note: we finally have a logo and new branding!  Check out the website (Software Artifact, Metadata, and Information Repository (Open Source) · Artificer), Twitter (Artificer (@ArtificerRepo) | Twitter), etc. to see it in action.  We'd love to hear your thoughts.  Oh, and you deserve to purchase yourself a T-Shirt: Artificer on Twitter: "Also, Artificer mugs, buttons, and t-shirts make great birthday gifts for your loved ones http…

 

Download: http://downloads.jboss.org/artificer/artificer-1.0.0.Beta1.zip

Docs: Index of /artificer/1.0.0.Beta1

Souce: Release artificer-1.0.0.Beta1 · ArtificerRepo/artificer · GitHub

Docker: (coming soon)

To learn more about Artificer, check out the website: artificer.jboss.org.  Full documentation, including a "Getting Started" guide, is available at docs.jboss.org/artificer


At first glance, this may sound like it's a bit of a stretch.  But bear with me -- the concept turned out much more useful than I had originally hoped...


"Getting Things Done", by David Allen, is a well known and well respected system for managing projects, tasks, and reference information, for both personal and professional uses. The system results in a large amount information chunks, many of which are related and dependent. Custom tags are also needed, as well as hierarchical metadata.

 

Sound familiar? "Getting Things Done", meet Artificer...

 

The demo populates an Artificer repo with fake reference information, as well as a system for a "Getting Things Done" style of project/task management. It includes a custom OWL ontology with some ideas for hierarchical classifiers. The beauty of "Getting Things Done" is that it's purely a skeleton system that can be molded for your own uses. That's the same idea here. This certainly isn't complete or exhaustive, but simply demonstrates what's possible.

 

In this particular setup, we make heavy use of ontologies/classifiers for nearly every aspect of GTD. They're easily used and modified during runtime, in addition to having the benefit of being hierarchical in nature. However, note that this could also be implemented using generic properties (essentially making them 'tags') as an alternative.

 

Our GTD ontology (gettingthingsdone.owl.xml) consists of:

 

  • Actions: Tasks to be acted upon. Further broken down into classifiers based on the level of complexity:
    • Simple
    • Moderate
    • Complex
  • Waiting: Actions to be acted upon, but currently held up by an external expectation (a call/email from someone, etc.).
  • Someday: Actions that are on the back-burner (ie, the "tickler file").
  • Context: Productivity increases when actions are grouped by context, rather than constantly shifting your focus. Our's include:
    • Using The Computer
    • On The Phone
    • Out Running Errands

 

Note that GTD utilizes an 'inbox', prior to organizing and classifying actions. You could certainly add that to the ontology, however we skip it here. The assumption is that other external sources (email, etc.) typical act as the inboxes.

 

Running the demo will push in an example dataset, using the ontologies to classify everything in a GTD way. Once its run, play around with the UI to see how it turned out.  Pay special attention to the Classifiers area within the sidebar filters.  That will allow you to build queries, such as:

/s-ramp[s-ramp:classifiedByAllOf(.,'http://artificer.jboss.org/gettingthingsdone.owl#ModerateAction','http://artificer.jboss.org/gettingthingsdone.owl#ComputerContext')]

IE, find all actions, moderate in complexity, that can be done while using the computer.  Or:

/s-ramp[GTDProjectRel[@name='Bathroom Remodel']]

IE, find all actions associated with the 'Bathroom Remodel' project.

 

Are those queries verbose?  Absolutely.  But that's why the UI and other interfaces exist!

 

For more info, check out the demo code itself: artificer/demos/end-to-end-use-case/getting-things-done at master · ArtificerRepo/artificer · GitHub

We've released another quick-turnaround alpha build for Artificer!  This releases adds a handful of features, improvements, and bug fixes.  In all likelihood, this should be considered the feature-freeze for 1.0, and therefore the last alpha.  I'm hoping to have a beta release out fairly quickly!

 

Highlights:

 


For the full list of changes, please see the release notes in JIRA: Release Notes - JBoss Issue Tracker


As always, we'd love to hear from you.  Check out Metadata and Software Artifact Repository (Open Source) · Artificer to learn how to get involved and how to follow the project's progress!


Download: http://downloads.jboss.org/artificer/artificer-1.0.0.Alpha2.zip

Docker: artificer/wildfly82:1.0.0.Alpha2

Docs: Index of /artificer/1.0.0.Alpha2

Souce: ArtificerRepo/artificer at artificer-1.0.0.Alpha2 · GitHub

Finally!  Artificer 1.0.0.Alpha1 has just been released!  This is several "firsts" for the project.  1.) Obviously, this is the first 1.0 release!  2.) We're finally able to claim full conformance with the OASIS S-RAMP specification.  3.) This is the first release under the new "Artificer" name!

 

There are several caveats that will need addressed when upgrading from Overlord S-RAMP to Artificer 1.0.  Check out the migration guide: Chapter 13. Migration Guides.  Note that there are a few migration tasks that have been purposefully left unfinished.  If you bump up against any of them, please contact us ASAP.  We're more than willing to work on additional migration tools, but are handling them case-by-case.

 

New feature and enhancement Highlights:

 


For the full list of changes, please see the release notes in JIRA: Release Notes - JBoss Issue Tracker.


As always, we'd love to hear from you.  Check out Metadata and Software Artifact Repository (Open Source) · Artificer to learn how to get involved and how to follow the project's progress!


Download: http://downloads.jboss.org/artificer/artificer-1.0.0.Alpha1.zip

Docs: http://docs.jboss.org/artificer/1.0.0.Alpha1

Souce: ArtificerRepo/artificer at artificer-1.0.0.Alpha1 · GitHub

To learn more about Artificer, check out the website: artificer.jboss.org.  Full documentation, including a "Getting Started" guide, is available at docs.jboss.org/artificer.

 

For this scenario, we pretend that we're a part of a large software development company, focused on providing SOA web services.  This company attempts to reuse artifacts as much as possible, reducing redundancy and increasing consistency.  Reuse is a good thing, right?  What if I, a member of a development team within the company, want to update, replace, or create a new version of that schema?  What services and what teams will be impacted?  Even more specifically, what will be impacted if I change a single type in the schema?


This is where Artificer starts to shine.  Further, this need was one of the main use cases driving the original OASIS S-RAMP specification.


In this example, assume we have two primary artifacts:

  • sample.xsd: A common schema used throughout the organization.
  • sample.wsdl: A specific WSDL controlled and implemented by another team in the organization.  Imports sample.xsd and utilizes its types.


When the XSD is uploaded to Artificer, the following artifacts are created:

  • sample.xsd (XsdDocument): The "primary" artifact
  • extInputType (ComplexTypeDeclaration): A "derived" artifact, representing a complex type declaration
  • extOutputType (ComplexTypeDeclaration): A "derived" artifact, representing a complex type declaration
  • extInput (ElementDeclaration): A "derived" artifact, representing an element declaration

 

Uploading the WSDL results in numerous artifacts.  I'm summarizing, for the sake of brevity:

  • sample.wsdl (WsdlDocument): The "primary" artifact
  • Bindings
  • BindingOperations
  • BindingOperationFault
  • BindingOperationInputs
  • BindingOperationOutputs
  • Faults
  • Messages
  • Operations
  • OperationInputs
  • OperationOutputs
  • Parts
  • Ports
  • PortTypes
  • SoapAddresses
  • SoapBindings


Artificer automatically recognizes that the WSDL imports the XSD and utilizes its types.  When that's the case, several useful "relationships" are automatically created.  Note that these relationships are bi-directional!  Traversing the tree, up and down, is supported by Artificer.

  • All derived artifacts are automatically linked to the primary/parent artifact through a "relatedDocument" relationship.  Ex: extOutputType -> "relatedDocument" -> XSD
  • WSDL -> "importedXsds" -> XSD
  • WSDL Part -> "type" -> XSD ComplexTypeDeclaration
  • WSDL Part -> "element" XSD Element


It's important to note that relationships can be defined between primary and derived artifacts!  The fine granularity differentiates Artificer and the S-RAMP spec from other alternatives.  Through this, full-blown impact analysis becomes possible.


The XSD and WSDL spaces only get us so far.  We still need additional metadata to link and describe the artifacts in other useful ways.  This is where S-RAMP's "ServiceImplementation" and "SOA" models come into play.  The models define dozens of logical, metadata-only artifacts, based on The Open Group SOA Ontology.  See the S-RAMP spec's Modeling SOA Concepts section for more info.  As a really simple example, here are a few that could be used for for our scenario:

  • Organization: Describes a specific organization or team ("actor") within the company
  • ServiceEndpoint: Describes a specific SOA endpoint


In this case, a few relationships would be helpful to at least know 1.) what services are in production, 2.) what services utilize given artifacts, and 3.) who is responsible for the services.

  • Organization -> "provides" -> ServiceEndpoint
  • ServiceEndpoint -> "definedBy" -> WSDL


Note that we're barely scratching the surface of the SOA and ServiceImplementation models -- there is a lot available in both.  However, all of the above information is enough to at least give an idea of what's possible.


The following demo code shows the above data model, example processes, and specific queries used to traverse the bi-directional relationship tree.  Feel free to use it as a basic template for coarse and fine grained impact analysis within your organization's artifacts!  Note that, although this specific demo uses the Java client, the concepts are possible in any Artificer interface!  https://github.com/ArtificerRepo/artificer/tree/master/demos/end-to-end-use-case/xsd-wsdl-impact-analysis


Running that demo results in the output, below.  Although it's a bit verbose, it attempts to adequately describe the actions.


*** Running Demo ***

 

Artificer Endpoint: http://localhost:8080/artificer-server

Artificer User: admin

 

Uploading XSD artifact (sample.xsd)...

Uploading WSDL artifact (sample.wsdl...)

 

Take a look at the primary artifacts, derived artifacts, and relationships through the UI: http://[HOST]:[PORT]/artificer-ui/index.html

 

Retrieving a specific type declaration (needed later), which was derived from the XSD...

*** Query: /s-ramp/xsd/ComplexTypeDeclaration[@name='extOutputType']

ComplexTypeDeclaration: extOutputType

 

Retrieving the Service artifact (needed later), which was derived from the WSDL...

*** Query: /s-ramp/wsdl/WsdlService[relatedDocument[@uuid='60a87a04-502c-4962-b8fd-6bea7bcfcbca']]

Service: SampleService

 

Showing that the WSDL imported the XSD and a relationship is defined...

*** Query: /s-ramp/wsdl/WsdlDocument[@uuid = '60a87a04-502c-4962-b8fd-6bea7bcfcbca']/importedXsds

*** Result: sample.wsdl imports sample.xsd

 

Showing all artifact relationships that target the given XSD (should include the 'importedXsds' from the WSDL)...

NOTE: This is an Artificer-specific endpoint, outside of the S-RAMP spec, allowing reverse/bi-directional traversal!

*** Endpoint: /artificer-server/artificer/reverseRelationships/d0abaf7e-1ff5-4151-ad0d-df230704d9e5

*** Result: sample.wsdl --> 'importedXsds' --> sample.xsd

*** Result: extOutputType --> 'relatedDocument' --> sample.xsd

*** Result: extInput --> 'relatedDocument' --> sample.xsd

*** Result: extInputType --> 'relatedDocument' --> sample.xsd

 

Creating SOA & ServiceImplementation logical artifacts:

Creating ServiceEndpoint 'ABC', which has a 'definedBy' relationship to the WSDL service...

Creating Organization 'Team XYZ', which has a 'provides' relationship to the ServiceEndpoint...

 

Ok, I want to update the content of the schema. But how do I know who/what that impacts?

Traverse the chain of relationships, starting at the XSD, to find out which organization to contact about the change!

NOTE: This is easily done in the UI (see link, above). Click on the schema, then on its 'Relationships' tab, to start.

 

Find, once again, all artifacts targeting the XSD with a relationship...

*** Result: 'importedXsds' <-- sample.wsdl (WsdlDocument)

*** Result: 'relatedDocument' <-- extOutputType (ComplexTypeDeclaration)

*** Result: 'relatedDocument' <-- extInput (ElementDeclaration)

*** Result: 'relatedDocument' <-- extInputType (ComplexTypeDeclaration)

 

Ok, looks like a WSDL is importing it. Find all relationships targeting that WSDL...

*** Result: 'relatedDocument' <-- soap:binding (SoapBinding)

*** Result: 'relatedDocument' <-- SampleBinding (Binding)

*** Result: 'relatedDocument' <-- errorFault (BindingOperationFault)

*** Result: 'relatedDocument' <-- findRequest (OperationInput)

*** Result: 'relatedDocument' <-- parameters (Part)

*** Result: 'relatedDocument' <-- findResponse (Message)

*** Result: 'relatedDocument' <-- findResponse (OperationOutput)

*** Result: 'relatedDocument' <-- emptyMessage (Message)

*** Result: 'relatedDocument' <-- parameters (Part)

*** Result: 'relatedDocument' <-- findRequest (Message)

*** Result: 'relatedDocument' <-- SampleService (WsdlService)

*** Result: 'relatedDocument' <-- find (BindingOperation)

*** Result: 'relatedDocument' <-- errorFault (Fault)

*** Result: 'relatedDocument' <-- find (Operation)

*** Result: 'relatedDocument' <-- SamplePort (Port)

*** Result: 'relatedDocument' <-- unknownFault (BindingOperationFault)

*** Result: 'relatedDocument' <-- SamplePortType (PortType)

*** Result: 'relatedDocument' <-- findResponse (BindingOperationOutput)

*** Result: 'relatedDocument' <-- soap:address (SoapAddress)

*** Result: 'relatedDocument' <-- findRequest (BindingOperationInput)

*** Result: 'relatedDocument' <-- message (Part)

*** Result: 'relatedDocument' <-- unknownFault (Fault)

*** Result: 'relatedDocument' <-- faultMessage (Message)

 

Wow, there's a lot to that WSDL. It looks like it defines a specific service. Let's check that out next.

*** Result: 'endpointDefinedBy' <-- Service Endpoint ABC (ServiceEndpoint)

 

Looks like 'Service Endpoint ABC' is defined by that WSDL service. Who is responsible for the endpoint?

*** Result: 'provides' <-- Team XYZ (Organization)

 

Alright, so 'Team XYZ' is responsible for 'Service Endpoint ABC'.

Better contact them and let them know why things are about to break...

 

More specifically, I only want to change one single type declaration: 'extOutputType'.

Within *all* WSDLs in the repo, what exactly is using that type? Again, look at the relationships targeting it.

*** Result: 'type' <-- parameters (Part)

Ok, one Part. Keep going. Which Message?

*** Result: 'part' <-- findResponse (Message)

Found the Message. Now where is it used?

*** Result: 'message' <-- findResponse (OperationOutput)

An OperationOutput. Which Operation?

*** Result: 'output' <-- find (Operation)

 

Now I know exactly which pieces of the WSDL will be affected!

 

Ok, ok, I get it. Artificer is pretty sweet...

 

*** Demo Completed ***

Once in a while, an old misconception pops up: early Overlord S-RAMP versions were sometimes seen as a simple extension of ModeShape.  It's true that there is a some overlap, when considering capabilities like ModeShape's sequencers.  But especially now, that's definitely not the truth.


JCR/ModeShape happened to be the first backing storage solution that was used.  And for good reason -- they're fantastic and the hierarchical nature is exactly what Artificer needed.  However, the architecture was purposefully abstracted in order support any other solution in the future.  For example, we're currently discussing the possibility of Cassandra, straight RDBMS, etc.  So, all capabilities need to be provided by Artificer itself, rather than strongly tying to any one backing platform and the built-in tools it may provide.

 

Further, Artificer's concepts are much more in-depth, compared to simple metadata.  For example, a WSDL can be deployed, from which we derive the Ports, Messages, Services, Bindings, etc.  Those derived artifacts can then be targeted by relationships from the SOA and ServiceImplementation models, allowing concepts like responsible Organizations to be linked to a given service endpoint.  Dependency connections can also be defined, creating the possibility for full-blown impact analysis.  And those are just a couple of examples.  The point is that the connections between artifacts, the rich metadata, the ontologies/classifications, etc. are why Artificer exists and where the possibilities start to get really powerful.

 

Just throwing that out there!  If there are ways we could better emphasize that within the community, I'm all ears!

As a user of the Artificer command line interface, do you have a set of scripts that you repeatedly run?  Tired of re-typing them?  You can obviously list those commands in a "batch file", then later execute it through the CLI.  Simply use the "-f /path/to/cli-commands.txt" argument.

 

However, a lesser-known feature of the Artificer shell is the "Log-to-File" option.  When starting the CLI, use the "-l /path/to/cli-commands.txt" argument.  All commands executed during the CLI session will be piped, line-by-line, to the given batch file.  We've also included a new "echo" command that will let your script print out useful logs.  So, rather than creating batch files by-hand, create them dynamically as you execute commands in real-time.

 

Simple, yep.  Handy, yep.