Video: http://youtu.be/mf-cJ5D4w_E

(Apologies for the quality of the video - hopefully will be good enough to get an idea of the concepts.)

I’ve been working on a prototype for a new sub-project (called Scepta) aimed at providing common ‘policy authoring and execution’ capabilities for use within Runtime and Design Time Governance tooling.

As well as reviewing what common capabilities could be shared between design and runtime governance solutions, the other objective has been to better align with other RedHat/JBoss open source projects.

For the prototype, Apache Camel was selected as the runtime framework on which to build this policy based governance solution for the following reasons:

• It is well supported by Red Hat
• It has a large number of connectors, which is ideal for a design time governance solution where interacting with many source and target systems is useful

However it has the following drawbacks when considering its applicability for defining policies:

• It combines operational considerations with the logic
• When using multiple simple routes, that communicate via JMS (activemq) for scalability, it is not efficient enough to process the activity events generated for runtime governance one event at a time - and manually enhancing the routes to provide batch/retry capabilities adds complexity to the individual routes

The solution that has been prototyped is to abstract out most (if not all) of the operational considerations (including support for batch/retry) and allow the routes describing the policies to be as simple as possible. The operational considerations can therefore be managed independently (by appropriate users/administrators) and applied to the policy/route when generating the executable representation.

Defining Policies

Underpinning the policy mechanism is camel, and therefore the policies are defined using a camel route (using the Spring XML DSL). In the initial prototype this is specified as XML - however the intention is to provide:

• graphical (drag n drop) editor for creating the route (with source view to enable direct editing if experienced user wants to edit xml text)
• templating - the policy (camel route) and associated resource files can be defined with replaceable fields, allowing business users to ‘instantiate’ a template being presented with a form to fill in the values for those fields
• language preference focused editing - essentially if the user prefers to define policies using a particular language (e.g. groovy scripts) then the policy editor could be focused on defining the policy using that notation, with additional information about the input and output endpoints.

Integration with Hawkular

Also looking at integration with Hawkular (new management project). Areas of current consideration:

• Metrics - for displaying performance feedback from policy execution
• Accounts - uses same ‘organization’ concept based on KeyCloak (which would be the SSO solution of choice for Scepta anyway)
• Cassandra - prototype currently uses cassandra (although other dbs could be used), but this would make it compatible with a hawkular environment
• Bus - based on activemq - so Scepta’s policies could use activemq to interact with hawkular’s components via the bus
• Alerts - RTGov uses the concept of Situations as a way to notify areas of concern. Hawkular has an alert mechanism used for the same purpose, so should investigate overlap.

Integration with APIMan

APIMan is an API Management project being developed at Red Hat. It provides the ability for custom policies to be “plugged in” to govern how an API is used.

An investigation should be carried out to determine whether policies defined in Scepta (including templated) could be deployed to APIMan and provide the basis for more easily including adhoc policies within an API management context.

Policy Deployment Options

• Artificer - deploying into a S-RAMP repository provides options for an organization to manage and deploy the policies
• Docker - generation of docker images, in support of…...
• Kubernetes/Fabric8/Openshift - management of policy deployments in a cloud environment - useful for obtaining performance feedback and being able to adjust operational aspects of the policies and/or deployed configuration of policies to improve performance. Scepta prototype UI uses hawt.io so makes integrating into fabric8/openshift console easier.

More on Characteristics

Endpoint Characteristics (Not explained too well in the video so here is some additional info):

Endpoints are essentially a logical name (used in the policy/route) that maps onto a physical camel URI - along with appropriate consumer and producer options.

Characteristics are more about how the endpoint will be used by consumers and producers.

The characteristic referenced in the video is “BatchWithRetryOnFailure”, which as the name suggests, batches up events being produced and handles them as a batch within a consumer - and while processing the individual events in the consumer, if an error occurs, then the individual event is batched up in a retry list, to be sent back to the consumer for a further attempt.

Other characteristics that could be supported are:

• pub/sub or queue semantics, governing whether multiple consumers should be supported (within a policy group) on the same endpoint.
• idempotent consumer - for example if we have a polling consumer (such as github) then in a clustered environment we may receive the same event multiple times. Adding the  idempotent consumer characteristic could prevent two nodes from processing the same event.
• idempotent producer - again in a clustered environment, we may want multiple instances of the same CEP rule to receive the same source events, but only want one of them to produce the results - this is necessary for temporal rules where the state is accumulated over a time period

These qualities are operational issues and therefore should not be defined within a policy, but instead by administrative staff that understand the environment in which the policies are going to be deployed.

Implications for RTGov

RTGov is essentially made up of three parts - the embedded activity collector (including validator and event pre-processor), the activity server and the UI. What is demonstrated in the video represents a potential replacement for the ‘activity server’, thus the collector and UI would still remain.

However it would also be interesting to explore how the activity validator functionality, currently triggered during activity collection in the client environment, could leverage scepta policies.

Implications for DTGov

Currently DTGov listens for events from the Artificer (S-RAMP) repository and if relevant processes those events using a BPMN2 process. This capability could be replicated in Scepta by listening for, or being notified of, events from Artificer and invoking a BPMN2 process - although currently there is no camel connector for this, so would need to build something.

More important for DTGov is the wide range of new source and target systems that can now be used within policies, and the fact that policies can be defined in other ways (rather than BPMN2). So for DTGov it is more about the additional management reporting tooling that could be built as an extension to the policies being executed in Scepta.

Implications for Artificer

TODO

Special thanks to Brett Meyer for implementing the Github and Jira camel connectors, and providing the Java implementation of the DTGov example upon which the ValidatePR policy was based.