Introduction

When reading this document please cast aside preconcieved ideas of what a rule engine is or how it works. The document is conceptual in nature and at this stage I would like conceptual input on how to progress the ideas, and not spend 95% of the time argueing over which symbols or keywords to use, or whether it's needed (see http://www.bikeshed.com). Expect many rough sketches, incomplete syntax etc, it is conceptual brain storming and not an implementation spec.

Production Rule Systems have a very rigid structure that hasn't changed much in 30 years. In essence they have an agenda + conflict resolution strategy. Simple "groups" have been added by various systems be it a single push/pop stacks or rufeflows. Further execution control has been hard coded via attributes.

The following is an attempt to remove this hard coded functionality and instead provide a base level of abstractions and life cycles which the user can extend and hook into. Existing PRD behaviour can be easily provided however it is hoped via "macro's" power users can implement other useful types of behaviours and provide those to end users encapsulated with the macro. It also allows the engine to used beyond the ideas conceived by the core development team. I hope to also introduce a macro concept for rules themselves and  the general "macro" concept will allow a more pattern oriented approach to rule engine development.

Module

The module concept can be introduced as generic container for rules. The module consists of a name and arguments. Parenthesis can be ommitted if no arguments are specified. There are also further member variables, internal to the module. A module supports attributes. A module can specify a single parent module, there is a MAIN module which is the default parent of all modules. A module may optional extend other modules

<@attribute(....)>*
module <module_name> ( <arguments>*)
   <parent module_name>?
    <extends <module_name>+>*
    <variables plus optional initialisation>*

A DRL can have multiple modules specified. Within the DRL file all rules that come after that module name are associated with the module.

module A
   rule r1....
   end

   rule r2...
   end

module B(String s, Person p)
    var Book b

    rule r3...
    end

    rule r4...
    end

NOTE) Can modules span namespaces? What does a namespce mean for a module, does it live within the namespace or is it global? For now assuming scoped to namespace.:

A module can be called like a java Runnable, the caller's method signature must match the specified arguments or a runtime exception is thrown

Module m = kruntime.packages["org.domain"].modules["A"]
ModuleHandle bhA = m.run()

m = kruntime.packages["org.domain"].modules["B"]
ModuleHandle mhB =m.run( "S", new Person("darth") );

The parameter variables are available to the rules inside of the module, in a similar manner that globals are. An array pattern referencing the arguments is injected as the root pattern to each rule. This not only makes the variables available but acts as a control object for when rules can fire with those variables. The ModuleHandle has a "stop" method that results in that array element being retracted, and thus the rule cannot fire any more for those variables. Even rules associated with that module that do not depend on those variables has the root control fact injected.

mhB.stop()

Note that a module can be called multple times with different arguments, each results in a different ModuleHandle. Should a module be called with existing arguments the previous ModuleHandle is returned. So you can consider a module instance module + variables.

Passed module arguments may or may not be facts, if they are facts they may be modified (unlike globals) and patterns will respect and react to that change as normal. The module's called signature is updated.

The role of member variables is for scoped data avaliable to events, such as counters or intermediary objects.

NOTE) If we want to make member variables available to rules, we will need to think carefuly about the behaviour, as they have more potentially more complications than globals. Currently they are only available as fields on events, see the activation-group variance.

Match

A Match is the same as the traditional Activation concept in PRD systems. It has an array of FactHandle's for the matched objects. But we recognise the Match can be active or dormant, and use a boolean to represent this, and that even information on dormant matches can be useful in reasoning systems. A Match is considered active if it is ellible for for foring and has not yet done so.

Match
    FactHandle[] facts
    boolean active

NOTE) Maybe instead of "boolean active" we should have an enum, and possible also including "running" as a state. ACTIVE, RUNNING, DORMANT.

NOTE) It's possible for a rule + facts to fire multiple times without relaxing (made false). We should optional keep a counter for the number of times fired, and even each execution can be time stamped.

Module Properties

Module Properties
activeSize // number of active Matches, queries can be used for more complex cases using rule attributes
dormantSize // number of dormant Matches, queries can be used for more complex cases using rule attributes

activeMatches // Collection of active Matches
dormantMatches // Collection of dormant Matches

NOTE) The size and and collection properties only contain relevance for eager matching algorithms that compute an entire cross product for each WME change. Future versions of Drools may optionally implement some lazy matching and in those situations the user's understanding of those properties may be a problem.

Module Methods

Module Methods
cancel() // all active matches are made dormant
cancel( Match ) // cancels a match, i.e. sets it dormant, if already dormant this method does nothing.

refresh() // all dormant activates are made active, however filters such as calendars, enabled etc are still obeyed
refresh( Match ) // refreshes a match, i.e. sets it active if it's dormant, if it's already active it is ignored

halt() // The state of the module is preserved, but no 
continue()// 

addListener() // type inference adds the listener implementation to the correct list(s). 
                    // Listener composition for  single instance is allowed, meaning it is added to multiple listener lists.

start(Object[] args)
stop() 

Module Events and Lifecycle

A Module has a life cycle to which listeners may be added.

Module Events
onEnter // when a module is called
onExit  // when a module is stopped
onMatch // when a rule is matched
onRematch // when a rule is matched and is matched again, without relaxing first (via update)
onUnmatch // when a rule stops being matched
onBeforeFire // before a rule for this module fires
onAfterFire // after a rule for this module fires
onHalt // tells the listener halt has been callled
onResume // tells the listeners resume ahs been called
onEmpty // when the size == 0, all Matches are dormant

-- other potential events --
onBeforeRuleEvaluation // before a wme insert/update/delete for this module
onAfterRuleEvaluation// aftera wme insert/update/delete for this module

onEmpty will only be triggered after the first rule evaluation phase. i.e. it does not fire after onEnter, before the rule evaluation phase has had a chance to execute.

NOTE) Other potential events are before/after rule evaluation. Each WME insert/modify/update causes a rule evaluation, and could be listend to via onBeforeRuleEvaluation and OnAfterRuleEvaluation. This poses problems with concurrency where potentially multiple insertions/updates/modifies could be happening at the same time. So at best it's a listener scoped to a fact, we cannot guarantee the resutling matches are assocated with this event, unless serial rule evaluation is enforced - which may be a possible configuration for a module.

DRL will support the ability to declare literal functions attached to these listeners, the exact syntax for this is TBD. But the keyword will probaby be "on.". We will support .Net style delegate operators for = and += when adding or setting listeners.

on.Enter += {

}

Those listeners can also be added from java code.

Simulation the Agenda

A Module itself doesn't do anything other than obey the life cycle of the prescribed events, it doesn't even fire a rule, making it dormant. The only thing the Module does is maintain the list of active and dormant matches.

However this then allows a much more flexible system to which the end user can customise the behaviour. For instance the traditional agenda+conflict resolution strategy can be implemented via the onMatch/onRematch/onUnmatch listeners.

class Agenda implements onMatch, onUnmatch, onRematch {
    ....
}

The above class implements a composition of event listener interfaces, but it only needs to be added to the module once

module.addListener( new Agenda() );

This means each module may have it's own conflict resolution strategy. Some may want more traditional lifo execution, but anything else is possible. Such as rule definition order, or async execution. Listeners can be combined, but that means that order IS important. Some listeners may want to preevent a rule from firing, others may want to prever other rules from firing after the current rule has fired.

Async rule execution

Fire each match asyncronously as it matches.

onMatch + { asyncFire( match ); }

Manual Interaction Agenda

Because the listeners are now fully pluggable and whether a rule fires or does not, or is cancelled is fully pragmatic it is possible to just expose the Module via an interactive GUI and the end user can see the conflict set and specify which rule(s) to fire.

As WME actions occur the users GUI will be updated showing the active matches, however none will fire without user selection. This means the user can interrogate the state of each Match and the fact it contains to select whch rules to fire. The user can also see dormant matches and "refresh" them so they can become available for firing again.

This could be taken a step further where a user sees a dash board of modules and all are inactive and the user can manually activate modules they wish to see evaluated.

Simulating existing rule execution behviours

activation-group plus variance

Other rule attribute behaviours can be implemented by combining listeners, although listener order is obviously important. For instance this listener can be added AFTER the agenda listener, and it simulates the existing "activation-group" behaviour, such that only the first activation fires for a conflict.

activation-group

onMatch += {
    module.cancel(); // cancels all other matches
}

We could do a variance on this that fires the first 3 activations, and cancels the rest

onEnter += { count = 0 } // count is a module member variable
onMatch += {
    if ( count++ == 3 ) {
         module.cancel(); // cancels all other matches
    }
}

ruleflow-group

ruleflow-group behaviour is very simple to simulate. jBPM injects a trigger member variable, that on exit calls telling jBPM to trigger the next nodes in the flow, which may be ruleflow-group nodes or other jBPM nodes

onEmpty { module.stop() } // stopping the module causes onExit to fire

onExit + {

       node.triggerCompleted()

}

agenda-groups

agenda-groups implement a push/pop stack behaviour. Only the stack tip executes, the others are considered "halted". To acehive this the consequence of any rule needs to call the following three methods:

module1.halt();

module2.onExit += { module1.resume() };

module2.call();

So that halts the current module and executes "pushes" the next module, when that module finishes "pops" the caller is resumed.

Simulating agenda-filters

If a match is cancelled, and made dormant it is not propagated to the next listener in the onMatch event.

onMatch += {

    if ( <boolean expr> ) {

        module.cancel( match );

    }

}

Events as facts

Events are also inserted as facts, so can be matched in rules too. Member variables are fields on the events

rule  activationgroupvariance when
    m : OnMatch( count < 3 )
then
    m.count++
end
 
Parent Modules and Module Scoping
Modules can contain sub modules, they can ony be called from within the scope of the parent module.
Sub modules will have access othe parent modules member variables and parameter variables.
 
Module Reuse and Extension
A module can extend other modules. The exact semantics of this will need a lot more extensive though, but i'll outline something to get started. 
When a module extends another module it effectively copies the definitions of all the contained rules.
However those rules will match and fire completely indepedantly of the source.
 
Macro's to abstract module configuration

All this manual configuration would become very cumbersome to the end user, and potentially far too complex. The idea is that all the above complexity is for power users, this is then encapsulated via macro's for end users. I don't intend to outline a Macro system for Drools yet, as that is a different subject in itself. For now I'll use <macro "name">, pontentially these can be listed. But how macro's are defined and applied are completely open to debate. These macro's are then expanded at complie time to provide all the required listener behaviour:

module xxxx
    macro activationGroup( "g1" )

module xxx
    macro agenda

module xxx
    macro agenda activationGroup( "g1 )

module xxx
    macro default

Concurrency, Data Isolation and Transactions

That modules can be called concurrently opens up a whole can of worms. For modules to achieve their full potential I believe they need to support MVCC like data modules, to isolate their changes from other modules via transactions, combined with optimistic or pessemestic locking strategies and synchronization techniques. However what are the transaction boundaries and how are they defined? How do modules/rules handle locking strategies? This is a huge TBD