Quick design notes.
adrian.brock Oct 5, 2006 7:14 AMI don't think this was ever posted publicly in one unit?
Bits of it are spread around different threads. :-)
Deployers Design ---------------- This is split into three sections: 1) Definitions - to try to avoid the "Humpty Dumpty syndrome" 2) Requirements - defines what we are trying to achive 3) Issues/Design - complications that result from the requirements (some very basic attempt at design) 4) Priorities - which doesn't actually prioritize anything :-), it just states the initial "must haves". WARNING: This is very much a quick brain dump that brings together thoughts from previous discussions (both verbally and in the forums). There are certainly bits missing, templates and "use case" configuration spring to mind. :-) Definitions: Archive - a packed deployment, e.g. a jar or zip Aspect - Cross cutting concern, see orthogonal Component - a part of a deployment, e.g. an ejb deployment can have many ejbs, a web deployment can have many servlets Deployer - a plugin that does the real deployment work Developer - JBoss developer or experienced user Explicit Rule - a rule that is defined in meta data to say "this is what I want" when an implicit rule doesn't work for them. Factory Settings - the configuration as delivered by JBoss or the vendor of the deployment Implicit Rule - a rule that is a default to ease the users life. i.e. a convention that uses a default behaviour without them having to specify anything JSR77 - the JavaEE management/monitoring/stats spec based on a hierarchical namespace model similar to JMX. JSR88 - the JavaEE deployment spec based on the idea that xpaths (xml locations) are modified in a GUI Metadata - The data applicable to a deployment aspect. I use this term to mean the xml, e.g. ejb-jar.xml, jboss.xml, web.xml, etc., but this also includes the annotations from Javaee4 which potentially can be defined by the generic metadata, see Metadata(2). Metadata(2) - Generic metadata (annotations/objects) attached to a scope. e.g. you can attach an @TransactionTimeout(300) at the server scope or an @TransactionTimeout(600) for a whole deployment. Meta data location - the place to look for meta data, e.g. META-INF or WEB-INF Managed Object - The object model (java object) equivalent of a piece of Meta Data. Main Deployer - The entry point/broker for the deployment system Orthogonal - The idea that a complicated system can be broken down into non-interacting parts by carefully identifying those parts (in general not possible and can lead to overcomplication when all possible interations of those parts, regardless of likelihood, is catered for). "Open the gates" - a quote from Baron Munchausen, a more generic "barrier" or "valve" process. Profile - a list of preconfigured deployments with potentially overrides Scanner - a simple "profile" that determines the deployments by looking at a directory (emulates the old JBoss4 behaviour) Scope - the identity/context of a component e.g. an ejb, ejb deployment, resource adapter, connection factory, servlet, web deployment, etc. But can also be more general like: * Subsystem - JCA, EJB, WEB, etc. * Server * Cluster * etc. These are obviously hierarchical, but it is not a trivial hierarchy. :-) e.g. a deployment defines which cluster it is part of not the server. Structure - breaking a deployment into "contexts" or subdeployments that are processed "independently". Structural Deployer - a deployer that calculates what is deployable, i.e. determines the structure Unpacked - a directory structure that pretends to an archive VDF - Virtual Deployment Framework (explained in the text) VFS - Virtual File System Requirements - philosophy: 1) Aspectization A developer when writing a deployer should only have to deal with their aspect. Other aspects, e.g. aop.xml, classloader.xml, security.xml are not their concern and should be orthogonal. 2) Transparency A developer should not be concerned with the underlying structure of the deployment. They should be presented with a logical model of the deployment structure that works whether the deployment is * an archive * a part of an archive, * is unpacked * is semi-unpacked (e.g jar in an unpacked deployment) * a logical archive (scattered around an ide) * etc. This requires a logical model that includes: * Where to look for metadata (aop.xml, etc.) * What is the classloader to load classes * How to build the classloader from the structure (see 3) 3) Structure Further to (2) the structural part of a deployment is a different concern. It can be: * Predetermined - you are told the structure * Calculated - you are given the "root" and structural deployers determine the subdeployments determing the model * Something in between Similarly, the classloader can be calculated or predetermined, the later is most useful in unit tests. 4) Raw Metadata vs managed object (JSR88) Each deployer should be fundamentally based on a "managed object". Each deployer should know how to build the managed object from the meta data. It should be possible to deploy the following scenarios: a) Predetermined structure + managed objects b) Predetermined structure + meta data locations c) Calculated structure + meta data locations d) Others? * For the full profile/JBoss ON we want (a). * JBoss4 mechansism is (c) * Unit tests would likely use (b or c) 5) Deployment should be width first To ease the implicit rules, deployments should be done in levels rather than per deployment. e.g. If the deployers are aop, classloader, ejb (gross simplification) via a deployment scanner then the protocol should be something like: scanner -> add all deployments to main deployer scanner -> main deployer - now process main deployer -> determine structures (where relevant) main deployer -> do aop for each deployment main deployer -> create classloaders main deployer -> create ejbs 6) Two phase/atomic deployments Internally the deployment should be two phase Phase 1: prepare the deployment, i.e. bring it to the state where it is ready to go Phase 2: "open the gates" which makes it available to receive requests, i.e. bind to jndi, link to transports, etc. This enables automatic (re)deployments where a redeploy can do: a) New deployment do phase 1 b) Old deployment reverse phase 2 c) New deployment do phase 2 d) Old deployment reverse phase 1 Ideally (b) and (c) should be the same process atomic process. 7) Two phase/JSR88 Different but related to (6) we need to support the distribute and activate of JSR88 from a client across a cluster. 8) Profile The profile should be a list of preconfigured services with overrides. The profile service can deploy multiple profiles to make up a running instance. i.e. preconfigured services == managed objects (factory settings) overrides == changes to factory settings Can take at least two forms (see 4) a) Real managed objects (serialized in some way) b) Raw deployments where one of the structure/managed objects is determined at runtime 9) Hot deployment Deployments should be hot deployable this includes the deployers which introduces extra complications :-) 10) Management/Monitoring Each deployment/component should be assigned a scope which links back through "deployment aspects" to the JMX/JSR77 layer. This should be tranparent to the deployers, except as a contextual object that they use to construct their components and their own "simple" decision of which part of the JSR77 model they construct. 11) Embedded/Server The implementation should cater for simple bootstrap in embedded mode (including unit tests) which might not have the JMX, JSR77, JSR88, profile, hot deployment, etc. and the full server mode which has everything. 12) On demand If I don't deploy a WAR then Tomcat shouldn't be started Issues - complications or implementation choices: (numbers correspond to requirements above) 1) Aspect This is based on a Deployer which has at least the following prepare() - bring the deployment to the nearly ready state commit() - "opens the gates" managedObject() - determines managed objects from meta data + openmbean style definition jsr77MetaData() - jsr77 type/context and stats mapping, etc. jsr88MetaData() - maps managed object to the jsr88 xpath model 2) Transparency This is based on a VFS model that maps the underlying structure protocols to a single VirtualFile model but also on a VDF that picks out certain parts of the VFS to provide things like the meta data location or the parts used for classloading. e.g. WEB-INF for wars or META-INF for most others 3) Structure There are essentially three structural deployers: * JAR - classloading is the root, the meta data location is META-INF jars in the jar are subdeployments. This caters for most things like jars, zips, sars, rars, ejbs, etc. * WEB - classloading in WEB-INF/classes and jars in WEB-INF/lib, the meta data locations is WEB-INF, no subdeployments! * EAR - application.xml and jboss-app.xml defines the subdeployments/classloading the meta data location is META-INF The main complication comes from things like: a) An unpacked directory with no META-INF can be a deployment, e.g. just classes but a subdirectory is not. b) An xxx-service.xml or xxx-ds.xml in a jar is a valid subdeployment, but in most circumstances an xml file in the root of a jar is not a subdeployment. 4) Meta data, managed objects, JSR88 a) Need to define the protocol that lets a deployer know a managed object exists or whether it should try to generate it "on the fly". b) The same generation mechanism needs to be exposed to the profile service c) The managed object needs to be mapped to JSR88 style xpaths d) The managed object needs to be defined using a mechanism similar to JMX's OpenMBeans/XMBeans 5) Width first deployment Need to be able to define the order for width first. Choices (not necessarily exclusive): a) Ordering, e.g. aop=100, classloading=200 (like JBoss4) b) Dependency, e.g classloading depends aop (better but what if aop is not deployed?) c) something else? 6) Atomic deployments Beyond getting this protocol defined this is not a requrement for the initial JBoss5 7) JSR88 deployment protocol Spec requirement 8) Profiles This is really a seperate topic/design, this should drive the deployment layer via the MainDeployer from 9) Hot Deployment Lots of complications here, e.g. a) adding/removing deployers b) Reprocess previous deployments c) Dependencies help in what needs to get reprocessed (see 5). d) One deployment creates another deployment, e.g. old JBoss4 deployment scanner or farm service Other issues including providing a protocol where a managed object's property doesn't require a redeploy. e.g. changing a transaction timeout for a method can be done "on the fly" without redeploying an ejb. 10) Scoping JSR77 This is largely a deployment aspect, see the JMX *prototype* in the Microcontiner. This links in heavily with the metadata(2) repository and its scopes (incomplete work). 11) Embedded/Server These will both need to be developed. a) The embedded so we can write some unit tests without bootstrapping a JBossAS instance to test different pieces/configs b) The server for JBoss5 12) On demand This is basically an extension to the Microcontainer behaviour. If the deployers are designed properly then this is virtually automatic, however it does require the seperation of things like: a) The Web Deployer (always there) b) The Web Container, started when there is a deployment that references it Priorties: The initial priorities are (in no particular order): 1) "Reproducing" the JBoss4 behaviour 2) Passing the JavaEE5 TCK 3) Features for JBoss ON.