0 Replies Latest reply on Oct 5, 2006 7:14 AM by adrian.brock

    Quick design notes.

      I don't think this was ever posted publicly in one unit?
      Bits of it are spread around different threads. :-)

      Deployers Design
      ----------------
      
      This is split into three sections:
      1) Definitions - to try to avoid the
      "Humpty Dumpty syndrome"
      2) Requirements - defines what we
      are trying to achive
      3) Issues/Design - complications that result
      from the requirements
      (some very basic attempt at design)
      4) Priorities - which doesn't actually
      prioritize anything :-), it just states
      the initial "must haves".
      
      WARNING: This is very much a quick
      brain dump that brings together
      thoughts from previous discussions
      (both verbally and in the forums).
      
      There are certainly bits missing,
      templates and "use case" configuration
      spring to mind. :-)
      
      Definitions:
      
      Archive - a packed deployment, e.g.
      a jar or zip
      
      Aspect - Cross cutting concern, see orthogonal
      
      Component - a part of a deployment, e.g.
      an ejb deployment can have many ejbs,
      a web deployment can have many servlets
      
      Deployer - a plugin that does the real
      deployment work
      
      Developer - JBoss developer or experienced user
      
      Explicit Rule - a rule that is defined in
      meta data to say "this is what I want"
      when an implicit rule doesn't work for them.
      
      Factory Settings - the configuration as
      delivered by JBoss or the vendor of the
      deployment
      
      Implicit Rule - a rule that is a default
      to ease the users life. i.e. a convention
      that uses a default behaviour without them
      having to specify anything
      
      JSR77 - the JavaEE management/monitoring/stats
      spec based on a hierarchical namespace model
      similar to JMX.
      
      JSR88 - the JavaEE deployment spec based on
      the idea that xpaths (xml locations) are
      modified in a GUI
      
      Metadata - The data applicable to a deployment
      aspect. I use this term to mean the xml, e.g.
      ejb-jar.xml, jboss.xml, web.xml, etc.,
      but this also includes the annotations from
      Javaee4 which potentially can be defined
      by the generic metadata, see Metadata(2).
      
      Metadata(2) - Generic metadata (annotations/objects)
      attached to a scope. e.g. you can attach an
      @TransactionTimeout(300) at the server scope or
      an @TransactionTimeout(600) for a whole deployment.
      
      Meta data location - the place to look for
      meta data, e.g. META-INF or WEB-INF
      
      Managed Object - The object model (java object)
      equivalent of a piece of Meta Data.
      
      Main Deployer - The entry point/broker
      for the deployment system
      
      Orthogonal - The idea that a complicated system
      can be broken down into non-interacting parts
      by carefully identifying those parts (in general
      not possible and can lead to overcomplication
      when all possible interations of those parts,
      regardless of likelihood, is catered for).
      
      "Open the gates" - a quote from Baron Munchausen,
      a more generic "barrier" or "valve" process.
      
      Profile - a list of preconfigured deployments
      with potentially overrides
      
      Scanner - a simple "profile" that determines
      the deployments by looking at a directory
      (emulates the old JBoss4 behaviour)
      
      Scope - the identity/context of a component
      e.g. an ejb, ejb deployment,
      resource adapter, connection factory, servlet,
      web deployment, etc.
      But can also be more general like:
      * Subsystem - JCA, EJB, WEB, etc.
      * Server
      * Cluster
      * etc.
      These are obviously hierarchical, but it
      is not a trivial hierarchy. :-)
      e.g. a deployment defines which cluster
      it is part of not the server.
      
      Structure - breaking a deployment into
      "contexts" or subdeployments that are
      processed "independently".
      
      Structural Deployer - a deployer that
      calculates what is deployable, i.e.
      determines the structure
      
      Unpacked - a directory structure
      that pretends to an archive
      
      VDF - Virtual Deployment Framework
      (explained in the text)
      
      VFS - Virtual File System
      
      Requirements - philosophy:
      
      1) Aspectization
      A developer when writing a deployer should only
      have to deal with their aspect. Other aspects,
      e.g. aop.xml, classloader.xml, security.xml
      are not their concern and should be orthogonal.
      
      2) Transparency
      A developer should not be concerned with
      the underlying structure of the deployment.
      They should be presented with a logical
      model of the deployment structure
      that works whether the deployment is
      * an archive
      * a part of an archive,
      * is unpacked
      * is semi-unpacked (e.g jar in an unpacked deployment)
      * a logical archive (scattered around an ide)
      * etc.
      
      This requires a logical model that includes:
      * Where to look for metadata (aop.xml, etc.)
      * What is the classloader to load classes
      * How to build the classloader from the
      structure (see 3)
      
      3) Structure
      Further to (2) the structural part of
      a deployment is a different concern.
      It can be:
      * Predetermined - you are told the structure
      * Calculated - you are given the "root" and
      structural deployers determine the subdeployments
      determing the model
      * Something in between
      Similarly, the classloader can be calculated
      or predetermined, the later is most useful in
      unit tests.
      
      4) Raw Metadata vs managed object (JSR88)
      Each deployer should be fundamentally
      based on a "managed object".
      Each deployer should know how to build
      the managed object from the meta data.
      It should be possible to deploy the
      following scenarios:
      a) Predetermined structure + managed objects
      b) Predetermined structure + meta data locations
      c) Calculated structure + meta data locations
      d) Others?
      
      * For the full profile/JBoss ON we want (a).
      * JBoss4 mechansism is (c)
      * Unit tests would likely use (b or c)
      
      5) Deployment should be width first
      To ease the implicit rules, deployments
      should be done in levels rather than per
      deployment.
      e.g. If the deployers are aop, classloader, ejb
      (gross simplification) via a deployment scanner
      then the protocol should be something like:
      scanner -> add all deployments to main deployer
      scanner -> main deployer - now process
      main deployer -> determine structures (where relevant)
      main deployer -> do aop for each deployment
      main deployer -> create classloaders
      main deployer -> create ejbs
      
      6) Two phase/atomic deployments
      Internally the deployment should be two phase
      Phase 1: prepare the deployment, i.e. bring
      it to the state where it is ready to go
      Phase 2: "open the gates" which makes
      it available to receive requests, i.e.
      bind to jndi, link to transports, etc.
      This enables automatic (re)deployments
      where a redeploy can do:
      a) New deployment do phase 1
      b) Old deployment reverse phase 2
      c) New deployment do phase 2
      d) Old deployment reverse phase 1
      Ideally (b) and (c) should be the
      same process atomic process.
      
      7) Two phase/JSR88
      Different but related to (6) we need to support
      the distribute and activate of JSR88
      from a client across a cluster.
      
      8) Profile
      The profile should be a list of
      preconfigured services with overrides.
      
      The profile service can deploy multiple
      profiles to make up a running instance.
      
      i.e.
      preconfigured services == managed objects (factory settings)
      overrides == changes to factory settings
      
      Can take at least two forms (see 4)
      a) Real managed objects (serialized in some way)
      b) Raw deployments where one of the structure/managed objects
      is determined at runtime
      
      9) Hot deployment
      Deployments should be hot deployable
      this includes the deployers which introduces
      extra complications :-)
      
      10) Management/Monitoring
      Each deployment/component should be assigned
      a scope which links back through
      "deployment aspects" to the JMX/JSR77
      layer. This should be tranparent to the
      deployers, except as a contextual
      object that they use to construct their components
      and their own "simple" decision of which
      part of the JSR77 model they construct.
      
      11) Embedded/Server
      The implementation should cater for simple
      bootstrap in embedded mode (including unit tests)
      which might not have the JMX, JSR77, JSR88, profile,
      hot deployment, etc.
      and the full server mode which has everything.
      
      12) On demand
      If I don't deploy a WAR then Tomcat shouldn't
      be started
      
      Issues - complications or implementation choices:
      
      (numbers correspond to requirements above)
      
      1) Aspect
      This is based on a Deployer
      which has at least the following
      prepare() - bring the deployment to the
      nearly ready state
      commit() - "opens the gates"
      managedObject() - determines managed objects
      from meta data + openmbean style definition
      jsr77MetaData() - jsr77 type/context and
      stats mapping, etc.
      jsr88MetaData() - maps managed object to the
      jsr88 xpath model
      
      2) Transparency
      This is based on a VFS model that maps
      the underlying structure protocols
      to a single VirtualFile model
      but also on a VDF that picks out
      certain parts of the VFS to provide
      things like the meta data location
      or the parts used for classloading.
      e.g. WEB-INF for wars or META-INF
      for most others
      
      3) Structure
      There are essentially three structural
      deployers:
      * JAR - classloading is the root,
      the meta data location is META-INF
      jars in the jar are subdeployments.
      This caters for most things like jars,
      zips, sars, rars, ejbs, etc.
      * WEB - classloading in WEB-INF/classes
      and jars in WEB-INF/lib, the meta data
      locations is WEB-INF, no subdeployments!
      * EAR - application.xml and jboss-app.xml
      defines the subdeployments/classloading
      the meta data location is META-INF
      
      The main complication comes from
      things like:
      a) An unpacked directory with no META-INF
      can be a deployment, e.g. just classes
      but a subdirectory is not.
      b) An xxx-service.xml or xxx-ds.xml
      in a jar is a valid subdeployment,
      but in most circumstances an xml file
      in the root of a jar is not a subdeployment.
      
      4) Meta data, managed objects, JSR88
      a) Need to define the protocol that lets
      a deployer know a managed object exists
      or whether it should try to generate
      it "on the fly".
      b) The same generation mechanism needs
      to be exposed to the profile service
      c) The managed object needs to be
      mapped to JSR88 style xpaths
      d) The managed object needs to be
      defined using a mechanism similar
      to JMX's OpenMBeans/XMBeans
      
      5) Width first deployment
      Need to be able to define
      the order for width first.
      Choices (not necessarily exclusive):
      a) Ordering, e.g. aop=100, classloading=200
      (like JBoss4)
      b) Dependency, e.g classloading depends aop
      (better but what if aop is not deployed?)
      c) something else?
      
      6) Atomic deployments
      Beyond getting this protocol defined
      this is not a requrement for the initial JBoss5
      
      7) JSR88 deployment protocol
      Spec requirement
      
      8) Profiles
      This is really a seperate topic/design,
      this should drive the deployment layer
      via the MainDeployer from
      
      9) Hot Deployment
      Lots of complications here, e.g.
      a) adding/removing deployers
      b) Reprocess previous deployments
      c) Dependencies help in what needs
      to get reprocessed (see 5).
      d) One deployment creates another deployment,
      e.g. old JBoss4 deployment scanner or farm
      service
      
      Other issues including providing a protocol
      where a managed object's property doesn't
      require a redeploy. e.g. changing a
      transaction timeout for a method can
      be done "on the fly" without redeploying
      an ejb.
      
      10) Scoping JSR77
      This is largely a deployment aspect, see
      the JMX *prototype* in the Microcontiner.
      This links in heavily with the metadata(2)
      repository and its scopes (incomplete work).
      
      11) Embedded/Server
      These will both need to be developed.
      a) The embedded so we can write some unit tests
      without bootstrapping a JBossAS instance to test
      different pieces/configs
      b) The server for JBoss5
      
      12) On demand
      This is basically an extension to the Microcontainer
      behaviour. If the deployers are designed properly
      then this is virtually automatic, however it
      does require the seperation of things like:
      a) The Web Deployer (always there)
      b) The Web Container, started when there is a deployment
      that references it
      
      Priorties:
      The initial priorities are (in no particular order):
      1) "Reproducing" the JBoss4 behaviour
      2) Passing the JavaEE5 TCK
      3) Features for JBoss ON.