JBoss Messaging Configuration and Management

This document provides the base for a discussion on Messaging configuration and management facilities. Please post your comments on this forum http://www.jboss.org/index.html?module=bb&op=viewtopic&t=74846.

Overview of JBossMQ Configuration Facilities

A JBossMQ instance is an aggregation of inter-dependent MBeans whose start-up configuration is specified as attributes of <mbean> elements. A map of component MBeans, corresponding configuration files and configuration/management attributes.

Proposed Messaging Configuration and Management Facilities

Remoting

The JBossMQ InvocationLayer MBeans and their corresponding configuration files (uil2-service.xml, jvm-il-service.xml, etc.) are rendered irrelevant and replaced by Remoting configuration files.

Each Messaging server peer relies on a Remoting Connector instance. The connector's associated invokers receive and forward local and remote invocations to a JMSServerInvocationHandler installed by the server peer. Both the Remoting connector and the server peer have been written to be totally independent from each other, in that a server peer can use an arbitrary connector and a connector can forward invocation to an arbitrary number of server peers, provided that each peer uses a different remoting subsystem tag.

A JBoss 5.0 instance declares a singleton Connector in its main configuration file and by default, the server peer will use this default connector instance and its default configuration. At service start up time, the server peer will register its invocation handler (a JMSServerInvocationHandler instance) so it needs the connector's ObjectName, which will get via a <depends> element that also insures the connector is started. The Messaging server peer MBean may also exposes a convenience read-only managed attribute that returns the value of the connector's locator.

TODO: Provide a HTTP invoker configuration.

Client-side and server-side interceptor chains

The JBossMQ interceptors are replaced by standard AOP interceptors and aspects. The interceptors are not deployed as MBeans anymore so they are not configured via service deployment descriptors. The server instance advises dynamically at runtime client-side delegate classes and server-side endpoint classes, based on two AOP configuration files: aop-messaging-client.xml and aop-messaging-server.xml. Examples are provided below:

aop-messaging-server.xml:

        <?xml version="1.0" encoding="UTF-8"?>
        <!DOCTYPE aop PUBLIC
           "-//JBoss//DTD JBOSS AOP 1.0//EN"
           "http://www.jboss.org/aop/dtd/jboss-aop_1_0.dtd">
        <aop>

           <interceptor class="org.jboss.jms.server.container.LogInterceptor" scope="PER_VM"></interceptor> 
           <interceptor class="org.jboss.jms.server.container.CachingInterceptor" scope="PER_VM"></interceptor>    
           <aspect class="org.jboss.jms.server.container.SecurityAspect" scope="PER_VM"></aspect>
           <aspect class="org.jboss.jms.server.container.InjectionAspect" scope="PER_VM"></aspect>

           <!-- Connection -->
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.ConnectionAdvised->
                 $implementing{org.jboss.jms.server.endpoint.ConnectionEndpoint}(..))">
              <interceptor-ref name="org.jboss.jms.server.container.LogInterceptor"></interceptor-ref>
           </bind>

           <!-- Session -->
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.SessionAdvised->
                 $implementing{org.jboss.jms.server.endpoint.SessionEndpoint}(..))">
              <interceptor-ref name="org.jboss.jms.server.container.LogInterceptor"></interceptor-ref>
           </bind>
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.SessionAdvised->
                 createProducerDelegate(..))">
              <advice name="handleCreateProducerDelegate" 
                      aspect="org.jboss.jms.server.container.SecurityAspect"></advice>
           </bind>
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.SessionAdvised->
                 createBrowserDelegate(..))">
              <advice name="handleCreateBrowserDelegate" 
                      aspect="org.jboss.jms.server.container.SecurityAspect"></advice>
           </bind>
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.SessionAdvised->
                 createConsumerDelegate(..))">
              <advice name="handleCreateConsumerDelegate" 
                      aspect="org.jboss.jms.server.container.SecurityAspect"></advice>
           </bind> 
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.SessionAdvised->
                 createConsumerDelegate(..))">
              <advice name="handleCreateConsumerDelegate" 
                      aspect="org.jboss.jms.server.container.InjectionAspect"></advice>
           </bind>

           <!-- Producer -->
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.ProducerAdvised->
                 $implementing{org.jboss.jms.server.endpoint.ProducerEndpoint}(..))">
              <interceptor-ref name="org.jboss.jms.server.container.LogInterceptor"></interceptor-ref>  
           </bind>
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.ProducerAdvised->
                 send(..))">
              <advice name="handleSend" aspect="org.jboss.jms.server.container.SecurityAspect"></advice>
           </bind>

           <!-- Consumer -->
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.ConsumerAdvised->
                 $implementing{org.jboss.jms.server.endpoint.ConsumerEndpoint}(..))">
              <interceptor-ref name="org.jboss.jms.server.container.LogInterceptor"></interceptor-ref>  
           </bind>

           <!-- Browser -->
           <bind pointcut="execution(* org.jboss.jms.server.endpoint.advised.BrowserAdvised->
                 $implementing{org.jboss.jms.server.endpoint.BrowserEndpoint}(..))">
              <interceptor-ref name="org.jboss.jms.server.container.LogInterceptor"></interceptor-ref>  
           </bind>
         
        </aop>

The AOP configuration is loaded as resource during the service startup, so they must be in the deployment's UCL classpath.

TODO: Adding a custom client-side or server-side interceptor involves modifying one of these files. Even if they are conceptually straightforward, they look daunting and difficult to change without tool support, so we need to provide examples of how to write and configure a new client-side or server-side interceptor.

Server Peer

The ServerPeer instance is the hub into which various components plug in at startup. Currently, the server peer maintains direct references to

• DestinationManager - manages destination maps

• ClientManager - manages the active connection endpoint map

• SecurityManager - stores per-destination security configuration and maintains direct references to the AuthenticationManager and RealMapping delegates

• TransactionRepository - manages local JMS transactions

• * JMSServerInvocationHandler* - receives invocations from Remoting and forwards them to the server

• PersistenceManager (the future TransactionLogDelegate/LazyDelegate, http://www.jboss.org/index.html?module=bb&op=viewtopic&t=73650 and http://www.jboss.org/index.html?module=bb&op=viewtopic&t=73137) - O/R mapper. Must be pluggable and each implementation may have its own arbitrary configuration.

• MessageStore - references/dereferences messages. Must be pluggable and each implementation may have its own arbitrary configuration.

• StateManager - manages the durable subscription map. Must be pluggable and each implementation may have its own arbitrary configuration.

• ThreadPool - manages threads used to deliver messages to consumers. Must be pluggable and each implementation may have its own arbitrary configuration.

Currently DestinationManager and StateManager are registered as MBeans with the MBeanServer during the server startup.

In my opinion DestinationManager, ClientManager, SecurityManager, TransactionRepository and JMSServerInvocationHandler must not be declared as independent services (MBeans), nor registered with the MBeanServer. There is no reason for that, as they will never be independently re-deployed or have dependencies on their own. MBeans are useful to achieve decoupling, but these service do not need to be decoupled. They form a larger functional unit, under ServerPeer's umbrella.

This is an initial matrix of per-component JMX features that should migrate to ServerPeer management interface:

PersistenceManager, MessageStore, StateManager and ThreadPool's implementations must be pluggable. Since it is reasonable to assume that various implementations will require different configuration attributes, the best way to deploy the plugins is as services, so they automatically expose their custom management interface. The server peer should, however, get direct references to these services through their management interface (getInstance()) and invoke contract methods directly on the instance. No need for dynamic proxies, since we don't plan on hot-redeploying a PersistenceManager or ThreadPool (or do we?)

These services should be implemented as XMBeans.

As a general rule, for each service configuration parameter should be clarified whether it makes sense to be changed at runtime. If only makes sense for that parameter to be set up at start up time, then the service constructor should provide for it, and the correspoinding manangement attribute should be read-only. If it makes sense to be changed at run-time, then it should be present in the constructor and it should also have a read-write managed attribute.

Configuration parameters/management interfaces:

ThreadPool

Contract (mostly irrelevant for the configuration discussion, but interesting nonetheless):

     TODO

Management interface:

This partial list of managed atribute had been compiled based on the concurrent's PooledExecutor implementation. JBossMQ has its own implementation of a thread pool that comes with its own configuration parameters (MinimumPoolSize, MaximumPoolSize, QueueSize, MaximumQueueSize, BlockingMode, ThreadGroupName, KeepAliveTime). Just these two examples should be sufficient to demonstrate the usefulness of deploying the thread pool as separate MBeans: an XMBean's (or a simple StandardMBean's) management interface is generated dynamically so it won't be necessary to modify ServerPeer's management interface to accomodate a new ThreadPool implementation.

Again, the fact that an arbitrary ThreadPool implementation is deployed as a service doesn't mean the ThreadPool will be hot redeployable, nor does it need to be. The effects of a thread pool hot redeployment on the messaging core are hard to predict and no such feature is necessary. The ServerPeer will get a hard reference to the ThreadPool during its start() sequence, and that would be all.

PersistenceManager

TBD

MessageStore

TBD

StateManager

TBD

Putting together all these, this is how a messaging-server.xml file should look like

<server>

   <mbean code="org.jboss.jms.server.ServerPeer" name="jboss.messaging:service=ServerPeer"
          xmbean-dd="xmdesc/ServerPeer-xmbean.xml">

      <depends optional-attribute-name="Connector">jboss.remoting:service=Connector,transport=socket</depends>    
      <depends optional-attribute-name="PersistenceManager">jboss.messaging:service=PersistenceManager</depends>
      <depends optional-attribute-name="MessageStore">jboss.messaging:service=MessageStore</depends>
      <depends optional-attribute-name="StateManager">jboss.messaging:service=StateManager</depends>
      <depends optional-attribute-name="ThreadPool">jboss.messaging:service=ThreadPool</depends>
      <depends>jboss:service=TransactionManager</depends>

      <attribute name="ID">peer0</attribute>
      <attribute name="DefaultDestinationConfig">
        <destination>              
            <maximum-depth>-1</maximum-depth>
            <redelivery-delay>500</redelivery-delay>
            <redelivery-limit>10</redelivery-limit>
            <recovery-retries>10</recovery-retries>
        </destination>
      </attribute>
      <attribute name="SecurityDomain">java:/jaas/messaging</attribute>
      <attribute name="DefaultSecurityConfig">
        <security>              
            <role name="guest" read="true" write="true" create="true"></role>
        </security>
      </attribute>
   </mbean>
 

   <!-- Persistence Manager -->
   <mbean code="org.jboss.jms.server.plugins.SomePersistenceManager" 
          name="jboss.messaging:type=plugin,service=PersistenceManager"
          xmbean-dd="xmdesc/PersistenceManager-xmbean.xml">

      <depends optional-attribute-name="ConnectionManager">
          jboss.jca:service=DataSourceBinding,name=DefaultDS</depends>
     
      <!-- implementation-specific attributes -->
      .....
   </mbean>


   <!-- Message Store -->
   <mbean code="org.jboss.jms.server.plugins.SomeMessageStoreImplementation" 
          name="jboss.messaging:type=plugin,service=MessageStore"
          xmbean-dd="xmdesc/MessageStore-xmbean.xml">

      <depends optional-attribute-name="ConnectionManager">
          jboss.jca:service=DataSourceBinding,name=DefaultDS</depends>
     
      <!-- implementation-specific attributes -->
      .....
   </mbean>

 
   <!-- State Manager -->
   <mbean code="org.jboss.jms.server.plugins.SomeStateManagerImplementation" 
          name="jboss.messaging:type=plugin,service=StateManager"
          xmbean-dd="xmdesc/StateManager-xmbean.xml">

      <depends optional-attribute-name="ConnectionManager">
          jboss.jca:service=DataSourceBinding,name=DefaultDS</depends>
     
      <!-- implementation-specific attributes -->
      .....
   </mbean>


   <!-- Thread Pool -->
   <mbean code="...." name="jboss.messaging:type=plugin,service=ThreadPool">
      <!-- implementation-specific attributes -->
      .....
   </mbean>


   <!-- DLQ -->
   <mbean code="org.jboss.jms.server.jmx.Queue" 
          name="jboss.messaging.destination:service=Queue,name=DLQ"></mbean>

   <!-- Other destinations  -->
</server>

Client-side Configuration

First of all, the "Client-side Configuration" title could be misleading and needs a clarification. This paragraph describes the configurable parameters of the client-side JMS facade. However, the files that contain initial values for these parameters are located on the server-side, and JMS client code must do nothing and should not attempt to access these file or get/set configuration. All JMS client code need to do is to lookup a connection factory in JNDI and use it to create a connection. The client-side configurable parameters will be embedded in the connection factory instance or will be transparently get by the client-side JMS facade in conversation with the server.

Configurable client-side JMS facade parameters:

• Connection/Session/MessageProducer/MessageConsumer/QueueBrowser's aspect chain

• ConnectionState's thread pool implementing class and custom parameters

• SessionState's thread pool implementing class and custom parameters (TODO: do we really need so many  thread pools (n + 1)?)

• etc

The aspect configuration chains and the associated pointcuts are specified in aop-messaging-server.xml on the server side.  This configuration file is loaded by the server peer as resource during the service startup, so it must be in the deployment's UCL classpath. Upon connection creation, the client-side AOP configuration is transferred as byte array on the client, where it is loaded by a custom aspect loader.

An example of aop-messaging-client.xml is available here.

All other client-side configuration parameters should be all specified in the deployment descriptor of the ConnectionFactory that was used to create the connection. There are cases when an aspect initialization value can be specified both in the aop.xml file and the connection factory deployment descriptor. Example: the number of messages to buffer in a BrowserInterceptor. Both ways to configure the value are valid, and the value specified in the connection factory deployment descriptor takes precedence.

Connection Factories

A connection factory is deployed as an MBean. The MBean's deployment descriptor should contain:

• The connection factory JNDI name.

• The remoting protocol to use (provided that more than one protocol is made available by the connector)

• Custom client configuration (see above).

Destinations

They don't vastly differ from JBossMQ destinations. Some adjustments to map to the current implementation may be needed, but the general structure stays the same.

Destinations may be created either administratively as a result of a service deployment or programmatically, by invoking managed operations on server peer.

Deployed Destinations

During deployment, a Destination service will need to make calls into the ServerPeer instance to make itself noticed and then registered by the server. The registration will result in creation of server-managed destination state (a core channel instance plus additional state).

A Destination service interacts with the server peer via server's DestinationManager interface. A Destination service instance sends invocations into the server either via a typed dynamic proxy or via a typed hard reference (preferred, since I don't envision hot redeployment of the server peer relative to its deployed destinations).

The DestinationManager interface must not be exposed as part of the management interface. This is not for public consumption by a JMX Client.

   DestinationManager
   {
      registerDestination(...)
      unregisterDestination(...)
   }

However, a destination may be hot redeployed relative to its server peer, so the server peer will send JMX invocations into its destination services (total decoupling)

TODO: Unit test for a destination hot re-deployment.

Dynamically Created Destinations

The user must be able to create destination using the Server's JMX interface (interactively using a JMX management client, or programmatically, via a JMX calls).

Destination management methods:

     createDestination(...)

No destroyDestination() is necessary. A newly created destination can be further managed (including shutting it down) via its own JMX management interface, now that the destination is a service on its own.

Such a destination creation must end in a new Destination service being deployed (?), so the user may continue to interact with the new created destination no differently than with a destination that was deployed the usual way.

It would be good if the programmatically created destinations could save their state and survive a server restart.

Clustering

TODO