In order to understand JBoss Messaging clustering configuration, we will start with a short clustering architectural overview, where we will identify "configuration areas", meaning architectural elements that have corresponding configuration files, and whose behavior can be changed through configuration.
One of the fundamental Messaging Core building blocks is the "Post Office". A JBoss Messaging Post Office is message routing component, which accepts messages for delivery and synchronously forwards them to their destination queues or topic subscriptions.
There is a single Post Office instance per JBoss Messaging server (cluster node). Both queues and topics deployed on a JBoss Messaging node are "plugged" into that Post Office instance. Internally JBoss Messaging only deals with the concepts of queues, and considers a topic to just be a set of queues (one for each subscription). Depending on the type of subscription - durable or non-durable - the corresponding queue saves messages to persistent storage or it just holds messages in memory and discards them when the non-durable subscription is closed.
Therefore, for a JMS queue, the Post Office routes messages to one and only one core queue, depending on the queue name, whereas for a JMS topic, the Post Office routes a message to a set of core queues, one for each topic subscription, depending on the topic name.
Clustering across multiple address spaces is achieved by clustering Post Office instances. Each JBoss Messaging cluster node runs a Clustered Post Office instance, which is aware of the presence of all other clustered Post Offices in the cluster. There is an one-to-one relationship between cluster nodes and clustered Post Office instances. So, naturally, the most important piece of clustering configuration is the clustered Post Office service configuration, covered in detail below.
Clustered Post Office instances connect to each other via JGroups and they heavily rely on JGroups group management and notification mechanisms. JGroups stack configuration is an essential part of JBoss Messaging clustering configuration. JGroups configuration is only briefly addressed in this guide. Detailed information on JGroups can be found in JGroups release documentation or on-line at http://www.jgroups.org or http://wiki.jboss.org/wiki/Wiki.jsp?page=JGroups
When routing messages, a clustered Post Office has a choice of forwarding the message to local queues or remote queues, plugged into remote Post Office instances that are part of the same cluster. Local queues are usually preferred, but if a local queue is part of a distributed queue, has no consumers, and other local queues part of the same distributed queue have consumers, messages can be automatically redistributed, subject of the message redistribution policy in effect. This allows us to create distributed queues and distributed topics. Message redistribution configuration is another subject that we will insist on.
Please note that some elements of clustering configuration are likely to change before the 1.2 final release. In particular we will try to add the ability for JBoss Messaging to use a pre-configured JGroups multiplex channel when used inside JBoss Application Server, but this is subject to availability of a AS clustering service supporting a multiplexed JGroups channel; such a service is currently being worked on by the AS Clustering team.
In JBoss Messaging 1.2.0, the JGroups configuration for each clustered Post Office instance is specified in the Post Office MBean service configuration element. The Post Office configuration is present in the persistence configuration file corresponding to the specific database you're using for the cluster instance.
So, if you are using a MySQL database instance as shared persistent storage for your cluster, the Post Office configuration for a specific node instance is available in $JBOSS_HOME/server/messaging-nodeX/deploy/jboss-messaging.sar/clustered-mysql-persistence-service.xml
An example of a Clustered Post Office configuration is shown below:
<mbean code="org.jboss.messaging.core.plugin.ClusteredPostOfficeService" name="jboss.messaging:service=PostOffice" xmbean-dd="xmdesc/ClusteredPostOffice-xmbean.xml"> <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends> <depends>jboss.jca:service=DataSourceBinding,name=DefaultDS</depends> <depends optional-attribute-name="TransactionManager">jboss:service=TransactionManager</depends> <attribute name="PostOfficeName">Clustered JMS</attribute> <attribute name="DataSource">java:/DefaultDS</attribute> <attribute name="CreateTablesOnStartup">true</attribute> <attribute name="SqlProperties"><!![CDATA[ CREATE_POSTOFFICE_TABLE=CREATE TABLE JBM_POSTOFFICE (POSTOFFICE_NAME VARCHAR(255), NODE_ID INTEGER, QUEUE_NAME VARCHAR(1023), COND VARCHAR(1023), SELECTOR VARCHAR(1023), CHANNEL_ID BIGINT, IS_FAILED_OVER CHAR(1), FAILED_NODE_ID INTEGER) INSERT_BINDING=INSERT INTO JBM_POSTOFFICE (POSTOFFICE_NAME, NODE_ID, QUEUE_NAME, COND, SELECTOR, CHANNEL_ID, IS_FAILED_OVER, FAILED_NODE_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?) DELETE_BINDING=DELETE FROM JBM_POSTOFFICE WHERE POSTOFFICE_NAME=? AND NODE_ID=? AND QUEUE_NAME=? LOAD_BINDINGS=SELECT NODE_ID, QUEUE_NAME, COND, SELECTOR, CHANNEL_ID, IS_FAILED_OVER, FAILED_NODE_ID FROM JBM_POSTOFFICE WHERE POSTOFFICE_NAME = ? ]]></attribute> <attribute name="GroupName">DefaultPostOffice</attribute> <attribute name="StateTimeout">5000</attribute> <attribute name="CastTimeout">5000</attribute> <attribute name="StatsSendPeriod">10000</attribute> <attribute name="MessagePullPolicy">org.jboss.messaging.core.plugin.postoffice.cluster.NullMessagePullPolicy</attribute> <attribute name="ClusterRouterFactory">org.jboss.messaging.core.plugin.postoffice.cluster.DefaultRouterFactory</attribute> <attribute name="AsyncChannelConfig"> <config> <UDP mcast_recv_buf_size="500000" down_thread="false" ip_mcast="true" mcast_send_buf_size="32000" mcast_port="45567" ucast_recv_buf_size="500000" use_incoming_packet_handler="false" mcast_addr="228.8.8.8" use_outgoing_packet_handler="true" loopback="true" ucast_send_buf_size="32000" ip_ttl="32" bind_addr="127.0.0.1"/> <AUTOCONF down_thread="false" up_thread="false"/> <PING timeout="2000" down_thread="false" num_initial_members="3" up_thread="false"/> <MERGE2 max_interval="10000" down_thread="false" min_interval="5000" up_thread="false"/> <FD timeout="2000" max_tries="3" down_thread="false" up_thread="false" shun="true"/> <VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/> <pbcast.NAKACK max_xmit_size="8192" down_thread="false" use_mcast_xmit="true" gc_lag="50" up_thread="false" retransmit_timeout="100,200,600,1200,2400,4800"/> <UNICAST timeout="1200,2400,3600" down_thread="false" up_thread="false"/> <pbcast.STABLE stability_delay="1000" desired_avg_gossip="20000" down_thread="false" max_bytes="0" up_thread="false"/> <FRAG frag_size="8192" down_thread="false" up_thread="false"/> <VIEW_SYNC avg_send_interval="60000" down_thread="false" up_thread="false" /> <pbcast.GMS print_local_addr="true" join_timeout="3000" down_thread="false" join_retry_timeout="2000" up_thread="false" shun="true"/> </config> </attribute> <attribute name="SyncChannelConfig"> <config> <UDP mcast_recv_buf_size="500000" down_thread="false" ip_mcast="true" mcast_send_buf_size="32000" mcast_port="45568" ucast_recv_buf_size="500000" use_incoming_packet_handler="false" mcast_addr="228.8.8.8" use_outgoing_packet_handler="true" loopback="true" ucast_send_buf_size="32000" ip_ttl="32" bind_addr="127.0.0.1"/> <AUTOCONF down_thread="false" up_thread="false"/> <PING timeout="2000" down_thread="false" num_initial_members="3" up_thread="false"/> <MERGE2 max_interval="10000" down_thread="false" min_interval="5000" up_thread="false"/> <FD timeout="2000" max_tries="3" down_thread="false" up_thread="false" shun="true"/> <VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/> <pbcast.NAKACK max_xmit_size="8192" down_thread="false" use_mcast_xmit="true" gc_lag="50" up_thread="false" retransmit_timeout="100,200,600,1200,2400,4800"/> <UNICAST timeout="1200,2400,3600" down_thread="false" up_thread="false"/> <pbcast.STABLE stability_delay="1000" desired_avg_gossip="20000" down_thread="false" max_bytes="0" up_thread="false"/> <FRAG frag_size="8192" down_thread="false" up_thread="false"/> <VIEW_SYNC avg_send_interval="60000" down_thread="false" up_thread="false" /> <pbcast.GMS print_local_addr="true" join_timeout="3000" down_thread="false" join_retry_timeout="2000" up_thread="false" shun="true"/> <pbcast.STATE_TRANSFER down_thread="false" up_thread="false"/> </config> </attribute> </mbean>
Relevant clustered Post Office configuration attributes are discussed below:
This identifies the JGroups group the clustered Post Office will connect to. All clustered Post Office instances with the same group name will connect to that group.
The maximum amount of time in milliseconds to wait when making a request to get the group state and waiting for the result. You will not normally need to change this value. Default is 5000 ms.
The maximum amount of time in milliseconds to wait when casting a message and waiting for a result. You will not normally need to change this value. Default is 5000 ms.
The period in milliseconds between much statistics for each queue will be cast across the cluster.
The fully qualified class name of the class that implements a factory for creating message routers. For different message routing policies this can be changed. Default is org.jboss.messaging.core.plugin.postoffice.cluster.DefaultRouterFactory. This factory creates routers that always favor local queues.
The fully qualified class name of the class that implements the MessagePullPolicy. The message pull policy specifies how messages are redistributed after they leave the Post Office and are delivered to queues. You can find more on message redistribution policy in the dedicated section below. By default, the message redistribution policy is org.jboss.messaging.core.plugin.postoffice.cluster.DefaultMessagePullPolicy.
The details of the JGroups configuration won't be discussed here since it is standard JGroups configuration. Detailed information on JGroups can be found in JGroups release documentation or on-line at http://www.jgroups.org or http://wiki.jboss.org/wiki/Wiki.jsp?page=JGroups
Each clustered Post Office instance can be configured to use a customizable message redistribution policy. The message redistribution policy dictates what happens with messages that are delivered to a local queue that is part of a distributed queue or a distributed subscription. In order to use a specific message redistribution policy, use the fully qualified class name of the class that implements the MessagePullPolicy.
By default, the message redistribution policy is org.jboss.messaging.core.plugin.postoffice.cluster.DefaultMessagePullPolicy which tries to redistribute messages depending on receivers's consumption speed.
To disable message redistribution completely, specify org.jboss.messaging.core.plugin.postoffice.cluster.NullMessagePullPolicy as the value of MessagePullPolicy attribute. In this case, a message is not redistributed, even if the local queue that has been delivered to has no consumers.