2 Replies Latest reply on Feb 1, 2011 2:31 PM by Mike M

    Clustered queues hide messages when server is down

    Mike M Newbie

      Here is my setup:
      - JBoss AS 5.1.0.GA
      - JBoss Messaging 1.4.3.GA server
      - Node (server) A
      - Node (server) B

      - Unique  -Djboss.messaging.ServerPeerID set per node (1 and 2 respectively)
      - Clustering is successfully working for me (i.e. nodes are finding one another correctly and applications in deploy-hasingleton as succesfully deployed/undeployed as expected)
      - Both nodes have queue Q deployed in farm, with Clustered=true
      - See other configuration details below
      - I've read through all of the related documentation for jbossAS and jboss messaging

       

      This is my test case:
      1. Node A is running, Node B is not running
      2. Send JMS messages to node A, using aaaa:1100 (HA-JNDI lookup) and custom connection factory (ServiceConnectionFactory below)
      - JMX Console for Node A shows messages in queue
      4. Shut down node A
      5. Start up node B
      6. Attempt to receive messages from node B, using same connection as above, except using bbbb:1100
      - no messages are received
      - JMX Console for Node B shows zero messages in queue

       

      My question is what I might have configured incorrectly. My goal is for messages to be received at 6 above.

       

      Here are the relevent configuration files:

       

      - farm/jms/queue-service.xml
      {code:xml}
      <server>

         <mbean code="org.jboss.jms.server.destination.QueueService"

            name="jboss.messaging.destination:service=Queue,name=com.mycompany.MyQueue"

            xmbean-dd="xmdesc/Queue-xmbean.xml">

            <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer

            </depends>

            <depends>jboss.messaging:service=PostOffice</depends>

       

            <attribute name="Clustered">true</attribute>

       

            <attribute name="SecurityConfig">

               <security>

                  <role name="my-service-queue-sub" read="true" write="false"

                     create="false" />

                  <role name="my-service-queue-pub" read="false" write="true"

                     create="false" />

               </security>

            </attribute>

         </mbean>

      </server>

      {code}

       

      - farm/jms/ServiceConnectionFactory-service.xml
      {code:xml}
      <connection-factories>

          <mbean code="org.jboss.jms.server.connectionfactory.ConnectionFactory"

              name="jboss.messaging.connectionfactory:service=ServiceConnectionFactory"

              xmbean-dd="xmdesc/ConnectionFactory-xmbean.xml">

              <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer

              </depends>

              <depends optional-attribute-name="Connector">jboss.messaging:service=Connector,transport=bisocket

              </depends>

              <depends>jboss.messaging:service=PostOffice</depends>

       

              <attribute name="DisableRemotingChecks">true</attribute>

              <attribute name="JNDIBindings">

                  <bindings>

                      <binding>/ServiceConnectionFactory</binding>

                      <binding>java:/ServiceConnectionFactory</binding>

                  </bindings>

              </attribute>

       

              <attribute name="SupportsFailover">true</attribute>

              <attribute name="SupportsLoadBalancing">true</attribute>

       

              <attribute name="SlowConsumers">true</attribute>

          </mbean>

       

      </connection-factories>

      {code}

       

      - deploy/messaging/messaging-service.xml
      {code:xml}
      <server>

       

         <!-- ServerPeer MBean configuration

              ============================== -->

       

         <mbean code="org.jboss.jms.server.ServerPeer"

            name="jboss.messaging:service=ServerPeer"

            xmbean-dd="xmdesc/ServerPeer-xmbean.xml">

       

            <!-- The unique id of the server peer - in a cluster each node MUST have a unique value - must be an integer -->

       

            <attribute name="ServerPeerID">${jboss.messaging.ServerPeerID:0}</attribute>

       

            <!-- The default JNDI context to use for queues when they are deployed without specifying one -->

       

            <attribute name="DefaultQueueJNDIContext">/queue</attribute>

       

            <!-- The default JNDI context to use for topics when they are deployed without specifying one -->

       

            <attribute name="DefaultTopicJNDIContext">/topic</attribute>

       

            <attribute name="PostOffice">jboss.messaging:service=PostOffice</attribute>

       

            <!-- The default Dead Letter Queue (DLQ) to use for destinations.

                 This can be overridden on a per destinatin basis -->

       

            <attribute name="DefaultDLQ">jboss.messaging.destination:service=Queue,name=DLQ</attribute>

       

            <!-- The default maximum number of times to attempt delivery of a message before sending to the DLQ (if configured).

                 This can be overridden on a per destinatin basis -->

       

            <attribute name="DefaultMaxDeliveryAttempts">10</attribute>

       

            <!-- The default Expiry Queue to use for destinations. This can be overridden on a per destinatin basis -->

       

            <attribute name="DefaultExpiryQueue">jboss.messaging.destination:service=Queue,name=ExpiryQueue</attribute>

       

            <!-- The default redelivery delay to impose. This can be overridden on a per destination basis -->

       

            <attribute name="DefaultRedeliveryDelay">0</attribute>

       

            <!-- The periodicity of the message counter manager enquiring on queues for statistics -->

       

            <attribute name="MessageCounterSamplePeriod">5000</attribute>

       

            <!-- The maximum amount of time for a client to wait for failover to start on the server side after

                 it has detected failure -->

       

            <attribute name="FailoverStartTimeout">60000</attribute>

       

            <!-- The maximum amount of time for a client to wait for failover to complete on the server side after

                 it has detected failure -->

       

            <attribute name="FailoverCompleteTimeout">300000</attribute>

       

            <attribute name="StrictTck">false</attribute>

       

            <!-- The maximum number of days results to maintain in the message counter history -->

       

            <attribute name="DefaultMessageCounterHistoryDayLimit">-1</attribute>

       

            <!-- The name of the connection factory to use for creating connections between nodes to pull messages -->

           
            <attribute name="ClusterPullConnectionFactoryName">jboss.messaging.connectionfactory:service=ClusterPullConnectionFactory</attribute>

       

            <!-- When redistributing messages in the cluster. Do we need to preserve the order of messages received

                  by a particular consumer from a particular producer? -->

       

            <attribute name="DefaultPreserveOrdering">true</attribute>

       

            <!-- Max. time to hold previously delivered messages back waiting for clients to reconnect after failover -->

       

            <attribute name="RecoverDeliveriesTimeout">300000</attribute>

       

            <!-- Set to true to enable message counters that can be viewed via JMX -->

       

            <attribute name="EnableMessageCounters">false</attribute>

       

            <!-- The password used by the message sucker connections to create connections.

                 THIS SHOULD ALWAYS BE CHANGED AT INSTALL TIME TO SECURE SYSTEM

            -->

            <attribute name="SuckerPassword">omitted</attribute>

       

            <!-- The name of the server aspects configuration resource

            <attribute name="ServerAopConfig">aop/jboss-aop-messaging-server.xml</attribute>

            -->

            <!-- The name of the client aspects configuration resource

              <attribute name="ClientAopConfig">aop/jboss-aop-messaging-client.xml</attribute>

            -->

       

            <depends optional-attribute-name="PersistenceManager">jboss.messaging:service=PersistenceManager</depends>

       

            <depends optional-attribute-name="JMSUserManager">jboss.messaging:service=JMSUserManager</depends>

       

            <depends>jboss.messaging:service=Connector,transport=bisocket</depends>

            <depends optional-attribute-name="SecurityStore"

                proxy-type="org.jboss.jms.server.SecurityStore">jboss.messaging:service=SecurityStore</depends>

         </mbean>

       

      </server>

      {code}

       

      - deploy/messaging/messaging-jboss-beans.xml
      {code:xml}
      <deployment xmlns="urn:jboss:bean-deployer:2.0">

       

         <bean name="SecurityStore" class="org.jboss.jms.server.jbosssx.JBossASSecurityMetadataStore">

            <!-- default security configuration -->

            <property name="defaultSecurityConfig">

               <![CDATA[

                  <security>

                     <role name="jboss-jms-guest" read="true" write="true" create="true"/>

                  </security>

               ]]>

            </property>

            <property name="suckerPassword">omitted</property>

            <property name="securityDomain">my-security-domain</property>

            <property name="securityManagement"><inject bean="JNDIBasedSecurityManagement"/></property>

            <!-- @JMX annotation to export the management view of this bean -->

            <annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss.messaging:service=SecurityStore",exposedInterface=org.jboss.jms.server.jbosssx.JBossASSecurityMetadataStoreMBean.class)</annotation>

            <!-- Password Annotation to inject the password from the common password utility

             <annotation>@org.jboss.security.integration.password.Password(securityDomain="my-security-domain",methodName="setSuckerPassword")</annotation>

             -->

         </bean>

       

         <bean name="MessagingDeploymentTemplateInfoFactory"

            class="org.jboss.managed.plugins.factory.DeploymentTemplateInfoFactory"/>

       

         <bean name="QueueTemplate" class="org.jboss.profileservice.management.templates.JmsDestinationTemplate">

            <property name="info"><inject bean="QueueTemplateInfo"/></property>

         </bean>

         <bean name="QueueTemplateInfo"

            class="org.jboss.profileservice.management.templates.JmsDestinationTemplateInfo">

            <constructor factoryMethod="createTemplateInfo">

               <factory bean="DSDeploymentTemplateInfoFactory"/>

               <parameter class="java.lang.Class">org.jboss.profileservice.management.templates.JmsDestinationTemplateInfo</parameter>

               <parameter class="java.lang.Class">org.jboss.jms.server.destination.QueueServiceMO</parameter>

               <parameter class="java.lang.String">QueueTemplate</parameter>

               <parameter class="java.lang.String">A template for JMS queue *-service.xml deployments</parameter>

            </constructor>

            <property name="destinationType">QueueTemplate</property>

         </bean>

       

         <bean name="TopicTemplate" class="org.jboss.profileservice.management.templates.JmsDestinationTemplate">

            <property name="info"><inject bean="TopicTemplateInfo"/></property>

         </bean>

         <bean name="TopicTemplateInfo"

            class="org.jboss.profileservice.management.templates.JmsDestinationTemplateInfo">

            <constructor factoryMethod="createTemplateInfo">

               <factory bean="DSDeploymentTemplateInfoFactory"/>

               <parameter class="java.lang.Class">org.jboss.profileservice.management.templates.JmsDestinationTemplateInfo</parameter>

               <parameter class="java.lang.Class">org.jboss.jms.server.destination.TopicServiceMO</parameter>

               <parameter class="java.lang.String">TopicTemplate</parameter>

               <parameter class="java.lang.String">A template for JMS topic *-service.xml deployments</parameter>

            </constructor>

            <property name="destinationType">TopicTemplate</property>

         </bean>

       

      </deployment>

      {code}

       

      - deploy/messaging/oracle-persistence-service.xml
      {code:xml}
      <server>

       

         <!-- Persistence Manager MBean configuration
             ======================================== -->

       

         <mbean code="org.jboss.messaging.core.jmx.JDBCPersistenceManagerService"

            name="jboss.messaging:service=PersistenceManager"

            xmbean-dd="xmdesc/JDBCPersistenceManager-xmbean.xml">

       

            <depends>jboss.jca:service=DataSourceBinding,name=appserverUserDS</depends>

       

            <depends optional-attribute-name="TransactionManager">jboss:service=TransactionManager</depends>

       

            <!-- The datasource to use for the persistence manager -->

       

            <attribute name="DataSource">java:/appserverUserDS</attribute>

       

            <!-- If true will attempt to create tables and indexes on every start-up -->

       

            <attribute name="CreateTablesOnStartup">false</attribute>

       

            <!-- If true then we will automatically detect and reject duplicate messages sent during failover -->

       

            <attribute name="DetectDuplicates">true</attribute>

       

            <!-- The size of the id cache to use when detecting duplicate messages -->

       

            <attribute name="IDCacheSize">500</attribute>

       

            <!-- Some databases don't support inserting blobs using INSERT INTO ... SELECT FROM -->

       

            <attribute name="SupportsBlobOnSelect">false</attribute>

       

            <attribute name="SqlProperties"><![CDATA[

               ...

            ]]></attribute>

       

            <!-- The maximum number of parameters to include in a prepared statement -->

       

            <attribute name="MaxParams">500</attribute>

       

         </mbean>

       

         <!-- Messaging Post Office MBean configuration
              ========================================= -->

       

         <mbean code="org.jboss.messaging.core.jmx.MessagingPostOfficeService"

            name="jboss.messaging:service=PostOffice"

            xmbean-dd="xmdesc/MessagingPostOffice-xmbean.xml">

       

            <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>

       

            <depends>jboss.jca:service=DataSourceBinding,name=appserverUserDS</depends>

       

            <depends optional-attribute-name="TransactionManager">jboss:service=TransactionManager</depends>

       

            <!-- The name of the post office -->

       

            <attribute name="PostOfficeName">JMS post office</attribute>

       

            <!-- The datasource used by the post office to access it's binding information -->

       

            <attribute name="DataSource">java:/appserverUserDS</attribute>

       

            <!-- If true will attempt to create tables and indexes on every start-up -->

       

            <attribute name="CreateTablesOnStartup">false</attribute>

       

            <attribute name="SqlProperties"><![CDATA[

               ...

            ]]></attribute>

       

            <!-- This post office is non clustered. If you want a clustered post office then set to true -->

           
            <attribute name="Clustered">true</attribute>

       

            <!-- All the remaining properties only have to be specified if the post office is clustered.

                 You can safely comment them out if your post office is non clustered -->

       

            <!-- The JGroups group name that the post office will use -->

       

            <attribute name="GroupName">${jboss.messaging.groupname:MessagingPostOffice}</attribute>

       

            <!-- Max time to wait for state to arrive when the post office joins the cluster -->

       

            <attribute name="StateTimeout">30000</attribute>

       

            <!-- Max time to wait for a synchronous call to node members using the MessageDispatcher -->

       

            <attribute name="CastTimeout">30000</attribute>

       

            <!-- Set this to true if you want failover of connections to occur when a node is shut down -->

       

            <attribute name="FailoverOnNodeLeave">true</attribute>

       

            <attribute name="ChannelFactoryName">jboss.jgroups:service=ChannelFactory</attribute>

            <attribute name="ControlChannelName">jbm-control</attribute>

            <attribute name="DataChannelName">jbm-data</attribute>

            <attribute name="ChannelPartitionName">${jboss.partition.name:DefaultPartition}-JMS</attribute>

         </mbean>

       

         <!-- Messaging JMS User Manager MBean config

              ======================================= -->

       

         <mbean code="org.jboss.jms.server.plugin.JDBCJMSUserManagerService"

            name="jboss.messaging:service=JMSUserManager"

            xmbean-dd="xmdesc/JMSUserManager-xmbean.xml">

       

            <depends>jboss.jca:service=DataSourceBinding,name=appserverUserDS</depends>

       

            <depends optional-attribute-name="TransactionManager">jboss:service=TransactionManager</depends>

       

            <attribute name="DataSource">java:/appserverUserDS</attribute>

       

            <attribute name="CreateTablesOnStartup">false</attribute>

       

            <attribute name="SqlProperties"><![CDATA[

               ...

            ]]></attribute>

         </mbean>

       

      </server>
      {code}