5 Replies Latest reply on Apr 6, 2005 3:40 PM by Sean Boltman

    There is no database failover capability in JMS PersistenceM

    Sean Boltman Newbie


      My research has shown me that this is a true statement:

      There is NO database failover capability in the PersustenceManager or StateManager for the JMS services under JBOSS.

      If the database that these Managers use happens to crash, you are hosed.

      HAJMS architecture does not attempt to solve this problem. HAJMS solves redundant access to a Queue, not the survival of the Queue through a database failure.

      There is not "second chance" JNDI name for the definition of the PersistenceManager and StateManager MBeans.... on that could be used when primary datasource failure is detected, and an attempt to flip over to a replicated datastore is attempted.

      Can someone can refute this statement? And show me the errors of my ways? I truely hope so. I would greatly appreciate it.

        • 1. Re: There is no database failover capability in JMS Persiste
          Elias Ross Master


          Options:

          You can do failover on the database side, i.e. with Oracle.

          You can create a new persistence manager to take two persistence managers. It would be trivial to code something that does:

           Tx createPersistentTx() {
           try {
           return pm1.createPTx();
           } catch (Exception e) {
           return pm2...
           }
           }
          


          • 2. Re: There is no database failover capability in JMS Persiste
            Adrian Brock Master

            This properly belongs in the jdbc driver/database layer.

            There is a feature request to add a simple version of this feature to JBoss's
            jdbc rar.

            • 3. Re: There is no database failover capability in JMS Persiste
              Sean Boltman Newbie

              Well, rather than wait for a JBOSS that supports this, or workaround suggestions from the Chief Scientist, we had to solve this problem *today*, and here is what we came up with that works:


              JBOSS JMS Failover

              The TDX is designed to use JBOSS JMS services when deployed under JBOSS.
              We configure this JMS service to use the TDX database for it?s state and persistence management. The TDX deployment declares a JNDI DataSource definition named ?jdbc/EpokDB? for the primary database TDX looks for and uses upon startup. This same JNDI DataSource name is referenced in the JBOSS JMS configuration found in these three files:

              /usr/local/jboss/server/EpokTDX/conf/login-config.xml
              /usr/local/jboss/server/EpokTDX/server/deploy-hasingleton/jms/mysql-jdbc2-service.xml
              /usr/local/jboss/server/EpokTDX/server/deploy-hasingleton/jms/mysql-jdbc-state-service.xml

              This configuration above represents the State Manager and Persistence Manager of the JMS service, along with Security Domain Manager of the JBOSS message queuing.
              When the TDX uses the JMS services, tables are created and maintained on the TDX primary database.

              So what happens when the primary database goes down? The TDX provides functionality to failover activity to a designated slave database defined with JNDI DataSource named ?jdbc/EpokDBFailover?. But how do you get JBOSS JMS services to failover? Currently, JBOSS 4.0.1 does not offer any failover capability for it?s JMS services. There are upcoming feature requests, but no existing functionality. Here is the solution we have implemented in our TDX to coordinate JBOSS JMS failover:

              TDX JMS client detects JBOSS JMS service failures and initiates failover processing for JBOSS JMS services. This procedure is to:
              1. Shutdown all existing JMS Queue Listeners
              2. Get handles to follow JMX Services:
              a. jboss.mq:service=StateManager
              b. jboss.mq:service=PersistenceManager
              c. jboss.security:service=XMLLoginConfig
              3. We issue a ?stop? to all three services.
              4. We change the ?ConnectionManager? attribute in the State and Persistence Managers to now use the JNDI DataSource named ?jdbc/EpokDBFailover?
              5. We call out to a shell script that changes ?jdbc/EpokDB? to ?jdbc/EpokDBFailover? in the three .xml configuration files listed above.
              6. We issue a ?start? to the three JMX services, essentially restarting them with new configuration parameters.
              7. TDX JMS client re-resolves JMS ConnectionManager and Queue, and reestablishes new JMS Queue Listeners on them.

              That completes the JBOSS JMS Failover processing. The JBOSS JMS services are now interacting with the database defined by the JNDI DataSource named ?jdbc/EpokDBFailover?.

              Note in Step 5 above that we modify JBOSS .xml configuration files. You will see that we first make a backup of each file with a ?.orig? file extension into the same directory.

              When you restore your failed master database (which results in the JNDI DataSource named ?jdbc/EpokDB? again being the primary database), you will need to follow the regular steps outlined for restoring to a Master plus moving the ?.orig? versions of these three JBOSS configuration files back to their original names (hence enabling ?jdbc/EpokDB? as again the JNDI DataSource to use for JBOSS JMS services).

              IMPORTANT: The above functionality is based on the use of Epok?s decribed procedure and configuration for setting up Slave databases for database replication. What is key here is on our testing, the JMS tables created by JBOSS are replicated to the Slave just like are the regular TDX tables.



              Feel free to contact me if you have questions on this.
              sean.boltman@epok.net

              • 4. Re: There is no database failover capability in JMS Persiste
                Adrian Brock Master

                That is one solution, sounds a bit complicated :-)

                A simpler version would be to stop the MCF
                jboss.jca:service=ManagedConnectionFactory,name=???
                (this will automatically shutdown the connection pool and other dependent services, e.g. JMS)
                change the connection url by invoking setManagedConnectionFactoryAttribute
                on the above MCF mbean
                then start the MBean again.

                • 5. Re: There is no database failover capability in JMS Persiste
                  Sean Boltman Newbie

                  Thats great to hear... Now if this suggestion had only posted sooner than 12 days later then the original questions regarding this topic of JMS failover...