1 2 Previous Next 26 Replies Latest reply on Aug 11, 2004 11:24 AM by Adrian Brock

    Want to help develop a Distributed HA-JMS

    Marcelo Klein Newbie

       

      "mkprim" wrote:
      I want to help to make a distribuetd HA-JMS, so anyone could be able to share the same queue in any server, and if the server fails, anyone can take the remote client (that is moved transparently to another server). Something like the Clustered-EJB services.
      How can I help?


        • 1. DatabaseServerLoginModule - getUserPrincipal returns role...
          Ivelin Ivanov Expert

          As far as I can tell, I've met the requirements per the documentation for using this module. I can authenticate without issue, but when I call request.getUserPrincipal I am returned one of the roles associated with the user instead of the user name. I'm likely missing something very subtle here - a look over would be apprecated:

          Jboss 3.2.6

          Login using david/david.

          A call to request.getUserPrincipal().getName() returns "caller_java".

          From my login-config.xml:

          <module-option name="principalsQuery">
          select password from principal where name =?
          </module-option>
          <module-option name="rolesQuery">
          select r.role, r.rolegroup from role r, principal p where p.name =? and r.principal_id = p.id
          </module-option>



          Using MaxDB:

          //
          create table principal (
          id serial primary key,
          name varchar(64) not null,
          description varchar(100) default null,
          password varchar(64) not null,
          create_date timestamp not null default timestamp,
          modify_date timestamp not null default timestamp constraint create_date <= modify_date,
          unique (name)
          )
          //
          create table role (
          id serial primary key,
          principal_id fixed(10) not null,
          role varchar(64) not null,
          rolegroup varchar(64) not null,
          create_date timestamp not null default timestamp,
          modify_date timestamp not null default timestamp constraint create_date <= modify_date,
          unique (principal_id, role, rolegroup),
          foreign key role_principal (principal_id) references principal (id)
          )
          //
          INSERT INTO principal (id, name, password) values (1, 'david', 'david')
          //
          INSERT INTO role (principal_id, role, rolegroup) values (1, 'Echo', 'Roles')
          //
          INSERT INTO role (principal_id, role, rolegroup) VALUES (1, 'caller_java', 'CallerPrincipal')

          • 2. Re: Wrote Entity-Command for DB2 Identity Column - how to co
            Adrian Brock Master

             

            "humandoing" wrote:
            I have written an entity-command for use with DB2 (tested on JBoss 3.2.2 and DB2 8.1) that utilizes the IDENTITY_VAL_LOCAL() function (DB2 specific) to retrieve and populate the primary key field of an entity bean during the create process. This works with tables in DB2 that have been created with an IDENTITY column.

            This is a small, typical subclass of JDBCIdentityColumnCreateCommand, and I am wondering if I could contribute it to the Group? If someone who has "Authority" :) would like to drop it into the repository, I would be more than happy to email it to them.

            Let me know, I'll be watching this topic.
            -daniel


            • 3. Re: Want to help develop a Distributed HA-JMS
              Ivelin Ivanov Expert


              Yes, you can help.

              The current HA-JMS implementation leaves it up to the client to register a Connection Exception listener. When the current master singleton server fails, the client will receive a callback with this exception, at which point it has to reconnect again via the HAConnectionFactory in order to get the new master singleton server.

              You can try to implement the logic in the client portion of the HAIL transport that keeps track of the changes in the back end and automatically fails over to the new master server. There are at least two ways to do this.

              1) The preferred one is to reuse the latest work done in the JBoss detached remote invokation layer. This way you will effectively acomplish two pending tasks, one is improving the client fail-over, the other is unifying the invokation layers.
              2) Another way to do the client fail-over is to borrow logic from the smart RMI proxies, which keeps track of the available servers.

              In either case you will need to add extra logic to the client to stay informed about the latest master server.


              Not a piece of cake. Quite a bit of work and research ...

              Let me know if you will be working on this.

              Ivelin

              PS: A caveat is that this works for non-durable sessions. If a session is durable there needs to be even more work done to ensure that there is no message loss during fail-over. This a problem that can be taken care of separately on the server side though.

              • 4. Re: Want to help develop a Distributed HA-JMS
                Marcelo Klein Newbie

                Sorry, but I was on vacation. Now I'm back.
                First, I want you to know a couple of things:

                1. I already have the jboss-3.2.3-src source code, built it with buildmagic and integrated it into my eclipse framework.
                2. I'm new in this world of contributing and this will be my first project that I'll work in the JBoss Server development
                3. I may sometimes need some guidelines to know the modules wich have the implementation o something like that.

                If you don't have any trouble with this, I'll be glad to join this project!

                Tech Questions:

                Ways to solve the reconnection
                You said :"1) The preferred one is to reuse the latest work done in the JBoss detached remote invokation layer. This way you will effectively acomplish two pending tasks, one is improving the client fail-over, the other is unifying the invokation layers."

                I Cannot find the detached invoker class in the code


                2) Another way to do the client fail-over is to borrow logic from the smart RMI proxies, which keeps track of the available servers.

                I did not keep track of this one because I'll try the first one.


                In either case you will need to add extra logic to the client to stay informed about the latest master server.

                Not a piece of cake. Quite a bit of work and research ...

                Let me know if you will be working on this.

                Ivelin


                Can U help me to start?

                • 5. Re: Want to help develop a Distributed HA-JMS
                  Bill Burke Master

                  The hard part with HA-JMS is durability as you have to synchronize state. Ivelin wrote something based on HASingletonService that works I guess. You should talk to him.

                  Glad you are interested though!

                  • 6. Re: Want to help develop a Distributed HA-JMS
                    Adrian Brock Master

                    FYI: Ivelin is working on a configuration for 3.2.4 that will provide HA-JMS

                    The real issues are not in the connectivity but in recovering from a broken
                    JMS server.

                    The plan is that 3.2.4 wil provide a singleton JMS server "out-of-the-box"
                    in the "all" configuration.

                    Most of the code is already present, the issues are mainly configuration.

                    Regards,
                    Adrian

                    • 7. Re: Want to help develop a Distributed HA-JMS
                      Ivelin Ivanov Expert

                      Yes, I am on this one. The work is not big enough to split effectively between two of us.

                      If you'd like to help, you can write a test plan and example client code for the upcoming HA-JMS improvements. The deliverables should include a document with step-by-step instructions along with source code and binaries that will allow someone to verify that HA-JMS is running fine.
                      An example scenario:

                      -- start of example --
                      1) Install and configure two identical instances of JBoss 3.2.4 or later on two computers that reside on the same subnet

                      2) Configure the DefaultDS on both JBoss instances to point to a single database, preferably PostgreSQL, MySQL or another production grade database

                      3) Start JBoss in configuration "all" on both servers

                      4) Run the provided example client code (this is where you come in, mkprim)

                      3) You should see the following output:
                      - Connection established successfully to HA-JMS via HAJNDI
                      - Successfully registered a durable subscribtion for topic TestHAJMSTopic
                      - Successfully registered publisher for topic TestHAJMSTopic
                      - Sent test message "TestMessage1"
                      - Received test message "TestMessage1"
                      - Disconnecting without unregistering subscriber in order to test durable subscription fail-over
                      - Connecting and registering another publisher
                      - Sending message for durable subscription "DurableTestMessage1". Message sent.

                      4) Now login to the jmx console on one of the two JBoss servers and inspect the MBean named jboss.mq:service=HAJMSSingletonController. See the value of the attribute CurrentSingletonHost. The attribute should indicate which of the two servers is the acting singleton.

                      5) Stop the singleton JMS server

                      6) In the console executing the client code, you should see the following messages:
                      - Received JMS connection listener exception. Reconnecting...
                      - Connection established successfully to HA-JMS via HAJNDI
                      - Successfully registered a durable subscribtion for topic TestHAJMSTopic
                      - Received durable failover test message "DurableTestMessage1"
                      - HA-JMS example passed!

                      If all the steps above passed as described, then you have a working HA-JMS deployment.

                      -- end of example --

                      ... and so on. 5 or 6 more test scenarios like this would be a great addition to the JBoss wiki, once 3.2.4 goes out:
                      http://jboss.org/wiki/Wiki.jsp


                      Ivelin


                      • 8. Re: Want to help develop a Distributed HA-JMS
                        Marcelo Klein Newbie

                        OK, understood, now I have downloaded the latest 3.2 nightly build so I suppose there is the 3.2.4 HAJMS code so i can test it and build the client example code.

                        What do you mean as a "Test Plan"? Is this the wiki and the client code, or something else?

                        I will need to know how to invoke a HA-JMS connection ('cause i only downloaded the last sources). Should I use UIL2ConnectionFactory? HAJMSConnectionFactory?
                        Maybe I'll need some tips from you to get better involved in the HA-JMS implementation and working, to do the right tests.

                        I hope I'm not being a pain in the back instead of a solution.
                        Thanks,
                        Marcelo

                        • 9. Re: Want to help develop a Distributed HA-JMS
                          Ivelin Ivanov Expert


                          Read this document:
                          http://cocoonhive.org/articles/jboss/20031125/JBoss32-hajms.html
                          Get JBoss 3.2 from CVS, build and try the examples.

                          Optionally, you can port this document to the JBoss wiki:
                          http://jboss.org/wiki/Wiki.jsp?page=JBossMQ

                          Let me know when you do all this and you feel more comfortable with my original suggestions.

                          Ivelin

                          • 10. Re: Want to help develop a Distributed HA-JMS
                            Marcelo Klein Newbie

                            OK, now i've written the test. But I thougth that the client could run out of the server (a remote client outside the JBoss server).
                            So, the code is done, works with any normal IL, but when I use the HAIL, it throws a security exception (no securityManager specified).
                            So I suppose I must run the client from a JBoss server.
                            Maybe you can help me finish the test, or give me some clues.
                            Thank you,
                            Marcelo

                            • 11. Re: Want to help develop a Distributed HA-JMS
                              Ivelin Ivanov Expert

                              There is also another very interesting task that would be real nice to have for 3.2.4. See if you'd rather work on it:
                              http://jboss.org/index.html?module=bb&op=viewtopic&t=46302

                              There will be minimal overlap with my work.

                              Ivelin

                              • 12. Re: Want to help develop a Distributed HA-JMS
                                Ivelin Ivanov Expert

                                Quick update: I am finished with these HAJMS improvements. Doing some clean up now. Should be checking in later today or tomorrow.

                                Ivelin

                                • 13. Re: Want to help develop a Distributed HA-JMS
                                  Adrian Brock Master

                                  Congrats Ivelin.

                                  If you modify the messaging module can you also update jboss-head.
                                  I spent a lot of time merging the two codebases a couple of
                                  weeks ago. I would hate for them to diverge again so soon :-)

                                  But you might have to wait until DR3 is out, since it
                                  is currently under a feature freeze. Bill?

                                  I also wrote a jndi double alias so you can have different bindings
                                  for remote/local usage. (currently only in head).

                                   <mbean code="org.jboss.naming.LinkRefPairService"
                                   name="jboss.jmx:alias=XAConnectionFactory">
                                   <attribute name="JndiName">jms/XAConnectionFactory</attribute>
                                   <attribute name="RemoteJndiName">XAConnectionFactory</attribute>
                                   <attribute name="LocalJndiName">java:/XAConnectionFactory</attribute>
                                   </mbean>
                                  


                                  Then on the provider we just have:

                                   <mbean code="org.jboss.jms.jndi.JMSProviderLoader"
                                   name="jboss.mq:service=JMSProviderLoader,name=DefaultJMSProvider">
                                   <attribute name="ProviderName">DefaultJMSProvider</attribute>
                                   <attribute name="ProviderAdapterClass">org.jboss.jms.jndi.JBossMQProvider</attribute>
                                   <attribute name="QueueFactoryRef">jms/XAConnectionFactory</attribute>
                                   <attribute name="TopicFactoryRef">jms/XAConnectionFactory</attribute>
                                   </mbean>
                                  


                                  Then if the JMSServer is local, we use the in memory connection factory
                                  "java:/ConnectionFactory" otherwise we use the remote version "ConnectionFactory".

                                  Regards,
                                  Adrian

                                  • 14. Re: Want to help develop a Distributed HA-JMS
                                    Ivelin Ivanov Expert

                                    Will keep CVS HEAD in mind for post DR3.

                                    I am now working on populating the user&roles tables with "guest" account when the JMS tables are created for the first time. This will allow friendly behavior out of the box and can be altered later.
                                    I am modifying JDBCStateManager to read additional SqlProperties and implement the logic.

                                    Ivelin

                                    1 2 Previous Next