1 2 Previous Next 25 Replies Latest reply on Jul 5, 2012 1:48 PM by clebert.suconic Go to original post
      • 15. Re: Pluggable cluster discovery
        gaohoward

        Hi Tomo,

         

        Sorry for the delay. I just completed a prototype of the refactoring. You can have a look at the code here:

         

        https://github.com/gaohoward/hornetq/tree/tomo_hq316_readonly

         

        It's a temporary commit on top of yours. Below is a summary:

         

        1. The BroadcastingEndpoing interface.

         

        This is a generic interface that any pluggable broadcasting technique must implement. Like JGroups. It is defined as follows:

         

        public interface BroadcastEndpoint

        {

           void init(Map<String, Object> params) throws Exception;

         

           void broadcast(byte[] data) throws Exception;

         

           byte[] receiveBroadcast() throws Exception;

         

           //mode: true is broadcasting, false receiving.

           void start(boolean broadcasting) throws Exception;

          

           void stop() throws Exception;

        }

         

        The endpoint has two functionalities, first it can broadcast any byte[] data. Second it can receive any byte[] data.

         

        When a BroadcastGroup starts, it initializes a BroadcastEndpoint (based on configuration) and use it to broadcast connectors information by calling its broadcast(byte[]) method. When a DiscoveryGroup starts it calls receiveBroadcast() to receive data.

         

        I have a sample implementation for UDP broadcasting, see org.hornetq.core.server.cluster.impl.UDPBroadcastEndpoint

         

        2. The configuration

         

        To add a new Broadcast endpoint, you need to add an entry to the configuration file, like:

         

           <broadcast-endpoints>

              <broadcast-endpoint name="udp" class="org.hornetq.core.server.cluster.impl.UDPBroadcastEndpoint">

                 <param key="group-address" value="231.7.7.7"/>

                 <param key="group-port" value="9876"/>

              </broadcast-endpoint>

           </broadcast-endpoints>

         

        The above configuration tells hornetq there is a broadcast endpoint named 'udp', its implementation is org.hornetq.core.server.cluster.impl.UDPBroadcastEndpoint and it has two parameters 'group-address' and 'group-port'.

         

        To let the BroadcastGroup and DiscoveryGroup use this endpoint, you need to give the name of the endpoint in their configurations, like:

         

           <broadcast-groups>

              <broadcast-group name="bg-group1" endpoint="udp">

                 <broadcast-period>5000</broadcast-period>

                 <connector-ref>netty</connector-ref>

              </broadcast-group>

           </broadcast-groups>

         

           <discovery-groups>

              <discovery-group name="dg-group1" endpoint="udp">

                 <refresh-timeout>10000</refresh-timeout>

              </discovery-group>

           </discovery-groups>

         

         

        3. Compatibility with old configurations

         

        The old way of broadcasting and discovery is fully supported. Internally we create a new type of BroadcastGroup and DiscoveryGroup, i.e. PluggableBroadcastGroup and PluggableDiscoveryGroup. If the configuration doesn't use the 'BroadcastEndpoing' way, HornetQ will use the existing BroadcastGroupImpl and DiscoveryGroupImpl. If the 'BroadcastEndpoint' is configured, HornetQ will use the Pluggable classes, which use the endpoint to send and receive broadcast.

         

        4. Pros & cons

         

        The good thing about this design is that we keep the backward compatibility. We don't need to change the ServerLocator implementations. Adding any 3rd party broadcasting is as simple as implementing a interface, without understanding HornetQ's internal data structures. HornetQ core doesn't need to add extra dependency on 3rd party jars.

         

        The bad thing is we have to specify full qualified class names in the config file. Which I think shouldn't be a big problem as we already did so with connectors, etc.

         

        5. Unfinished works.

         

        Next I'll take a look at your JGroups Broadcast code and change it to be a 'JGroupsBroadcastEndpoint', and considering the impact on AS7 integration (which I think should not be very difficult to fit in). Then all the tests.

         

        Any thoughts please let me know.

         

        Thanks

        Howard

        • 16. Re: Pluggable cluster discovery
          gaohoward

          Status update:

           

          I'm thinking about this task as two implementations, one is to implement a 'pluggable' discovery/broadcasting mechanism to allow any other techniques than UDP broadcasting. And the other is to implement a 'jgroups' plugin to show the mechanism actually works with jgroups. The first should be part of core code base, the other should be like a reference implementation -- for which the best place is a folder under examples dir, together with an example that shows how to use/configure jgroups to form a cluster.

           

          Howard

          • 17. Re: Pluggable cluster discovery
            gaohoward

            Hi Tomo,

             

            I have finished the code (finally :)). Can you take time to give it a review?

             

            Currently I have the code on two different branches:

             

            https://github.com/gaohoward/hornetq/commits/Branch_2_2_EAP --this is for EAP 5

            https://github.com/gaohoward/hornetq/commits/hq316_premerge5 -- this is for trunk

             

            They are basically same with some minor differences like logging. The document is added to the last chapter of user's manual.

            You can build it yourself. For convenience I attached it here.

             

            Any questions please ping me.

             

            Thanks

            Howard

            • 18. Re: Pluggable cluster discovery
              clebert.suconic

              @Howard: Where is the JGroups implementation?

              • 19. Re: Pluggable cluster discovery
                clebert.suconic

                ok.. here's my take on the branch:

                 

                - I don't see the jGroupps implementation (or S3 bucket if you want to go straight)

                 

                - there are non comments whatsoever. please add the copyright Apache License information to the headers.

                 

                - Are you sure the ServerLocator will be serializable compatible with previous versions?

                • 20. Re: Pluggable cluster discovery
                  clebert.suconic

                  @Howard: I can finish the implementation but I first need to understand what happened to the JGroups implementation.

                  • 21. Re: Pluggable cluster discovery
                    gaohoward

                    Hi Clebert,

                     

                    The JGroups implementation is given as an example. It's in examples/cluster-jgroups.

                     

                    Thanks

                    Howard

                    • 22. Re: Pluggable cluster discovery
                      gaohoward

                      Clebert Suconic wrote:

                       

                       

                      - Are you sure the ServerLocator will be serializable compatible with previous versions?

                       

                      I added a new member in its discoveryGroupConfiguration. That may affect the serialization of the ServerLocator. Probably I can create a NewDiscoveryGroupConfiguration and keep the old class intact.

                       

                       

                      Howard

                      • 23. Re: Pluggable cluster discovery
                        clebert.suconic

                        User's are not expected to develop any JGroups integration in order to use the JGropus plugin. The JGroups endpoint or any class required to make its usage has to be in the main codebase.

                         

                         

                        I will move it over.. I couldn't find it before.

                        • 24. Re: Pluggable cluster discovery
                          gaohoward

                          OK agreed.

                          • 25. Re: Pluggable cluster discovery
                            clebert.suconic

                            One other change I'm doing is:

                             

                            These classes are duplicated: Theres DiscoveryGroup and PluggableDiscoveryGroup... We don't need two implementations of the same thing.. I will refactor it and just keep one.

                            1 2 Previous Next