1 2 Previous Next 18 Replies Latest reply on Oct 11, 2010 4:35 PM by jesper.pedersen

    Enhancements to improve for the HornetQ/AS7 integration

    starksm64

      Hi, I wanted to start a discussion of some integration issues that I would like to address to both simplify, speed up the boot time, and have more flexilibity with regard to which pieces can be started/stopped independently. The main issues I have run across are:

       

          1.    Agreement on the domain specific language for messaging to expose to admins. Should all of the hornetq-configuration.xsd be used or are there pieces that expose implementation details admins should not see? At a minimum, some simple alias for the standard connector/acceptor etc. factory names is needed so as not to expose class names to admins.
          2.    Need to separate the start of the connectors and acceptors to allow for lazy loading of the hornetq service without starting the network layer elements.
          3.    Need a quiesce shutdown notion that allows the transport layer to be stopped while allowing in flight message transmissions to complete
          4.    Need an api for setting the root class loader rather than assuming the thread context class loader has been set correctly.
          5.    Need an ability to avoid creating any journal files. Currently a minimum of two is needed.

        • 1. Re: Enhancements to improve for the HornetQ/AS7 integration
          clebert.suconic

          1. I'm not sure about what you're asking on this one. we have a separation between jms and core layers.

           

             Anyway, we could extend the model to add a type (invm/remote) and the admin would only use the className case something more specific was developed by the user

           

          2, 3:I'm not really sure what that means

           

           

          4: Why? I thought the TCL was the API for that.

           

          5: This has been raised by Dimitris in another thread but that's not something we want for production. (you would hit the first message sent with a large latency)

           

          http://community.jboss.org/thread/157108?tstart=0

          • 2. Re: Enhancements to improve for the HornetQ/AS7 integration
            clebert.suconic

            Anyway, the next scheduled version is 2.2, and we won't be able to add any enhancements like this until 2.3

            • 3. Re: Enhancements to improve for the HornetQ/AS7 integration
              starksm64

              Clebert Suconic wrote:

               

              1. I'm not sure about what you're asking on this one. we have a separation between jms and core layers.

               

                 Anyway, we could extend the model to add a type (invm/remote) and the admin would only use the className case something more specific was developed by the user

               

              Right, this is about whether the full configuration model is approriate for exposing to admins and including whether there are implementation details that will change across versions. Exposing class names of connectors potentially goes against both goals, so having a more abstract notion of the different transport types such as invm/remote is what is needed. The question is mostly how much of the admin insulation do you want in HornetQ vs the integration layer in ASx. If you think the HornetQ configuration model satisfies your user base needs, then we just need to have validation that the domain model view of messaging configuration in ASx can be mapped onto HornetQ configuration notions going forward.

              • 4. Re: Enhancements to improve for the HornetQ/AS7 integration
                starksm64

                Clebert Suconic wrote:

                 

                 

                2, 3:I'm not really sure what that means

                Right now the ASx integration code is using the org.hornetq.core.server.HornetQServer class passing in a org.hornetq.core.config.Configuration instance to start the messaging subsystem. There is a MSC service wrapper around the HornetQServer that has dependencies on port and interface binding objects to pull in domain level values for connector/acceptor ports and interfaces.

                 

                We would also like to be able to separate out the transport level services and have those injected into the HornetQServer to allow transport layers to be started lazily. A related issue is that when a server is shutting down, we would like to be able to take down the transport layers and wait for in progress requests to finish to allow for graceful shutdowns.

                 

                • 5. Re: Enhancements to improve for the HornetQ/AS7 integration
                  starksm64

                  Clebert Suconic wrote:

                   

                  4: Why? I thought the TCL was the API for that.

                   

                   

                  Jason, DML, etc. would like a better contract for integrating subsystems.

                   

                  As we move to the MSC modular class loading, the big problem is that it's really a poor contract, especially in a modular environment. You don't really know what set it, when they set, what it points to. It can be anything really. Wed have to have some kind of contract about what the value of it should be, but if we have to have a contract then its better expressed in terms of an API/SPI.

                   

                  As an example, How often does HQ read the TCL? Do you expect the same consistent result whenever HQ code is looking it up?

                   

                  In the AS we could say that every subsystem gets XXX for the TCL, but that would only be for the duraction of activate. If a subsystem is leveraging thread pools or interacting with other layers, the TCL needs to be reset to the HQ module class loader or there needs to be another way to ensure HQ class loading happens in the correct scope.

                   

                  • 6. Re: Enhancements to improve for the HornetQ/AS7 integration
                    clebert.suconic

                    4. HornetQServerImpl will get the TCL once and apply it to ThreadFactory used by the Executors:

                     

                    From HornetQServerimpl::

                       private void initialisePart1() throws Exception

                       {

                          // Create the pools - we have two pools - one for non scheduled - and another for scheduled

                     

                          ThreadFactory tFactory = new HornetQThreadFactory("HornetQ-server-threads" + System.identityHashCode(this), false, getThisClassLoader());

                     

                     

                    There are a couple of other thread pools that are doing the same (for instance the journal has another executor, so this is probably done 2 times on the server, but we could do it only once).

                     

                    There's also the Client's side. ClientSessionFactoryImpl will also start a thread pool, but on that case it has to be the TCL as that's living on the client's side.

                    • 7. Re: Enhancements to improve for the HornetQ/AS7 integration
                      clebert.suconic

                      with 2 and 3 we may not even need the lazy journal startup, you just have a lazy server initialization done at the first connection.

                       

                      I'm not sure yet how this all will be done...  We need to make sure HornetQ to be standalone also, and be runnable in other application servers (that's the goal at least).

                       

                      We are now focusing on the 2.2 release that's due this year.

                       

                      I'm sure we do stuff like this later with 2.3+ and I will need to get up to date with design on the AS7 and have some design talks over IRC, etc... (maybe a face 2 face meeting if really needed).

                      • 8. Re: Enhancements to improve for the HornetQ/AS7 integration
                        timfox

                        Scott Stark wrote:

                         

                        Clebert Suconic wrote:

                         

                        1. I'm not sure about what you're asking on this one. we have a separation between jms and core layers.

                         

                           Anyway, we could extend the model to add a type (invm/remote) and the admin would only use the className case something more specific was developed by the user

                         

                        Right, this is about whether the full configuration model is approriate for exposing to admins and including whether there are implementation details that will change across versions. Exposing class names of connectors potentially goes against both goals, so having a more abstract notion of the different transport types such as invm/remote is what is needed. The question is mostly how much of the admin insulation do you want in HornetQ vs the integration layer in ASx. If you think the HornetQ configuration model satisfies your user base needs, then we just need to have validation that the domain model view of messaging configuration in ASx can be mapped onto HornetQ configuration notions going forward.

                        HornetQ configuration should not be exposing any "plumbing" details. We were careful from the beginning to avoid exposing such plumbing to the user .

                         

                        E.g. you'll see queues are declared as:

                        <queue name="something">

                           ...

                        </queue>

                         

                        not:

                         

                        <jboss-internal-mbean-thingy="xyzryghehe", load-on-startup="false>

                            <bean-description="queue-type" file="/def/queue.description">

                        </jboss-internal-mbean-thingy>

                         

                        Or such like

                         

                        The classnames on the connectors/acceptors is deliberate, since we want to allow users to be able to specify their own conectors/acceptors if they want to change the transport. In practice this rarely happens.

                         

                        I'm not sure these classes would be considered plumbing, more a hook point for the user. We also allow the user to specify classes so they can add their own interceptor or transformer classes to the message flow.

                        • 9. Re: Enhancements to improve for the HornetQ/AS7 integration
                          timfox

                          BTW, in previous mail when I use "plumbing" I'm talking about any JBoss internal plumbing required for JBoss AS to start up HornetQ properly - this should not be exposed in the hornetq configuration.

                           

                          I'm not talking about application level message plumbing, e.g. specifying transformers, transport types, and interceptors which is part of the user application design so is valid user app config.

                          • 10. Re: Enhancements to improve for the HornetQ/AS7 integration
                            starksm64

                            Which of the HornetQ_2_1_2_Final/examples show these interceptor/transformer usecase configuration?

                            • 11. Re: Enhancements to improve for the HornetQ/AS7 integration
                              jason.greene

                              Clebert Suconic wrote:

                               

                              4. HornetQServerImpl will get the TCL once and apply it to ThreadFactory used by the Executors:

                               

                              From HornetQServerimpl::

                                 private void initialisePart1() throws Exception

                                 {

                                    // Create the pools - we have two pools - one for non scheduled - and another for scheduled

                               

                                    ThreadFactory tFactory = new HornetQThreadFactory("HornetQ-server-threads" + System.identityHashCode(this), false, getThisClassLoader());

                               

                               

                              There are a couple of other thread pools that are doing the same (for instance the journal has another executor, so this is probably done 2 times on the server, but we could do it only once).

                               

                              There's also the Client's side. ClientSessionFactoryImpl will also start a thread pool, but on that case it has to be the TCL as that's living on the client's side.

                               

                              OK so it looks like this CL is only used to see HQ classes? If thats the case you can just have them use Class.forName() without a CL, or MyHqClass.class.getClassLoader(). Or is it used to find a user's custom classes? If it's to find user classes then under what conditions are they looked up? If it's supposed to be in the context of an EE deployment (say a war doing a jms request), then TCCL will be set as you expect. If however it's supposed to reference things like HQ extensions or app server things, then you would need to have an explicit classloader for that.

                              • 12. Re: Enhancements to improve for the HornetQ/AS7 integration
                                jason.greene

                                Tim Fox wrote:

                                 

                                The classnames on the connectors/acceptors is deliberate, since we want to allow users to be able to specify their own conectors/acceptors if they want to change the transport. In practice this rarely happens.

                                 

                                I'm not sure these classes would be considered plumbing, more a hook point for the user. We also allow the user to specify classes so they can add their own interceptor or transformer classes to the message flow.

                                Ah ok, so you consider these to be part of the HQ API? So does that mean that for example, the netty API thats used to define them is going to be BC for the lifespan of an EAP major release?

                                 

                                If thats the case then it would make sense to expose them, although perhaps the solution we are doing for JCA would make sense? What we are doing there is allowing anything but defining static tags for the most common data source configs. E.g. there will be an <oracle-dataousrce> that knows how to load the proper jdbc driver, and a <generic-jdbc-datasource driver-module-id="blah>

                                 

                                Note that any time we do have a support class name reference, it needs to be a module identifier as opposed to a class name. The rationale for this btw is that AS7 has a pure modular CL model, so that we can have a clean separation between our internals and a user's applications (an EAP PRD req). The basic notion is that all libraries that make up the AS are "modules" which have a clear set of imports and exports (sort of like OSGi). This means multiple modules could exist for the same classspace but at different versions etc. To EE users this will be transpartent, by default they will see the EE apis and some set of JBoss APIS (HQ, infinispan, etc). Our implementations can even use libraries that are in conflict with what an application uses (e.g. hq could use google-collections 1.3 internally, and the app could use 7.0 internally, and it would all just work) However, this requires that we at all times manage carefully what classloader is being used.

                                 

                                So what does this mean to you? It just means that if you have an extension mechanism thats support and in the domain, you need to take a ClassLoader along with a class name. We can map everything to that so you dont have to use our module apis [although that is an isolated component with 0 deps if for some reason you ever did want to use it. Although I don't think you have a need or want to]

                                • 13. Re: Enhancements to improve for the HornetQ/AS7 integration
                                  jason.greene

                                  Jason Greene wrote:

                                   

                                  So what does this mean to you? It just means that if you have an extension mechanism thats support and in the domain, you need to take a ClassLoader along with a class name. We can map everything to that so you dont have to use our module apis [although that is an isolated component with 0 deps if for some reason you ever did want to use it. Although I don't think you have a need or want to]

                                  Or it can also be a class instance object, we could then load that up for you.

                                  • 14. Re: Enhancements to improve for the HornetQ/AS7 integration
                                    clebert.suconic

                                    We only started using an TCCL on the executor because of Security needed the TCCL to load a class. (to read the properties files with the Users/Password).

                                     

                                    <Changed this post a bit after its original content>

                                    1 2 Previous Next