14 Replies Latest reply on Apr 8, 2010 11:39 AM by jmesnil

    Deployment issues

    jmesnil
      While investigating https://jira.jboss.org/jira/browse/HORNETQ-350, I found some issues related to our deployment phases.
      I checked it running HornetQ in standalone but aiui this applies too when it is integrated in the AS.
      First, configured Core queues are not deployed (https://jira.jboss.org/jira/browse/HORNETQ-352) that I am fixing now.
      Then, queues, adress settings and security settings are now deployed twice when file deployment is enabled:
         - First, in HornetQServerImpl.initialisePart2() from the configuration object (which, in the standalone case, is loaded with the content of hornetq-configuration.xml)
         - Then, they are deployed again by the various XmlDeployer (AddressSettingsDeployer, QueueDeployer, SecurityDeployer).
      I'll fix this by:
         - deploying from the various XmlDeployer if file deployment is enabled
         - deploying from the configuration object *only if file deployment is disabled*
      Do you see a use case which would cause trouble?
        • 1. Re: Deployment issues
          jmesnil
          I've looked extensively at our deployments steps and here is a summary of the current situation.
          HornetQServerImpl has 2 different ways to deploy "resources"
          - from the Configuration object
          - if file deployment is enabled, we have deployers (for queues, address settings, and security settings) which looks for configuration files in the classpath and deploy resources from them.
          Currently, in the trunk, we always deploy from the Configuration object and we start the deployers if file deployment is enabled.
          If HornetQ is run in standalone, the Configuration object was itself creating from a configuration file => its resources will be deployed twice:
          1. through the Configuration object
          2. and again by the deployers.
          To prevent that, I'm making sure w
          e deploy resources from the Configuration object only if file deployment is disabled.
          If file deployment is enabled, the deployers are the only one deploying resources.
          What are these resources that can be deployed like this?
          - core queues
          - address settings
          - security settings
          The differences between these resources and others in the Configuration object (e.g. diverts, bridges, connectors, acceptors, etc.) is that they
          are the only resources that can be deployed on a running server.
          Adding a new <queue> to hornetq-configuration.xml will deploy the queue in the server. Adding a new <divert> will have no effect until server is restarted.
          I wanted to make sure this would also be the correct behaviour for our AS6 integration code.
          At first, I was confused when looking at the hornetq-int code for AS 6. The HornetQCoreDeployment object will use HornetQ's Configuration object
          as the POJO used to deploy queue/address/security settings. But this configuration object is never passed to HornetQ server, it is just a holder. This is not intuitive.
          HornetQ config API should made explicit what are the "deployable" resources in its Configuration.
          E.g. having a DeployableConfiguration interface with attributes for address settings, security settings and queues only.
          Otherwise, other AS projects have no way to know what resources can be deployable from HornetQ. For example, if TorqueBox adds a divert to the Configuation object passed to HornetQCoreDeployment, this will have no effect.
          On a side note, the JMSConfiguration object is cleaner in that regard: it exposes the JMS resources that can be deployed by HornetQ.
          I'm still not 100% clear on AS6 deployment. For resources defined in hornetq-configuration.xml, how many times will they be deployed?
          AIUI, they should be parsed by HornetQCoreConfigRealDeployer and deployed only once inside HornetQServerImpl.start().
          But are they deployed a 2nd time through HornetQCoreDeployment. The workflow here is not clear to me at all (besides adding logging statements to see what's going on...).
          Clebert, I looked at the AS integration thread but I did not find THE definitive description of how deployment is supposed to work with AS6. It could make for a awesome wiki page
          A few more thoughts on deployment:
          We must also support deployment of diverts & bridges through Management API (https://jira.jboss.org/jira/browse/HORNETQ-89).
          To be consistent, this means that diverts & bridges would also become deployable resources that needs to be supported by the AS6 deployers. Again, having an explicit API for these resources configuration would help.
          This also means that HornetQ needs to add (internal) diverts & bridges deployers which watches configuration files and deploy/undeploy the resources when their configuration changes.
          • 2. Re: Deployment issues
            timfox

            Translation of Jeff's post:

             

            All the deployment crap that was added for AS6 integration is a big fucking inconsistent mess.

             

            Who volunteers to fix it?

            • 3. Re: Deployment issues
              jmesnil

              I've been designated volunteer

               

              Clebert, I'd like your input here but my idea to clear things up is to have a proper POJO to identify deployable resources handled by deployers.

              Let's call it
              DeployableConfiguration (or a better name)
                 - getQueueConfigurations()
                 - getSecurityRoles()
                 - getAddressSettings()
              This will be the POJO used by AS6 deployers. If Bob wants to add Core queues from TorqueBox, he will use this object instead of HornetQ's Configuration object.
              Deployers are not dealing with HornetQ server config, they must not have to use HornetQ Configuration interface.
              We need also to address the standalone case. By diffing the trunk with 2.0.0.GA, it appears we can now configure and deploy address & security settings without requiring XML config files. We should make this feature consistent with what's been done by the JMSServerManagerImpl.
              Its constructor either requires a config file name (and JMS resources will be deployed by the JMSServerDeployer) or a JMSServerConfiguration object (and JMS resources will be deployed from this object).
              We could have something similar for HornetQServerImpl: if it takes a "DeployableConfiguration", we deploy resources from this object. If it does not take it, resources will be deployed by our own deployers (QueueDeployer, AddressSettingsDeployer, SecurityDeployer).
              The key thing are:
              - make explicit what resources are deployable (now we have queues, security, address settings + JMS resources, later we'll add diverts, bridges, etc.)
              - make sure that resources are deployed only once (regardless of their origins)
              - make deployment design of JMS and Core resources consistent
              wdyt?

              • 4. Re: Deployment issues
                clebert.suconic

                The AS deployer don't use any of our deployers.

                 

                As I said, the only thing I did for the integration was to extract the parsers from the deployers, and keep whatever was the previous standalone way of deploying it.

                 

                On the AS integration, the HornetQ deployers should be disabled

                • 5. Re: Deployment issues
                  clebert.suconic

                  Tim Fox wrote:

                   

                  Translation of Jeff's post:

                   

                  All the deployment crap that was added for AS6 integration is a big fucking inconsistent mess.

                   

                  Who volunteers to fix it?

                  It was already inconsistent before.

                   

                  The AS integration only extracted the parsing that was only done inside the XMLDeployers.

                  • 6. Re: Deployment issues
                    clebert.suconic


                    The only inconsistencies I read here are:

                     

                    - Things being deployed twice. But you said you will only do this if deployment is disabled, right?

                     

                    There's a problem if you do that. Especially if it's an address setting. The server will be running for a few moments with the wrong settings. If the server receives a message before the deployment kicked in, you could reject a valid message.. or something like that.

                     

                    I don't think I changed anything around here BTW.

                     

                    - We could add diverts and bridges to be deployed by the AS deployers also.

                    • 7. Re: Deployment issues
                      clebert.suconic

                      Jeff, Only thing I need is to be able to parse a XML into any POJO format.

                       

                      The AS deployers are using Management operations to create Queues, AddressSettings.. etc. They are not going to use any deployment methods existent on standalone besides the parsing.

                       

                      The only reason I'm using the Configuration objects for now is because those are the only representation we have about the XMLs.

                       

                      If you refactor around this is fine. We will just have to update the AS deployers before next release.

                      • 8. Re: Deployment issues
                        jmesnil
                        I am trying to make sure I understand how AS deployers are working. The only good description I found was on Bob's oddthesis blog. Is there a better source of information?
                        As I understand it, HornetQConfigParserDeployer parses hornetq-configuration.xml and returns a Configuration object, the "metadata" generated by the deployer. We needed to have a POJO representation of our XML schema, so we took Configuration. Is that correct?
                        <bean name="HornetQConfigParserDeployerFile1">
                             <constructor>
                              <parameter>hornetq-configuration.xml</parameter>
                           </constructor>
                        </bean>
                        
                        Then, we have HornetQCoreConfigRealDeployer which will attach the "metadata" Configuration object and the HornetQServer bean to the
                        HornetQCoreDeployment object.
                        When the AS start, the MC will start the HornetQCoreDeployment object which will trigger start of HornetQServer first and then call
                        HornetQCoreDeployment.start(). Is that correct?
                        I'm confused by this. HornetQ AS6 configuration in hornetq-jboss-beans.xml already associated HornetQ server with hornetq-configuration.xml:
                        <!-- The core configuration -->
                        <bean name="Configuration">
                           <property name="configurationUrl">${jboss.server.home.url}${/}deploy${/}hornetq${/}hornetq-configuration.xml</property>
                        </bean>
                         
                        <!-- The core server -->
                        <bean name="HornetQServer">
                           <constructor>
                              <parameter>
                                 <inject bean="Configuration"/>
                              </parameter>
                              ...
                        </bean>
                        
                        When HornetQServer bean is started, it will deploy the core queues, address settings and security settings from this file.
                        Why do we need to deploy them *again* in HornetQCoreDeployment.start()?
                        It seems unnecessary to use AS6 deployers to deploy *HornetQ server configuration*!
                        Now, is the real intent of the AS6 deployers to support *hornetq-queues.xml* configuration to deploy additional core queues (with their associated address and security settings)?
                        Is that the case?
                        If that's the case, we need to make this explicit:
                        - use deployers only for hornetq-queues.xml config files (and not hornetq-configuration.xml)
                        - have a POJO representing only what can be deploy by this (core queues, address settings, security settings)
                          We can have a XML schema for hornetq-queues.xml which represents the deployable resources and use JAXB on the AS6 integration code to create the corresponding POJO.
                        - document somewhere how all this pieces fit together, who does what, etc.
                        Things are more clear on the JMS side.
                        We have a deployer (HornetQJMSParserDeployer) which parses hornetq-jms.xml and creates a JMSConfiguration POJO
                        We have a deployer (HornetQJMSRealDeployer) which takes a JMSConfiguration POJO and deploys it using the JMSServerManager bean.
                        If we want to be consistent, we should rename our hornetq-queues.xml files to hornetq-core.xml and make it a separate schema with deployable
                        Core resources (core queues, address & security settings, later diverts, bridges, etc.)
                        The only thing the AS6 user needs to know is that he can deploy Core resources by adding a hornetq-core.xml file and JMS resources by adding a hornetq-jms.xml file. Simple like that.
                        (As an aside, I look at TorqueBox to see how Bob was deploying JMS resources and he did not use JMSConfiguration & HornetQJMSRealDeployer but the JMSServerManager bean directly... I am not sure why, did he do that before we committed our AS6 deployers? I thought he did the TorqueBox work at the same time)
                        I am late to the party and I'm notoriously slow to pick things up.
                        As I am still figuring out how all these pieces fit together (am I the only one?), if we had a clear view of the global picture, it'd be help us make sure we offer a consistent (and simple) story for deployment.
                        • 9. Re: Deployment issues
                          jmesnil

                          I hate this: I'm adding line breaks to clear my writings and they're removed by the forum software... sorry for that, it's already tedious to read all this stuff and this does not help...

                          • 10. Re: Deployment issues
                            jmesnil

                            Clebert Suconic wrote:

                            - Things being deployed twice. But you said you will only do this if deployment is disabled, right?

                             

                            There's a problem if you do that. Especially if it's an address setting. The server will be running for a few moments with the wrong settings. If the server receives a message before the deployment kicked in, you could reject a valid message.. or something like that.

                            I don't understand this use case. Could you be more specific?

                            • 11. Re: Deployment issues
                              clebert.suconic

                              Jeff wrote : "When HornetQServer bean is started, it will deploy the core queues, address settings and security settings from this file.

                              Why do we need to deploy them *again* in HornetQCoreDeployment.start()?:"

                               

                               

                              As far as I know, HornetQ won't be able to scan that file again after the server is started. Only the AS scanner will be able to identify changes.
                              I was going to disable HQ deployers on next integration version.

                               

                               

                              "It seems unnecessary to use AS6 deployers to deploy *HornetQ server configuration*!

                               

                              Now, is the real intent of the AS6 deployers to support *hornetq-queues.xml* configuration to deploy additional core "

                               

                               

                              hornetq-queues.xml doesn't include address-settings or security-settings. My initial intent was to only support hornetq-queues.xml however we still need the settings. An user could choose to add hornetq-queues.xml and the settings at his deployment unit.

                               

                               

                               

                              "If we want to be consistent, we should rename our hornetq-queues.xml files to hornetq-core.xml and make it a separate schema with deployable"

                               

                               

                               

                              +1

                               

                               

                               

                              "As an aside, I look at TorqueBox to see how Bob was deploying JMS resources and he did not use JMSConfiguration & HornetQJMSRealDeployer but the JMSServerManager bean directly... I am not sure why, did he do that before we committed our AS6 deployers? I thought he did the TorqueBox work at the same time)"

                               

                               

                               

                              Not yet, but he will eventually

                               

                               

                              Another reason we have the separation of parsing and POJOs on AS is ** Dependencies **

                               

                              The scanner on the deploy dirs will happen way before HornetQ is started.

                               

                              If I add a dependency between the deployer and HOrneTQ, the scanner won't call the deployers again.

                              • 12. Re: Deployment issues
                                clebert.suconic

                                Jeff Mesnil wrote:

                                 

                                I hate this: I'm adding line breaks to clear my writings and they're removed by the forum software... sorry for that, it's already tedious to read all this stuff and this does not help...

                                 

                                It sucks... I had to post-edit my post and go to the HTML view and add </p>

                                • 13. Re: Deployment issues
                                  clebert.suconic

                                  Jeff Mesnil wrote:

                                   

                                  Clebert Suconic wrote:

                                  - Things being deployed twice. But you said you will only do this if deployment is disabled, right?

                                   

                                  There's a problem if you do that. Especially if it's an address setting. The server will be running for a few moments with the wrong settings. If the server receives a message before the deployment kicked in, you could reject a valid message.. or something like that.

                                  I don't understand this use case. Could you be more specific?

                                   

                                  Say you start the server. At this point core-queues are already available.

                                   

                                  Now say, after you start the server, you are going to deploy security settings, authorizing users to the queue.

                                   

                                  It's unlikely, but I guess there's a chance a produced message could come in before the security setting was applied.

                                  • 14. Re: Deployment issues
                                    jmesnil

                                    Clebert Suconic wrote:

                                     

                                    Jeff Mesnil wrote:

                                     

                                    Clebert Suconic wrote:

                                    - Things being deployed twice. But you said you will only do this if deployment is disabled, right?

                                     

                                    There's a problem if you do that. Especially if it's an address setting. The server will be running for a few moments with the wrong settings. If the server receives a message before the deployment kicked in, you could reject a valid message.. or something like that.

                                    I don't understand this use case. Could you be more specific?

                                     

                                    Say you start the server. At this point core-queues are already available.

                                     

                                    Now say, after you start the server, you are going to deploy security settings, authorizing users to the queue.

                                     

                                    It's unlikely, but I guess there's a chance a produced message could come in before the security setting was applied.

                                    For a same configuration file, we first apply address & security settings before deploying the queues.

                                    Your scenario could happen if the user:

                                    - define a core queue in 1 hornetq-queues.xml file

                                    - defines corresponding address & security settings in another hornetq-queues.xml

                                     

                                    This could happen but I'm expecting the user to specify related configuration in the same config file.

                                     

                                    But this use cas is not relevatnt to the issue: if file deployment is enabled, we deploy from the configuration files instead of the Configuration object but this does not change the order of deployment.