1 2 Previous Next 16 Replies Latest reply on Nov 27, 2006 6:36 AM by marklittle

    ESB configuration

    slaboure

      Hello,

      Part II of my discussion with MarkL: the configuration of the ESB.

      Today, it seems that most of the behaviour is wrapped inside an EAR and that some configuration parameters are set at the JVM level using some System Properties put in the conf directory.

      However, it is extremly easy and much more convenient to put these parameters inside MBean attributes i.e. setup an MBean per object to be configured and put these MBean definitions inside the SAR file of it corresponding class files. ALSO, most of the additional behaviour (like setting up a transformation, etc.) could be done as part of a specific MBean instead of relying on custom code/script. Such MBean definitions could be put in sample files in the /deploy directory (all entries would be commented/disabled) so that setting up a new ESB behaviour could be done in a very efficient way without relying too much on the documentation: just follow the examples. It is worth mentionning that MBean properties are not limited to primitive type: they can be any XML snippet. The new Microcontainer provides further improved features in that regard.

      Also, in order to make it easy for developers to change the default behaviour, it would certainly be usefull to use "unpacked SAR" i.e. do not put the JARs and their corresponding META-INF/jboss-services.xml file in a SAR file but, instead, but them in a xxx.sar DIRECTORY: that way any configuration item can be changed without having to go through an unpack-repack cycle of the archive.

      Concerning the MBeans, it would certainly be usefull to provide both a POJO and a related MBean: that way, the same code could be re-used both to work on the microkernel and the new microcontainer.

      A side effect of this is that you could setup two completely different ESB bus as part of the same JVM: they would not have to share their configuration (i.e. System Properties) anymore (i.e. JVM-wide vs. MBean-wide).

      I would be surprised if any of the ESB configuration would be restricted to code: most behaviours should be available as an MBean definition/POJO API that would cover the public API of the ESB.

      Cheers,



      Sacha

        • 1. Re: ESB configuration
          kurtstam

          Hi Sacha,

          I agree that an MBean has a natural fit with our needs. However, the current codebase does not require any support from the AS, and using MBeans will tie us to an AS/MK/MC again. A JMS provider is the only EE component we current need. So I'm not sure Mark willing to compromise any of the current 'leightness' of the solution. Also keep in mind that we will also suppport JBI in the future, which supports similar deployment features as you described for the mbeans, where code and configuration are deployed as one unit.

          On the plane ride home I had some time to think about our current deployment configuration file and I came to the following example xml:

          <jbossesb>
           <hosts>
           <host name="filebank" dnsName="localhost"/>
           <host name="jmsbank" dnsName="localhost"/>
           <host name="loanbroker" dnsName="localhost"/>
           <host name="jms-provider" dnsName="localhost"/>
           </hosts>
           <servers>
           <server name="fileBankServer" host="fileBank" appserver="jboss-4.0.3SP1">
           <properties>
           <property name="java.naming.provider.url" value="localhost:1099"/>
           <property name="java.naming.factory.initial" value="org.jnp.interfaces.NamingContextFactory"/>
           <property name="java.naming.factory.url.pkgs" value="org.jboss.naming:org.jnp.interfaces"/>
           </properties>
           </server>
           <server name="inhouse-jboss-4.0.4" host="jms-provider" appserver="jboss-4.0.4">
           <properties>
           <property name="java.naming.provider.url" value="localhost:1099"/>
           <property name="java.naming.factory.initial" value="org.jnp.interfaces.NamingContextFactory"/>
           <property name="java.naming.factory.url.pkgs" value="org.jboss.naming:org.jnp.interfaces"/>
           </properties>
           </server>
           <server name="loanbroker-listener" host="loanbroker" appserver="jbossesb"/>
           </servers>
           <channels>
           <channel name="bank-jms-channel" server="fileBankServer" resourceType="QUEUE" userName="" password=""/>
           <channel name="bank-ftp-channel" server="inhouse-jboss-4.0.4" resourceType="FTP" userName="kurt" password="secret"/>
           </channels>
           <services>
           <service name="filebank-gateway" category="gateway" server="loanbroker-listener" description="This listener picks up files deposited by the fileBank"
           class="org.jboss.soa.esb.FileBankGateway">
           <listeners>
           <listener description="A File-Based listener" channel="bank-ftp-channel"/>
           <listener description="A JMS-Based listener" channel="bank-jms-channel"/>
           </listeners>
           <actions>
           <action name="TestDefaultRouteAction" process="route" class="org.jboss.soa.esb.actions.CbrProxyAction" service-category="MessageRouting"
           service-name="ContentBasedRoutingService" />
           </actions>
           </service>
           </services>
          </jbossesb>
          


          Basically what I did is to 'normalize' the current listener and action configuration and to add that configuration to either a 'service', 'channel', 'server' or 'host'.

          I kept 1 configuration file for ALL the esb configuration (yes there apart from the properties set through the PropertyManager). This allows use to:


          1. generate the boiled down configuration for each server. (similar to the one above, but specific for that server)
          2. draw a picture of the esb configuaration as a whole, along with the connections (channels) and the servers they run on (along with info about their physical host).
          3. I'm working on the xsd for the config, so we can do a level0 sanity check.
          4. We can check if that all services are registered in the registry, before continuing (rather then build in a Thread sleep). I know we could do this with MBean dependency too if we decide to go this way.
          5. Each server can check if the channels they are supposed to access are up and running: 'can connect to ftp server' or 'can connect to jms-server'
          6. We can send in test messages a la nagios (synthetic tests) on startup or on a regular basis.



          For RC I think we need 1 through 4 and the tasks should be performed by the esb bootstrapper, which also reads the properties file.

          BTW I think when the ESB tries to connect to a channel we should keep on trying, log on failure, but keep on trying so that we can recover from nodes being brought up out of order (or temporary network failure).

          I think this should greatly reduce the complexity of configuring the esb and we can provide detail error messages, of which we are in dire need.

          I'd love to hear your feedback. I will follow up with my ideas on packaging on the other thread. We can go many ways on this!

          --Kurt

          • 2. Re: ESB configuration
            tfennelly

            Hey Kurt.

            I like your example config file there - it reads and has a structure which can be validated.

            On the other hand, I'd also be interested in seeing what the MC/MK can bring to this. I think doing this type of thing in a way that is consistent with other JBoss products is something that would be nice and a big selling feature. I'd imagine that this would be something that would appeal greatly to existing MC/MK users. Of course, this all assumes that it could meet our needs. Add to this the fact that MC/MK is already there and proven to work. If we didn't use this (e.g. because we wanted to stay "light"), what would we fill the gap with other than something we'd write ourselves i.e. effectively our own "lightweight" container, which we'd have to test, maintain etc. I think it's always easy to underestimate the LOE on something like this.

            • 3. Re: ESB configuration
              marklittle

              Microcontainer is pretty lightweight. Last time I looked it was only a few classes. The inference seems to be that binding to the MC is somehow too heavyweight for us, but I don't see that at all. A default container, such as MC, would help a lot. Hence the reason for moving towards it in the next release.

              • 4. Re: ESB configuration
                kconner

                Both MC and OSGi are lightweight and provide the functionality we would otherwise need to write. It is better for us to leverage these frameworks than it is to reinvent.

                Part of the initial requirements was the ability to be embedded in other containers so that we enable partners/users to take advantage of our ESB within their environments.

                Kev

                • 5. Re: ESB configuration
                  marklittle

                  Last time I spoke with Sacha/Adrian, it seemed like MC could be embedded in other containers too. Anyway, not an issue for the 4.0 release of ESB!

                  • 6. Re: ESB configuration
                    kconner

                    MC can be embedded but we should be striving for independence from the containers. If a small form device already has an OSGi framework then we should be able to use it without embedding another framework.

                    I want ESB on my phone :-)

                    Kev

                    • 7. Re: ESB configuration
                      tfennelly

                       

                      "Kevin.Conner@jboss.com" wrote:
                      Both MC and OSGi are lightweight and provide the functionality we would otherwise need to write. It is better for us to leverage these frameworks than it is to reinvent.


                      +1

                      Perhaps I missread Kurt anyway. Sorry if I did Kurt :-)

                      Of course, one downside to using an "off the shelf" container is usually the generic "one fits all" configuration which can become very verbose. Not sure if that's also the case with MC. I know that Spring 2.x supports domain specific configurations on top of its container, which means you can tailor the config file to be something more meaningful to your application, while at the same time get the benefits of the container. Of course I'm not suggesting we use Spring, but I do think that's a nice container level feature.

                      • 8. Re: ESB configuration
                        kconner

                        Spring is definitely something we have to bear in mind. While it may not be one of our supported configurations it may be something that our partners/users would choose.

                        • 9. Re: ESB configuration
                          marklittle

                           

                          "Kevin.Conner@jboss.com" wrote:
                          MC can be embedded but we should be striving for independence from the containers. If a small form device already has an OSGi framework then we should be able to use it without embedding another framework.

                          I want ESB on my phone :-)

                          Kev


                          I want ESB everywhere too, but we need to walk before we can run. And 4.0 isn't out of the door yet, so let's concentrate on that first.

                          • 10. Re: ESB configuration
                            tfennelly

                             

                            "Kevin.Conner@jboss.com" wrote:
                            MC can be embedded but we should be striving for independence from the containers. If a small form device already has an OSGi framework then we should be able to use it without embedding another framework.

                            I want ESB on my phone :-)

                            Kev


                            Isn't this just a classic layering issue. We need to keep the core ESB code 100% deployment agnostic. That even includes MC and OSGi. That way, these integration/deployment layers can be layered on top, removed, replaced etc without placing any additional requirements on the core ESB code. Of course, that's always easier in theory than in practice :-)

                            • 11. Re: ESB configuration
                              alesj

                               

                              "tfennelly" wrote:

                              Of course, one downside to using an "off the shelf" container is usually the generic "one fits all" configuration which can become very verbose. Not sure if that's also the case with MC. I know that Spring 2.x supports domain specific configurations on top of its container, which means you can tailor the config file to be something more meaningful to your application, while at the same time get the benefits of the container. Of course I'm not suggesting we use Spring, but I do think that's a nice container level feature.


                              MC already supports this.
                              There is some code to write, to bind you specific configuration and container, but it is trivial.



                              • 12. Re: ESB configuration
                                kconner

                                 

                                Isn't this just a classic layering issue. We need to keep the core ESB code 100% deployment agnostic.

                                Precisely. The core ESB will, ideally, be nothing more than a set of interfaces for well defined services. The framework (whichever it is) should be responsible for bootstrapping the implementations of those services into the configuration required by the deployment.

                                But as Mark says, this is for the next release :-)

                                Kev


                                • 13. Re: ESB configuration
                                  tfennelly

                                   

                                  "alesj" wrote:
                                  "tfennelly" wrote:

                                  Of course, one downside to using an "off the shelf" container is usually the generic "one fits all" configuration which can become very verbose. Not sure if that's also the case with MC. I know that Spring 2.x supports domain specific configurations on top of its container, which means you can tailor the config file to be something more meaningful to your application, while at the same time get the benefits of the container. Of course I'm not suggesting we use Spring, but I do think that's a nice container level feature.


                                  MC already supports this.
                                  There is some code to write, to bind you specific configuration and container, but it is trivial.



                                  Lovely Jobly :-)


                                  • 14. Re: ESB configuration
                                    alesj

                                     

                                    "Kevin.Conner@jboss.com" wrote:
                                    Spring is definitely something we have to bear in mind. While it may not be one of our supported configurations it may be something that our partners/users would choose.


                                    Already task in JIRA - http://jira.jboss.com/jira/browse/JBMICROCONT-26.

                                    Another idea is to support already existing Spring xml files and interpret them as our MC beans - not as Spring container managed beans, but as MC managed beans.
                                    Similar to what you want with your ESB configuration - domain specific configuration, but managed by MC.


                                    1 2 Previous Next