JBossESB - JBossESBDocumentation - JBossESB Stateless Service Clustering
In mission critical systems it is important to design with redundancy in mind. JBossESB 4.2.GA is the first version with built-in fail-over, load balancing and delayed message redelivery to help you build a robust architecture. When you use SOA it is implied that the Service has become the building unit. JBossESB allows you to replicate identical services across many nodes. Where each node can be a virtual or physical machine running an instance of JBossESB. The collective of all these JBossESB instances is called "The Bus". Services within the bus use different delivery channels to exchange messages. In ESB terminology one such channel maybe JMS, FTP, HTTP, etc. These different "protocols" are provided by systems external to the ESB; the JMS-provider, the FTP server, etc. Services can be configured to listen to one or more protocols. For each protocol that it is configured to listen on, it creates an End Point Reference (EPR) in the Registry.
Service, EPRs, listeners and actions
In the jboss-esb.xml is service element consists of one or more listeners and one or more actions. Let's take a look at the JBossESBHelloworld example. The configuration fragment below is loosely based on the configuration of the JBossESBHelloworld example, but I left out gateways since I want to focus on Bus-internal message exchange only. When the service initializes it registers the category, name and description to the UDDI registry. Also for each listener element it will register a ServiceBinding to UDDI, in which it stores an EPR xml which is much like a URI, but somewhat richer. In this case it will register a JMSEPR for this service, as it is a jms-listener. The jms specific like queue name etc are not shown, but appeared at the top of the jboss-esb.xml where you can find the 'provider' section. In the jms-listener we can simply reference the "quickstartEsbChannel" in the busidref attribute.
... <service category="FirstServiceESB" name="SimpleListener" description="Hello World"> <listeners> <jms-listener name="helloWorld" busidref="quickstartEsbChannel" maxThreads="1"></jms-listener> </listeners> <actions> <action name="action1" class="org.jboss.soa.esb.actions.SystemPrintln"></action> </actions> </service> ...
Figure 1. Hello World configuration fragment, one service instance on one node.
Given the category and service name, another service can send a message to our Hello World Service by looking
up the Service in the Registry. It will receive the JMSEPR and it can use that to send a message to. All this
heavy lifting is done in the ServiceInvoker class. When our HelloWorld Service receives a message over the
quickstartEsbChannel, it will hand this message to the process method of the first action in the ActionPipeline, which is the SystemPrintln action.
In our example we have this service running on let's say Node1. What happens if we simply take the helloworld.esb and deploy it to Node2 as well (see figure 2)? Let's say we're using jUDDI for our Registry and we have configured all our nodes to access one central jUDDI database (and yes it is recommended to use a clustered database for that). Node2 will find that the FirstServiceESB - SimpleListener Service is already registered! It will simply add a second ServiceBinding to this service. So now we have 2 ServiceBindings for this Service. We now have our first distributed Service! If Node1 goes down, Node2 will keep on working.
Figure 2. Two service instance each on a different node.
You will get load balancing as both service instances listen to the same queue. However this means that we still have a single point of failure in our setup. This is where Protocol Clustering maybe an option.
Some JMS providers can be clustered. JBossMessaging is one of these providers, which is why we use this as our default JMS provider in JBossESB. When you cluster JMS you remove a single point of failure from your architecture, see Figure 3.
Figure 3. Protocol clustering: Here we cluster JMS.
Please read the documentation on Clustering for JBossMessaging if you want to enable JMS clustering. Other examples of Protocol Clustering would be a NAS for the FileSystem protocol, but what if your provider simply cannot provide any clustering? Well in that case you can add multiple listeners to your service, and use multiple (JMS) providers. However this will require fail-over and load-balancing across providers which leads us to the next section.
Channel Fail-over and Load Balancing
Our HelloWorld Service can listen to more then 1 protocol. Here we have add an ftp channel.
... <service category="FirstServiceESB" name="SimpleListener" description="Hello World"> <listeners> <jms-listener name="helloWorld" busidref="quickstartEsbChannel" maxThreads="1"></jms-listener> <jms-listener name="helloWorld2" busidref="quickstartFtpChannel2" maxThreads="1"></jms-listener> </listeners> ...
Figure 4. Adding another JMS channel.
Now our Service is simultaneously listening to two JMS queues. Now these queues can be
provided by JMS providers on different physical boxes! So we now have a made a redundant
JMS connection between two services. We can even mix protocols in this setup, so we can also add
and ftp-listener to the mix.
... <service category="FirstServiceESB" name="SimpleListener" description="Hello World"> <listeners> <jms-listener name="helloWorld" busidref="quickstartEsbChannel" maxThreads="1"></jms-listener> <jms-listener name="helloWorld2" busidref="quickstartJmsChannel2" maxThreads="1"></jms-listener> <ftp-listener name="helloWorld3" busidref="quickstartFtpChannel3" maxThreads="1"></ftp-listener> <ftp-listener name="helloWorld4" busidref="quickstartFtpChannel3" maxThreads="1"></ftp-listener> </listeners> ...
Figure 5. Adding an 2 FTP servers to the mix.
When the ServiceInvoker tries to deliver a message to our Service it will get a choice of 8 EPRs now (4 EPRs from Node1 and 4 EPRs from Node2). How will it decide which one to use? For that you can configure a Policy. In the jbossesb-properties.xml you can set the 'org.jboss.soa.esb.loadbalancer.policy'. Right now three Policies
are provided, or you can create your own.
First Available. If a healthy ServiceBinding is found it will be used unless it dies, and it will move to the next EPR in the list. This Policy does not provide any load balancing between the two service instances.
Round Robin. Typical Load Balance Policy where each EPR is hit in order of the list.
Random Robin. Like the other Robin but then ramdom.
The EPR list the Policy works with may get smaller over time as dead EPR will be removed from the (cached) list. When the list is exhausted or the time-to-live of the list cache is exceeded, the ServiceInvoker will obtain a fresh list of EPRs from the Registry. The 'org.jboss.soa.esb.registry.cache.life' can be set in the jbossesb-properties file, and is defaulted to 60,000 milliseconds. All good right? Well.. what if none of the EPRs work at the moment? This is where we may use Message Redelivery Service.
If the list of EPRs contains nothing but dead EPRs the ServiceInvoker can do one of two things:
1. If you are trying to deliver the message synchronously it will send the message to the DeadLetterService, which by default will store to the DLQ MessageStore, and it will send a failure back to the caller. Processing will stop. Note that you can configure the DeadLetterService in the jbossesb.esb if for instance you want it to go to a JMS queue, or if you want to receive a notification.
2. If you are trying to deliver the message asychronously (recommended), it too will send the message to the DeadLetterService, but the message will get stored to the RDLVR MessageStore. The Redeliver Service (jbossesb.esb) will retry sending the message until the maximum number of redelivery attempts is exceeded. In that case the message will get stored to the DLQ MessageStore and processing will stop.
Figure 6. If all the EPRs are bad at a given moment, async requests can be store in the MessageStore for redelivery at a later time.
Note that DeadLetterService is turned on by default, however in the jbossesb-properties.xml you could set org.jboss.soa.esb.dls.redeliver to false to turn off its use.