Message bridge performance in WAN

[jjacobwip]

Hi,

We have an ActiveMQ queue which gets posted with 65K+ messages in an hour. These messages needs to be transfered over the WAN to a local queue created in Jboss messaging. We have created a message bridge for this where the source of the bridge is the Remote ActiveMQ queue and target is a local Jboss messaging queue.

We are seeing a very low performance here where in only 1000 messages are reaching the local queue in 1 minute. MaxBatchSize is set to 50 and MaxBatchTime is set to 1.

The question is -

1. How does MaxBatchSize work? Will the bridge pick up all 50 msgs and transfer it over WAN or will it transfer the messages one by one over the WAN and create a batch and send it to the local queue?

2. How can we get a better performance? We know that n/w speed is an important factor here and we are working on it, but just wanted to know if there is any configurations which we can tweak to get a better performance.

Env:
JBoss 5.0.1
Jboss Messaging 1.4.1 GA

Message Bridge Config:

<mbean code="org.jboss.jms.server.bridge.BridgeService"
name="com.nordpool:service=Bridge,name=BridgeFromOM"
xmbean-dd="xmdesc/Bridge-xmbean.xml">

<attribute name="SourceProviderLoader">jboss.mq:service=JMSProviderLoader,name=ActiveMQJMSProvider</attribute>

<attribute name="TargetProviderLoader">jboss.messaging:service=JMSProviderLoader,name=JMSProvider</attribute>

<attribute name="SourceDestinationLookup">inbound</attribute>

<attribute name="TargetDestinationLookup">queue/LocalQueue</attribute>

<attribute name="SourceUsername">xx</attribute>

<attribute name="SourcePassword">xx</attribute>

<!--
<attribute name="TargetUsername">xx/attribute>
<attribute name="TargetPassword">xx</attribute>
-->

<attribute name="QualityOfServiceMode">0</attribute>

<attribute name="Selector"></attribute>

<attribute name="MaxBatchSize">50</attribute>

<attribute name="MaxBatchTime">4000</attribute>

<attribute name="SubName"></attribute>

<attribute name="ClientID"></attribute>

<attribute name="FailureRetryInterval">1000</attribute>

<attribute name="MaxRetries">2</attribute>

</mbean>

Thanks

[jjacobwip]

Here is the bridge config.

<mbean code="org.jboss.jms.server.bridge.BridgeService" name="com.nordpool:service=Bridge,name=BridgeFromOM" xmbean-dd="xmdesc/Bridge-xmbean.xml">
<attribute name="SourceProviderLoader">jboss.mq:service=JMSProviderLoader,name=ActiveMQJMSProvider</attribute>
<attribute name="TargetProviderLoader">jboss.messaging:service=JMSProviderLoader,name=JMSProvider</attribute>
<attribute name="SourceDestinationLookup">inbound</attribute>
<attribute name="TargetDestinationLookup">queue/LocalQueue</attribute>
<attribute name="SourceUsername">xx</attribute>
<attribute name="SourcePassword">xx</attribute>
<!--
<attribute name="TargetUsername">xx/attribute>
<attribute name="TargetPassword">xx</attribute>
-->
<attribute name="QualityOfServiceMode">0</attribute>
<attribute name="Selector"></attribute>
<attribute name="MaxBatchSize">50</attribute>
<attribute name="MaxBatchTime">4000</attribute>
<attribute name="SubName"></attribute>
<attribute name="ClientID"></attribute>
<attribute name="FailureRetryInterval">1000</attribute>
<attribute name="MaxRetries">2</attribute>
</mbean>

[gaohoward]

Hi, here is what JBM doc says

9.3.11. MaxBatchSize
This attribute specifies the maximum number of messages to consume from the source destination before sending
them in a batch to the target destination. It's value must >= 1

9.3.12. MaxBatchTime
This attribute specifies the maximum number of milliseconds to wait before sending a batch to target, even if the
number of messages consumed has not reached MaxBatchSize. It's value must can be -1 to represent 'wait forever',
or >=1 to specify an actual time.

Please note the unit of MaxBatchTime value is milliseconds. I think give it 1 is too small.

[jjacobwip]

Thanks, we had already gone through JBM docs.

What we noticed is that when MaxtBatchTime is set as 1 milli second, bridge is sending only one message. According to that in one second, bridge should send 1000 messages which is not the case. What we would like to know is why the bridge is taking that much time to consume the messages. For 50 messages, bridge is taking almost 4 secs to consume. Our requirement is to process 650K+ messages over the WAN in 1 hour.

We tried with higher values for batchtime, but the through put was still low.

[timfox]

If you are sending a lot of messages in a fairly constant stream it doesn't make a lot sense to me why you are setting maxBatchSize to a value other than -1, since the batch will be sent anyway when maxBatchSize is reached.

Regarding the speed of bridging, this is a function of many factors including:

The size of your messages
Are the messages persistent?
The bandwidth of your network
The round trip time of your network
The speed of the ActiveMQ server
The Quality of service mode you are using.
The speed of the JBM server

So it's very hard to quantify.

[gaohoward]

hi,

where did you deploy the bridge? at the ActiveMQ side or the JBoss Messaging side?

[jjacobwip]

Thanks for quick response. The bridge is deployed at JBoss Messaging side

[jjacobwip]

In our case, the messages (1000) are already available in the ActiveMQ queue. i.e for testing purpose, the messages are already posted to the queue and left their without consuming. When the bridge is started, it is actually waiting for secs and consuming the messgaes - there is a delay here eventhough the messages are already available in the source queue.