Upgrade JBoss AS 7.1.1 to hornetq 2.2.22

[ajanz]

we use JBoss AS 7.1.1 in production and got problems with messaging. sometime messaging just hangs.

for example at

at java.util.concurrent.Semaphore.acquire(Semaphore.java:472)

at org.hornetq.core.client.impl.ClientProducerCreditsImpl.acquireCredits(ClientProducerCreditsImpl.java:81)

at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:305)

at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)

at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)

at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:199)

at org.hornetq.ra.HornetQRAMessageProducer.send(HornetQRAMessageProducer.java:221)

i read that in hornetq 2.2.13 ( = AS 7.1.1.) there were some bugs.

on [JBPAPP6-947] HornetQ producer hang in ClientProducerCreditsImpl.acquireCredits after connection loss - JBoss Issue Trac…

it is recommended to upgrade to hornetq 2.2.22

so i tried to do this .

i changed the jar files in the modules folder. but at startup i get

11:14:54,812 WARN  [org.jboss.modules] (MSC service thread 1-6) Failed to define class org.jboss.as.messaging.jms.AS7RecoveryRegistry in Module "org.jboss.as.messaging:main" from l

ocal module loader @2f6d549e (roots: D:\projekte\jboss-as-7.1.1.Final\modules): java.lang.LinkageError: Failed to link org/jboss/as/messaging/jms/AS7RecoveryRegistry (Module

"org.jboss.as.messaging:main" from local module loader @2f6d549e (roots: D:\projekte\jboss-as-7.1.1.Final\modules))

        at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:396)

        at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:243)

        at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:73)

        at org.jboss.modules.Module.loadModuleClass(Module.java:517)

        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:182)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

        at org.jboss.as.messaging.jms.PooledConnectionFactoryService.createService(PooledConnectionFactoryService.java:245) [jboss-as-messaging-7.1.1.Final.jar:7.1.1.Final]

        at org.jboss.as.messaging.jms.PooledConnectionFactoryService.start(PooledConnectionFactoryService.java:180) [jboss-as-messaging-7.1.1.Final.jar:7.1.1.Final]

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]

        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_55]

Caused by: java.lang.NoClassDefFoundError: org/hornetq/jms/server/recovery/RecoveryRegistry

        at java.lang.ClassLoader.defineClass1(Native Method) [rt.jar:1.7.0_55]

        at java.lang.ClassLoader.defineClass(ClassLoader.java:800) [rt.jar:1.7.0_55]

        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) [rt.jar:1.7.0_55]

        at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:327)

        at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:391)

        ... 15 more

Caused by: java.lang.ClassNotFoundException: org.hornetq.jms.server.recovery.RecoveryRegistry from [Module "org.jboss.as.messaging:main" from local module loader @2f6d549e (roots:

D:\projekte\jboss-as-7.1.1.Final\modules)]

        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

        ... 20 more

so what is missing there ?

[jbertram]

I'm not sure you need to upgrade due to the "hang" you are seeing because the "hang" looks to be simply related to normal producer flow control (i.e. the client has to acquire credits from the server before it is allowed to send more messages).

In any case, the HornetQ version is fairly tightly coupled to the AS version due to the messaging subsystem.  If you really need to run AS 7 in production then I suggest you move to JBoss EAP 6 and also consider getting a Red Hat support subscription.

[ajanz]

i agree with that AS Version and HornetQ Version are tightly coupled. I tried to update today. but there were hard changes, some classe do no longer exist, or were renamed. so i stopped this.

there were 20 threads, which acquired credits.  so this could indeed be an issue of the producer flow control.  so may be we will try a larger window size.  at the moment the window size is not configured, so the default value will be active.

[ajanz]

we use hornetq inside JBoss with the following connection factory

<pooled-connection-factory name="hornetq-ra">

                        <transaction mode="xa"/>

                        <connectors>

                            <connector-ref connector-name="in-vm"/>

                        </connectors>

                        <entries>

                            <entry name="java:/JmsXA"/>

                        </entries>

                        <min-large-message-size>25242880</min-large-message-size>

                    </pooled-connection-factory>

this is not HornetQConnectionFactory, it's HornetRAQConnectionFactoryImpl. There i found no producerwindowsize.

but i found this in documentation.

   <address-settings>

                    <address-setting match="#">

                        <dead-letter-address>jms.queue.DLQ</dead-letter-address>

                        <expiry-address>jms.queue.ExpiryQueue</expiry-address>

                        <redelivery-delay>0</redelivery-delay>

                        <max-size-bytes>1048576</max-size-bytes>

                        <address-full-policy>BLOCK</address-full-policy>

                        <message-counter-history-day-limit>10</message-counter-history-day-limit>

                    </address-setting>

                </address-settings>

so we got a maximum of 10 MB for our topic. But this is a multiuser system, and we send images and "large" xml files over the topic.

so we will increase max-size-bytes

[jbertram]

this is not HornetQConnectionFactory, it's HornetRAQConnectionFactoryImpl. There i found no producerwindowsize.

I'm not sure what you mean.  The pooled-connection-factory in AS 7.1.1.Final supports the "producer-window-size" configuration element.

That said, if you have your address-full-policy set to BLOCK and the address has reached max-size-bytes then your producers will simply have to wait for clients to consume the messages before they are allowed to send more.