-
1. Re: Hornetq ejb3 mdb reconnect issue
timfox Dec 3, 2009 11:13 AM (in response to sv_srinivaas)Please provide full details of how to replicate this issue and someone will take a look.
Or just wait a short time until the next release is out since it's probably been fixed weeks ago anyway on trunk. -
2. Re: Hornetq ejb3 mdb reconnect issue
sv_srinivaas Dec 7, 2009 3:01 AM (in response to sv_srinivaas)HI i tried this time with the latest version Hornetq 2.0.0.CR1 and still having issues with mdb not reconnecting in case of a network outage.
Environment details:
JBoss 5.0.0 GA
Hornetq 2.0.0.CR1
OS: Windows Xp (service pack2)
Installed Jboss 5.0.0. GA and Hornetq. Executed jca-config example that comes with HornetQ after making the below changes to run the example using the ip address instead of localhost (that comes by default).
Steps to replicate the issue:
1. Modified jca-config\build.xml to run the jboss server using -b option to specify the ip address as shown below
2. Modified jca-config\config\jndi.properties to use the ip address instead of loaclhost as below
java.naming.provider.url=jnp://xxx.yy.zzz.aaa:1199
3. Modified jca-config\server2\hornetq-configuration.xml to use the ip address for both the acceptor and connector (as i'm running the server and mdb client in the same machine)
4. Modified jca-config\server2\jms-remote-ds.xml to use the ip address for connection parameters
<config-property name="ConnectionParameters" type="java.lang.String">host=xxx.yy.zzz.aaa;port=5445</config-property>
5. Modified jca-config\server2\ra.xml to use the ip address for acceptor and connector
<config-property>
<config-property-name>ConnectionParameters</config-property-name>
<config-property-type>java.lang.String</config-property-type>
<config-property-value> host=xxx.yy.zzz.aaa;port=5445</config-property-value>
</config-property>
6. Then using the \jca-config\build.xml, started both the servers 1 and 2 using the commands "build.bat deploy" and then "build.bat deploy2". The consumer count also increased from 0 to 15 for queue B in server2.
7. Sent a message to queue B using "build.bat run".
8. Message was sent successfully and in the server 2 console (as I’m interested only in remote adapter test case) I was able to see the info that " (MDBQueueB.java) Message received using the remote adapter. Message = "Hello there MDB!" that confirmed that the MDB picked the message from remote queue B
9. Now I disconnected the network cable few seconds and I could the see the below exception in server2 console
[exec] 12:14:44,958 ERROR [STDERR] Dec 7, 2009 12:14:44 PM org.hornetq.core
.logging.impl.JULLogDelegate warn
[exec] WARNING: Handling JMS exception failure: org.hornetq.ra.HornetQRAMan
agedConnection@488831
[exec] javax.jms.JMSException: HornetQException[errorCode=0 message=Netty e
xception]
[exec] at org.hornetq.jms.client.HornetQConnection$JMSFailureListener.c
onnectionFailed(HornetQConnection.java:582)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.callFailureL
isteners(FailoverManagerImpl.java:741)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.failoverOrRe
connect(FailoverManagerImpl.java:703)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.handleConnec
tionFailure(FailoverManagerImpl.java:554)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.connectionEx
ception(FailoverManagerImpl.java:296)
[exec] at org.hornetq.integration.transports.netty.NettyConnector$Liste
ner$2.run(NettyConnector.java:632)
[exec] at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.ru
n(OrderedExecutorFactory.java:96)
[exec] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Thread
PoolExecutor.java:885)
[exec] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:907)
[exec] at java.lang.Thread.run(Thread.java:619)
[exec] Caused by: HornetQException[errorCode=0 message=Netty exception]
[exec] at org.hornetq.integration.transports.netty.HornetQChannelHandle
r.exceptionCaught(HornetQChannelHandler.java:105)
[exec] at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(S
impleChannelHandler.java:140)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:560)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannel
HandlerContext.sendUpstream(DefaultChannelPipeline.java:796)
[exec] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptio
nCaught(SimpleChannelUpstreamHandler.java:148)
[exec] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUp
stream(SimpleChannelUpstreamHandler.java:122)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:560)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:555)
[exec] at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels
.java:619)
[exec] at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.ja
va:86)
[exec] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamin
gRunnable.java:108)
[exec] at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRu
nnable.java:46)
[exec] at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunn
able.run(VirtualExecutorService.java:179)
[exec] ... 3 more
[exec] Caused by: java.net.SocketException: Connection reset
[exec] at java.net.SocketInputStream.read(SocketInputStream.java:168)
[exec] at java.net.SocketInputStream.read(SocketInputStream.java:182)
[exec] at java.io.FilterInputStream.read(FilterInputStream.java:66)
[exec] at java.io.PushbackInputStream.read(PushbackInputStream.java:122
)
[exec] at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.ja
va:77)
[exec] ... 5 more
[exec] 12:14:44,958 WARN [TxConnectionManager] Connection error occured: o
rg.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListene
r@11768ab[state=NORMAL mc=org.hornetq.ra.HornetQRAManagedConnection@488831 handl
es=0 lastUse=1260168260708 permit=false trackByTx=false mcp=org.jboss.resource.c
onnectionmanager.JBossManagedConnectionPool$OnePool@17f75f1 context=org.jboss.re
source.connectionmanager.InternalManagedConnectionPool@1182c3d xaResource=org.ho
rnetq.ra.HornetQRAXAResource@1d8c699 txSync=null]
[exec] javax.jms.JMSException: HornetQException[errorCode=0 message=Netty e
xception]
[exec] at org.hornetq.jms.client.HornetQConnection$JMSFailureListener.c
onnectionFailed(HornetQConnection.java:582)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.callFailureL
isteners(FailoverManagerImpl.java:741)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.failoverOrRe
connect(FailoverManagerImpl.java:703)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.handleConnec
tionFailure(FailoverManagerImpl.java:554)
[exec] at org.hornetq.core.client.impl.FailoverManagerImpl.connectionEx
ception(FailoverManagerImpl.java:296)
[exec] at org.hornetq.integration.transports.netty.NettyConnector$Liste
ner$2.run(NettyConnector.java:632)
[exec] at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.ru
n(OrderedExecutorFactory.java:96)
[exec] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Thread
PoolExecutor.java:885)
[exec] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:907)
[exec] at java.lang.Thread.run(Thread.java:619)
[exec] Caused by: HornetQException[errorCode=0 message=Netty exception]
[exec] at org.hornetq.integration.transports.netty.HornetQChannelHandle
r.exceptionCaught(HornetQChannelHandler.java:105)
[exec] at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(S
impleChannelHandler.java:140)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:560)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannel
HandlerContext.sendUpstream(DefaultChannelPipeline.java:796)
[exec] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptio
nCaught(SimpleChannelUpstreamHandler.java:148)
[exec] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUp
stream(SimpleChannelUpstreamHandler.java:122)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:560)
[exec] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(D
efaultChannelPipeline.java:555)
[exec] at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels
.java:619)
[exec] at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.ja
va:86)
[exec] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamin
gRunnable.java:108)
[exec] at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRu
nnable.java:46)
[exec] at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunn
able.run(VirtualExecutorService.java:179)
[exec] ... 3 more
[exec] Caused by: java.net.SocketException: Connection reset
[exec] at java.net.SocketInputStream.read(SocketInputStream.java:168)
[exec] at java.net.SocketInputStream.read(SocketInputStream.java:182)
[exec] at java.io.FilterInputStream.read(FilterInputStream.java:66)
[exec] at java.io.PushbackInputStream.read(PushbackInputStream.java:122
)
[exec] at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.ja
va:77)
[exec] ... 5 more
10. Reconnected the network cable and sent a new message using the "build.bat run" command and the jmx console confirmed that the message was sent to queue B in server2 but the MDB consumer count was showing as 0 and the message was NOT picked by the MDB.
11. Also the server 2 console displayed the below error message
[exec] 13:08:28,812 ERROR [STDERR] Dec 7, 2009 1:08:28 PM org.hornetq.core.
logging.impl.JULLogDelegate warn
[exec] WARNING: Connection failure has been detected: Did not receive ping
from /172.18.164.138:1895. It is likely the client has exited or crashed without
closing its connection, or the network between the server and client has failed
. The connection will now be closed. [code=3]
[exec] 13:08:28,812 ERROR [STDERR] Dec 7, 2009 1:08:28 PM org.hornetq.core.
logging.impl.JULLogDelegate warn
[exec] WARNING: Client connection failed, clearing up resources for session
3aeb9c8f-e303-11de-a4e1-002170a2b761
12. When I restarted server2 MDB started working and picked the message. -
3. Re: Hornetq ejb3 mdb reconnect issue
ataylor Dec 7, 2009 5:11 AM (in response to sv_srinivaas)i will take a look
-
4. Re: Hornetq ejb3 mdb reconnect issue
timfox Dec 7, 2009 8:05 AM (in response to sv_srinivaas)What did you set, reconnectAttempts to in the jca config? Default is zero
-
5. Re: Hornetq ejb3 mdb reconnect issue
sv_srinivaas Dec 7, 2009 10:19 PM (in response to sv_srinivaas)reconnectAttempts is set to -1 in hornetq-jms.xml as below. I dont see much difference with or without this setting.
<connection-factory name="ConnectionFactory">
<connector-ref connector-name="netty"/>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<!-- Try reconnecting an unlimited number of times (-1 means "unlimited") -->
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory> -
6. Re: Hornetq ejb3 mdb reconnect issue
timfox Dec 8, 2009 4:59 AM (in response to sv_srinivaas)That's not the JCA config.
It's the JCA adaptor making a connection, so you need to set reconnect attempts in there. -
7. Re: Hornetq ejb3 mdb reconnect issue
sv_srinivaas Dec 8, 2009 5:15 AM (in response to sv_srinivaas)Tim, can you pls let me know where exactly should I make this change? Is it in the ra.xml under hornetq-ra.rar\META-INF and if so, is it in the inbound-resourceadapter? Thanks!
-
8. Re: Hornetq ejb3 mdb reconnect issue
ataylor Dec 8, 2009 5:26 AM (in response to sv_srinivaas)you need to configure server2/ra.xml and server2/jms-ds.xml. Server2 ids the remote erver that sends and consumes from server1
-
9. Re: Hornetq ejb3 mdb reconnect issue
ataylor Dec 8, 2009 9:25 AM (in response to sv_srinivaas)fyi i have changed the default configuration to always try reconnection, see http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4269537#4269537
-
10. Re: Hornetq ejb3 mdb reconnect issue
sv_srinivaas Dec 9, 2009 1:33 AM (in response to sv_srinivaas)Hi, clicking on the above url refreshes the same page. Pls help.
-
11. Re: Hornetq ejb3 mdb reconnect issue
timfox Dec 9, 2009 3:59 AM (in response to sv_srinivaas)"sv_srinivaas" wrote:
Hi, clicking on the above url refreshes the same page. Pls help.
Andy was going to add something to the user manual regarding this, so it should help you out. -
12. Re: Hornetq ejb3 mdb reconnect issue
ataylor Dec 9, 2009 4:11 AM (in response to sv_srinivaas)sorry i posted the wrong link, it was meant to be for the Jira which is now fixed in trunk. https://jira.jboss.org/jira/browse/HORNETQ-240.
If you follow my config that i provided earlier this should work fine. I have tested this myself. -
13. Re: Hornetq ejb3 mdb reconnect issue
sv_srinivaas Dec 10, 2009 9:29 AM (in response to sv_srinivaas)Andy, I have not worked in jca before, can you pls paste the code for ra.xml and jms-ds xml for this fix? I tried based on your comment but i dont see any difference in the output and i'm sure i'm missing something.
-
14. Re: Hornetq ejb3 mdb reconnect issue
timfox Dec 10, 2009 9:31 AM (in response to sv_srinivaas)Probably a good place to start is to read the chapter on JCA in the user manual.
That explains how to configure the resource adaptor.