HornetQ fails to connect server then node crash
shailendra.soni Jan 13, 2014 2:17 AMEnvironmental Information
--------------------------------------------
Jboss Application Server :- 7.2 Final
Operating System :- Solaris Sparc
JVM :- Sun Java 1.6_25 64 bit.
RAM :- 32 GB
@@
Recently, we have migrated from jboss 6.1 to jboss 7.2 Final in our two solaris servers.
We have 76.12 server and 76.13 servers in same network and jboss 7.2 on each server. Each jboss 7.2 is having different application and different queue. Also We are configured jboss 7.2 as domain mode with 5 instance on each server.
So like
Server :- 76.12 We have jboss 7.2 and application name is SRSEar and one Queue name is SRSSchedule. This application itself as client and MDB, client send message to queue and MDB consume it and process it. And We have 5 domain instaces.
Server :- 76.13 We have jboss 7.2 and application name is IMSEar and one Queue name is IMSSchedule. This application itself as client and MDB, client send message to queue and MDB consume it and process it. And We have 5 domain instaces.
Now problem statements.
- We have configured hornetQ as cluster mode in both jboss servers. In this Jboss 7.2 create bridge between 76.12 and 76.13 and then after some time any of the node got crashed with given message…
10:54:15,416 WARN [org.hornetq.core.client] (hornetq-failure-check-thread) HQ212107: Connection failure has been detected: HQ119034: Did not receive data from invm:0. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. You also might have configured connection-ttl and client-failure-check-period incorrectly. Please check user manual for more information. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
10:54:15,417 WARN [org.hornetq.core.server] (hornetq-failure-check-thread) HQ222100: Client connection failed, clearing up resources for session c6a1c0d0-7ba0-11e3-8169-f7da8b19e073
10:54:15,512 WARN [org.hornetq.core.server] (hornetq-failure-check-thread) HQ222158: Cleared up resources for session c6a1c0d0-7ba0-11e3-8169-f7da8b19e073
Once above error comes, then jboss 7.2 have different kind of error comes. Like,
11:26:05,063 WARN [org.hornetq.jms.server] (Thread-492 (HornetQ-client-global-threads-1881806294)) HQ122017: Notified of connection failure in xa recovery connectionFactory for provider ClientSessionFactoryImpl [serverLocator=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=dc132230-7b9d-11e3-b692-e5ebefde02b7, factory=org-hornetq-core-remoting-impl-invm-InVMConnectorFactory) ?server-id=0], discoveryGroupConfiguration=null], connectorConfig=TransportConfiguration(name=dc132230-7b9d-11e3-b692-e5ebefde02b7, factory=org-hornetq-core-remoting-impl-invm-InVMConnectorFactory) ?server-id=0, backupConfig=null] will attempt reconnect on next pass: HornetQException[errorType=NOT_CONNECTED message=HQ119025: Channel disconnected]
12:01:55,637 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016027: Local XARecoveryModule.xaRecovery got XA exception XAException.XAER_RMERR: javax.transaction.xa.XAException: Error trying to connect to any providers for xa recovery
at org.hornetq.jms.server.recovery.HornetQXAResourceWrapper.getDelegate(HornetQXAResourceWrapper.java:314) [hornetq-jms-server-2.3.0.CR1.jar:]
My first question :- How do I stop bridge? As I do not want to jboss 7.2 connect to another jboss 7.2 in same network.
- On overcome above problem, I have also removed cluster mode in horentQ and then also I have remove “dg-group1” reference from cluster-connections ( I think if cluster false then HornetQ will not use cluster-connections, may be silly change) , after that I have removed data and tmp directory of each and every nodes. And after restart the server, I got the following exception, after few hours my two nodes are crashed.
10:54:15,416 WARN [org.hornetq.core.client] (hornetq-failure-check-thread) HQ212107: Connection failure has been detected: HQ119034: Did not receive data from invm:0. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. You also might have configured connection-ttl and client-failure-check-period incorrectly. Please check user manual for more information. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
10:54:15,417 WARN [org.hornetq.core.server] (hornetq-failure-check-thread) HQ222100: Client connection failed, clearing up resources for session c6a1c0d0-7ba0-11e3-8169-f7da8b19e073
10:54:15,512 WARN [org.hornetq.core.server] (hornetq-failure-check-thread) HQ222158: Cleared up resources for session c6a1c0d0-7ba0-11e3-8169-f7da8b19e073
So I need to two solutions,
- How do I stop bridge so it can work as cluster on same machine?
- And how do I stop the exception comes in point 2?
Please let me know if you would require more detail.
-
domain.xml 74.4 KB