Getting ConnectionFactory not found sometimes - HornetQ server not started in HA, collocated topology with replication mode
Hi,
We are using Jboss7.2 with HornetQ2.4. Topology is collocated HA using in-vm connection factory.
Randomly while doing failover/failback, sometimes ConnectionFactory is not getting bound to JNDI name and it continuously gives the error
javax.naming.NameNotFoundException: ConnectionFactory -- service jboss.naming.context.java.ConnectionFactory
at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
at org.jboss.as.naming.InitialContext.lookup(InitialContext.java:120)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:183)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
After restarting jboss things work fine. Below are the messages in server.log at the time of failure
INFO [org.hornetq.jms.server] (ServerService Thread Pool -- 122) HQ121004: JMS Server Manager Caching command for destroyConnectionFactory for RemoteConnectionFactory since the JMS Server is not active yet
HQ121004: JMS Server Manager Caching command for createQueue for XXX since the JMS Server is not active yet
(MSC service thread 1-17) HQ122018: Could not start recovery discovery on XARecoveryConfig [transportConfiguration = [TransportConfiguration(name=8bfca16f-baf5-11e3-bcee-cb0c6044933c, factory=org-hornetq-core-remoting-impl-invm-InVMConnectorFactory) ?server-id=0], discoveryConfiguration = null, username=null, password=****], we will retry every recovery scan until the server is available
INFO [org.hornetq.ra] (default-threads - 2) HQ151005: awaiting HornetQ Server availability
ERROR [org.hornetq.core.server] (HQ119000: Activation for server HornetQServerImpl::serverUUID=null) HQ224000: Failure in initialisation: HornetQIllegalStateException[errorType=ILLEGAL_STATE message=HQ119026: Backup Server was not yet in sync with live]
at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingBackupActivation.run(HornetQServerImpl.java:2523) [hornetq-server-2.4.0-SNAPSHOT.jar:]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
ERROR [stderr] (HQ119000: Activation for server HornetQServerImpl::serverUUID=null) HornetQIllegalStateException[errorType=ILLEGAL_STATE message=HQ119026: Backup Server was not yet in sync with live]
ERROR [stderr] (HQ119000: Activation for server HornetQServerImpl::serverUUID=null) at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingBackupActivation.run(HornetQServerImpl.java:2523)
ERROR [stderr] (HQ119000: Activation for server HornetQServerImpl::serverUUID=null) at java.lang.Thread.run(Thread.java:744)
INFO [org.hornetq.core.server] (Thread-3 (HornetQ-client-netty-threads-194345171)) HQ221024: Backup server HornetQServerImpl::serverUUID=7884359f-b8d2-11e3-b862-9f538e14a75f is synchronized with live-server.
INFO [org.hornetq.core.server] (Thread-1 (HornetQ-server-HornetQServerImpl::serverUUID=null-585850032)) HQ221031: backup announced
What could cause this issue?
In production environment, it is not possible to restart app server multiple times.
Is there a way to solve it?
Regards,
Veena
