HornetQ live-Backup failover on JBoss AS 7
hsichuan Jan 11, 2012 3:52 AMHello everyone, I am not a native English speaker and I'm new to here. I really need some help, please help me~thanks.
I had spent a lot of time on the HornetQ User Manual but it seems like that there are some problems happened.
What I want to do is that live server can failover to backup server when client is sending messages.
My problem is: when I killed the live server, my backup server could not take over and become live.
I have 4 servers -
A: HornetQ Live Server on JBoss AS7 (jboss-as-7.0.1.Final)
deploy mdb-ear-1.0.0.ear (andytaylor-HornetQ-AS7-examples-1a5935b , it's a mdb example)
B: HornetQ Backup Server on JBoss AS7
deploy mdb-ear-1.0.0.ear , but backup server is lock, this deployment will not be active until backup server become live
C: a client running MDB example, send 2000 of messages to live server (andytaylor-HornetQ-AS7-examples-1a5935b/mdb/mdb-client)
D: shared store - NFS, to store A&B's message journal
The method of executing server:
A: ./standalone.sh --server-config=standalone-preview-ha-live.xml
B: ./standalone.sh --server-config=standalone-preview-ha-back.xml
C: mvn -Pclient test
Here is the configurations changed on A & B
standalone-preview-ha-live.xml & standalone-preview-ha-back.xml are modified from standalone-preview-ha.xml
where the font color is red is my modified for configurations, others are the same as standalone-preview-ha.xml
A: live server -
<subsystem xmlns="urn:jboss:domain:messaging:1.0">
<!-- Default journal file size is 10Mb, reduced here to 100k for faster first boot -->
<journal-file-size>102400</journal-file-size>
<journal-min-files>2</journal-min-files>
<journal-type>NIO</journal-type>
<!-- disable messaging persistence -->
<persistence-enabled>false</persistence-enabled>
<!--add hornetq-configuration set live here-->
<clustered>true</clustered>
<failover-on-shutdown>false</failover-on-shutdown>
<shared-store>true</shared-store>
<!--end here-->
---
skip
---
<jms-destinations>
<jms-queue name="testQueue">
<entry name="queue/test"/>
</jms-queue>
<jms-queue name="mdbQueue">
<entry name="queue/mdbQueue"/>
</jms-queue>
<jms-queue name="mdbReplyQueue">
<entry name="queue/mdbReplyQueue"/>
</jms-queue>
<jms-topic name="testTopic">
<entry name="topic/test"/>
</jms-topic>
</jms-destinations>
---
skip
---
<interfaces>
<interface name="management">
<inet-address value="140.XXX.XX.231"/>
</interface>
<interface name="public">
<inet-address value="140.XXX.XX.231"/>
</interface>
</interfaces>
B: backup server -
<subsystem xmlns="urn:jboss:domain:messaging:1.0">
<!-- Default journal file size is 10Mb, reduced here to 100k for faster first boot -->
<journal-file-size>102400</journal-file-size>
<journal-min-files>2</journal-min-files>
<journal-type>NIO</journal-type>
<!-- disable messaging persistence -->
<persistence-enabled>false</persistence-enabled>
<!--add hornetq-configuration set backup here -->
<clustered>true</clustered>
<failover-on-shutdown>false</failover-on-shutdown>
<backup>true</backup>
<shared-store>true</shared-store>
<!--end here-->
<connectors>
<netty-connector name="netty" socket-binding="messaging" />
<netty-connector name="netty-throughput" socket-binding="messaging-throughput">
<param key="batch-delay" value="50"/>
</netty-connector>
<in-vm-connector name="in-vm" server-id="0" />
</connectors>
---
skip
---
<jms-destinations>
<jms-queue name="testQueue">
<entry name="queue/test"/>
</jms-queue>
<jms-queue name="mdbQueue">
<entry name="queue/mdbQueue"/>
</jms-queue>
<jms-queue name="mdbReplyQueue">
<entry name="queue/mdbReplyQueue"/>
</jms-queue>
<jms-topic name="testTopic">
<entry name="topic/test"/>
</jms-topic>
</jms-destinations>
---
skip
---
<interfaces>
<interface name="management">
<inet-address value="140.XXX.XX.232"/>
</interface>
<interface name="public">
<inet-address value="140.XXX.XX.232"/>
</interface>
</interfaces>
C: I modified "MDBClient.java", in order to send messages to the live server.
final int numMessages = 2000;
HashMap map = new HashMap();
map.put("host", "140.XXX.XX.231");
map.put("port", 5445);
TransportConfiguration transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class.getName(),map);
---skip---
TextMessage message=null;
//Step 7. Create a Text Message
for (int i = 0; i < numMessages; i++)
{
message = session.createTextMessage("This is a text message"+ i);
//Step 8. Send the Message
producer.send(message);
System.out.println("Sent message: " + message.getText());
}
D: shared store - NFS
The NFS directory is /home/user/jboss-as-7.0.1.Final/standalone/data
==================================
Now, after live & backup server is started, I executed "MDBClient.java", sending 2000 messages to the live server.
Then, I went to the live server and killed its process id.
jps
kill XXXXX
My backup server did not become live, but after some minutes, there are a lot of WARN on live server's boot.log & backup server's server.log
WARN [com.arjuna.ats.jta] ARJUNA16038: No XAResource to recover < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ebf, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ec0, eis_name=unknown eis name >
Here is the server.log on live server.
14:32:55,668 INFO [org.apache.catalina.core.AprLifecycleListener] (MSC service thread 1-4) The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /opt/jdk1.6.0_24/jre/lib/amd64/server:/opt/jdk1.6.0_24/jre/lib/amd64:/opt/jdk1.6.0_24/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
14:32:55,701 INFO [org.jboss.as.jmx.JMXConnectorService] (MSC service thread 1-1) Starting remote JMX connector
14:32:55,707 INFO [org.jboss.as.remoting] (MSC service thread 1-7) Listening on /140.XXX.XX.231:9999
14:32:55,738 INFO [org.jboss.as.ee] (Controller Boot Thread) Activating EE subsystem
14:32:55,742 INFO [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-5) Starting Coyote HTTP/1.1 on http-WMC-Cloud-A-140.XXX.XX.231-8080
14:32:55,751 INFO [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-6) JBoss Web Services - Stack CXF Server 4.0.0.Beta3
14:32:55,889 INFO [org.jboss.as.modcluster] (MSC service thread 1-4) Mod_cluster uses default load balancer provider
14:32:55,963 INFO [org.jboss.modcluster.ModClusterService] (MSC service thread 1-4) Initializing mod_cluster 1.1.3.Final
14:32:55,982 INFO [org.jboss.as.connector] (MSC service thread 1-2) Starting JCA Subsystem (JBoss IronJacamar 1.0.3.Final)
14:32:55,986 INFO [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-4) Listening to proxy advertisements on 224.0.1.105:23364
14:32:56,045 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) Bound data source [java:jboss/datasources/ExampleDS]
14:32:56,056 INFO [org.jboss.as.jacorb] (MSC service thread 1-3) CORBA ORB Service Started
14:32:56,126 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingjournal,bindingsDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingbindings,largeMessagesDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messaginglargemessages,pagingDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingpaging)
14:32:56,161 \u8b66\u544a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.
14:32:56,172 INFO [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) Registered connection factory java:/JmsXA
14:32:56,193 INFO [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-1) HornetQ resource adaptor started
14:32:56,193 INFO [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatorhornetq-ra
14:32:56,209 INFO [org.jboss.as.jacorb] (MSC service thread 1-3) CORBA Naming Service Started
14:32:56,304 INFO [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-5) Started Netty Acceptor version 3.2.3.Final-r${buildNumber} WMC-Cloud-A:5455 for CORE protocol
14:32:56,305 INFO [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-5) Started Netty Acceptor version 3.2.3.Final-r${buildNumber} WMC-Cloud-A:5445 for CORE protocol
14:32:56,306 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) Server is now live
14:32:56,306 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) HornetQ Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [b4c2c38f-3c18-11e1-8dd2-e41f13246ff6] started
14:32:56,308 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-8) trying to deploy queue jms.topic.testTopic
14:32:56,384 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-8) Bound messaging object to jndi name java:/topic/test
14:32:56,395 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-2) Bound messaging object to jndi name java:/RemoteConnectionFactory
14:32:56,397 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-1) Bound messaging object to jndi name java:/ConnectionFactory
14:32:56,397 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) trying to deploy queue jms.queue.mdbQueue
14:32:56,421 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-5) Bound messaging object to jndi name java:/queue/mdbQueue
14:32:56,422 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-7) trying to deploy queue jms.queue.testQueue
14:32:56,435 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-7) Bound messaging object to jndi name java:/queue/test
14:32:56,435 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) trying to deploy queue jms.queue.mdbReplyQueue
14:32:56,446 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-4) Bound messaging object to jndi name java:/queue/mdbReplyQueue
14:32:56,623 INFO [org.jboss.as.deployment] (MSC service thread 1-4) Started FileSystemDeploymentService for directory /home/wmc/jboss-as-7.0.1.Final/standalone/deployments
14:32:56,641 INFO [org.jboss.as] (Controller Boot Thread) JBoss AS 7.0.1.Final "Zap" started in 2665ms - Started 128 of 208 services (80 services are passive or on-demand)
14:32:56,652 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2) Starting deployment of "my-webapp.war"
14:32:56,652 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) Starting deployment of "mdb-ear-1.0.0.ear"
14:32:56,692 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) Starting deployment of "mdb-jar-1.0.0.jar"
14:32:56,717 INFO [org.jboss.as.jpa] (MSC service thread 1-3) added javax.persistence.api dependency to my-webapp.war
14:32:56,720 INFO [org.jboss.as.jpa] (MSC service thread 1-5) added javax.persistence.api dependency to mdb-ear-1.0.0.ear
14:32:56,723 INFO [org.jboss.as.jpa] (MSC service thread 1-5) added javax.persistence.api dependency to mdb-jar-1.0.0.jar
14:32:56,875 INFO [org.jboss.web] (MSC service thread 1-4) registering web context: /my-webapp
14:32:56,923 INFO [org.jboss.as.server.controller] (DeploymentScanner-threads - 2) Deployed "mdb-ear-1.0.0.ear"
14:32:56,923 INFO [org.jboss.as.server.controller] (DeploymentScanner-threads - 2) Deployed "my-webapp.war"
14:33:05,921 INFO [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Engine [jboss.web] will use jvmRoute: d883de1f-fe17-3214-8442-4e4880a8781a
---after client send messages---
14:34:10,802 INFO [stdout] (Thread-3 (group:HornetQ-client-global-threads-2000502626)) message This is a text message2
14:34:10,805 INFO [stdout] (Thread-3 (group:HornetQ-client-global-threads-2000502626)) connection created
14:34:10,815 INFO [stdout] (Thread-9 (group:HornetQ-client-global-threads-2000502626)) message This is a text message8
14:34:10,815 INFO [stdout] (Thread-9 (group:HornetQ-client-global-threads-2000502626)) connection created
14:34:10,815 INFO [stdout] (Thread-2 (group:HornetQ-client-global-threads-2000502626)) message This is a text message1
14:34:10,816 INFO [stdout] (Thread-2 (group:HornetQ-client-global-threads-2000502626)) connection created
---skip---kill the live server
14:36:32,377 INFO [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-5) Stopping Coyote HTTP/1.1 on http-WMC-Cloud-A-140.116.82.231-8080
14:36:32,378 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-7) Unbound messaging object from jndi name java:/RemoteConnectionFactory
14:36:32,380 INFO [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-6) Unbound messaging object from jndi name java:/topic/test
boot.log on live server (because live server is shutdown)
14:32:55,322 INFO [org.jboss.as.logging] Removing bootstrap log handlers
The followings are the WARNs and ERRORs after the live server killed.
14:36:32,398 INFO [org.jboss.as.logging] Restored bootstrap log handlers
14:36:32,415 INFO [org.jboss.modcluster.ModClusterService] All active sessions drained from context [/my-webapp] in 0 seconds
14:36:32,421 INFO [org.jboss.as.jmx.JMXConnectorService] JMX remote connector stopped
14:36:32,423 INFO [stdout] message This is a text message415
14:36:32,423 INFO [stdout] connection created
14:36:32,426 INFO [org.jboss.modcluster.ModClusterService] All active sessions drained from context [/my-webapp] in 0 seconds
14:36:32,429 INFO [stdout] session created
14:36:32,430 ERROR [stderr] javax.jms.InvalidDestinationException: Destination mdbReplyQueue does not exist
14:36:32,430 ERROR [stderr] at org.hornetq.jms.client.HornetQSession.createProducer(HornetQSession.java:337)
14:36:32,430 ERROR [stderr] at org.hornetq.ra.HornetQRASession.createProducer(HornetQRASession.java:1163)
14:36:32,430 ERROR [stderr] at org.hornetq.example.MDBSendExample.onMessage(MDBSendExample.java:77)
--skip--
14:36:32,608 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] HornetQ Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [b4c2c38f-3c18-11e1-8dd2-e41f13246ff6] stopped
14:37:02,575 WARN [com.arjuna.ats.jta] ARJUNA16086: TransactionImple.enlistResource setTransactionTimeout on XAResource < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ee7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ee8, eis_name=unknown eis name > threw: XAException.XAER_RMERR: javax.transaction.xa.XAException
--skip
14:37:02,599 ERROR [org.hornetq.core.client.impl.ClientSessionImpl] Caught jmsexecptione : HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 52]
--skip
14:37:02,605 WARN [com.arjuna.ats.jta] ARJUNA16041: prepare on < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:edf, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ee0, eis_name=unknown eis name > (org.hornetq.core.client.impl.DelegatingSession@4aa594e1) failed with exception XAException.XAER_RMERR: javax.transaction.xa.XAException
--skip
14:37:02,610 WARN [com.arjuna.ats.arjuna] ARJUNA12073: BasicAction.End() - prepare phase of action-id 0:ffff8c7452e7:7386542f:4f0d2d17:edf failed.
14:37:02,610 WARN [com.arjuna.ats.arjuna] ARJUNA12075: Action Aborting
14:37:02,633 WARN [org.hornetq.core.client.impl.ClientSessionImpl] failover occured during commit throwing XAException.XA_RETRY
14:37:02,634 WARN [com.arjuna.ats.jta] ARJUNA16036: commit on < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:eb7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:eb8, eis_name=unknown eis name > (org.hornetq.core.client.impl.DelegatingSession@58fe210a) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException
Here is the server.log on backup server.
14:35:36,363 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-5) backup server is starting with configuration HornetQ Configuration (clustered=true,backup=true,sharedStore=true,journalDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingjournal,bindingsDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingbindings,largeMessagesDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messaginglargemessages,pagingDirectory=/home/wmc/jboss-as-7.0.1.Final/standalone/data/messagingpaging)
14:35:36,365 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-5) Caching command for createQueue for mdbReplyQueue since the JMS Server is not active yet
14:35:36,367 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-5) Caching command for createConnectionFactory for RemoteConnectionFactory since the JMS Server is not active yet
14:35:36,367 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-5) Caching command for createQueue for testQueue since the JMS Server is not active yet
14:35:36,367 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-5) Caching command for createQueue for mdbQueue since the JMS Server is not active yet
14:35:36,368 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-6) Caching command for createConnectionFactory for InVmConnectionFactory since the JMS Server is not active yet
14:35:36,368 INFO [org.hornetq.core.server.impl.FileLockNodeManager] (Thread-18) Waiting to become backup node
14:35:36,368 \u8cc7\u8a0a [org.hornetq.jms.server.impl.JMSServerManagerImpl] (MSC service thread 1-2) Caching command for createTopic for testTopic since the JMS Server is not active yet
14:35:36,396 INFO [org.jboss.as.jacorb] (MSC service thread 1-1) CORBA Naming Service Started
14:35:36,401 INFO [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-3) Registered connection factory java:/JmsXA
14:35:36,407 INFO [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-3) HornetQ resource adaptor started
14:35:36,408 INFO [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-3) IJ020002: Deployed: file://RaActivatorhornetq-ra
14:35:36,427 INFO [org.hornetq.core.server.impl.FileLockNodeManager] (Thread-18) ** got backup lock
14:35:36,443 \u8b66\u544a [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-18) Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.
14:35:36,487 \u8cc7\u8a0a [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-18) HornetQ Backup Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [b4c2c38f-3c18-11e1-8dd2-e41f13246ff6] started, waiting live to fail before it gets active
14:35:36,843 INFO [org.jboss.as.deployment] (MSC service thread 1-5) Started FileSystemDeploymentService for directory /home/wmc/jboss-as-7.0.1.Final/standalone/deployments
14:35:36,860 INFO [org.jboss.as] (Controller Boot Thread) JBoss AS 7.0.1.Final "Zap" started in 2572ms - Started 128 of 208 services (80 services are passive or on-demand)
14:35:46,162 INFO [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Engine [jboss.web] will use jvmRoute: a8c846d5-8c74-327b-ba50-723177f332e2
--the following are the WARNs after live server killed,
14:37:56,286 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA16037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed8, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@78221c75 >
14:37:56,287 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA16037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:edc, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5271b383 >
14:37:56,288 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA16038: No XAResource to recover < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed8, eis_name=unknown eis name >
14:37:56,288 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA16038: No XAResource to recover < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffff8c7452e7:7386542f:4f0d2d17:ed7, node_name=1, branch_uid=0:ffff8c7452e7:7386542f:4f0d2d17:edc, eis_name=unknown eis name >
--it will receive the same WARN repeately---
=================================
The live & backup server will always receive the same WARNs repeately, it seems like a deadlock.
However, the backup server did not become live.
Do I miss any other configurations on live & backup server?
Does anyone knows how to successfully set live & backup server failover?
I'm sorry about that I post a long description of my problem, if there is any question about my problem, just ask.
Thanks for your help!