Two questions: failover, persistent distributed state
Hello!
I started to implement a HA singleton service. After some first failover tests I recognized that it takes about 3 minutes until the new master node is elected. It seems to me that JBoss first trys to migrate the whole JMS stuff which also lays in directory "deploy-hasingleton" (I use the 3.2.4 source snapshot from 26. May 2004).
There are some exceptions on the console, see below. Btw, my service hasn't any dependency to a JMS Queue or Topic. Is the delay caused by these exceptions and does JBoss migrate singleton services sequentially where one has to wait until the one before was migrated.
The second question is about the DistributedState. I looked at the code of DistributedStateImpl and it seems to me that everything which is stored there is not persistent. Are there any plans for the future to implement a persistent distributed state?
Regards,
Thorsten
09:27:36,140 WARN [Connection] Connection failure: org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketException: Connection reset) at org.jboss.mq.Connection.asynchFailure(Connection.java:429) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:398) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:332) at java.lang.Thread.run(Thread.java:534) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:168) at java.io.BufferedInputStream.fill(BufferedInputStream.java:183) at java.io.BufferedInputStream.read(BufferedInputStream.java:201) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313) at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380) at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452) at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601) at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:279) ... 1 more 09:27:36,156 WARN [JMSContainerInvoker] JMS provider failure detected: org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketException: Connection reset) at org.jboss.mq.Connection.asynchFailure(Connection.java:429) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:398) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:332) at java.lang.Thread.run(Thread.java:534) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:168) at java.io.BufferedInputStream.fill(BufferedInputStream.java:183) at java.io.BufferedInputStream.read(BufferedInputStream.java:201) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313) at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380) at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452) at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601) at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:279) ... 1 more 09:27:36,187 INFO [JMSContainerInvoker] Trying to reconnect to JMS provider 09:27:46,187 ERROR [JMSContainerInvoker] Could not stop JMS connection org.jboss.mq.SpyJMSException: Cannot disable the connection with the JMS server; - nested throwable: (java.io.IOException: Client is not connected) at org.jboss.mq.Connection.doStop(Connection.java:1278) at org.jboss.mq.Connection.stop(Connection.java:711) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerStop(JMSContainerInvoker.java:832) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1265) at org.jboss.mq.Connection.asynchFailure(Connection.java:439) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:398) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:332) at java.lang.Thread.run(Thread.java:534) Caused by: java.io.IOException: Client is not connected at org.jboss.mq.il.uil2.SocketManager.internalSendMessage(SocketManager.java:226) at org.jboss.mq.il.uil2.SocketManager.sendMessage(SocketManager.java:192) at org.jboss.mq.il.uil2.UILServerIL.setEnabled(UILServerIL.java:162) at org.jboss.mq.Connection.doStop(Connection.java:1274) ... 8 more 09:28:46,734 ERROR [DLQHandler] Initialization failed DLQHandler javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.mq.Connection.asynchFailure(Connection.java:439) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:398) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:332) at java.lang.Thread.run(Thread.java:534) 09:28:46,781 INFO [JMSContainerInvoker] Reconnected to JMS provider 09:28:46,796 WARN [JMSContainerInvoker] JMS provider failure detected: javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.mq.Connection.asynchFailure(Connection.java:439) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:398) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:332) at java.lang.Thread.run(Thread.java:534) 09:28:46,796 INFO [JMSContainerInvoker] Trying to reconnect to JMS provider 09:29:56,937 ERROR [DLQHandler] Initialization failed DLQHandler javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$1.run(JMSContainerInvoker.java:776) 09:29:56,937 INFO [JMSContainerInvoker] Reconnected to JMS provider 09:29:56,937 WARN [JMSContainerInvoker] JMS provider failure detected: javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$1.run(JMSContainerInvoker.java:776) 09:29:56,937 INFO [JMSContainerInvoker] Trying to reconnect to JMS provider 09:30:37,031 INFO [fupMailQueue] Bound to JNDI name: queue/fupMailQueue 09:30:37,062 INFO [OIL2ServerILService] JBossMQ OIL2 service available at : /0.0.0.0:8092 09:30:37,171 INFO [UILServerILService] JBossMQ UIL service available at : /0.0.0.0:8093 09:30:37,250 INFO [DLQ] Bound to JNDI name: queue/DLQ 09:30:37,921 ERROR [DLQHandler] Initialization failed DLQHandler javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$1.run(JMSContainerInvoker.java:776) 09:30:37,921 INFO [JMSContainerInvoker] Reconnected to JMS provider 09:30:37,937 WARN [JMSContainerInvoker] JMS provider failure detected: javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542) at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1267) at org.jboss.ejb.plugins.jms.JMSContainerInvoker$1.run(JMSContainerInvoker.java:776) 09:30:37,937 INFO [JMSContainerInvoker] Trying to reconnect to JMS provider 09:30:38,000 INFO [DefaultPartition] New cluster view (id: 10, delta: -1) : [192.168.2.25:1099] 09:30:38,015 INFO [DefaultPartition:ReplicantManager] Dead members: 1 09:30:48,390 INFO [JMSContainerInvoker] Reconnected to JMS provider
