4 Replies Latest reply on Nov 30, 2007 4:12 AM by Gian Marco Brusaca'

    long time to reload messages in fail over state

    Gian Marco Brusaca' Newbie

      Hi,
      I have done some tests like the exemples distribueted. In my fail over test I have make 2 clustered istances of Jboss 4.2.2 + JBM 1.4.0 SP1 and I have sended 100 messages in to node 1. In my condition test the balancer have put all (!?) messages in node 2 and afther few second I have manualy killed node 2. Follow the log about this step:

      12:55:36,518 INFO [FailoverCommandCenter] JBoss Messaging server failure detected - waiting for failover to complete...
      12:55:36,705 INFO [TreeCache] viewAccepted(): [192.168.10.48:32790|4] [192.168.10.48:32790]
      12:55:36,720 INFO [TreeCache] viewAccepted(): [192.168.10.48:32792|4] [192.168.10.48:32792]
      12:55:36,726 INFO [TreeCache] viewAccepted(): [192.168.10.48:32783|4] [192.168.10.48:32783]
      12:55:36,799 INFO [MessagingPostOffice] JBoss Messaging is failing over for failed node 48. If there are many messages to reload this may take some time...
      13:27:08,793 INFO [MessageMdb] MESSAGE BEAN: IntProperty iteration: 63
      


      You can see: the node 1 takes 32 minutes to start to consume messages. In this time the CPUs are at 0%... Is it normal?
      When I have restarted node 2 it throws the follow exception:

      14:23:21,934 INFO [TreeCache] viewAccepted(): [192.168.10.48:32783|5] [192.168.10.48:32783, 192.168.10.72:1961]
      14:23:22,037 INFO [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)14:23:57,576 WARN [Client] unable to remove remote callback handler: Can not remove callback listener from target server with id of 3j011-vuslhj-f9l6ile8-1-f9l6pat0-8+3j011-vuslhj-f9l6ile8-1-f9l6payb-b as it does not exist as a registered callback listener.
      14:24:06,520 ERROR [FailoverCommandCenter] Failover failed
      org.jboss.jms.exception.MessagingShutdownException: Cannot handle invocation since messaging server is not active (it is either starting up or shutting down)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:133)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:769)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:573) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:387)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:166)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:163)
       at org.jboss.remoting.Client.invoke(Client.java:1634)
       at org.jboss.remoting.Client.invoke(Client.java:548)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:167)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:83)
       at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
       at org.jboss.jms.client.container.ClusteringAspect.handleCreateConnectionDelegate(ClusteringAspect.java:134)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientClusteredConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.createConnectionDelegate(ClientClusteredConnectionFactoryDelegate.java)
       at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:129)
       at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
       at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
       at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:452)
      14:24:06,522 ERROR [ConnectionFailureListener] Caught exception in handling failure
      org.jboss.jms.exception.MessagingShutdownException: Cannot handle invocation since messaging server is not active (it is either starting up or shutting down)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:133)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:769)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:573) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:387)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:166)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:163)
       at org.jboss.remoting.Client.invoke(Client.java:1634)
       at org.jboss.remoting.Client.invoke(Client.java:548)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:167)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:83)
       at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
       at org.jboss.jms.client.container.ClusteringAspect.handleCreateConnectionDelegate(ClusteringAspect.java:134)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientClusteredConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
       at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.createConnectionDelegate(ClientClusteredConnectionFactoryDelegate.java)
       at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:129)
       at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
       at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
       at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:452)
      ^[[5~^[[5;5~^[[5;5~^[[5~14:25:02,748 INFO [TreeCache] viewAccepted(): [192.168.10.48:32790|5] [192.168.10.48:32790, 192.168.10.72:1978]
      14:25:05,280 INFO [TreeCache] viewAccepted(): [192.168.10.48:32792|5] [192.168.10.48:32792, 192.168.10.72:1983]
      14:26:59,230 INFO [STDOUT]
      -------------------------------------------------------
      GMS: address is 192.168.10.48:32796
      -------------------------------------------------------
      
      


      Is it a bug? Thank you.