6 Replies Latest reply on Nov 1, 2012 12:18 AM by Clebert Suconic

    Journal file replication NPE issue

    yong deng Newbie

      HornetQ: 2.3 BETA

      Topology: Two HornetQ has been configured to form HA live backup group. The HA mode is date replication

       

      Steps:

       

      1. Start master

      2. Start backup

      3. Send some durable messages to master

      4. Stop master

      5. backup take over and then stop the backup

      6. Retry some rounds of above

       

      Finally, seems HornetQ journal data on two servers will be kind of out of sync.

      During journal file replication, there will be NPE thrown in backup server.

       

       

       

       

       

      18:26:37,455 INFO  [org.hornetq.integration.bootstrap] HQ101001: Starting HornetQ Server

       

      18:26:39,080 INFO  [org.hornetq.core.server] HQ111045: Configuration option clustered is deprecated. Consult the manual for details.

       

      18:26:39,096 WARN  [org.hornetq.core.server] HQ112054: AIO was not located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal

       

      18:26:39,205 INFO  [org.hornetq.core.server] HQ111001: backup server is starting with configuration HornetQ Configuration (clustered=true,backup=true,sharedStore=false,journalDirectory=../data/journal,bindingsDirectory=../data/bindings,largeMessagesDirectory=../data/large-messages,pagingDirectory=../data/paging)

       

      18:26:39,236 WARN  [org.hornetq.core.server] HQ112216: Moving data directory ../data/bindings to ..\data\bindings1

       

      18:26:39,236 WARN  [org.hornetq.core.server] HQ112216: Moving data directory ../data/paging to ..\data\paging1

       

      18:26:39,236 WARN  [org.hornetq.core.server] HQ112216: Moving data directory ../data/large-messages to ..\data\large-messages1

       

      18:26:39,299 INFO  [org.hornetq.core.server] HQ111017: Using NIO Journal

       

      18:26:39,314 WARN  [org.hornetq.core.server] HQ112008: Security risk! HornetQ is running with the default cluster admin user and default password. Please see the HornetQ user guide, cluster chapter, for instructions on how to change this.

       

      18:26:42,314 INFO  [org.hornetq.core.server] HQ111037: Waiting to become backup node

       

      18:26:42,330 INFO  [org.hornetq.core.server] HQ111038: ** got backup lock

       

      18:26:42,330 INFO  [org.hornetq.core.server] HQ001111: HornetQ Backup Server version 2.3.0.BETA1 (HornetQ sting, 122) [3a6fc34f-1dc2-11e2-a6fc-c979bbfccd4c] started, waiting live to fail before it gets active

       

      18:26:44,611 WARN  [org.hornetq.core.server] HQ112130: error handling packet PACKET(ReplicationStartSyncMessage)[type=120, channelID=2, packetObject=ReplicationStartSyncMessage] for replication: HornetQException[errorType=IO_ERROR message=HQ149001: failed to rename file hornetq-data-2.hq.tmp to hornetq-data-2.hq]

          at org.hornetq.core.journal.impl.AbstractSequentialFile.renameTo(AbstractSequentialFile.java:159) [hornetq-journal.jar:]

          at org.hornetq.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:624) [hornetq-journal.jar:]

          at org.hornetq.core.journal.impl.JournalFilesRepository.createRemoteBackupSyncFile(JournalFilesRepository.java:572) [hornetq-journal.jar:]

          at org.hornetq.core.journal.impl.JournalImpl.createFilesForBackupSync(JournalImpl.java:3067) [hornetq-journal.jar:]

          at org.hornetq.core.replication.ReplicationEndpoint.handleStartReplicationSynchronization(ReplicationEndpoint.java:585) [hornetq-core.jar:]

          at org.hornetq.core.replication.ReplicationEndpoint.handlePacket(ReplicationEndpoint.java:214) [hornetq-core.jar:]

          at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:548) [hornetq-core-client.jar:]

          at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:550) [hornetq-core-client.jar:]

          at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:511) [hornetq-core-client.jar:]

          at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1627) [hornetq-core-client.jar:]

          at org.hornetq.core.remoting.impl.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:72) [hornetq-core-client.jar:]

          at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:95) [jboss-mc.jar:]

          at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [jboss-mc.jar:]

          at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792) [jboss-mc.jar:]

          at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:281) [jboss-mc.jar:]

          at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:169) [hornetq-core-client.jar:]

          at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:134) [hornetq-core-client.jar:]

          at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75) [jboss-mc.jar:]

          at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [jboss-mc.jar:]

          at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [jboss-mc.jar:]

          at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [jboss-mc.jar:]

          at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [jboss-mc.jar:]

          at org.jboss.netty.channel.socket.oio.OioWorker.process(OioWorker.java:72) [jboss-mc.jar:]

          at org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73) [jboss-mc.jar:]

          at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:52) [jboss-mc.jar:]

          at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102) [jboss-mc.jar:]

          at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [jboss-mc.jar:]

          at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:176) [jboss-mc.jar:]

          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_26]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_26]

          at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_26]