1 Reply Latest reply on Dec 20, 2011 10:10 PM by gaohoward

    Export and decode HornetQ journal

    jplacebo

      Hi all,

       

      At my company we have been using JBoss 5.1.0.GA and HornetQ 2.0.0.GA for some time and we are thinking about upgrading HornetQ to version 2.1.2. I'm already testing this version on a development machine and noticed several performance upgrades which are very welcome for us.

      One functionality that should be useful to us is the ability to export the HornetQ journal to text file, make changes to it and import it back. This is wanted because sometimes we get what probably is synchronisation errors:

       

      SEVERE [org.hornetq.ra.inflow.HornetQMessageHandler] (Thread-20237 (group:HornetQ-client-global-threads-724300297)) Failed to deliver message

      java.lang.IllegalStateException: This message endpoint + HistoricMessageBean is already in use by another thread Thread[Thread-17128 (group:HornetQ-client-global-threads-724300297),5,]

              at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:117)

              at $Proxy570.beforeDelivery(Unknown Source)

              at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:215)

              at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)

              at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)

              at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)

              at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)

              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

              at java.lang.Thread.run(Thread.java:619)

       

      This happens when I run stress tests with version 2.1.2, and occasionally in our production system with version 2.0.0.GA and here we have little traffic (about 4 requests/second max., only in ocasional peaks).

       

      I still don't know what is causing this, not sure if this is a HornetQ or EJB issue, but it would be great if I could edit the exported file, remove the problematic messages and restart the application without having to clean the entire journal.

      Here's a example of a segment of exported file from journal, where I had only 1 message in the queue:

       

      #File,JournalFileImpl: (hornetq-data-1.hq id = 1, recordID = 1)

      operation@AddRecordTX,txID@419,id@420,userRecordType@31,length@938,isUpdate@false,data@AAADNAAAAx-s7QAFc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAF3BAAAAApzcgApcHQucHRpbm92YWNhby5nd3QuY29yZS51dGlsLkhpc3RvcmljRXZlbnRdMthU0NplBwIABUwAGGV2ZW50Q29uc29saWRhdGlvblN0YXR1c3QAMUxwdC9wdGlub3ZhY2FvL2d3dC9jb3JlL3V0aWwvU3RhdHVzQ29uc29saWRhdGlvbjtMAAlldmVudERhdGV0ABBMamF2YS91dGlsL0RhdGU7TAAHZXZlbnRJZHQAEExqYXZhL2xhbmcvTG9uZztMAAtldmVudFN0YXR1c3QAN0xwdC9wdGlub3ZhY2FvL2d3dC9jb3JlL3V0aWwvSGlzdG9yaWNFdmVudCRFdmVudFN0YXR1cztMAAR0eXBldAA1THB0L3B0aW5vdmFjYW8vZ3d0L2NvcmUvdXRpbC9IaXN0b3JpY0V2ZW50JEV2ZW50VHlwZTt4cH5yAC9wdC5wdGlub3ZhY2FvLmd3dC5jb3JlLnV0aWwuU3RhdHVzQ29uc29saWRhdGlvbgAAAAAAAAAAEgAAeHIADmphdmEubGFuZy5FbnVtAAAAAAAAAAASAAB4cHQACEFXQUlUSU5Hc3IADmphdmEudXRpbC5EYXRlaGqBAUtZdBkDAAB4cHcIAAABNFtQE_d4c3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAACbwwfnIANXB0LnB0aW5vdmFjYW8uZ3d0LmNvcmUudXRpbC5IaXN0b3JpY0V2ZW50JEV2ZW50U3RhdHVzAAAAAAAAAAASAAB4cQB-AAp0AAxDUF9DT01NSVRURUR-cgAzcHQucHRpbm92YWNhby5nd3QuY29yZS51dGlsLkhpc3RvcmljRXZlbnQkRXZlbnRUeXBlAAAAAAAAAAASAAB4cQB-AAp0ABVDUkVBVEVfQ09ORklSTV9DSEFSR0V4AAADtwAAAAAAAAGkAQAAADRqAG0AcwAuAHEAdQBlAHUAZQAuAGcAdwB0AEgAaQBzAHQAbwByAGkAYwBRAHUAZQB1AGUAAAL_AAAAAAAAAAAAAAAAAAAAAAQBAAAAAQAAABJxAHUAZQB1AGUAVAB5AHAAZQAKAAAACkUAVgBFAE4AVAA=

      operation@UpdateTX,txID@419,id@420,userRecordType@32,length@8,isUpdate@true,data@AAAAAAAAAAw=

      operation@AddRecordTX,txID@419,id@421,userRecordType@31,length@227,isUpdate@false,data@AAAAZwAAAFKs7QAFc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAACbwwAAAA8AAAAAAAAAGlAQAAADRqAG0AcwAuAHEAdQBlAHUAZQAuAGcAdwB0AEMAaABhAHIAZwBpAG4AZwBRAHUAZQB1AGUAAAL_AAAAAAAAAAAAAAAAAAAAAAQBAAAAAQAAABJxAHUAZQB1AGUAVAB5AHAAZQAKAAAAEEMASABBAFIARwBJAE4ARwA=

      operation@UpdateTX,txID@419,id@421,userRecordType@32,length@8,isUpdate@true,data@AAAAAAAAAAg=

      operation@Prepare,txID@419,numberOfRecords@4,extraData@AAIAAwAAABhhNzAxYmJhOjk2Y2U6NGVmMDczODA6NDcAAAAaMS1hNzAxYmJhOjk2Y2U6NGVmMDczODA6NDE=

      operation@Commit,txID@419,numberOfRecords@4

      ...

       

      I am hoping you help me understand this. So can I decode the message on line #2 on the text above, after the string "data@"? Is that message convertible on readable text? I tried to decode using Base64 but wasn't lucky. I want to know what messages to delete because I can't relate the internal HornetQ message ID ("txID@...") to our requests transaction ID.

       

      Thank you in advance,

      Jose

        • 1. Re: Export and decode HornetQ journal
          gaohoward

          I think first you need to investigate the issue first, if it's a bug in HonetQ raise a Jira (but better before trying out the latest release).

          Second the journal data format is not supposed to be edited directly. It should be done by a tool. Please raise a request on the Jira.

           

          Howard