-
1. Re: hornetq2.5.5 messages lost after restart
jbertram Sep 24, 2014 4:38 PM (in response to bwitherell)Can you clarify a few things?
- What release are you using? HornetQ 2.5.5 hasn't been released yet. Perhaps you meant 2.2.5?
- Were the messages persistent?
-
2. Re: hornetq2.5.5 messages lost after restart
bwitherell Sep 24, 2014 5:54 PM (in response to jbertram)haha yes sorry about that. its 2.2.5. didnt realize i did that.
yes the messages were persistent.
Thanks for the quick response.
-
3. Re: hornetq2.5.5 messages lost after restart
jbertram Sep 24, 2014 6:01 PM (in response to bwitherell)I'm not familiar with any bugs that match the description of your symptoms. Was there a particular JIRA that was supposed to be fixed in 2.2.5.Final related to this?
Have you considered upgrading to a newer release? HornetQ 2.2.5.Final was released over 3 years ago now.
-
4. Re: hornetq2.5.5 messages lost after restart
bwitherell Sep 24, 2014 7:23 PM (in response to jbertram)it was well known that in earlier releases messeges didnt persist when hornetq was restarted. not sure on any specific jira ticket. but why would 2.2.5 do this? I didint see anything in the logs. the journal dir was huge 20gb though. is it possible it could have OOMed and somhow that caused the persistance issue i saw?
-
5. Re: hornetq2.5.5 messages lost after restart
jbertram Sep 24, 2014 10:36 PM (in response to bwitherell)it was well known that in earlier releases messeges didnt persist when hornetq was restarted. not sure on any specific jira ticket.
I have been either supporting or developing HornetQ for the last few years and I was not aware of this issue. If you could be more specific that would be helpful.
but why would 2.2.5 do this?
I have no idea.
I didint see anything in the logs. the journal dir was huge 20gb though.
Is it still huge? Perhaps you could run the PrintData tool on it and attach the output?
is it possible it could have OOMed and somhow that caused the persistance issue i saw?
In my experience the journal is robust and wouldn't fail in the case of an OOME such that messages wouldn't be read upon a server restart.
-
6. Re: hornetq2.5.5 messages lost after restart
bwitherell Sep 25, 2014 10:40 AM (in response to jbertram)So I ran the print tool as described by the discussion you linked to.
It returned 8,000 or so lines. Unfortunately I am prohibited from sending you the data.
If you could just give me a few helpful hints on how to interpret this and what I should look for I would appreciate it.
Thanks a lot for your continued help on this!
-
7. Re: hornetq2.5.5 messages lost after restart
jbertram Sep 25, 2014 4:03 PM (in response to bwitherell)I didn't write the PrintData tool so I'm not familiar enough with its output to direct you blindly. The main thing I'd recommend is simply looking at the results from the messaging journal, specifically the surviving records. At the end of the day, however, I don't think there's much that can be done since you can't submit the data for analysis and possible correction. Even if you could analyze the output from the tool it's unlikely you'd be able to resolve any issues you found.
Like I said, I'm not familiar with any bugs that would cause this kind of problem. I certainly haven't heard of any enterprise customers running into this, but then again we started enterprise support with a build of 2.2.10.
-
8. Re: hornetq2.5.5 messages lost after restart
bwitherell Sep 25, 2014 4:24 PM (in response to jbertram)Thanks! Is surviving data the stuff that should be put back on the queue after restart?
-
9. Re: hornetq2.5.5 messages lost after restart
jbertram Sep 25, 2014 5:22 PM (in response to bwitherell)I believe so, but again I'm not terribly familiar with the tool.