1 2 Previous Next 16 Replies Latest reply on Nov 7, 2012 8:29 AM by clebert.suconic

    Upgrading HornetQ version in JBoss7

    qtm

      Hi,

       

      I want to have the fix for the problem I've described in https://community.jboss.org/thread/206484?tstart=0 , so I've taken 2.2.21 from git and built it. If I test it in a HornetQ standalone mode I can see the bug has been fixed, but if I try to integrate it in JBoss7.1 by replacing the jars the issue is still happening. I guess I'm not completing all the necessary steps:

      at the moment I'm just replacing the jars(hornetq-ra.jar, hornetq-core.jar, hornetq-jms.jar) from [jboss_installation]/modules/org/hornetq and updating their modules so they point to the correct jar.

       

      Can someone shine some light on this issue?

       

      Thank you

        • 1. Re: Upgrading HornetQ version in JBoss7
          clebert.suconic

          Why don't you get Branch_2_2_AS7... that issue was fixed on 22.. and there's another issue with duplciate-ids that was just fixed there.

           

           

          git checkout Branch_2_2_AS7

          ./build.sh

           

           

          go to the hornetq module...

          remove the indexes...

          replcace the JARs (edit the XMLs to use the same name you have on the JARs)

           

           

          Do the same on hornetq-ra's module

          • 2. Re: Upgrading HornetQ version in JBoss7
            qtm

            Hi Clebert,

             

            I've tried the new version and the problem is worse. If I run the attached test and immediately after it's finished I restart one of 2 nodes I set up on my localhost I get duplicates again. Instead of 20k messages I end up with 22-23k.

            With 2.21 if I do this test I end up with 20k messages after the first try, but if I run it again and do another restart immediately after Test is done I get duplicates.. instead of 40k I have 42-43k...

            The weird part is that for 2.21 HornetQ standalone this does not happen (or at least I couldn't get such a situation), but when I replace the jars in Jboss7 I face this situation.

             

            For both HornetQ and Jboss7 I use

               <id-cache-size>15000</id-cache-size>

               <persist-id-cache>true</persist-id-cache>

             

            and duplicate detection = true on the cluster connection.

            • 3. Re: Upgrading HornetQ version in JBoss7
              clebert.suconic

              What version did you sue? a checkout from git?

              • 4. Re: Upgrading HornetQ version in JBoss7
                clebert.suconic

                Can you add the configs you used for your standalone on jboss 7?

                • 5. Re: Upgrading HornetQ version in JBoss7
                  qtm

                  Hi Clebert,

                   

                  I've attached the config files.

                   

                  I went to https://github.com/hornetq/hornetq/tree/Branch_2_2_AS7 and pressed the zip button and I got a version build on 31 October. I tried the test with 2.2.21 and 2.2.22 (downloaded them from https://github.com/hornetq/hornetq/tags ).

                   

                  Thanks for your help

                  • 6. Re: Upgrading HornetQ version in JBoss7
                    clebert.suconic

                    I wanted to get to this today, but another issue got most of my time today...

                     

                    I read your producer, and I see that you don't have a consumer...

                     

                     

                    one thing that could affect this is redistribution. Maybe redistribution delay is too small and some setting is making to redistribute like crazy up to the point you run our of IDs?

                     

                     

                    You may try increasing the ID size to like 10K (what I think it's reasonable), and maybe play with redistribution settings.

                     

                     

                    if you find something around that (which I think it has a great possibility).. than that's it.. otherwise I will get into it Monday.. or I may ask one of my collegues to play on this depending on our schedule.

                    • 7. Re: Upgrading HornetQ version in JBoss7
                      clebert.suconic

                      That's one thing that I know it's set differently by default between AS7 and standalone.. redistribution.. so.. that's a high possibility of being your issue.

                       

                       

                      Try playing with these settings I said.

                      • 8. Re: Upgrading HornetQ version in JBoss7
                        qtm

                        The ID size is increased to 15k and redistribution-delay=-1, so there is no redistribution.

                        • 9. Re: Upgrading HornetQ version in JBoss7
                          qtm

                          It's clearly related to paging and it's proportional to <max-size-bytes>. In order to reproduce it for big max-size-bytes I must increase the load, so the problem still remains, even if it shows up less often.

                          • 10. Re: Upgrading HornetQ version in JBoss7
                            qtm

                            Hi Clebert,

                             

                            Could you please try to run the examples? I've tried everything I could and the problem still persists.

                             

                            Thank you

                            • 11. Re: Upgrading HornetQ version in JBoss7
                              clebert.suconic

                              I will be looking on this this afternoon, ok?

                              • 12. Re: Upgrading HornetQ version in JBoss7
                                clebert.suconic

                                There's something weird going on that I'm not sure what's happening yet...

                                 

                                 

                                But there's at least one error on your test:

                                 

                                      TextMessage message = queueSession.createTextMessage();

                                 

                                      for (int i = 0; i < numberOfMessages; i++) {

                                         message.setJMSReplyTo(queue);

                                         message.setLongProperty("prop_long", 451l);

                                 

                                 

                                 

                                You created the message before sending the message... producer.send is an asynchronous operation... so the message may still being send while you are changing the body of the message.

                                 

                                session.commit(); will block you and wait for all the operations to finish.

                                 

                                 

                                I'm looking at some other stuff that seems wrong on hornetq.. some bug that you're hitting on clustering and paging.

                                • 13. Re: Upgrading HornetQ version in JBoss7
                                  clebert.suconic

                                  Nothing wrong...

                                   

                                   

                                  I just spent my day on your mistake

                                   

                                   

                                  If this solves your issue, you will own me a favour, ok?

                                   

                                   

                                  I decided to spent this time as I'm investigating another paging issue.. I thought this could be something related.. but this such a basic usage that would been caught by QA already.

                                   

                                  What confused both of you and me, is that the core-queues are not showed on the JMX Console. and the counters will not get up to date until the bridges have finished transferring data between the nodes. When you use paging over clustering... each message willl have to be committed individually during the transfer and that takes some extra time.

                                   

                                  Everything worked after I fixed the producer... (I actually remember making this kind of mistake years ago, even before I joined Jboss, using a different JMS Server back then).

                                   

                                   

                                  Follows your test with my modifications!

                                  • 14. Re: Upgrading HornetQ version in JBoss7
                                    clebert.suconic

                                    There was a small mistake on the previous one.. re-adding it here

                                    1 2 Previous Next