7 Replies Latest reply on Jul 30, 2008 2:00 PM by clebert.suconic

    LinkedList Factor on the Journal...

    clebert.suconic

      On the journal, all the messages from all the destinations are stored on a single journal.

      Suppose this scenario where a customer is using two destinations:

      - FastDestination
      On this destination, a consumer will be aways there to listen to messages

      - LazyDestination
      On this destination, a consumer needs to take the action to connect itself (think about e-mails). So he will take days to connect itself again.

      Adds and deletes are coming for FastDestination but not for LazyDestination.

      For example, you could have something like that on the journal

      File1
      10 Add Records for Lazy
      8000 Add Records for Fast

      File2
      8000 Delete Records for Fast
      100 Add Records for Fast

      File3
      100 Delete Records for Fast
      300 Add Records for Fast

      File4
      599 Add Records for fast
      ...

      On this simple Example... even though all FastRecords from File1, 2 and 3 are gone, the Journal won't be able to reclaim any files until these 10 Lazy records are gone.

      If Lazy takes for instance 3 days to read its destinations, we will need probably 1 thousand journal files on the journal (or even more).

      It will be fairly difficult to eventually reload the journal (It would require a lot of memory).

        • 1. Re: LinkedList Factor on the Journal...
          clebert.suconic

          Few solutions I have thought for this:

          I - Separate CurrentFile by Destination type.
          We would add/delete records per destination.
          We would still be able to commit without a problem

          I have written this code already, and it has one side effect. If the customer has 1000 lazy destinations you will have 1000 opened files, what is not possible on AIO.


          II - Separate CurrentFile by Destination speed.
          We could have a destination configured as Lazy, what would keep records in separate.
          We would still be able to commit without a problem, as any commit record aways carries the filesIds and number of elements used.

          This would use the code I already wrote on (i).

          III - Depending on the number of dependencies found on Reclaiming, we could start a FullReclaiming logic (think of a fullGC) that would move records around, facilitating how the files are reclaiming.
          This has performance implications. It's listed here just as a brain storm.


          I think option II would be a reasonable one.

          • 2. Re: LinkedList Factor on the Journal...
            timfox

            Yes this is a known issue. If you have messages hanging around in a destination for a long time, then sure, they'll stop the file from being reclaimed.

            Although I'm not convinced this is a major problem, more of an edge case.

            In later JBM versions, we should probably remedy this by adding compacting to files to.

            • 3. Re: LinkedList Factor on the Journal...
              gozilla

              Tim,

              Sadly enough, this is not an edge case: when you have a decently sized messaging server, you have easily thousands of destinations used by hundreds of apps.

              We had cases that apps used our MQ server to talk to some batch jobs, which were only triggered once a day.

              Cheers,

              François

              • 4. Re: LinkedList Factor on the Journal...
                timfox

                I don't understand your point, we can support many hundreds and thousands of destinations.

                • 5. Re: LinkedList Factor on the Journal...
                  clebert.suconic

                   

                  "gozilla" wrote:
                  Tim,

                  Sadly enough, this is not an edge case: when you have a decently sized messaging server, you have easily thousands of destinations used by hundreds of apps.

                  We had cases that apps used our MQ server to talk to some batch jobs, which were only triggered once a day.

                  Cheers,

                  François


                  Francois (user gozilla),

                  The case of many files would be a side effect of one of my suggestions to fix the problem, which I used as an argument to suggest II. That's not a real problem.

                  We already support many destinations without any problem.

                  • 6. Re: LinkedList Factor on the Journal...
                    gozilla

                     

                    "timfox" wrote:
                    Yes this is a known issue. If you have messages hanging around in a destination for a long time, then sure, they'll stop the file from being reclaimed.

                    Although I'm not convinced this is a major problem, more of an edge case.


                    My point was that having message hanging around in destinations for "long" time is happening frequently with a sized installation.

                    Francois

                    • 7. Re: LinkedList Factor on the Journal...
                      clebert.suconic

                       

                      "gozilla" wrote:
                      "timfox" wrote:
                      Yes this is a known issue. If you have messages hanging around in a destination for a long time, then sure, they'll stop the file from being reclaimed.

                      Although I'm not convinced this is a major problem, more of an edge case.


                      My point was that having message hanging around in destinations for "long" time is happening frequently with a sized installation.

                      Francois


                      I added a patch with a proposed fix on

                      https://jira.jboss.org/jira/browse/JBMESSAGING-1342

                      We should do it before Beta.

                      Just the fact of having Messages on DLQ for instance would break reclaiming at the moment. So I think it is a major issue.