10 Replies Latest reply on Jun 21, 2011 10:47 PM by saifeldeen

    java.lang.IllegalStateException: Cannot find add info

    jeremystone

      Sometimes we get the following WARNING logged:

       

      java.lang.IllegalStateException: Cannot find add info <some number>

              at org.hornetq.core.journal.impl.JournalImpl.appendUpdateRecord(JournalImpl.java:899)

              at org.hornetq.core.persistence.impl.journal.JournalStorageManager.updateScheduledDeliveryTime(JournalStorageManager.java:527)

              at org.hornetq.core.server.impl.QueueImpl.checkDLQ(QueueImpl.java:982)

              at org.hornetq.core.server.impl.QueueImpl$RefsOperation.afterRollback(QueueImpl.java:1505)

       

      Unfortunately it occurs infrequently and apparently only under moderate load.

       

      This looks similar to http://community.jboss.org/thread/154006 but we are using HornetQ 2.1.2  where  https://issues.jboss.org/browse/HORNETQ-440 is fixed. (Also see it after building the branch_2_1 code). We do not see any of the other stack traces reported at the start of that topic, just the checkDLQ one.

       

      Any ideas?

       

      Regards,

       

      Jeremy Stone

        • 1. java.lang.IllegalStateException: Cannot find add info
          clebert.suconic

          This same question was asked yesterday: http://community.jboss.org/thread/161812?tstart=0

          • 2. java.lang.IllegalStateException: Cannot find add info
            jeremystone

            Thanks for the prompt reply.

             

            This is a different stack trace from that posted in yesterday's question.

             

            Is our case also fixed, then? If so in which release will it appear? 2.2?

            • 3. java.lang.IllegalStateException: Cannot find add info
              clebert.suconic

              It's the same issue.. just through a different record.

              1 of 1 people found this helpful
              • 4. java.lang.IllegalStateException: Cannot find add info
                jeremystone

                That's good to know.

                 

                Do you know what files changed for this (is there a JIRA - not HORNETQ-440, I guess?) so that we can make a build incorporating the fix but one that is compatible with 2.1.2?

                • 5. Re: java.lang.IllegalStateException: Cannot find add info
                  saifeldeen

                  Hi all,

                   

                  We are facing the same issue, but with HornetQ v2.2.2.  Our setup is as follows:

                  Hornetq: v2.2.2-GA

                  JRE: v1.6.17

                  JBoss: v4.2.3 GA

                  OS: Windows Server 2003 R2, 32 bit

                   

                  The same error was noticed in our logs, i.e.

                  2011-05-31 23:23:19.435+1000 WARN  [Old I/O server worker (parentId: 5437246, [id: 0x0052f73e, xxx-app-01.vb.tv/1.0.0.27:5445])] QueueImpl.warn:76 - Error on checkDLQ

                  java.lang.IllegalStateException: Cannot find add info 29877

                            at org.hornetq.core.journal.impl.JournalImpl.appendUpdateRecord(JournalImpl.java:909)

                            at org.hornetq.core.persistence.impl.journal.JournalStorageManager.updateDeliveryCount(JournalStorageManager.java:755)

                            at org.hornetq.core.server.impl.QueueImpl.checkRedelivery(QueueImpl.java:1612)

                            at org.hornetq.core.server.impl.QueueImpl$RefsOperation.afterRollback(QueueImpl.java:2013)

                  ...

                  2011-05-31 23:23:19.435+1000 WARN  [Old I/O server worker (parentId: 5437246, [id: 0x0052f73e, xxx-app-01.vb.tv/1.0.0.27:5445])] QueueImpl.warn:71 - Message has reached maximum delivery attempts, sending it to Dead Letter Address jms.queue.DLQ from jms.queue.applicationQueue

                   

                  From all of the posts relating to this issue, the following trends are:

                  • JBoss 4.2.3 GA
                  • System is being heavily loaded

                   

                  I am unable to reliably reproduce this problem (which is annoying) but I know that it generally happens under significant loads...

                   

                  Any ideas on why this might be happening?  Are there any fixes for this?  As I said earlier, we are using HornetQ v2.2.2...

                  • 6. Re: java.lang.IllegalStateException: Cannot find add info
                    clebert.suconic

                    This is fixed on the next version. But this is an ignorable warn.

                     

                    For some reason the rollback will try to redelivery but the message was already gone by the time it tried to update it.

                    • 7. Re: java.lang.IllegalStateException: Cannot find add info
                      greyfairer2

                      Hi all,

                       

                      I had the same exception, but with a different trace:

                       

                      2011-06-14 13:56:46,308 ERROR [org.hornetq.core.protocol.core.ServerSessionPacketHandler] (ajp-0.0.0.0-8009-9) Caught unexpected exception: java.lang.IllegalStateException: Cannot find add info 770146

                                at org.hornetq.core.journal.impl.JournalImpl.appendUpdateRecord(JournalImpl.java:909) [:]

                                at org.hornetq.core.persistence.impl.journal.JournalStorageManager.storeAcknowledge(JournalStorageManager.java:519) [:]

                                at org.hornetq.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:742) [:]

                                at org.hornetq.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:571) [:]

                                at org.hornetq.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:553) [:]

                                at org.hornetq.core.protocol.core.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:268) [:]

                                at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:474) [:]

                                at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:496) [:]

                      [...]

                                at org.jboss.netty.channel.socket.http.HttpTunnelingServlet.service(HttpTunnelingServlet.java:179) [:]

                       

                      The message concerned was sent by a client who disconnected suddenly, maybe before it received the reply to the send() command.

                       

                      We got this message about 24 times in a row when a different client connected and started a MessageConsumer. The major problem is that the Consumer received the same message also 24 times. The Consumer uses Auto-Acknowledgement, but maybe the Ack failed on the server, and the message got redelivered somehow...

                       

                      Which version was this fixed in? We have HornetQ 2.2.2.Final with Netty 3.2.4.Final now.

                      • 8. Re: java.lang.IllegalStateException: Cannot find add info
                        clebert.suconic

                        I have fixed this exception on Branch_2_2_EAP. It was an ignorable exception when I fixed it.

                         

                         

                        Can you give it a try?

                         

                         

                        We are just fixing a latest JIRA to have another release.

                        • 9. Re: java.lang.IllegalStateException: Cannot find add info
                          greyfairer2

                          Retested with r10789 from Branch_2_2_EAP, but that didn't help.

                          Apparantly it is not related to the previous one, so I created a new JIRA issue with more details: HORNETQ-722

                           

                          Greets, Geert.

                          • 10. Re: java.lang.IllegalStateException: Cannot find add info
                            saifeldeen

                            Hi all,

                             

                            We figured out why we were getting the "java.lang.IllegalStateException: Cannot find add info" errors.  It had to do with long running transactions on the client side.  One important fact I forgot to mention above (which I thought was insignficant at the time) was that we had 2 nodes listening on a single queue.  What was happening was that when the queue was reasonably large, messages were getting buffered on the client side and then getting discarded as the load increased.  The transactions on the client side are long running, hence, it wasn't able to keep up with the demand.  HornetQ by default buffers the messages on the client:

                            By default, the consumer-window-size is set to 1 MiB (1024 * 1024 bytes).

                             

                            See documentation: http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html/flow-control.html

                             

                            Therefore, on the client side, we used a non-buffered connection factory (i.e. setting consumer-window-size = 0 in the connection-factory settings in the hornetq-jms.xml).  The client nodes now appear to be behaving correctly.

                             

                            Hope that helps someone else...  Thanks Clebert for looking into it.