12 Replies Latest reply on Dec 4, 2006 8:35 AM by alchemista

    org.jboss.mq.SpyJMSException:  OutOfMemoryError

    regenwetter

      Hi,

      I'm using JBossMQ (PTP) and everything seems to be working fine except the fact that during a load test the following error occured:


      org.jboss.mq.SpyJMSException: Exiting on unexpected error in read task; - nested throwable: (java.lang.reflect.UndeclaredThrowableException)
      at org.jboss.mq.Connection.asynchFailure(Connection.java:436)
      at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:147)
      at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:405)
      at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:347)
      at java.lang.Thread.run(Thread.java:534)
      Caused by: java.lang.reflect.UndeclaredThrowableException
      ... 4 more
      Caused by: java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start(Native Method)
      at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.addThread(PooledExecutor.java:520)
      at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(PooledExecutor.java:880)
      at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:301)
      ... 1 more
      2005-04-11 10:57:48,596 INFO [STDOUT] org.jboss.mq.SpyJMSException: Cannot get a client ID; - nested throwable: (java.lang.reflect.UndeclaredThrowableException)


      Later on I got frequently the following warning message:


      2005-04-12 00:00:15,014 WARN [org.jboss.mq.Connection] Connection failure:
      org.jboss.mq.SpyJMSException: Connection Failed; - nested throwable: (java.io.IOException: ping timeout.)
      at org.jboss.mq.Connection.asynchFailure(Connection.java:436)
      at org.jboss.mq.Connection$PingTask.run(Connection.java:1385)
      at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
      at java.lang.Thread.run(Thread.java:534)
      Caused by: java.io.IOException: ping timeout.
      at org.jboss.mq.Connection$PingTask.run(Connection.java:1377)
      ... 2 more


      Thanks in advance for any help.

        • 1. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
          shiming

          I got the similar error with single JMS client and JMS listener. I just send 1000 messages continuously (sleep 50ms before send a message) . The listener stops receiving message after about 600 messages. The listener doesn't really do anything. It just receives message and print out timestamp. The message size is 1K. Jboss version 4.0.1sp1. Tested with OpenJMS fine with 10,000 messages with 100K message size.

          • 2. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
            regenwetter

            Did you find any solution ?

            • 3. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
              disrael

              Are you using connection pooling? It seems like you are not and are running out of ability to create new tcp sockets. Would be interested if your test still fails if you use only one connection for all messaging.

              • 4. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                regenwetter

                The JMS Client does not use connection pooling. It does something like that:

                public void send(...){
                lookupJMSResourcen();
                connection = cf.createConnection();
                session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
                ...
                }
                
                private static void lookupJMSResourcen() throws Exception{
                 if (ic == null)
                 ic = new InitialContext();
                 if (cf == null)
                 cf = (ConnectionFactory) ic.lookup(CONNECTION_FACTORY);
                 if (destination == null)
                 destination = (Destination) ic.lookup(QUEUE);
                }
                
                


                The "JMS Server" is a message driven bean.

                It is not easy to reproduce the error.

                Any ideas ?

                • 5. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                  disrael

                  Move the connection = cf.createConnection();
                  outside of the send.

                  • 6. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                    regenwetter

                    I have already thought of it. But I am closing the connection in a finally block within the send method. I can't do it any longer if I move the statement

                    connection = cf.createConnection();

                    outside the send method because of IllegalStateException (or something similar like that). But I have to do it. I think I have two possibilities:

                    1.) Open for each request a new connection which I have to close after sending the message (current situation). This should work fine (and it does without heavy load. I am not even able to reproduce the error. Sometimes it happens, sometimes it doesn't happen)

                    2.) Implementing a connection pool.

                    What do you think ?

                    • 7. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                      disrael

                      We were doing 1) but it is very expensive. I am switching over to 2) which is easy until you play around with server disconnect problem.

                      • 8. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                        regenwetter

                        I thought it is expensitive too until I measured the time. After getting the connection the first time it always takes just 1-2 ms (at maximum 15 ms). It seems the ConnectionFactory is caching the connection. In my opinion everything should work fine if they wouldn't be that strange error I reported.

                        I'm not pretty sure what do mean with "play around with server disconnect problem" ? Therefore I would like to ask you what is your advice ?

                        • 9. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                          felderr

                          Did you already solved the problem? We get the same error, but in combination with an OutOfMemoryError:

                          Caused by: org.jboss.mq.SpyJMSException: Cannot get a client ID; - nested throwable: (java.lang.reflect.UndeclaredThrowableException)
                          at org.jboss.mq.Connection.askForAnID(Connection.java:1119)
                          at org.jboss.mq.Connection.checkClientID(Connection.java:1092)
                          at org.jboss.mq.SpyConnection.createQueueSession(SpyConnection.java:175)
                          at at.sozvers.eweko.service.impl.DomeaJMSClientServiceImpl.createQueueSession(DomeaJMSClientServiceImpl.java:69)
                          at at.sozvers.eweko.service.impl.DomeaJMSClientServiceImpl.forwardDoc(DomeaJMSClientServiceImpl.java:113)
                          ... 73 more
                          Caused by: java.lang.reflect.UndeclaredThrowableException
                          at org.jboss.mq.il.uil2.msgs.BaseMsg.setError(BaseMsg.java:301)
                          at org.jboss.mq.il.uil2.SocketManager$ReadTask.replyAll(SocketManager.java:445)
                          at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:399)
                          at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:344)
                          ... 1 more
                          Caused by: java.lang.OutOfMemoryError: unable to create new native thread
                          at java.lang.Thread.start(Native Method)
                          at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.addThread(PooledExecutor.java:520)
                          at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(PooledExecutor.java:880)
                          at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:298)
                          ... 1 more

                          Any Idea?

                          • 11. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                            huonghovan

                            Hi all,

                            I got the same problem with you guys if i just close only the Session. But after i do the close for all include the Sender and Connection it looks OK. I was ran out of Memory after 1000 message post to the queue. Here is what i do with my sendObject:

                            public static void sendObject(Serializable object, String jvmURL, String queueName)
                            throws NamingException, JMSException {
                            PSServiceLocator locator = new PSServiceLocator(jvmURL);
                            //lookup the resources
                            QueueConnectionFactory connectionFactory =
                            (QueueConnectionFactory)locator.lookupObject(CONNECTION_FACTORY);
                            Queue queue = (Queue)locator.lookupObject(queueName);
                            //safe jms resources
                            QueueConnection queueConnection = null;
                            QueueSession queueSession = null;
                            QueueSender queueSender = null;
                            try {
                            //create the session connection
                            queueConnection = connectionFactory.createQueueConnection();
                            //instance the queue session
                            queueSession = queueConnection.createQueueSession(false,
                            Session.AUTO_ACKNOWLEDGE);

                            //create queue and send the message
                            queueSender = queueSession.createSender(queue);
                            Message message = queueSession.createObjectMessage(object);
                            queueSender.send(message);
                            }finally {
                            //free all created objects
                            if(queueSender != null) {
                            queueSender.close();
                            }
                            if(queueSession != null) {
                            queueSession.close();
                            }
                            if(queueConnection != null) {
                            queueConnection.close();
                            }
                            }
                            }


                            Hope it helps you all,

                            • 12. Re: org.jboss.mq.SpyJMSException:  OutOfMemoryError
                              alchemista

                              I use my JMS client as an appender for log4j. In log4j, since append is called very frequently, it's not recommended to close the sender/session/connection each time.

                              Apache's log4j JMS Appender does not close the publisher/session/connection with each log send. The doAppend method is synchronized luckily :)