3 Replies Latest reply on Apr 16, 2013 11:25 AM by jmesnil

    hundreds of HornetQ threads on EAR multiple redeployments

    karomann

      Hi,

       

      recently I have experiences issued when multiple my EAR file onto JBoss EAP 6 multiple times in a short period of time (trying to set up integration testing with Arquillian) - after 10-20 redeployments I get a few hundreds of HornetQ threads running leading to server hang up. After some time the threads are no longer there (inactivity timeouts?), but automating the integration testing is not possible at all, as usually the threads have already caused OutOfMemoryError.

       

      What can be the root cause of this? Is there any workaround like e.g. setting shorter timeout period for the threads? I have searched my project for any references to javax.jms.* and I have found only one class containing them, I have pasted its code below. It seems to cache some JMS resources, however they should be freed in the @PreDestroy method.

       

       

      @Stateless

      public class JmsResponder {


                @Resource(lookup = "java:/queue/outqueue")

                private Queue outQueue;

       

                private String userName;

                private String password;

                private int jmsPortUsed;

       

                private HashMap<String, JMSParams> connCache;

       

       

                @PostConstruct

                private void init(){

                          userName = "myusername";

                          password = "mypassword";

                          jmsPortUsed = 1234;

                          connCache = new HashMap<String, JMSParams>();

                }

       

       

                @TransactionAttribute(TransactionAttributeType.NEVER)

                public void reply(String messageText, String address) {

           

                          QueueSender sender = null;

       

       

                          try {

                                    QueueSession sess = getSession(address);

                                    Message msg = sess.createMessage();

       

       

                                    msg.setStringProperty("messageText", messageText);

                                    sender = sess.createSender(outQueue);

                                    sender.send(msg);

                          } catch (JMSException e) {

                                         e.printStackTrace();

                          } finally {

                                    try {

                                              sender.close();

                                    } catch (JMSException e) {

                                              e.printStackTrace();

                                    }

                          }

                }

       

                private QueueSession getSession(String address) {

                          if (!connCache.containsKey(address)){

                                    JMSParams params = new JMSParams();

                                    try {

                                              Map<String, Object> connectionParams = new HashMap<String, Object>();

                                              connectionParams.put(TransportConstants.PORT_PROP_NAME, jmsPortUsed);

                                              connectionParams.put(TransportConstants.HOST_PROP_NAME, address);

                                              TransportConfiguration transportConfiguration = new TransportConfiguration("org.hornetq.core.remoting.impl.netty.NettyConnectorFactory", connectionParams);

                                              params.connectionFactory = HornetQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.CF, transportConfiguration);

                                              params.conn = params.connectionFactory.createQueueConnection(userName, password);

                                              params.sess = params.conn.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);

                                              connCache.put(address, params);

                                    } catch (Exception e){

                                              LOG.error(e, e);

                                              if (params.sess != null){

                                                        try {

                                                                  params.sess.close();

                                                        } catch (JMSException e1) {

                                                                  LOG.error(e1, e1);

                                                        }

                                              }

                                              if (params.conn != null){

                                                        try {

                                                                  params.conn.close();

                                                        } catch (JMSException e1) {

                                                                  LOG.error(e1, e1);

                                                        }

                                              }

                                              if (params.connectionFactory != null){

                                                        params.connectionFactory.close();

                                              }

                                    }

                          }

                          return connCache.get(address).sess;

                }

       

                private static class JMSParams {

                          QueueConnection conn = null;

                          QueueSession sess = null;

                          HornetQConnectionFactory connectionFactory = null;

                }

       

       

                @PreDestroy

                private void destruct(){

                          for (JMSParams param : connCache.values()){

                                    if (param.sess != null){

                                              try {

                                                        param.sess.close();

                                              } catch (JMSException e1) {

                                                        LOG.error(e1, e1);

                                              }

                                    }

                                    if (param.conn != null){

                                              try {

                                                        param.conn.close();

                                              } catch (JMSException e1) {

                                                        LOG.error(e1, e1);

                                              }

                                    }

                                    if (param.connectionFactory != null){

                                              param.connectionFactory.close();

                                    }

                          }

                }

      }

       

      I am also attaching messaging subsystem configuration from my standalone.xml (messaging.txt) as well as thread dump from a sample reproduction of the issue (stack.txt). The sample thread dump contains only 45 HornetQ threads but on the machine I run it it was enough to hand the JBoss up. On stronger machine this number can reach a few hundrends. The tests that are run with Arquillin in the EAR does not use the JMS API at all.