2 Replies Latest reply on Jan 29, 2006 10:46 AM by amitka

    Complex JMS queue problem

      Hello,

      I am using jboss-3.2.3, which is runing on Solaris 8 and with Sun JVM 1.4.1, and PostgreSQL.
      And the hsqldb-jdbc2-service.xml is used. Each jms text message size is about 500bytes .
      And all testing components are running in the same machine.
      The following problem happened for a few times and is repeatable.

      In the following, i list all information which i think is important.

      (1)RMI clients call a remote method(sendXmlStringIntoQueue(...)) of a stateless session bean. And this
      stateless session bean itself sends messages passed from the RMI clients to a message queue.
      The corresponding configuration of the jms-ds.xml is:
      < mbean code="org.jboss.jms.jndi.JMSProviderLoader" name="jboss.mq:service=JMSProviderLoader,name=JBossMQProvider" >
      < attribute name="ProviderName" >DefaultJMSProvider< /attribute >
      < attribute name="ProviderAdapterClass" >org.jboss.jms.jndi.JBossMQProvider< /attribute >
      < attribute name="QueueFactoryRef" >java:/ConnectionFactory< /attribute >
      < attribute name="TopicFactoryRef" >java:/ConnectionFactory< /attribute >
      < /mbean >

      < no-tx-connection-factory >
      < jndi-name >Jms< /jndi-name >
      < adapter-display-name >JMS Adapter< /adapter-display-name >
      < config-property name="SessionDefaultType" type="java.lang.String" >javax.jms.Topic< /config-property >
      < security-domain-and-application >JmsRealm</security-domain-and-application>
      < min-pool-size >50< /min-pool-size >
      < max-pool-size >50< /max-pool-size >
      < /no-tx-connection-factory >

      (2)in this stateless session bean, i use following method to send message to the queue:
      (Note: I used non-persistent message.see: jmsMessage.setJMSDeliveryMode(DeliveryMode.NON_PERSISTENT);)

      public void sendXmlStringIntoQueue(
      String qcfjndi,
      String qjndi,
      String xmlString,
      String type,
      String messageselector)
      throws EJBException {
      log.debug("sendXmlStringIntoQueue() is called");
      log.debug("qcfjndi = " + qcfjndi);
      log.debug("qjndi = " + qjndi);
      log.debug("messageselector = " + messageselector);
      QueueConnection qc = null;
      QueueSession queueSession = null;
      try {
      Context enc = (Context) iniCtx.lookup("java:comp/env");
      QueueConnectionFactory qcf =
      (QueueConnectionFactory) enc.lookup(qcfjndi);
      //like "jms/QueueConnectionFactory"
      qc = qcf.createQueueConnection();
      queueSession =
      qc.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
      Queue queue = (Queue) enc.lookup(qjndi); //like "jms/Queue"
      QueueSender queueSender = queueSession.createSender(queue);
      TextMessage jmsMessage = queueSession.createTextMessage(xmlString);
      jmsMessage.setStringProperty("type", type == null ? "" : type);
      if (messageselector != null) {
      jmsMessage.setStringProperty(
      "messageselector",
      messageselector);
      }
      jmsMessage.setJMSDeliveryMode(DeliveryMode.NON_PERSISTENT);
      queueSender.send(jmsMessage);
      log.debug("msg = " + xmlString);
      log.debug("message is sent to queue " + qcfjndi);
      } catch (Exception e) {
      log.error("sendXmlStringIntoQueue() throws exception", e);
      } finally {
      try {
      if (queueSession != null)
      queueSession.close();
      if (qc != null)
      qc.close();
      } catch (Exception ex) {
      log.error("Failed during when closing session and qc", ex);
      throw new EJBException();
      }
      }
      }
      (3)Only one message listener is running in local machine but in different process from the jboss server.
      The initialization code for this listener is :
      private void init() throws Exception {
      Hashtable t = new Hashtable();
      t.put(
      Context.INITIAL_CONTEXT_FACTORY,
      "org.jnp.interfaces.NamingContextFactory");
      t.put(Context.PROVIDER_URL, url);
      t.put(Context.URL_PKG_PREFIXES, "org.jboss.naming:org.jnp.interfaces");
      Context ctx = new InitialContext(t);
      queueConnectionFactory =
      (QueueConnectionFactory) ctx.lookup("UIL2ConnectionFactory");
      queue = (Queue) ctx.lookup(queueName);
      queueConnection = queueConnectionFactory.createQueueConnection();
      queueConnection.setExceptionListener(this);
      queueSession =
      queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
      queueSession.createReceiver(queue).setMessageListener(this);
      queueConnection.start();
      }
      and the onMessage(...) did very few things because i try to make sure the thread will return
      as soon as possible.

      (4)I did the stress test by sending 80 messages/second from the RMI clients(in other word, the
      jboss server received 80 messages/second) . After one day runing,
      the following error is logged:

      RROR [org.jboss.resource.adapter.jms.JmsSessionFactoryImpl] (RMI TCP Connection(297143)-172.16.103.238:) could not create s
      ession
      javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 5000 [ms] )
      at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.ja
      va:250)
      at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool
      .java:496)
      at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:425)
      at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:477)
      at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnect
      ionManager2.java:814)
      at org.jboss.resource.adapter.jms.JmsSessionFactoryImpl.createQueueSession(JmsSessionFactoryImpl.java:119)
      at net.jwcc.jabber.xmppcore.XMLStanzaRouterEJB.sendXmlStringIntoQueue(XMLStanzaRouterEJB.java:367)
      at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683)
      at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:185)
      at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
      at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
      at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:240)
      at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:128)
      at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:118)
      at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
      at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
      at org.jboss.ejb.Container.invoke(Container.java:700)
      at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
      at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
      at org.jboss.invocation.jrmp.server.JRMPInvoker.invoke(JRMPInvoker.java:367)
      at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:261)
      at sun.rmi.transport.Transport$1.run(Transport.java:148)
      at java.security.AccessController.doPrivileged(Native Method)

      (5) Because (4) always happens, I try to switch from the hsqldb-jdbc2-service.xml to postgres-jdbc2-service.xml today.
      It seems that the messages are written into jms_messages table even though i set the deliver mode to be non-persistent (see above).
      This obversation is from both the jmx-console's MessageCache and the jms_messages table of the postgresql.
      The MessageCache has the following configuration (partial):

      < mbean code="org.jboss.mq.server.jmx.DestinationManager" name="jboss.mq:service=DestinationManager" >
      < depends optional-attribute-name="MessageCache">jboss.mq:service=MessageCache< /depends >
      < depends optional-attribute-name="PersistenceManager" >jboss.mq:service=PersistenceManager< /depends >
      < depends optional-attribute-name="StateManager" >jboss.mq:service=StateManager< /depends >
      < /mbean >

      < mbean code="org.jboss.mq.server.MessageCache"
      name="jboss.mq:service=MessageCache" >
      < attribute name="HighMemoryMark" >150< /attribute >
      < attribute name="MaxMemoryMark" >200< /attribute >
      < attribute name="CacheStore" >jboss.mq:service=PersistenceManager< /attribute >
      < /mbean >

      From jmx-console, i found that even the "CurrentMemoryUsage" is less than the "HighMemoryMark",
      the messages are still written into jms_messages table. However, I expect that those messages should be kept
      in memory until CurrentMemoryUsage >= HighMemoryMark because the messages are non-persistent.


      Any one can help? And what is the possible reasons? And where is the bottleneck?
      Is this due to the only one message listener session (I didn't tried yet and just suspected)?

      yang

        • 1. Re: Complex JMS queue problem

          Post the log, you will find information on how to configure it in "Read Me First"

          Do the entries in JMS_MESSAGES have operation "A" or "T"?

          Is 5 seconds long enough to wait in a busy period? Consider whether you are seeing
          long garbage collections, paging or some other process starving JBoss of CPU.

          You might also try JBoss3.2.4RC1. I made a lnumber of improvements
          to the JMS Resource Adapter in that version. If that fixes your problem you should
          be able to copy jms-ra.rar into 3.2.3 (I haven't tried it though)?

          Regards,
          Adrian

          • 2. Re: Complex JMS queue problem

            I have the same problem.
            Which log can help? Is it the jboss.log?
            The entry in JMS_MESSAGES has a TXOP = "A".

            I'm using 3.2.6.

            Thanks.