3 Replies Latest reply on Sep 14, 2012 11:47 AM by jbertram

JMS Messages are sent successfully and disappear

dc-williams Sep 14, 2012 10:34 AM

Hi,

Summary

We are running JBOSS AS v5.1.0.GA-v2.12-1 and a separate instance of stand-alone, non-clustered hornetq 2.2.14-FINAL on the same machine. An application running in the JBOSS 5 AS sends messages via JMS to a queue in hornetq. These are consumed by a stand-alone application (also JAVA and also on the same machine but in a third JVM).

All messages are sent without throwing exceptions however some never show up on the queue (messageCount does not increment for these messages at any point). About 1 in every 10 messages disappear this way. It only happens when there are multiple producers sending to the queue. If there is only one producer sending, it does not happen. If we send, for example, 10x2 message (20 in total but 10 from each producer), the problem is unlikely to happen. Same for 40. Anything over 50 and it still may not happen but the chances get higher and for 100 messages, we are guaranteed to lose some.

Whether the consumer is running at the time of sending or not makes no difference, the messages will still be lost.

The configurations are all below but if, for example, producer A sends 50 messages to QUEUE_A and producer B simultaneously sends 50 messages to QUEUE_A, ultimately, they seem to lose an even number (about 4 messages) each. If we send an uneven number then one will lose an even number and the other an odd number). When refreshing the MBean view in jConsole in the case of sending 2x50 (100), we only see 92 in the messageCount. Subsequently, the sole consumer on QUEUE_A will not see any of the lost messages. They seem to get lost in the sending.

We are using JDK 1.6.0 for all three JVMs and connecting via TCP even though the JVMs are on the same host (no other way to do it anyway I guess).

The configuration for the hornetq connectivity FROM JBOSS is:

hornetq-ds.xml

<?xml version="1.0" encoding="UTF-8"?>

<connection-factories>

<!--

JMS XA Resource adapter, use this for outbound JMS connections.

Inbound connections are defined at the @MDB activation or at the resource-adapter properties.

-->

<tx-connection-factory>

<jndi-name>RemoteJmsXA</jndi-name>

<xa-transaction/>

<rar-name>hornetq-ra.rar</rar-name>

<connection-definition>org.hornetq.ra.HornetQRAConnectionFactory</connection-definition>

<config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>

<config-property name="ConnectorClassName" type="java.lang.String">org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</config-property>

<config-property name="ConnectionParameters" type="java.lang.String">host=ourhost.ourdomain.com;port=5445</config-property>

<max-pool-size>20</max-pool-size>

</tx-connection-factory>

</connection-factories>

jms-ds.xml

<?xml version="1.0" encoding="UTF-8"?>

<connection-factories>

<!--

The JMS provider loader. Currently pointing to a non-clustered ConnectionFactory. Need to

be replaced with a clustered non-load-balanced ConnectionFactory when it becomes available.

See http://jira.jboss.org/jira/browse/JBMESSAGING-843.

-->

<mbean code="org.jboss.jms.jndi.JMSProviderLoader"

name="jboss.messaging:service=JMSProviderLoader,name=JMSProvider">

<attribute name="ProviderName">DefaultJMSProvider</attribute>

<attribute name="ProviderAdapterClass">org.jboss.jms.jndi.JNDIProviderAdapter</attribute>

<attribute name="FactoryRef">java:/XAConnectionFactory</attribute>

<attribute name="QueueFactoryRef">java:/XAConnectionFactory</attribute>

<attribute name="TopicFactoryRef">java:/XAConnectionFactory</attribute>

</mbean>

<attribute name="ProviderName">HornetQJMSProvider</attribute>

<attribute name="ProviderAdapterClass">org.jboss.jms.jndi.JNDIProviderAdapter

</attribute>

<attribute name="FactoryRef">XAConnectionFactory</attribute>

<attribute name="QueueFactoryRef">XAConnectionFactory</attribute>

<attribute name="TopicFactoryRef">XAConnectionFactory</attribute>

java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory

java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces

java.naming.provider.url=ourhost.ourdomain.com:7099

</attribute>

</mbean>

<tx-connection-factory>

<jndi-name>JmsXA</jndi-name>

<xa-transaction/>

<rar-name>jms-ra.rar</rar-name>

	<connection-definition>org.jboss.resource.adapter.jms.JmsConnectionFactory</connection-definition>
	<config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>
	<config-property name="JmsProviderAdapterJNDI" type="java.lang.String">java:/DefaultJMSProvider</config-property>
	<max-pool-size>20</max-pool-size>
	<security-domain-and-application>JmsXARealm</security-domain-and-application>
	<depends>jboss.messaging:service=ServerPeer</depends>

	<!-- turn off txn interleaving -->
	<track-connection-by-tx/>

</tx-connection-factory>

</connection-factories>

The actual relevant hornetq configuration is (queue / topic names simplified):

hornetq-jms.xml

<configuration xmlns="urn:hornetq"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">

<connection-factory name="NettyConnectionFactory">

<connector-ref connector-name="netty"/>

</connectors>

</entries>

</connection-factory>

<connection-factory name="NettyConnectionFactory">

<xa>false</xa>

<connector-ref connector-name="netty"/>

</connectors>

<block-on-durable-send>true</block-on-durable-send>

<block-on-non-durable-send>true</block-on-non-durable-send>

</entries>

</connection-factory>

<connection-factory name="NettyThroughputConnectionFactory">

<connector-ref connector-name="netty-throughput"/>

</connectors>

</entries>

</connection-factory>

<connection-factory name="NettyThroughputConnectionFactory">

<xa>false</xa>

<connector-ref connector-name="netty-throughput"/>

</connectors>

</entries>

</connection-factory>

</queue>

</queue>

</queue>

</topic>

</topic>

</configuration>

hornetq-configuration.xml

<configuration xmlns="urn:hornetq"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">

<paging-directory>${data.dir:../data}/paging</paging-directory>

<bindings-directory>${data.dir:../data}/bindings</bindings-directory>

<journal-directory>${data.dir:../data}/journal</journal-directory>

<journal-min-files>10</journal-min-files>

<large-messages-directory>${data.dir:../data}/large-messages</large-messages-directory>

<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>

</connector>

<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>

</connector>

</connectors>

<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>

</acceptor>

<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>

</acceptor>

</acceptors>

<security-settings>

<security-setting match="#">

</security-setting>

</security-settings>

<address-settings>

<address-setting match="#">

<dead-letter-address>jms.queue.DLQ</dead-letter-address>

<expiry-address>jms.queue.ExpiryQueue</expiry-address>

<redelivery-delay>0</redelivery-delay>

<max-size-bytes>40485760</max-size-bytes>

<page-size-bytes>20485760</page-size-bytes>

<message-counter-history-day-limit>10</message-counter-history-day-limit>

<address-full-policy>PAGE</address-full-policy>

</address-setting>

</address-settings>

</configuration>

Any ideas appreciated!

Dan

1. Re: JMS Messages are sent successfully and disappear

jbertram Sep 14, 2012 10:43 AM (in response to dc-williams)

Can you paste the code sending the messages?
Actions
2. Re: JMS Messages are sent successfully and disappear

dc-williams Sep 14, 2012 11:37 AM (in response to jbertram)

Hi Justin,
here you go (this is not compling code but it gives you enough to see what classes are being used and modes being set setc):

ConnectionFactory connectionFactory = null;
try {

    Properties props = new Properties();

    props.setProperty(Context.INITIAL_CONTEXT_FACTORY, CONTEXT_FACTORY);
    props.setProperty(Context.URL_PKG_PREFIXES, URL_PKG_PREFIXES);
    props.setProperty(Context.PROVIDER_URL, "jbas5:1099");

    Context context = new InitialContext(props);

    connectionFactory = (ConnectionFactory) context.lookup("java:/RemoteJmsXA");
} catch (NamingException e) {
    throw e;
}

Destination destination = null;
try {
    Properties props = new Properties();

    props.setProperty(Context.INITIAL_CONTEXT_FACTORY, CONTEXT_FACTORY);
    props.setProperty(Context.URL_PKG_PREFIXES, URL_PKG_PREFIXES);
    props.setProperty(Context.PROVIDER_URL, "hornetq:7099");

    Context context = new InitialContext(props);

    destination = (Destination) context.lookup("QUEUE_A");
} catch (NamingException e) {
    throw e;
}

Connection conn = null;
Session session = null;
try {

    conn = connectionFactory.createConnection("user", "password"));
    session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
    conn.start();

    MessageProducer producer = session.createProducer(destination);
    Message message = createMessage(session, object);

    try {
        producer.send(message);
        jmsCorrelationId = message.getJMSMessageID();
    } finally {
        try {
            if (producer != null)
                producer.close();
        } catch (JMSException e) {
        }
    }

} catch (JMSException e) {
    throw e;
} finally {
    try {
        if (session != null)
            session.close();
    } catch (JMSException e) {
    }

    try {
        if (conn != null)
            conn.close();
    } catch (JMSException e) {
    }
}
Actions
3. Re: JMS Messages are sent successfully and disappear

jbertram Sep 14, 2012 11:47 AM (in response to dc-williams)

Quick question...

When you lookup "java:/RemoteJmsXA" you are setting a Context.PROVIDER_URL of "jbas5:1099" in the InitialContext. Isn't this look-up local (i.e. in the same VM as the application client)? If so, why are you not using an empty InitialContext?
Actions

Go to original post