losing messages publishing to topics

[cballew]

We were doing some stress testing on some code that we're getting ready to deploy to production soon. Basically for brevity I've done my code home work and searches and read the wiki's, done the googles, etc. If I missed, my apologies because I tried my best and yes I looked to the jboss source to try to discover the issue. I am using the released 4.0.0 build. default configuration with the only variance being a switch using the doc/examples sql server stuff for hooking up to sql server and no change trying against hsql.

I have narrowed the issue to the client/publish part of the code. I have established that everything that makes it into jms_messages gets delivered to the onMessage on the other side and we've tested against both sql server and hsql and have used the jtds driver and also tried the ms sql jdbc driver. we have validated that that I get to the code after the actual publish without exception for each message slated to be delivered. The test senario had us trying 5k messages that compress before sending and sent as bytemessage for each instance of our publisher. I have 40 publishers sending to 40 topic with 40 receivers. the counts have been in blocks 20K, 10K, 5K and a few runs of just 10 messages. I spread the spread of messages over the 40 at 5 per second randomly, but evenly spread so everyone of them gets hit ( of course not the case with the 10 :-) No matter what we do I either lose between 8 - 14 or go over by 2 to 4 messages. this I verfied by turning off the durable subscribers on several of the runs and without exception, we get everything that makes it to the jboss.mq store, I always see the shortages or overages in the store itself. I will post the code for the send on request and we definitely get to the end of the send code without an exception. we are auto ack on the publish. Only 1 durable subscriber to a topic. BTW, JBoss rocks and this is the only real issue I've had, but it's a biggie. I am persistent on the send and I have tried setting the delivery mode directly in the publish method and using the setDeliveryMode on the TopicPublisher. The call on the publish looks like the following:

tPub.publish(t,msg,DeliveryMode.PERSISTENT,4,0);

No exception appears in the logs, everything looks normal, but for some reason a few of the messages never make it or make it more than once. the number of times I get to the end of my publish code exactly matches the number sent, never more or less. All the code lives as jboss servicembeans and is hosted on the same instance of jboss.

thanks in advance for any aide.

[cballew]

Now I have tried the Sun JVM on this issue with the same results. Different machines too. I can't escape the conclusion any more that for some reason even though I have called publish n number of times I never get a perfect n in the jms_messages store table, I'm either over or under every time. I used 1.4.2_06 from Sun and the jrockit version I had been using was 1.4.2_04.

Here's my publish code and I create the topicsession and topic connection in the wrapper and have a finally for them similar to the one for the topicpublisher in this one.

private boolean Send(String status, String topicName, String MessageID,
 String Alias, String Message, TopicConnection conn,
 TopicSession ts) throws Exception
 {
 TopicPublisher tPub=null;
 try
 {
 Context ctx = new InitialContext();
 // for saftey...
 Topic t = (Topic)ctx.lookup("topic/" + topicName);
 if(t==null)
 {
 t = ts.createTopic(topicName);
 }

 tPub = ts.createPublisher(t);
 BytesMessage msg = (BytesMessage)ts.createBytesMessage();
 // compress for sending
 CompressedMessage cmsg =new CompressedMessage(Message);
 msg.writeBytes(cmsg.getCompressedBytes());

 msg.setStringProperty("alias",Alias);
 msg.setStringProperty("status", status);
 msg.setStringProperty("messageid", MessageID);
 this.lastProcessedMessageID = MessageID;
 msg.setStringProperty("senttime",getTimeString());
 this.lastMessageProcessedAt = getTimeString();
 msg.setStringProperty("topicname",topicName);
 msg.setStringProperty("compressiontype","gzip/text");
 msg.setIntProperty("compressedsize",cmsg.length());

 tPub.publish(t,msg,DeliveryMode.PERSISTENT,4,0);
 messageCount++;

 return true;
 }
 catch(Exception e)
 {
 logger.error(e);
 throw e;
 }
 finally
 {
 try
 {
 tPub.close();
 }
 catch(Throwable doNothing)
 {
 }
 }
 }

Comments and logging removed for brevity.

[cballew]

OK, I guess I'll continue my monologue to chronical my experience. I switched today to using localhost UIL2ConnectionFactory instead of the internal connection factory, I get exactly the same behavior by dropping messages and/or too many messages. I cannot think of a reason that I should ever get more or less messages than I sent. This is just wrong. I also move my memory on the JVM from max 512 MB to 1024 MB with no effect. Exactly the same results. Everything else seems to be functioning normally. I have no choice, but to conclude jboss.mq is just broke on the publish side and not reliable and thus cannot be used for enterprise level messaging. I don't really believe this, but I can't establish any other possibility. I am more than willing to accept it could be something I'm doing, but that does not appear to be the case. Similar scenarios against SonicMQ and WebLogic do not yield the same results. My preference because of the nice JSR 77 implementation by JBoss is JBoss, but how can I use it if I cannot rely on the messaging. Sorry, I don't mean to be frustrated, but we're a big WebLogic Shop and risked a lot to try to get our management to let our team give JBoss a shot. Now all I'm wearing is egg.

[starksm64]

Create a bug report on sourceforge with the test program and configuration details so someone can look into the details.

http://sourceforge.net/tracker/?group_id=22866&atid=376685

[adrian.brock]

More importantly, use "READ THIS FIRST" to produce TRACE logging
so we can see what is really happening rather relying on your assertions
or trying to guess what the problem is.

As it stands, this problem is unanswerable because it contains no relevent
information. Hence, you were ignored.