Occasional NPE on ack to XA producer

[tory_mckeag]

I'm seeing intermittent null pointer exceptions in HornetQ 2.0.0.GA as messages are produced, with the exceptions occurring under the following conditions:

-HornetQ is running in standalone mode.

-The producer is a web app that is producing messages using Apache CXF, using the XA connection provider configured through the HornetQ RA packaging in JBoss AS 5.

-The producer is producing a relatively low volume of messages (~2-3 a second), which are all pretty small (5-10K).

This only seems to happen occasionally, and even then it seems to take awhile before I start seeing the errors (I left it running overnight).

The error stack I get is

[New I/O server worker #1-1] 15:34:57,065 SEVERE [org.hornetq.core.server.impl.ServerSessionImpl]  Failed to acknowledge
java.lang.NullPointerException
at org.hornetq.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:544)
at org.hornetq.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:527)
at org.hornetq.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:540)
at org.hornetq.core.server.impl.ServerSessionImpl.handleAcknowledge(ServerSessionImpl.java:648)
at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:157)
at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:466)
at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:423)
at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:389)
at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:432)
at org.hornetq.integration.transports.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:67)
at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:98)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:358)
at org.hornetq.integration.transports.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:171)
at org.hornetq.integration.transports.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:136)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323)
at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
 

After having a brief look at the code, I would speculate that the root of the problem might be occurring in org.hornetq.core.server.impl.ServerSessionImpl, where it appears that if handleXAEnd is called before handleAcknowledge, in which case the transaction would be null when HornetQ is trying to acknowledge it.  Maybe there's a race condition?

Any help / advice would be greatly appreciated.  It appears that when these exceptions occur, I think they may also leave some resources open, as the memory seems to balloon over time (although for all I know that is unrelated stupidity on my part)

Meanwhile, I will try to get some time to create a simple standalone app that can reproduce the error.

[timfox]

Does this happen with TRUNK?

This looks somewhat familiar, perhaps it's already fixed. Did you check in JIRA?

[tory_mckeag]

I did a dig through JIRA and didn't find anything that looks like the same thing.  I'll try building trunk and see if the problem goes away though...