Possible issues with netty bridge?

[dhawes]

I have noticed a couple of issues that seem to arise when using a netty bridge.

The first is that sometimes messages are never sent across the bridge.  Other messages can still be enqueued and will be sent over the bridge when this happens, but the "stuck" messages just sit there (the jmx-console count never goes to 0).  These messages will remain in this state until the source JBoss is restarted (a restart on the target JBoss does nothing), at which point they will be sent across the bridge.

The second issue that I see is that the source queue sometimes hangs until it is restarted.  When this happens I cannot view anything about the queue through the jmx-console (the webpage simply hangs on load).  Messages can still be enqueued, but they are never sent across the bridge.  A restart of the source JBoss is necessary to resume sending the enqueued messages.

I noticed this issue:

http://community.jboss.org/thread/153084 (JMS Bridge stops retrying even with unlimited retry configured)

and have set both the jnp.timeout and jnp.sotimeout on a whim, but have still noticed this occuring.

I occasionally see the following in the logs, but I am not sure if it is related:

2010-06-18 09:44:13,557 WARN  [org.jboss.resource.connectionmanager.TxConnectionManager] (Thread-49) Connection error occured: org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener@56d94a5f[state=NORMAL mc=org.hornetq.ra.HornetQRAManagedConnection@539c5c48 handles=0 lastUse=1276868340599 permit=false trackByTx=false mcp=org.jboss.resource.connectionmanager.JBossManagedConnectionPool$OnePool@38808450 context=org.jboss.resource.connectionmanager.InternalManagedConnectionPool@7d6ebd2a xaResource=org.hornetq.ra.HornetQRAXAResource@53a1cf84 txSync=null]
javax.jms.JMSException: HornetQException[errorCode=2 message=Channel disconnected]
        at org.hornetq.jms.client.HornetQConnection$JMSFailureListener.connectionFailed(HornetQConnection.java:603)

Has anyone seem similar issues with netty bridges?

[clebert.suconic]

There's an issue with paging when receiving messages over the bridge that I'm aware of.

https://jira.jboss.org/browse/HORNETQ-382

Besides this, I have seen a few issues on the Bridge fixed on past releases. Are you using the latest release? (you didn't state what version you're using now)

[clebert.suconic]

382 is being fixed now. There's a fix committed that jeff will confirm tomorrow if that's fixing the issue or not.

[dhawes]

We are using 2.1.0 Final.

[dhawes]

I have noticed a third issue.

Sometimes messages can be enqueued (MessageCount increases), but they are never sent to the target.  When then happens the ConsumerCount is 0.  A restart of the source causes everything to work properly (though I have not determined if the messages in the queue before the restart ever make it to the target).

I am wondering if these logs are related (hostnames and ips changed):

source 2010-07-07 23:39:10,569 ERROR [org.hornetq.core.client.impl.ClientSessionImpl] (Thread-23 (group:HornetQ-client-global-threads-236948990)) Failed to handle failover
HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 32]
    at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
    at org.hornetq.core.client.impl.ClientSessionImpl.handleFailover(ClientSessionImpl.java:874)
    at org.hornetq.core.client.impl.FailoverManagerImpl.reconnectSessions(FailoverManagerImpl.java:814)
    at org.hornetq.core.client.impl.FailoverManagerImpl.failoverOrReconnect(FailoverManagerImpl.java:715)
    at org.hornetq.core.client.impl.FailoverManagerImpl.handleConnectionFailure(FailoverManagerImpl.java:577)
    at org.hornetq.core.client.impl.FailoverManagerImpl.access$600(FailoverManagerImpl.java:70)
    at org.hornetq.core.client.impl.FailoverManagerImpl$DelegatingFailureListener.connectionFailed(FailoverManagerImpl.java:1140)
    at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:482)
    at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:254)
    at org.hornetq.core.client.impl.FailoverManagerImpl$PingRunnable$1.run(FailoverManagerImpl.java:1198)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
source 2010-07-07 23:39:10,569 WARN  [org.hornetq.core.protocol.core.impl.RemotingConnectionImpl] (Thread-15 (group:HornetQ-client-global-threads-236948990)) Connection failure has been detected: Did not receive data from server for org.hornetq.core.remoting.impl.netty.NettyConnection@25e6ed81[local= /source:60262, remote=/target:5445] [code=3]

This is with HornetQ 2.1.0.  We are upgrading to 2.1.1 in the near future.  Do any of the fixes for 2.1.1 apply here?