4 Replies Latest reply on Nov 4, 2008 9:54 PM by jerry.zhao

    Messaging 1.3GA lock ups

    colomb

      Apologies for the long stack trace ahead of time. We are evaluating jboss 1.3.0GA w/JBoss AS 4.2.0 and are seeing the messaging server completely lock up. New connections just hang while connecting and current connections hand while sending/recieving messages. We are only seeing this happen when stepping through code on the client side. I could see that pausing a client side app would make it lose connectivity with the server (missing pings, etc), but why is the server locking up? If anyone could provide some insight, we would greatly appreciate it. Thank you.

      4:07:01,037 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2w-151xj5-f3608vvm-1-f366eupl-u
      14:07:01,037 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f366eupl-u. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,037 WARN [Dispatcher] Cannot find object with id 50 to register
      14:07:01,038 ERROR [ExceptionUtil] ConnectionEndpoint[48] close [a2s31a-nkbv1l-f3666m5m-1-f366ftbi-1n]
      javax.jms.IllegalStateException: Cannot find binding for topic sub with queue name: a2s31a-nkbv1l-f3666m5m-1-f366erdf-1b
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:490)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
       at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
       at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
       at org.jboss.remoting.Lease.access$300(Lease.java:39)
       at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:07:01,038 ERROR [SimpleConnectionManager] Failed to close connection
      javax.jms.IllegalStateException: Cannot find binding for topic sub with queue name: a2s31a-nkbv1l-f3666m5m-1-f366erdf-1b
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:490)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
       at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
       at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
       at org.jboss.remoting.Lease.access$300(Lease.java:39)
       at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:07:01,038 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608ynh-l. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,039 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608yco-h. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,039 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608ya3-b. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:12:16,874 WARN [BisocketClientInvoker] Unable to send ping: shutting down PingTimerTask
      java.net.SocketException: Broken pipe
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker$PingTimerTask.run(BisocketClientInvoker.java:497)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:12:26,579 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2j-dflkpp-f366gqob-1-f366gsia-b
      14:12:26,579 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2j-dflkpp-f366gqob-1-f366gsia-b. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:24:45,557 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2w-wlnmoy-f36701d9-1-f3670vvd-m
      14:24:45,557 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-wlnmoy-f36701d9-1-f3670vvd-m. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:25:56,702 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-wlnmoy-f36701d9-1-f367030j-4. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      
      [cssdev@DL04_46:~]$!vi
      vi MyContentGatewayClient.cpp
      [cssdev@DL04_46:~]$vi /var/log/css/jboss.log
       at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$close_N4742752445160157748.invokeNext(ConnectionAdvised$close_N4742752445160157748.java)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.close(ConnectionAdvised.java)
       at org.jboss.jms.wireformat.CloseRequest.serverInvoke(CloseRequest.java:66)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:141)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:383)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
      14:06:36,880 WARN [BisocketClientInvoker] Unable to send ping: shutting down PingTimerTask
      java.net.SocketException: Broken pipe
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker$PingTimerTask.run(BisocketClientInvoker.java:497)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:07:01,037 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2w-151xj5-f3608vvm-1-f366eupl-u
      14:07:01,037 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f366eupl-u. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,037 WARN [Dispatcher] Cannot find object with id 50 to register
      14:07:01,038 ERROR [ExceptionUtil] ConnectionEndpoint[48] close [a2s31a-nkbv1l-f3666m5m-1-f366ftbi-1n]
      javax.jms.IllegalStateException: Cannot find binding for topic sub with queue name: a2s31a-nkbv1l-f3666m5m-1-f366erdf-1b
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:490)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
       at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
       at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
       at org.jboss.remoting.Lease.access$300(Lease.java:39)
       at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:07:01,038 ERROR [SimpleConnectionManager] Failed to close connection
      javax.jms.IllegalStateException: Cannot find binding for topic sub with queue name: a2s31a-nkbv1l-f3666m5m-1-f366erdf-1b
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:490)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
       at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
       at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
       at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
       at org.jboss.remoting.Lease.access$300(Lease.java:39)
       at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:07:01,038 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608ynh-l. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,039 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608yco-h. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:07:01,039 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-151xj5-f3608vvm-1-f3608ya3-b. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:12:16,874 WARN [BisocketClientInvoker] Unable to send ping: shutting down PingTimerTask
      java.net.SocketException: Broken pipe
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker$PingTimerTask.run(BisocketClientInvoker.java:497)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      14:12:26,579 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2j-dflkpp-f366gqob-1-f366gsia-b
      14:12:26,579 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2j-dflkpp-f366gqob-1-f366gsia-b. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:24:45,557 WARN [SimpleConnectionManager] ConnectionManager[9fa2fb] cannot look up remoting session ID a2s2w-wlnmoy-f36701d9-1-f3670vvd-m
      14:24:45,557 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-wlnmoy-f36701d9-1-f3670vvd-m. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.
      14:25:56,702 WARN [SimpleConnectionManager] A problem has been detected with the connection to remote client a2s2w-wlnmoy-f36701d9-1-f367030j-4. It is possible the client has exited without closing its connection(s) or there is a network problem. All connection resources corresponding to that client process will now be removed.