4 Replies Latest reply on Jul 16, 2009 5:36 PM by ron_sigal

    Problems after Migration from WebSphere to JBoss (Broken JMS

    dahm

      Hello,

      unfortunately we are having severe problems after the migration from WebSphere 5 to JBoss 4.2.3.GA
      (+ upgrade to JBoss messaging 1.4.2.GA-SP1, + Remoting upgrade to 2.5.1).. However, I'm not sure whether those are related to Messaging or rather
      the underlying JBoss Remoting.

      We have the following configuration settings:

      Server:
      - OS: IBM AIX ubitest 3 5 00C0587E4C00
      - JRE:
      java version "1.5.0"
      Java(TM) 2 Runtime Environment, Standard Edition (build pap64dev-20080716b (SR8))
      IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 AIX ppc64-64 j9vmap6423-20080716 (JIT enabled)
      J9VM - 20080625_20583_BHdSMr
      JIT - 20080620_1845_r8
      GC - 200806_19)
      JCL - 20080716a

      Client:
      - OS: Windows XP
      - JRE
      java version "1.6.0_13"
      Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
      Java HotSpot(TM) Client VM (build 11.3-b02, mixed mode, sharing)

      Remarkable settings on the server side:

      in bisocket-remoting.xml: Enabled pings

      <attribute name="pingFrequency" isParam="true">60000</attribute>
      <attribute name="pingWindowFactor" isParam="true">2</attribute>
      


      connection-factories.xml:

      <attribute name="SendAcksAsync">true</attribute>
      <attribute name="DisableRemotingChecks">true</attribute>
      

      jboss-service.xml:

      Since we're running multiple instances, the bindings have been altered to use port 14457 instead of 4457 for remoting.
      I.e. all port numbers in that configuration have been "prefixed" with "1". We copied the above mentioned remoting settings to
      the bindings XML file.

      Now for the application setting:

      We're are running multiple fat clients that communicate with each other via messaging. I.e. one client changes data, calls
      an EJB (2.1) method which will not only store the data but also send a message to the given topic:

      <mbean code="org.jboss.jms.server.destination.TopicService" name="ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_Topic"
       xmbean-dd="xmdesc/Topic-xmbean.xml">
       <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer
       </depends>
       <depends>jboss.messaging:service=PostOffice</depends>
       <attribute name="JNDIName">/JMSTOPIC</attribute>
      
       <attribute name="MaxDeliveryAttempts">10</attribute>
       <attribute name="RedeliveryDelay">200</attribute>
       <attribute name="ExpiryQueue">ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_EXQ
       </attribute>
       <attribute name="DLQ">ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_DLQ
       </attribute>
       </mbean>
      


      Most of our service methods send such messages to notify changes of application data, thus it is vital the messages arrive...

      All clients connect at startup to the above mentioned topic and set a message listener just as described in the JBM examples.
      If it receives a message it will check if it is relevant for him. There are at most 40-50 clients at a time, but that was irrelevant
      for the test.

      Now the problem is that the connection BREAKS at random time:
      The client waits for messages eternally while the server has already shut down the connection. We checked the latter fact via the
      JMX console.
      We cannot reproduce the error (as always only the customer can ;-( ). The only way to surely trigger it, is to pull the network cable.

      We checked that with the firewall and network people and they said that idle connections will be closed after 1 hour.
      We built an keep alive client which sends a message to itself (via the server of course)
      every 3 minutes and waits for it 10 seconds. If it does not receive the message it will display a message to the user
      and restart the connection to the topic.

      Unfortunately this problem occurs to often to be ignored. We tried to enable JBM internal pings (s.a. configuration), but that
      didn't help very much. It seems to me that JBM reestablishes the connection but our keep alive client still does not receive
      his messages and reconnects.

      No the big question is: Do you have any clue or idea???
      Any help would be appreciated!

      Currently, I'm investigationg whether there are any known incompatibilities between SUN and IBM JDK.

      Thanks in advance
      Markus

      P.S. Attached are the log files of the client and server with TRACE enabled.

      Client:

      2009-04-24:
      
      16:41:12,750 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[ca66b
      4, $PING$]
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f]
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:13,125 TRACE [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] sending PING tied to lease
      16:41:13,125 TRACE [ConnectionValidator] pinging, sending InvocationRequest[f23ad3, self, $PING$] over SocketClientInvoker[6b93c5, biso
      cket://ubitest:14457]
      16:41:13,125 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457](1) invoking InvocationRequest[f23ad
      3, self, $PING$]
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] obtained semaphore: 49
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6]
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] reading version from input stream
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457](ClientSocketWrapper[Socket[addr=/19
      2.168.231.17,port=14457,localport=2089].14bc4e6]) got Exception: java.net.SocketException: Connection reset
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] released semaphore: 50
      16:41:13,578 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6] closing socket
      16:41:13,578 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6] closed socket
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] retries exhausted
      16:41:13,578 DEBUG [ConnectionValidator] ConnectionValidator failed to ping via SocketClientInvoker[6b93c5, bisocket://ubitest:14457]
      org.jboss.remoting.InvocationFailureException: Unable to perform invocation; nested exception is:
       java.net.SocketException: Connection reset
       at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:142)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:901)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.ConnectionValidator.doCheckConnectionWithLease(ConnectionValidator.java:650)
       at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:334)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1263)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:838)
       ... 6 more
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] SocketException received. This is likely due to client disconnec
      ting and resetting connection.
      java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:1038)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:673)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      16:41:13,593 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2093].148bd9d] closing s
      ocket
      16:41:13,593 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2093].148bd9d] closed so
      cket
      16:41:13,593 DEBUG [ServerThread] WorkerThread#0[192.168.231.17:13385] closed socketWrapper: ServerSocketWrapper[Socket[addr=ubitest/19
      2.168.231.17,port=13385,localport=2093].148bd9d]
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] removing itself from clientpool
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] returning itself to threadpool
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] begins to wait
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](ClientSocketWrapper[Socket[addr=/19
      2.168.231.17,port=14457,localport=2090].17fc44f]) got Exception: java.net.SocketException: Connection reset
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f] closing socket
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f] closed socket
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] retries exhausted
      16:41:13,593 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] failed to ping to server
      org.jboss.remoting.InvocationFailureException: Unable to perform invocation; nested exception is:
       java.net.SocketException: Connection reset
       at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:142)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:901)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:335)
       at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:20)
       at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:369)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1263)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:838)
       ... 7 more
      16:41:13,593 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms]'s connection is invalid
      16:41:13,593 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clie
      ntLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireforma
      t.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&time
      out=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {validatorPingPe
      riod=10000, socket.check_connection=false, stopLeaseOnFailure=true, marshaller=org.jboss.jms.wireformat.JMSWireFormat, unmarshaller=org
      .jboss.jms.wireformat.JMSWireFormat, clientLeasePeriod=10000, clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper, numb
      erOfCallRetries=1, dataType=jms, pingWindowFactor=2, connection_checker=true, NumberOfCallRetries=1, JBM_clientMaxPoolSize=200, pingFre
      quency=60000, timeout=5000, validatorPingTimeout=5000}
      16:41:13,593 DEBUG [InvokerRegistry] removed SocketClientInvoker[6b93c5, bisocket://ubitest:14457] from registry
      16:41:13,593 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[6b93c5, bisocket://ubitest:14457]
      16:41:13,593 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnecting ...
      16:41:13,593 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnecting ...
      16:41:13,593 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14446, enableTcpNoDelay timeout 0 ms, maxP
      oolSize=50]
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2092].514f7f] closing socket
      16:41:13,609 DEBUG [ConnectionValidator] Thread[Thread-12,5,main] calling ConnectionState[83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5].Consol
      idatedListener.handleConnectionException()
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2092].514f7f] closed socket
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2088].8dcd5d] closing socket
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2088].8dcd5d] closed socket
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=200]
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=1]
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=50]
      16:41:13,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnected
      16:41:13,625 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] stopped, returning true
      16:41:13,625 DEBUG [ConnectionValidator] org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask@1420a71 detected connect
      ion failure: stopping LeasePinger
      16:41:13,625 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] setting disconnect timeout to: -1
      16:41:13,625 TRACE [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] stopping lease timer
      16:41:13,625 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )]: disconnectTimeout: -1
      16:41:13,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[6fa30
      c, $DISCONNECT$]
      16:41:13,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:13,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] creating socket
      16:41:13,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      java.net.NoRouteToHostException: No route to host: connect
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.connect(SocketClientInvoker.java:289)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:206)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:433)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1089)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:762)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:138)
       at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:401)
       at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:797)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      16:41:13,640 DEBUG [SocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457]
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator
      [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSock
      etWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&s
      ocket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=
      10000&validatorPingTimeout=5000]
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:776)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:138)
       at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:401)
       at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:797)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.NoRouteToHostException: No route to host: connect
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.connect(SocketClientInvoker.java:289)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:206)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:433)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1089)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:762)
       ... 7 more
      16:41:13,640 DEBUG [MicroRemoteClientInvoker] error shutting down lease pinger
      16:41:13,640 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] shut down lease pinger
      16:41:42,812 TRACE [BisocketServerInvoker] checking connections
      16:41:42,812 TRACE [BisocketServerInvoker] elapsed: 59828
      16:41:43,625 INFO [JMSKeepAliveClient] Sending PING KeepAliveMessage: f43481c5-7d7d-4633-bf0f-10f347142ff2/Fri Apr 24 16:41:43 CEST 20
      09
      16:41:43,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446](1) invoking InvocationRequest[11c9ba1,
      INVOKER, org.jboss.invocation.Invocation@12d813e]
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] obtained semaphore: 49
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] creating socket
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] created socket: Socket[addr=/192.168.23
      1.17,port=14446,localport=2179]
      16:41:43,625 TRACE [SocketWrapper] constructing org.jboss.remoting.transport.socket.ClientSocketWrapper instance for Socket[addr=/192.1
      68.231.17,port=14446,localport=2179], using timeout 0
      16:41:43,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] setting timeout t
      o 0
      16:41:43,625 TRACE [ClientSocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] getting out
      put stream from Socket[addr=/192.168.231.17,port=14446,localport=2179], org.jboss.invocation.unified.marshall.InvocationMarshaller@1a5a
      f9f
      16:41:43,625 TRACE [JavaSerializationManager] Creating ObjectOutputStream
      16:41:43,625 TRACE [ClientSocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] getting inp
      ut stream from Socket[addr=/192.168.231.17,port=14446,localport=2179], org.jboss.invocation.unified.marshall.InvocationUnMarshaller@ad1
      57f
      16:41:43,625 TRACE [JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      16:41:43,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] setting timeout t
      o 0
      16:41:43,625 TRACE [ClientSocketWrapper] reset timeout: 0
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] writing version 22 on output stream
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] writing invocation to marshaller
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] done writing invocation to marshaller
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] reading version from input stream
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] read version 22 from input stream
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] reading response from unmarshaller
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] returned ClientSocketWrapper[Socket[add
      r=/192.168.231.17,port=14446,localport=2179].801b62] to pool
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] released semaphore: 50
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] received response InvocationResponse[1d
      3f905, org.jboss.invocation.MarshalledValueEX@fb57a7aa]
      16:41:43,640 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] received InvocationResponse so going to
       return response's return value of org.jboss.invocation.MarshalledValueEX@fb57a7aa
      16:41:53,640 ERROR [JMSKeepAliveClient] Keep alive message NOT RECEIVED within 10000 ms
      16:41:53,640 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Zugriff wird beendet.
      16:41:53,640 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[45021
      2, JMS, org.jboss.jms.wireformat.ConnectionStopRequest@8b2139]
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] creating socket
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] created socket: Socket[addr=/192.16
      8.231.17,port=14457,localport=2180]
      16:41:53,640 TRACE [SocketWrapper] constructing org.jboss.jms.client.remoting.ClientSocketWrapper instance for Socket[addr=/192.168.231
      .17,port=14457,localport=2180], using timeout 0
      16:41:53,640 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] setting timeout t
      o 0
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] read version 22 from input stream
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading response from unmarshaller
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket
      [addr=/192.168.231.17,port=14457,localport=2180].322a97] to pool
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received response InvocationRespons
      e[dd750e, java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5]
      16:41:53,656 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received a server-side exception as
       response to the invocation: java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf
      6o4c5
      16:41:53,656 ERROR [JBossMessagingEmpfangsServiceImpl] Fehler beim Anhalten des JMS-Empfangsservice
      org.jboss.jms.exception.MessagingJMSException: Failed to invoke
       at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:271)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:205)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$stop$aop(Clien
      tConnectionDelegate.java:258)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.ConnectionAspect.handleStop(ConnectionAspect.java:157)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.stop(ClientConnectionDelegate.java)
       at org.jboss.jms.client.JBossConnection.stop(JBossConnection.java:126)
       at de.bwb.ubi.technik.jms.JBossMessagingEmpfangsServiceImpl.herunterfahren(JBossMessagingEmpfangsServiceImpl.java:100)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.checkMessageReceived(JMSKeepAliveClient.java:81)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.run(JMSKeepAliveClient.java:47)
      Caused by: java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5
       at org.jboss.jms.wireformat.ConnectionStopRequest.serverInvoke(ConnectionStopRequest.java:64)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
       at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:524)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
       at org.jboss.remoting.Client.invoke(Client.java:1724)
       at org.jboss.remoting.Client.invoke(Client.java:629)
       at org.jboss.remoting.Client.invoke(Client.java:617)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
       ... 21 more
      16:41:53,656 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[13008
      53, JMS, ClosingRequest[ID=a3-kswmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5, ver=30]]
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97]
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] read version 22 from input stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading response from unmarshaller
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket
      [addr=/192.168.231.17,port=14457,localport=2180].322a97] to pool
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received response InvocationRespons
      e[1bbd743, java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3
      3-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.]
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received a server-side exception as
       response to the invocation: java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-
      ftwzmvcl-1-ftwzmx33-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.
      16:41:53,687 DEBUG [Client] unable to remove remote callback handler
      java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8+a2l1532
      -6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.
       at org.jboss.remoting.ServerInvoker.handleInternalInvocation(ServerInvoker.java:1685)
       at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.handleInternalInvocation(BisocketServerInvoker.java:863)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:878)
       at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
       at org.jboss.remoting.Client.invoke(Client.java:1724)
       at org.jboss.remoting.Client.invoke(Client.java:629)
       at org.jboss.remoting.Client.removeListener(Client.java:1162)
       at org.jboss.jms.client.remoting.JMSRemotingConnection.stop(JMSRemotingConnection.java:389)
       at org.jboss.jms.client.container.ConnectionAspect.handleClose(ConnectionAspect.java:192)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.close(ClientConnectionDelegate.java)
       at org.jboss.jms.client.JBossConnection.close(JBossConnection.java:132)
       at de.bwb.ubi.technik.jms.JBossMessagingEmpfangsServiceImpl.herunterfahren(JBossMessagingEmpfangsServiceImpl.java:113)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.checkMessageReceived(JMSKeepAliveClient.java:81)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.run(JMSKeepAliveClient.java:47)
      16:41:53,687 TRACE [LocalClientInvoker] Using local client invoker for invocation.
      16:41:53,687 TRACE [ServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] received InternalInvocation[131a3e3]
      16:41:53,687 TRACE [ServerInvoker] handling InternalInvocation where method name = removeClientListener
      16:41:53,687 DEBUG [ServerInvoker] ServerInvoker (SocketServerInvoker[10.93.41.110:1353850969]) removing client callback handler with s
      ession id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d.
      16:41:53,687 DEBUG [BisocketServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] shutting down control connection: a2l1532-6bwsc
      9-ftwzmvcl-1-ftwzmx6k-d
      16:41:53,687 TRACE [ServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] successfully dispatched invocation, returning null from
       subsystem 'JMS' to client a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://10.93.41.110:1353850969/callback?callbackServ
      erHost=10.93.41.110&callbackServerPort=1353850969&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jboss.jms.c
      lient.remoting.ClientSocketWrapper&datatype=jms&guid=a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx4u-c&isCallbackServer=true&onewayThreadPool=org.jb
      oss.jms.server.remoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper], config {}
      16:41:53,687 DEBUG [InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@1e7c41f from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting org.jboss.remoting.transport.local.LocalClientInvoker@1e7c41f
      16:41:53,687 TRACE [Connector] org.jboss.remoting.transport.Connector@1933acb shutting down server invoker
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clie
      ntLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireforma
      t.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&time
      out=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {enableLease=tru
      e, clientMaxPoolSize=200}
      16:41:53,687 DEBUG [InvokerRegistry] removed SocketClientInvoker[59a735, bisocket://ubitest:14457] from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[59a735, bisocket://ubitest:14457]
      16:41:53,687 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] closing socket
      16:41:53,687 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] closed socket
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnected
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {enableLease=false, clientMaxPoolSize=1}
      16:41:53,687 DEBUG [InvokerRegistry] removed SocketClientInvoker[1503458, bisocket://ubitest:14457] from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[1503458, bisocket://ubitest:14457]
      16:41:53,687 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnected
      16:41:53,687 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Connection erfolgreich geschlossen.
      16:41:53,703 INFO [JBossMessagingEmpfangsServiceImpl] JMS-EmpfangsService wird initialisiert.
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting maxPoolSize to 200
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting client socket wrapper class name to org.jboss.jms.client.remoting.ClientSocketWrapper
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting shouldCheckConnection to false
      16:41:53,703 DEBUG [SocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting timeout to 0
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] constructed
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting maxPoolSize to 200
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting client socket wrapper class name to org.jboss.jms.client.remoting.ClientSocketWrapper
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting shouldCheckConnection to false
      16:41:53,718 DEBUG [SocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting timeout to 0
      16:41:53,718 DEBUG [BisocketClientInvoker] Setting ping frequency to: 60000
      16:41:53,718 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] connecting
      16:41:53,718 DEBUG [MicroSocketClientInvoker] Creating semaphore with size 200
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] added new pool ([]) as ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, maxPoolSize=200]
      16:41:53,718 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] connected
      16:41:53,718 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] sending initial lease ping to server to determine if server has leasing enabled.
      16:41:53,718 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457](1) invoking InvocationRequest[f37f80, $PING$]
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] creating socket
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] created socket: Socket[addr=/192.168.231.17,port=14457,localport=2182]
      16:41:53,718 TRACE [SocketWrapper] constructing org.jboss.jms.client.remoting.ClientSocketWrapper instance for Socket[addr=/192.168.231.17,port=14457,localport=2182], using timeout 0
      16:41:53,718 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2182].37bf73] setting timeout to 0^
      ERKUNGEN, DIENSTSTELLE, ENDE, FAHRZEUGTECHNIK, KENNUNG, NAME FROM Q73.TERMINE WHERE anfang between ? AND ? AND auftrags_id is null AND DIENSTSTELLE =
       ?
      2009-04-24 16:42:34,498 DEBUG [de.bwb.ubi.technik.j2ee.AbstractSessionBean] Session context gesetzt auf: org.jboss.ejb.StatelessSessionEnterpriseCont
      ext$SessionContextImpl@5c2a5c2a
      2009-04-24 16:42:34,512 DEBUG [de.bwb.ubi.technik.ConnectionUtility] Lokale DataSource gefunden: org.jboss.resource.adapter.jdbc.WrapperDataSource@37
      913791
      2009-04-24 16:42:34,512 DEBUG [de.bwb.ubi.technik.ConnectionUtility] DataSource Datenbank: UBI, Datenbankreferenz: java:comp/env/jdbc/UbiDB, Benutzer
      : ubiusr erfolgreich erzeugt.
      2009-04-24 16:42:46,933 DEBUG [de.bwb.ubi.technik.JMSUtility] Hole Topic=ubi/AEKQ738/Topic, TopicConnectionFactory=java:/JmsXA
      2009-04-24 16:42:46,934 DEBUG [de.bwb.ubi.technik.JMSUtility] Verschicke: MaterialVeraendertMessage(Flags: 9, Absender: AE-K/WIL, Empfänger: AE-K/
      WIL, Zeitraum: null, Auftrag-IDs: [a0fdae5cf80ce351:-2094326382:89855076925351, a0fdae5cf80ce351:-1018186381:76354645994893, a0fdae5cf80ce351:-1282
      274781:21481706821686, a0fdae5cf80ce351:-939044953:37924726689438, a0fdae5cf80ce351:-939243582:-134878552536500, a0fdae5cf80ce351:-1018418057:-139856
      431212012, 1eb974d816ff1389:1325893456:-93814993645549, 1eb974d816ff1389:630862340:-94017820758430, 74e9f18d983f48c2:1780718555:-66563119250438, a0fd
      ae5cf80ce351:-2094309215:2560686883205, 03385af61fe66aa6:-2064776811:-10903780589557, 1eb974d816ff1389:-1250346218:44130378423354, a0fdae5cf80ce351:-
      1018224946:130187114064384, a0fdae5cf80ce351:940110330:-46318232595116, a0fdae5cf80ce351:-1282244218:-102644285825322, a0fdae5cf80ce351:-939282262:-1
      00108353681230, 134aa2eff4523781:-681801703:56415776020908, a0fdae5cf80ce351:-1273513471:-59524293388654, a0fdae5cf80ce351:-1282353882:-6863993162453
      4, a0fdae5cf80ce351:1610253884:131558533973281, 5f183970dad225d0:-1764300843:98919483708721, a0fdae5cf80ce351:-1282184444:-95074956205059, a0fdae5cf8
      0ce351:-1018449018:-40485279951369, 5f183970dad225d0:-1764441796:127755781475015, a0fdae5cf80ce351:-1018440316:25673384859557, a0fdae5cf80ce351:52558
      6569:72792207948094, a0fdae5cf80ce351:-1282341763:24272775747047], Einsatzplan-IDs: [ab34af72d4600e40:-695889591:-125795419357165], Mitarbeiter-IDs:
       [], Fahrzeug-IDs: [], Mobil-IDs: [], Stellungnahme-IDs: [])
      2009-04-24 16:42:46,935 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,937 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
      2009-04-24 16:42:46,937 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,938 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,938 INFO [de.bwb.ubi.kn.infrastruktur.service.AEKAbstractSessionBean] Versende Nachricht: MaterialVeraendertMessage(Flags: 9, A
      bsender: AE-K/WIL, Empfänger: AE-K/WIL, Zeitraum: null, Auftrag-IDs: [a0fdae5cf80ce351:-2094326382:89855076925351, a0fdae5cf80ce351:-1018186381:7
      6354645994893, a0fdae5cf80ce351:-1282274781:21481706821686, a0fdae5cf80ce351:-939044953:37924726689438, a0fdae5cf80ce351:-939243582:-134878552536500,
       a0fdae5cf80ce351:-1018418057:-139856431212012, 1eb974d816ff1389:1325893456:-93814993645549, 1eb974d816ff1389:630862340:-94017820758430, 74e9f18d983f
      48c2:1780718555:-66563119250438, a0fdae5cf80ce351:-2094309215:2560686883205, 03385af61fe66aa6:-2064776811:-10903780589557, 1eb974d816ff1389:-12503462
      18:44130378423354, a0fdae5cf80ce351:-1018224946:130187114064384, a0fdae5cf80ce351:940110330:-46318232595116, a0fdae5cf80ce351:-1282244218:-1026442858
      25322, a0fdae5cf80ce351:-939282262:-100108353681230, 134aa2eff4523781:-681801703:56415776020908, a0fdae5cf80ce351:-1273513471:-59524293388654, a0fdae
      5cf80ce351:-1282353882:-68639931624534, a0fdae5cf80ce351:1610253884:131558533973281, 5f183970dad225d0:-1764300843:98919483708721, a0fdae5cf80ce351:-1
      282184444:-95074956205059, a0fdae5cf80ce351:-1018449018:-40485279951369, 5f183970dad225d0:-1764441796:127755781475015, a0fdae5cf80ce351:-1018440316:2
      5673384859557, a0fdae5cf80ce351:525586569:72792207948094, a0fdae5cf80ce351:-1282341763:24272775747047], Einsatzplan-IDs: [ab34af72d4600e40:-69588959
      1:-125795419357165], Mitarbeiter-IDs: [], Fahrzeug-IDs: [], Mobil-IDs: [], Stellungnahme-IDs: [])
      2009-04-24 16:42:46,942 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,982 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,982 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834](1) invo
      king InvocationRequest[43c443c4, CALLBACK, OnewayInvocation[InternalInvocation[438e438e]]]
      2009-04-24 16:42:49,543 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:42:49,545 WARN [de.bwb.ubi.gis.MapxtremeServiceImpl] Anzahl (Finalize): 0
      2009-04-24 16:42:49,546 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:13
      53850969] disconnecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] disconn
      ecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] is not
      connected!
      2009-04-24 16:42:49,546 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:13
      53850969] disconnecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] disconn
      ecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] is not
      connected!
      2009-04-24 16:42:53,802 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=1
      4457].38fe38fe] closing socket
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=1
      4457].38fe38fe] closed socket
      2009-04-24 16:42:53,803 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#7[10.93.41.110:2182] closed socketWrapper: ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=14457].38fe38fe]
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2184,localport=1
      4457].4ed94ed9] closing socket
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2184,localport=1
      4457].4ed94ed9] closed socket
      2009-04-24 16:42:53,803 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#8[10.93.41.110:2184] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2184,localport=14457].4ed94ed9]
      2009-04-24 16:42:53,805 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:42:53,807 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] from reg
      istry
      2009-04-24 16:42:53,807 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:14
      32040834] disconnecting ...
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] disconn
      ecting ...
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2186,localport=1
      3385].1c271c27] closing socket
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2186,localport=1
      3385].1c271c27] closed socket
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] disconn
      ected
      2009-04-24 16:42:53,820 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l1532-6bwsc9-ftwzmvcl-1-ftwzr0h3-u
      2009-04-24 16:42:53,826 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2187,localport=1
      4457].51095109] closing socket
      2009-04-24 16:42:53,826 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2187,localport=1
      4457].51095109] closed socket
      2009-04-24 16:42:53,826 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#7[10.93.41.110:2187] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2187,localport=14457].51095109]
      2009-04-24 16:42:53,969 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,local
      port=14446].789a789a] unable to writing CLOSING byte
      java.net.SocketException: There is no process to read data written to a pipe.
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:105)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:149)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:88)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:146)
       at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1723)
       at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:689)
       at org.jboss.remoting.transport.socket.ServerSocketWrapper.close(ServerSocketWrapper.java:57)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:655)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      2009-04-24 16:42:53,970 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,localport=1
      4446].789a789a] closing socket
      2009-04-24 16:42:53,970 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,localport=1
      4446].789a789a] closed socket
      2009-04-24 16:42:53,971 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#6[10.93.41.110:2179] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2179,localport=14446].789a789a]
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectOutputStream
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000
      2009-04-24 16:43:05,121 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Verschicke Nachricht: KeepAliveMessage: cd7bc57d
      -f694-4465-adbc-29593e871a92/Fri Apr 24 16:43:06 CEST 2009
      2009-04-24 16:43:05,122 DEBUG [de.bwb.ubi.technik.JMSUtility] Hole Topic=ubi/AEKQ738/Topic, TopicConnectionFactory=java:/JmsXA
      2009-04-24 16:43:05,122 DEBUG [de.bwb.ubi.technik.JMSUtility] Verschicke: KeepAliveMessage: cd7bc57d-f694-4465-adbc-29593e871a92/Fri Apr 24 16:43:06
      CEST 2009
      2009-04-24 16:43:05,124 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,125 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
      2009-04-24 16:43:05,126 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,126 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,127 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Nachricht verschickt.
      2009-04-24 16:43:05,127 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:49,550 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,local
      port=14446].7a1c7a1c] wrote CLOSING
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,localport=1
      4446].7a1c7a1c] closing socket
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,localport=1
      4446].7a1c7a1c] closed socket
      2009-04-24 16:44:05,129 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#0[10.93.41.120:1720] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.120,port=1720,localport=14446].7a1c7a1c]
      2009-04-24 16:44:49,551 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:45:49,550 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:46:05,138 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectOutputStream
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000
      2009-04-24 16:46:05,141 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Verschicke Nachricht: KeepAliveMessage: 5de39747
      -35c8-4f8a-af3c-f07582f6baf1/Fri Apr 24 16:46:06 CEST 2009
      
      
      2009-04-26:
      
      
      
      08:00:15,593 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received InvocationResponse so going to return response's return value of InvocationResponse[b16742, true]
      08:00:15,593 TRACE [LeasePinger] LeasePinger[SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457](a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1i)] successfully pinged the server
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] read version -1 from input stream
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] EOFException received. This is likely due to client finishing communication.
      java.io.EOFException
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:693)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      08:00:15,843 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7] closing socket
      08:00:15,843 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7] closed socket
      08:00:15,843 DEBUG [ServerThread] WorkerThread#3[192.168.231.17:13385] closed socketWrapper: ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7]
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] removing itself from clientpool
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] returning itself to threadpool
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] begins to wait
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] read version 22 from input stream
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] reading response from unmarshaller
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=3029].11e64ba] to pool
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] released semaphore: 200
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received response InvocationResponse[f232ed, org.jboss.jms.wireformat.NullResponse@ed715f]
      08:00:15,875 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received InvocationResponse so going to return response's return value of org.jboss.jms.wireformat.NullResponse@ed715f
      08:00:15,875 DEBUG [ConnectionValidator] ConnectionValidator[710c8a:SocketClientInvoker[1ee6a83, bisocket://ubitest:14457], pingPeriod=10000 ms] shut down lease pinger
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] read version 22 from input stream
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] reading response from unmarshaller
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=3028].23b7d2] to pool
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] reading response from unmarshaller
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=3028].23b7d2] to pool
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] released semaphore: 50
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] received response InvocationResponse[ae50f8, org.jboss.invocation.MarshalledValueEX@fb57a7aa]
      08:00:16,453 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] received InvocationResponse so going to return response's return value of org.jboss.invocation.MarshalledValueEX@fb57a7aa
      08:00:26,453 ERROR [JMSKeepAliveClient] Keep alive message NOT RECEIVED within 10000 ms
      08:00:26,453 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Zugriff wird beendet.
      08:00:26,453 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457](1) invoking InvocationRequest[fbdc05, JMS, org.jboss.jms.wireformat.ConnectionStopRequest@129d0de]
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] obtained semaphore: 199
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] reusing pooled connection: ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2691].10e5a8a]
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] writing version 22 on output stream
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] writing invocation to marshaller
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] done writing invocation to marshaller
      
      08:00:26,796 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket[addr=/192.1


        • 1. Re: Problems after Migration from WebSphere to JBoss (Broken
          dahm

          Which is also suspicious is the message

          2009-04-26 08:00:15,229 WARN [org.jboss.jms.server.connectionmanager.SimpleConnectionManager] A problem has been detected with the connection to remote c
          lient a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1j, jmsClientID=f-d5jjnwtf-1-949jnwtf-h8ncvx-c351l2a. It is possible the client has exited without closing its co
          nnection(s) or the network has failed. All associated connection resources will be cleaned up.


          2009-04-26 08:00:15,124 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] from registry
          2009-04-26 08:00:15,125 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1g
          2009-04-26 08:00:15,128 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474
          083] disconnecting ...
          2009-04-26 08:00:15,128 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] disconnectin
          g ...
          2009-04-26 08:00:15,148 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1j
          2009-04-26 08:00:15,163 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,173 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.120,port=3426,localport=13385]
          .14811481] closing socket
          2009-04-26 08:00:15,173 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.120,port=3426,localport=13385]
          .14811481] closed socket
          2009-04-26 08:00:15,218 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] disconnected
          2009-04-26 08:00:15,218 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1
          j+a2l153c-xvcn8h-ftwnj949-1-ftyrxud0-1o] shut down
          2009-04-26 08:00:15,229 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,229 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@8840884 from registry
          2009-04-26 08:00:15,229 WARN [org.jboss.jms.server.connectionmanager.SimpleConnectionManager] A problem has been detected with the connection to remote c
          lient a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1j, jmsClientID=f-d5jjnwtf-1-949jnwtf-h8ncvx-c351l2a. It is possible the client has exited without closing its co
          nnection(s) or the network has failed. All associated connection resources will be cleaned up.
          2009-04-26 08:00:15,233 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,256 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.ServerInvoker] ServerInvoker (SocketServerInvoker[192.168.231.17:173216532]) removing client callback ha
          ndler with session id of 5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmns-1z6+5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmuk-1za.
          2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] unrecognized listener ID: 5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmuk-1
          za
          2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@711c711c from registry
          2009-04-26 08:00:15,265 TRACE [org.jboss.remoting.transport.Connector] org.jboss.remoting.transport.Connector@609b609b shutting down server invoker
          2009-04-26 08:00:15,304 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
          2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.InvokerRegistry] decremented org.jboss.remoting.transport.local.LocalClientInvoker@10581058's count, cur
          rent count 2
          2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] from registry
          2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924
          472] disconnecting ...
          2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] disconnectin
          g ...
          2009-04-26 08:00:15,305 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2694,localport=13385]
          .46604660] closing socket
          2009-04-26 08:00:15,305 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2694,localport=13385]
          .46604660] closed socket
          2009-04-26 08:00:15,322 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Nachricht verschickt.
          2009-04-26 08:00:15,322 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,343 TRACE [org.jboss.remoting.Lease] removed lease:a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1l
          2009-04-26 08:00:15,343 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] disconnected
          2009-04-26 08:00:15,343 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1
          g+a2l1532-vedftv-ftx1lgyf-1-ftyrxyai-1l] shut down
          2009-04-26 08:00:15,344 DEBUG [org.jboss.remoting.InvokerRegistry] decremented org.jboss.remoting.transport.local.LocalClientInvoker@21172117's count, cur
          rent count 2
          2009-04-26 08:00:15,565 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[SocketServerInvoker[ubitest:14457]] ig
          noring the callback error
          2009-04-26 08:00:15,565 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] Error handling callback
          java.lang.Exception: Can not make remoting client invocation due to not being connected to server.
           at org.jboss.remoting.Client.invoke(Client.java:1729)
           at org.jboss.remoting.Client.invoke(Client.java:629)
           at org.jboss.remoting.Client.invokeOneway(Client.java:678)
           at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:835)
           at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:708)
           at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1454)
           at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1366)
           at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:327)
           at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
           at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:583)
           at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
           at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:638)
           at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:506)
           at org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:971)
           at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
           at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
           at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionA
          dvised.java:101)
           at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
          650789275322226.java)
           at org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
           at sun.reflect.GeneratedMethodAccessor131.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:618)
           at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
           at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
          650789275322226.java)
           at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
           at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
          650789275322226.java)
           at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
           at org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
           at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
           at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
           at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:106)
           at org.jboss.remoting.Client.invoke(Client.java:1724)
           at org.jboss.remoting.Client.invoke(Client.java:629)
           at org.jboss.remoting.Client.invoke(Client.java:617)
           at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
           at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnect
          ionDelegate.java:221)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
          _N3268650789275322226.java)
           at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
           at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
          _N3268650789275322226.java)
           at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
          _N3268650789275322226.java)
           at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
           at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
          _N3268650789275322226.java)
           at org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
           at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:667)
           at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:367)
           at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:255)
           at org.jboss.resource.adapter.jms.JmsXAResource.commit(JmsXAResource.java:98)
           at org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
           at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:636)
           at com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2619)
           at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1779)
           at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
           at com.arjuna.ats.arjuna.AtomicAction.end(AtomicAction.java:216)
           at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commit(TransactionImple.java:240)
           at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:501)
           at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:361)
           at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:181)
           at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)
           at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)
           at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)
           at org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)
           at org.jboss.ejb.Container.invoke(Container.java:960)
           at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:618)
           at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
           at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
           at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
           at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
           at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
           at org.jboss.invocation.unified.server.UnifiedInvoker.invoke(UnifiedInvoker.java:231)
           at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:618)
           at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
           at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
           at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
           at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
           at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
           at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:216)
           at $Proxy15.invoke(Unknown Source)
           at org.jboss.remoting.ServerInvocationHandlerWrapper.invoke(ServerInvocationHandlerWrapper.java:77)
           at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
           at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
           at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
           at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:524)
           at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
          2009-04-26 08:00:15,884 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] did not find server socket factory configu
          ration as mbean service or classname. Creating default server socket factory.
           at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
          2009-04-26 08:00:15,884 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] did not find server socket factory configu
          ration as mbean service or classname. Creating default server socket factory.
          2009-04-26 08:00:15,818 INFO [de.bwb.ubi.kn.boot.GISKeepAliveJob] Starte GIS-Anfragen
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] created server socket factory javax.net.De
          faultServerSocketFactory@5bd65bd6
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] setting pingFre
          quency to 60000
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] setting pingWin
          dowFactor to 2
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.Connector] org.jboss.remoting.transport.Connector@786c786c started
          2009-04-26 08:00:15,910 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.ServerInvoker] ServerInvoker (SocketServerInvoker[192.168.231.17:1452030437]) added client callback hand
          ler CallbackManager[62476247] with session id of 5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6 and callback handle object
          of null.
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@266f266f from registry
          2009-04-26 08:00:15,910 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] Session id for callback handler is 5c4o6fh-iuij0f-ftwxi8xn-1-ftzb
          zvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2
          kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6] using callbackTimeout value 10000
          2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[UNITIALIZED] setting server invoker to
           SocketServerInvoker[ubitest:14457]
          2009-04-26 08:00:15,911 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[SocketServerInvoker[ubitest:14457]] se
          tting callback handler to ServerInvokerCallbackHandler[5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6]
          2009-04-26 08:00:15,911 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
          


          • 2. Re: Problems after Migration from WebSphere to JBoss (Broken
            ron_sigal

            Sorry for the delay. Is this problem still outstanding?

            • 3. Re: Problems after Migration from WebSphere to JBoss (Broken
              dahm

              Yes, unfortunately. We found a workaround, but it is still a problem.

              We found that using remoting 2.5.2 may be an issue, should we go back to the 2.2 branch?

              Cheers
              Markus

              • 4. Re: Problems after Migration from WebSphere to JBoss (Broken
                ron_sigal

                Hi Markus,

                I've seen some similar firewall issues lately, and I can make some suggestions:

                "dahm" wrote:

                Remarkable settings on the server side:

                in bisocket-remoting.xml: Enabled pings

                <attribute name="pingFrequency" isParam="true">60000</attribute>
                <attribute name="pingWindowFactor" isParam="true">2</attribute>
                



                These parameters turn on a control connection test facility in the bisocket transport, and we've seen problems with it lately. The fact is that the JBossMessaging code has never used it (they turned it off, in effect, by making the ping frequency "infinite"), so it's never really been tested in the context of JBossMessaging. I assume your goal in turning it on was to keep the connection alive in the context of the firewall, and I would suggest something like:

                 <attribute name="pingFrequency" isParam="true">60000</attribute>
                 <attribute name="pingWindowFactor" isParam="true">MAX_INT / 60000</attribute>
                


                (where, of course, MAX_INT / 60000 is replaced by an actual integer). This sets the window in which the ping is expected to "infinity", so that you get the pinging without the possibility of a late ping.

                "dahm" wrote:

                We checked that with the firewall and network people and they said that idle connections will be closed after 1 hour.
                We built an keep alive client which sends a message to itself (via the server of course)
                every 3 minutes and waits for it 10 seconds. If it does not receive the message it will display a message to the user
                and restart the connection to the topic.


                What's happening, I guess, is that the firewall is killing a connection, but the Remoting client tries to use it anyway, right? The problem with the "keep alive client", though, is that the Remoting bisocket transport uses a pool of connections, and there's no guarantee that the keep alive client will use, and keep alive, all of them.

                Here's an alternative suggestion. I made it recently in another context and haven't gotten any feedback yet, but it's something to try. If a Remoting invocation results in a SocketException, the invocation will be retried if the parameter "numberOfCallRetries" is set to a value greater than 1. Now, "numberOfCallRetries" is explicitly set to 1 in the "do not change" section of remoting-bisocket-service.xml, but it seems like a reasonable thing to try.

                Here's yet another untested suggestion. If you set the parameter "idleTimeout" to a value "n" > 0, then a thread on the server will periodically wake up and shut down any ServerThreads that have been idle for more than "n" seconds. Since the ServerThread's socket will get closed, the client should avoid using that connection.

                "dahm" wrote:

                We found that using remoting 2.5.2 may be an issue, should we go back to the 2.2 branch?


                The fact is that AS 4.2 hasn't been tested, by us, with Remoting 2.4/2.5. I'm not aware of any reason wh AS 4.2 shouldn't work with Remoting 2.4/2.5, but going back to the 2.2 branch is a viable strategy in the face of problems. However, in this case, I suspect you'd see the same firewall issues.

                Let us know how it goes.

                -Ron