8 Replies Latest reply on Apr 27, 2009 11:46 PM by dahm

    Problems after Migration from WebSphere to JBoss (Broken JMS

    dahm

      Hello,

      unfortunately we are having severe problems after the migration from WebSphere 5 to JBoss 4.2.3.GA
      (+ JBoss messaging 1.4.2.GA-SP1).. However, I'm not sure whether those are related to Messaging or rather
      the underlying JBoss Remoting.

      We have the following configuration settings:

      Server:
      - OS: IBM AIX ubitest 3 5 00C0587E4C00
      - JRE:
      java version "1.5.0"
      Java(TM) 2 Runtime Environment, Standard Edition (build pap64dev-20080716b (SR8))
      IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 AIX ppc64-64 j9vmap6423-20080716 (JIT enabled)
      J9VM - 20080625_20583_BHdSMr
      JIT - 20080620_1845_r8
      GC - 200806_19)
      JCL - 20080716a

      Client:
      - OS: Windows XP
      - JRE
      java version "1.6.0_13"
      Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
      Java HotSpot(TM) Client VM (build 11.3-b02, mixed mode, sharing)

      Remarkable settings on the server side:

      in bisocket-remoting.xml: Enabled pings

      <attribute name="pingFrequency" isParam="true">60000</attribute>
      <attribute name="pingWindowFactor" isParam="true">2</attribute>
      


      connection-factories.xml:

      <attribute name="SendAcksAsync">true</attribute>
      <attribute name="DisableRemotingChecks">true</attribute>
      

      jboss-service.xml:

      Since we're running multiple instances, the bindings have been altered to use port 14457 instead of 4457 for remoting.
      I.e. all port numbers in that configuration have been "prefixed" with "1". We copied the above mentioned remoting settings to
      the bindings XML file.

      Now for the application setting:

      We're are running multiple fat clients that communicate with each other via messaging. I.e. one client changes data, calls
      an EJB (2.1) method which will not only store the data but also send a message to the given topic:

      <mbean code="org.jboss.jms.server.destination.TopicService" name="ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_Topic"
       xmbean-dd="xmdesc/Topic-xmbean.xml">
       <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer
       </depends>
       <depends>jboss.messaging:service=PostOffice</depends>
       <attribute name="JNDIName">/JMSTOPIC</attribute>
      
       <attribute name="MaxDeliveryAttempts">10</attribute>
       <attribute name="RedeliveryDelay">200</attribute>
       <attribute name="ExpiryQueue">ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_EXQ
       </attribute>
       <attribute name="DLQ">ubi.aek.destination:service=Topic,name=ubi_APPLICATIONID_DLQ
       </attribute>
       </mbean>
      


      Most of our service methods send such messages to notify changes of application data, thus it is vital the messages arrive...

      All clients connect at startup to the above mentioned topic and set a message listener just as described in the JBM examples.
      If it receives a message it will check if it is relevant for him. There are at most 40-50 clients at a time, but that was irrelevant
      for the test.

      Now the problem is that the connection BREAKS at random time:
      The client waits for messages eternally while the server has already shut down the connection. We checked the latter fact via the
      JMX console.
      We cannot reproduce the error (as always only the customer can ;-( ). The only way to surely trigger it, is to pull the network cable.

      We checked that with the firewall and network people and they said that idle connections will be closed after 1 hour.
      We built an keep alive client which sends a message to itself (via the server of course)
      every 3 minutes and waits for it 10 seconds. If it does not receive the message it will display a message to the user
      and restart the connection to the topic.

      Unfortunately this problem occurs to often to be ignored. We tried to enable JBM internal pings (s.a. configuration), but that
      didn't help very much. It seems to me that JBM reestablishes the connection but our keep alive client still does not receive
      his messages and reconnects.

      No the big question is: Do you have any clue or idea???
      Any help would be appreciated!

      Currently, I'm investigationg whether there are any known incompatibilities between SUN and IBM JDK.

      Thanks in advance
      Markus

      P.S. Attached are the log files of the client and server with TRACE enabled.

      Client:

      2009-04-24:
      
      16:41:12,750 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[ca66b
      4, $PING$]
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f]
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:12,750 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:13,125 TRACE [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] sending PING tied to lease
      16:41:13,125 TRACE [ConnectionValidator] pinging, sending InvocationRequest[f23ad3, self, $PING$] over SocketClientInvoker[6b93c5, biso
      cket://ubitest:14457]
      16:41:13,125 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457](1) invoking InvocationRequest[f23ad
      3, self, $PING$]
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] obtained semaphore: 49
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6]
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:13,125 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] reading version from input stream
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457](ClientSocketWrapper[Socket[addr=/19
      2.168.231.17,port=14457,localport=2089].14bc4e6]) got Exception: java.net.SocketException: Connection reset
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] released semaphore: 50
      16:41:13,578 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6] closing socket
      16:41:13,578 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2089].14bc4e6] closed socket
      16:41:13,578 TRACE [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] retries exhausted
      16:41:13,578 DEBUG [ConnectionValidator] ConnectionValidator failed to ping via SocketClientInvoker[6b93c5, bisocket://ubitest:14457]
      org.jboss.remoting.InvocationFailureException: Unable to perform invocation; nested exception is:
       java.net.SocketException: Connection reset
       at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:142)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:901)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.ConnectionValidator.doCheckConnectionWithLease(ConnectionValidator.java:650)
       at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:334)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1263)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:838)
       ... 6 more
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] SocketException received. This is likely due to client disconnec
      ting and resetting connection.
      java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:1038)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:673)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      16:41:13,593 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2093].148bd9d] closing s
      ocket
      16:41:13,593 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2093].148bd9d] closed so
      cket
      16:41:13,593 DEBUG [ServerThread] WorkerThread#0[192.168.231.17:13385] closed socketWrapper: ServerSocketWrapper[Socket[addr=ubitest/19
      2.168.231.17,port=13385,localport=2093].148bd9d]
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] removing itself from clientpool
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] returning itself to threadpool
      16:41:13,593 TRACE [ServerThread] WorkerThread#0[192.168.231.17:13385] begins to wait
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](ClientSocketWrapper[Socket[addr=/19
      2.168.231.17,port=14457,localport=2090].17fc44f]) got Exception: java.net.SocketException: Connection reset
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f] closing socket
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2090].17fc44f] closed socket
      16:41:13,593 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] retries exhausted
      16:41:13,593 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] failed to ping to server
      org.jboss.remoting.InvocationFailureException: Unable to perform invocation; nested exception is:
       java.net.SocketException: Connection reset
       at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:142)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:901)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:335)
       at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:20)
       at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:369)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1263)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:838)
       ... 7 more
      16:41:13,593 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms]'s connection is invalid
      16:41:13,593 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clie
      ntLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireforma
      t.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&time
      out=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {validatorPingPe
      riod=10000, socket.check_connection=false, stopLeaseOnFailure=true, marshaller=org.jboss.jms.wireformat.JMSWireFormat, unmarshaller=org
      .jboss.jms.wireformat.JMSWireFormat, clientLeasePeriod=10000, clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper, numb
      erOfCallRetries=1, dataType=jms, pingWindowFactor=2, connection_checker=true, NumberOfCallRetries=1, JBM_clientMaxPoolSize=200, pingFre
      quency=60000, timeout=5000, validatorPingTimeout=5000}
      16:41:13,593 DEBUG [InvokerRegistry] removed SocketClientInvoker[6b93c5, bisocket://ubitest:14457] from registry
      16:41:13,593 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[6b93c5, bisocket://ubitest:14457]
      16:41:13,593 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnecting ...
      16:41:13,593 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnecting ...
      16:41:13,593 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14446, enableTcpNoDelay timeout 0 ms, maxP
      oolSize=50]
      16:41:13,593 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2092].514f7f] closing socket
      16:41:13,609 DEBUG [ConnectionValidator] Thread[Thread-12,5,main] calling ConnectionState[83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5].Consol
      idatedListener.handleConnectionException()
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2092].514f7f] closed socket
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2088].8dcd5d] closing socket
      16:41:13,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2088].8dcd5d] closed socket
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=200]
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=1]
      16:41:13,625 TRACE [MicroSocketClientInvoker] clearing pool for ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, m
      axPoolSize=50]
      16:41:13,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[6b93c5, bisocket://ubitest:14457] disconnected
      16:41:13,625 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] stopped, returning true
      16:41:13,625 DEBUG [ConnectionValidator] org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask@1420a71 detected connect
      ion failure: stopping LeasePinger
      16:41:13,625 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] setting disconnect timeout to: -1
      16:41:13,625 TRACE [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )] stopping lease timer
      16:41:13,625 DEBUG [LeasePinger] LeasePinger[SocketClientInvoker[59a735, bisocket://ubitest:14457](a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3j-a
      )]: disconnectTimeout: -1
      16:41:13,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[6fa30
      c, $DISCONNECT$]
      16:41:13,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:13,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] creating socket
      16:41:13,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      java.net.NoRouteToHostException: No route to host: connect
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.connect(SocketClientInvoker.java:289)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:206)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:433)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1089)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:762)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:138)
       at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:401)
       at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:797)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      16:41:13,640 DEBUG [SocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457]
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator
      [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSock
      etWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&s
      ocket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=
      10000&validatorPingTimeout=5000]
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:776)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:426)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
       at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:138)
       at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:401)
       at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:797)
       at java.util.TimerThread.mainLoop(Timer.java:512)
       at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.NoRouteToHostException: No route to host: connect
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.connect(SocketClientInvoker.java:289)
       at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:206)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:433)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1089)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:762)
       ... 7 more
      16:41:13,640 DEBUG [MicroRemoteClientInvoker] error shutting down lease pinger
      16:41:13,640 DEBUG [ConnectionValidator] ConnectionValidator[16d4b50:SocketClientInvoker[6b93c5, bisocket://ubitest:14457], pingPeriod=
      10000 ms] shut down lease pinger
      16:41:42,812 TRACE [BisocketServerInvoker] checking connections
      16:41:42,812 TRACE [BisocketServerInvoker] elapsed: 59828
      16:41:43,625 INFO [JMSKeepAliveClient] Sending PING KeepAliveMessage: f43481c5-7d7d-4633-bf0f-10f347142ff2/Fri Apr 24 16:41:43 CEST 20
      09
      16:41:43,625 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446](1) invoking InvocationRequest[11c9ba1,
      INVOKER, org.jboss.invocation.Invocation@12d813e]
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] obtained semaphore: 49
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] creating socket
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] created socket: Socket[addr=/192.168.23
      1.17,port=14446,localport=2179]
      16:41:43,625 TRACE [SocketWrapper] constructing org.jboss.remoting.transport.socket.ClientSocketWrapper instance for Socket[addr=/192.1
      68.231.17,port=14446,localport=2179], using timeout 0
      16:41:43,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] setting timeout t
      o 0
      16:41:43,625 TRACE [ClientSocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] getting out
      put stream from Socket[addr=/192.168.231.17,port=14446,localport=2179], org.jboss.invocation.unified.marshall.InvocationMarshaller@1a5a
      f9f
      16:41:43,625 TRACE [JavaSerializationManager] Creating ObjectOutputStream
      16:41:43,625 TRACE [ClientSocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] getting inp
      ut stream from Socket[addr=/192.168.231.17,port=14446,localport=2179], org.jboss.invocation.unified.marshall.InvocationUnMarshaller@ad1
      57f
      16:41:43,625 TRACE [JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      16:41:43,625 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=2179].801b62] setting timeout t
      o 0
      16:41:43,625 TRACE [ClientSocketWrapper] reset timeout: 0
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] writing version 22 on output stream
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] writing invocation to marshaller
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] done writing invocation to marshaller
      16:41:43,625 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] reading version from input stream
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] read version 22 from input stream
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] reading response from unmarshaller
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] returned ClientSocketWrapper[Socket[add
      r=/192.168.231.17,port=14446,localport=2179].801b62] to pool
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] released semaphore: 50
      16:41:43,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] received response InvocationResponse[1d
      3f905, org.jboss.invocation.MarshalledValueEX@fb57a7aa]
      16:41:43,640 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[9f0d, socket://ubitest:14446] received InvocationResponse so going to
       return response's return value of org.jboss.invocation.MarshalledValueEX@fb57a7aa
      16:41:53,640 ERROR [JMSKeepAliveClient] Keep alive message NOT RECEIVED within 10000 ms
      16:41:53,640 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Zugriff wird beendet.
      16:41:53,640 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[45021
      2, JMS, org.jboss.jms.wireformat.ConnectionStopRequest@8b2139]
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] creating socket
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] created socket: Socket[addr=/192.16
      8.231.17,port=14457,localport=2180]
      16:41:53,640 TRACE [SocketWrapper] constructing org.jboss.jms.client.remoting.ClientSocketWrapper instance for Socket[addr=/192.168.231
      .17,port=14457,localport=2180], using timeout 0
      16:41:53,640 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] setting timeout t
      o 0
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing invocation to marshaller
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] done writing invocation to marshall
      er
      16:41:53,640 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] read version 22 from input stream
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading response from unmarshaller
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket
      [addr=/192.168.231.17,port=14457,localport=2180].322a97] to pool
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received response InvocationRespons
      e[dd750e, java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5]
      16:41:53,656 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received a server-side exception as
       response to the invocation: java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf
      6o4c5
      16:41:53,656 ERROR [JBossMessagingEmpfangsServiceImpl] Fehler beim Anhalten des JMS-Empfangsservice
      org.jboss.jms.exception.MessagingJMSException: Failed to invoke
       at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:271)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:205)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$stop$aop(Clien
      tConnectionDelegate.java:258)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.ConnectionAspect.handleStop(ConnectionAspect.java:157)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$stop_N2856118408655404442.invokeNext(ClientConnectionDelegate$stop_N2
      856118408655404442.java)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.stop(ClientConnectionDelegate.java)
       at org.jboss.jms.client.JBossConnection.stop(JBossConnection.java:126)
       at de.bwb.ubi.technik.jms.JBossMessagingEmpfangsServiceImpl.herunterfahren(JBossMessagingEmpfangsServiceImpl.java:100)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.checkMessageReceived(JMSKeepAliveClient.java:81)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.run(JMSKeepAliveClient.java:47)
      Caused by: java.lang.IllegalStateException: Cannot find object in dispatcher with id 83-7iwmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5
       at org.jboss.jms.wireformat.ConnectionStopRequest.serverInvoke(ConnectionStopRequest.java:64)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
       at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:524)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
       at org.jboss.remoting.Client.invoke(Client.java:1724)
       at org.jboss.remoting.Client.invoke(Client.java:629)
       at org.jboss.remoting.Client.invoke(Client.java:617)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
       ... 21 more
      16:41:53,656 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457](1) invoking InvocationRequest[13008
      53, JMS, ClosingRequest[ID=a3-kswmzwtf-1-nx8ixwtf-f0jiui-hf6o4c5, ver=30]]
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reusing pooled connection: ClientSo
      cketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97]
      16:41:53,656 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] writing version 22 on output stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading version from input stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] read version 22 from input stream
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] reading response from unmarshaller
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket
      [addr=/192.168.231.17,port=14457,localport=2180].322a97] to pool
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] released semaphore: 200
      16:41:53,687 TRACE [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received response InvocationRespons
      e[1bbd743, java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx3
      3-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.]
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] received a server-side exception as
       response to the invocation: java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-
      ftwzmvcl-1-ftwzmx33-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.
      16:41:53,687 DEBUG [Client] unable to remove remote callback handler
      java.lang.RuntimeException: Can not remove callback listener from target server with id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8+a2l1532
      -6bwsc9-ftwzmvcl-1-ftwzmx6k-d as it does not exist as a registered callback listener.
       at org.jboss.remoting.ServerInvoker.handleInternalInvocation(ServerInvoker.java:1685)
       at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.handleInternalInvocation(BisocketServerInvoker.java:863)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:878)
       at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
       at org.jboss.remoting.Client.invoke(Client.java:1724)
       at org.jboss.remoting.Client.invoke(Client.java:629)
       at org.jboss.remoting.Client.removeListener(Client.java:1162)
       at org.jboss.jms.client.remoting.JMSRemotingConnection.stop(JMSRemotingConnection.java:389)
       at org.jboss.jms.client.container.ConnectionAspect.handleClose(ConnectionAspect.java:192)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$close_N4742752445160157748.invokeNext(ClientConnectionDelegate$close_
      N4742752445160157748.java)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.close(ClientConnectionDelegate.java)
       at org.jboss.jms.client.JBossConnection.close(JBossConnection.java:132)
       at de.bwb.ubi.technik.jms.JBossMessagingEmpfangsServiceImpl.herunterfahren(JBossMessagingEmpfangsServiceImpl.java:113)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.checkMessageReceived(JMSKeepAliveClient.java:81)
       at de.bwb.ubi.technik.jms.JMSKeepAliveClient.run(JMSKeepAliveClient.java:47)
      16:41:53,687 TRACE [LocalClientInvoker] Using local client invoker for invocation.
      16:41:53,687 TRACE [ServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] received InternalInvocation[131a3e3]
      16:41:53,687 TRACE [ServerInvoker] handling InternalInvocation where method name = removeClientListener
      16:41:53,687 DEBUG [ServerInvoker] ServerInvoker (SocketServerInvoker[10.93.41.110:1353850969]) removing client callback handler with s
      ession id of a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8+a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx6k-d.
      16:41:53,687 DEBUG [BisocketServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] shutting down control connection: a2l1532-6bwsc
      9-ftwzmvcl-1-ftwzmx6k-d
      16:41:53,687 TRACE [ServerInvoker] SocketServerInvoker[10.93.41.110:1353850969] successfully dispatched invocation, returning null from
       subsystem 'JMS' to client a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx33-8
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://10.93.41.110:1353850969/callback?callbackServ
      erHost=10.93.41.110&callbackServerPort=1353850969&callbackServerProtocol=bisocket&clientMaxPoolSize=1&clientSocketClass=org.jboss.jms.c
      lient.remoting.ClientSocketWrapper&datatype=jms&guid=a2l1532-6bwsc9-ftwzmvcl-1-ftwzmx4u-c&isCallbackServer=true&onewayThreadPool=org.jb
      oss.jms.server.remoting.DirectThreadPool&serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper], config {}
      16:41:53,687 DEBUG [InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@1e7c41f from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting org.jboss.remoting.transport.local.LocalClientInvoker@1e7c41f
      16:41:53,687 TRACE [Connector] org.jboss.remoting.transport.Connector@1933acb shutting down server invoker
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clie
      ntLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireforma
      t.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&time
      out=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {enableLease=tru
      e, clientMaxPoolSize=200}
      16:41:53,687 DEBUG [InvokerRegistry] removed SocketClientInvoker[59a735, bisocket://ubitest:14457] from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[59a735, bisocket://ubitest:14457]
      16:41:53,687 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] closing socket
      16:41:53,687 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2180].322a97] closed socket
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[59a735, bisocket://ubitest:14457] disconnected
      16:41:53,687 TRACE [InvokerRegistry] destroying client invoker InvokerLocator [bisocket://ubitest:14457/?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&pingFrequency=60000&pingWindowFactor=2&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&validatorPingPeriod=10000&validatorPingTimeout=5000], config {enableLease=false, clientMaxPoolSize=1}
      16:41:53,687 DEBUG [InvokerRegistry] removed SocketClientInvoker[1503458, bisocket://ubitest:14457] from registry
      16:41:53,687 TRACE [InvokerRegistry] disconnecting SocketClientInvoker[1503458, bisocket://ubitest:14457]
      16:41:53,687 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnecting ...
      16:41:53,687 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1503458, bisocket://ubitest:14457] disconnected
      16:41:53,687 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Connection erfolgreich geschlossen.
      16:41:53,703 INFO [JBossMessagingEmpfangsServiceImpl] JMS-EmpfangsService wird initialisiert.
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting maxPoolSize to 200
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting client socket wrapper class name to org.jboss.jms.client.remoting.ClientSocketWrapper
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting shouldCheckConnection to false
      16:41:53,703 DEBUG [SocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting timeout to 0
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] constructed
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting maxPoolSize to 200
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting client socket wrapper class name to org.jboss.jms.client.remoting.ClientSocketWrapper
      16:41:53,703 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting shouldCheckConnection to false
      16:41:53,718 DEBUG [SocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] setting timeout to 0
      16:41:53,718 DEBUG [BisocketClientInvoker] Setting ping frequency to: 60000
      16:41:53,718 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] connecting
      16:41:53,718 DEBUG [MicroSocketClientInvoker] Creating semaphore with size 200
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] added new pool ([]) as ServerAddress[192.168.231.17:14457, NO enableTcpNoDelay timeout 0 ms, maxPoolSize=200]
      16:41:53,718 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] connected
      16:41:53,718 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] sending initial lease ping to server to determine if server has leasing enabled.
      16:41:53,718 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457](1) invoking InvocationRequest[f37f80, $PING$]
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] obtained semaphore: 199
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] creating socket
      16:41:53,718 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7a7686, bisocket://ubitest:14457] created socket: Socket[addr=/192.168.231.17,port=14457,localport=2182]
      16:41:53,718 TRACE [SocketWrapper] constructing org.jboss.jms.client.remoting.ClientSocketWrapper instance for Socket[addr=/192.168.231.17,port=14457,localport=2182], using timeout 0
      16:41:53,718 TRACE [SocketWrapper] ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2182].37bf73] setting timeout to 0^
      ERKUNGEN, DIENSTSTELLE, ENDE, FAHRZEUGTECHNIK, KENNUNG, NAME FROM Q73.TERMINE WHERE anfang between ? AND ? AND auftrags_id is null AND DIENSTSTELLE =
       ?
      2009-04-24 16:42:34,498 DEBUG [de.bwb.ubi.technik.j2ee.AbstractSessionBean] Session context gesetzt auf: org.jboss.ejb.StatelessSessionEnterpriseCont
      ext$SessionContextImpl@5c2a5c2a
      2009-04-24 16:42:34,512 DEBUG [de.bwb.ubi.technik.ConnectionUtility] Lokale DataSource gefunden: org.jboss.resource.adapter.jdbc.WrapperDataSource@37
      913791
      2009-04-24 16:42:34,512 DEBUG [de.bwb.ubi.technik.ConnectionUtility] DataSource Datenbank: UBI, Datenbankreferenz: java:comp/env/jdbc/UbiDB, Benutzer
      : ubiusr erfolgreich erzeugt.
      2009-04-24 16:42:46,933 DEBUG [de.bwb.ubi.technik.JMSUtility] Hole Topic=ubi/AEKQ738/Topic, TopicConnectionFactory=java:/JmsXA
      2009-04-24 16:42:46,934 DEBUG [de.bwb.ubi.technik.JMSUtility] Verschicke: MaterialVeraendertMessage(Flags: 9, Absender: AE-K/WIL, Empfänger: AE-K/
      WIL, Zeitraum: null, Auftrag-IDs: [a0fdae5cf80ce351:-2094326382:89855076925351, a0fdae5cf80ce351:-1018186381:76354645994893, a0fdae5cf80ce351:-1282
      274781:21481706821686, a0fdae5cf80ce351:-939044953:37924726689438, a0fdae5cf80ce351:-939243582:-134878552536500, a0fdae5cf80ce351:-1018418057:-139856
      431212012, 1eb974d816ff1389:1325893456:-93814993645549, 1eb974d816ff1389:630862340:-94017820758430, 74e9f18d983f48c2:1780718555:-66563119250438, a0fd
      ae5cf80ce351:-2094309215:2560686883205, 03385af61fe66aa6:-2064776811:-10903780589557, 1eb974d816ff1389:-1250346218:44130378423354, a0fdae5cf80ce351:-
      1018224946:130187114064384, a0fdae5cf80ce351:940110330:-46318232595116, a0fdae5cf80ce351:-1282244218:-102644285825322, a0fdae5cf80ce351:-939282262:-1
      00108353681230, 134aa2eff4523781:-681801703:56415776020908, a0fdae5cf80ce351:-1273513471:-59524293388654, a0fdae5cf80ce351:-1282353882:-6863993162453
      4, a0fdae5cf80ce351:1610253884:131558533973281, 5f183970dad225d0:-1764300843:98919483708721, a0fdae5cf80ce351:-1282184444:-95074956205059, a0fdae5cf8
      0ce351:-1018449018:-40485279951369, 5f183970dad225d0:-1764441796:127755781475015, a0fdae5cf80ce351:-1018440316:25673384859557, a0fdae5cf80ce351:52558
      6569:72792207948094, a0fdae5cf80ce351:-1282341763:24272775747047], Einsatzplan-IDs: [ab34af72d4600e40:-695889591:-125795419357165], Mitarbeiter-IDs:
       [], Fahrzeug-IDs: [], Mobil-IDs: [], Stellungnahme-IDs: [])
      2009-04-24 16:42:46,935 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,937 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
      2009-04-24 16:42:46,937 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,938 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,938 INFO [de.bwb.ubi.kn.infrastruktur.service.AEKAbstractSessionBean] Versende Nachricht: MaterialVeraendertMessage(Flags: 9, A
      bsender: AE-K/WIL, Empfänger: AE-K/WIL, Zeitraum: null, Auftrag-IDs: [a0fdae5cf80ce351:-2094326382:89855076925351, a0fdae5cf80ce351:-1018186381:7
      6354645994893, a0fdae5cf80ce351:-1282274781:21481706821686, a0fdae5cf80ce351:-939044953:37924726689438, a0fdae5cf80ce351:-939243582:-134878552536500,
       a0fdae5cf80ce351:-1018418057:-139856431212012, 1eb974d816ff1389:1325893456:-93814993645549, 1eb974d816ff1389:630862340:-94017820758430, 74e9f18d983f
      48c2:1780718555:-66563119250438, a0fdae5cf80ce351:-2094309215:2560686883205, 03385af61fe66aa6:-2064776811:-10903780589557, 1eb974d816ff1389:-12503462
      18:44130378423354, a0fdae5cf80ce351:-1018224946:130187114064384, a0fdae5cf80ce351:940110330:-46318232595116, a0fdae5cf80ce351:-1282244218:-1026442858
      25322, a0fdae5cf80ce351:-939282262:-100108353681230, 134aa2eff4523781:-681801703:56415776020908, a0fdae5cf80ce351:-1273513471:-59524293388654, a0fdae
      5cf80ce351:-1282353882:-68639931624534, a0fdae5cf80ce351:1610253884:131558533973281, 5f183970dad225d0:-1764300843:98919483708721, a0fdae5cf80ce351:-1
      282184444:-95074956205059, a0fdae5cf80ce351:-1018449018:-40485279951369, 5f183970dad225d0:-1764441796:127755781475015, a0fdae5cf80ce351:-1018440316:2
      5673384859557, a0fdae5cf80ce351:525586569:72792207948094, a0fdae5cf80ce351:-1282341763:24272775747047], Einsatzplan-IDs: [ab34af72d4600e40:-69588959
      1:-125795419357165], Mitarbeiter-IDs: [], Fahrzeug-IDs: [], Mobil-IDs: [], Stellungnahme-IDs: [])
      2009-04-24 16:42:46,942 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,982 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:42:46,982 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834](1) invo
      king InvocationRequest[43c443c4, CALLBACK, OnewayInvocation[InternalInvocation[438e438e]]]
      2009-04-24 16:42:49,543 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:42:49,545 WARN [de.bwb.ubi.gis.MapxtremeServiceImpl] Anzahl (Finalize): 0
      2009-04-24 16:42:49,546 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:13
      53850969] disconnecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] disconn
      ecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] is not
      connected!
      2009-04-24 16:42:49,546 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:13
      53850969] disconnecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] disconn
      ecting ...
      2009-04-24 16:42:49,546 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[60fb60fb, bisocket://10.93.41.110:1353850969] is not
      connected!
      2009-04-24 16:42:53,802 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=1
      4457].38fe38fe] closing socket
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=1
      4457].38fe38fe] closed socket
      2009-04-24 16:42:53,803 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#7[10.93.41.110:2182] closed socketWrapper: ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2182,localport=14457].38fe38fe]
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2184,localport=1
      4457].4ed94ed9] closing socket
      2009-04-24 16:42:53,803 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2184,localport=1
      4457].4ed94ed9] closed socket
      2009-04-24 16:42:53,803 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#8[10.93.41.110:2184] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2184,localport=14457].4ed94ed9]
      2009-04-24 16:42:53,805 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:42:53,807 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] from reg
      istry
      2009-04-24 16:42:53,807 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:14
      32040834] disconnecting ...
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] disconn
      ecting ...
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2186,localport=1
      3385].1c271c27] closing socket
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2186,localport=1
      3385].1c271c27] closed socket
      2009-04-24 16:42:53,807 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5a0c5a0c, bisocket://10.93.41.110:1432040834] disconn
      ected
      2009-04-24 16:42:53,820 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l1532-6bwsc9-ftwzmvcl-1-ftwzr0h3-u
      2009-04-24 16:42:53,826 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2187,localport=1
      4457].51095109] closing socket
      2009-04-24 16:42:53,826 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2187,localport=1
      4457].51095109] closed socket
      2009-04-24 16:42:53,826 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#7[10.93.41.110:2187] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2187,localport=14457].51095109]
      2009-04-24 16:42:53,969 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,local
      port=14446].789a789a] unable to writing CLOSING byte
      java.net.SocketException: There is no process to read data written to a pipe.
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:105)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:149)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:88)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:146)
       at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1723)
       at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:689)
       at org.jboss.remoting.transport.socket.ServerSocketWrapper.close(ServerSocketWrapper.java:57)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:655)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      2009-04-24 16:42:53,970 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,localport=1
      4446].789a789a] closing socket
      2009-04-24 16:42:53,970 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.110,port=2179,localport=1
      4446].789a789a] closed socket
      2009-04-24 16:42:53,971 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#6[10.93.41.110:2179] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.110,port=2179,localport=14446].789a789a]
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectOutputStream
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      2009-04-24 16:43:05,118 TRACE [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000
      2009-04-24 16:43:05,121 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Verschicke Nachricht: KeepAliveMessage: cd7bc57d
      -f694-4465-adbc-29593e871a92/Fri Apr 24 16:43:06 CEST 2009
      2009-04-24 16:43:05,122 DEBUG [de.bwb.ubi.technik.JMSUtility] Hole Topic=ubi/AEKQ738/Topic, TopicConnectionFactory=java:/JmsXA
      2009-04-24 16:43:05,122 DEBUG [de.bwb.ubi.technik.JMSUtility] Verschicke: KeepAliveMessage: cd7bc57d-f694-4465-adbc-29593e871a92/Fri Apr 24 16:43:06
      CEST 2009
      2009-04-24 16:43:05,124 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,125 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
      2009-04-24 16:43:05,126 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,126 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:05,127 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Nachricht verschickt.
      2009-04-24 16:43:05,127 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
      2009-04-24 16:43:49,550 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,local
      port=14446].7a1c7a1c] wrote CLOSING
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,localport=1
      4446].7a1c7a1c] closing socket
      2009-04-24 16:44:05,129 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/10.93.41.120,port=1720,localport=1
      4446].7a1c7a1c] closed socket
      2009-04-24 16:44:05,129 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#0[10.93.41.120:1720] closed socketWrapper: ServerSocket
      Wrapper[Socket[addr=/10.93.41.120,port=1720,localport=14446].7a1c7a1c]
      2009-04-24 16:44:49,551 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:45:49,550 TRACE [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] checking connections
      2009-04-24 16:46:05,138 TRACE [org.jboss.remoting.transport.socket.SocketServerInvoker] passed through ServerSocketRefresh.release()
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectOutputStream
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.serialization.impl.java.JavaSerializationManager] Creating ObjectInputStreamWithClassLoader
      2009-04-24 16:46:05,139 TRACE [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000
      2009-04-24 16:46:05,141 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Verschicke Nachricht: KeepAliveMessage: 5de39747
      -35c8-4f8a-af3c-f07582f6baf1/Fri Apr 24 16:46:06 CEST 2009
      
      
      2009-04-26:
      
      
      
      08:00:15,593 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received InvocationResponse so going to return response's return value of InvocationResponse[b16742, true]
      08:00:15,593 TRACE [LeasePinger] LeasePinger[SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457](a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1i)] successfully pinged the server
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] read version -1 from input stream
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] EOFException received. This is likely due to client finishing communication.
      java.io.EOFException
       at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:693)
       at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
       at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
      08:00:15,843 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7] closing socket
      08:00:15,843 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7] closed socket
      08:00:15,843 DEBUG [ServerThread] WorkerThread#3[192.168.231.17:13385] closed socketWrapper: ServerSocketWrapper[Socket[addr=ubitest/192.168.231.17,port=13385,localport=2694].4469b7]
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] removing itself from clientpool
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] returning itself to threadpool
      08:00:15,843 TRACE [ServerThread] WorkerThread#3[192.168.231.17:13385] begins to wait
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] read version 22 from input stream
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] reading response from unmarshaller
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=3029].11e64ba] to pool
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] released semaphore: 200
      08:00:15,875 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received response InvocationResponse[f232ed, org.jboss.jms.wireformat.NullResponse@ed715f]
      08:00:15,875 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] received InvocationResponse so going to return response's return value of org.jboss.jms.wireformat.NullResponse@ed715f
      08:00:15,875 DEBUG [ConnectionValidator] ConnectionValidator[710c8a:SocketClientInvoker[1ee6a83, bisocket://ubitest:14457], pingPeriod=10000 ms] shut down lease pinger
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] read version 22 from input stream
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] reading response from unmarshaller
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=3028].23b7d2] to pool
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] reading response from unmarshaller
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14446,localport=3028].23b7d2] to pool
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] released semaphore: 50
      08:00:16,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] received response InvocationResponse[ae50f8, org.jboss.invocation.MarshalledValueEX@fb57a7aa]
      08:00:16,453 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[d58ce2, socket://ubitest:14446] received InvocationResponse so going to return response's return value of org.jboss.invocation.MarshalledValueEX@fb57a7aa
      08:00:26,453 ERROR [JMSKeepAliveClient] Keep alive message NOT RECEIVED within 10000 ms
      08:00:26,453 INFO [JBossMessagingEmpfangsServiceImpl] JMS-Zugriff wird beendet.
      08:00:26,453 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457](1) invoking InvocationRequest[fbdc05, JMS, org.jboss.jms.wireformat.ConnectionStopRequest@129d0de]
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] obtained semaphore: 199
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] reusing pooled connection: ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=2691].10e5a8a]
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] writing version 22 on output stream
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] writing invocation to marshaller
      08:00:26,453 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] done writing invocation to marshaller
      
      08:00:26,796 TRACE [MicroSocketClientInvoker] SocketClientInvoker[1b5bbf7, bisocket://ubitest:14457] returned ClientSocketWrapper[Socket[addr=/192.168.231.17,port=14457,localport=3029].11e


        • 1. Re: Problems after Migration from WebSphere to JBoss (Broken
          dahm

          BTW: We also updated the Remoting to version 2.5.1

          • 2. Re: Problems after Migration from WebSphere to JBoss (Broken
            gaohoward

            hi, did you set these parameters in your remoting config file?

            10000
            10000
            5000

            The server thinks a client connection is bad when it doesn't receive a ping from client for 'clientLeasePeriod'. The client will think the server connection is bad when it sends a ping but doesn't receive response from the server for 'validatorPingTimeout'.

            So they should be properly configured according to you network configuration.

            • 3. Re: Problems after Migration from WebSphere to JBoss (Broken
              gaohoward

              Sorry, post again.

              hi, did you set these parameters in your remoting config file?

               <attribute name="clientLeasePeriod" isParam="true">10000</attribute>
               <attribute name="validatorPingPeriod" isParam="true">10000</attribute>
               <attribute name="validatorPingTimeout" isParam="true">5000</attribute>
              

              The server thinks a client connection is bad when it doesn't receive a ping from client for 'clientLeasePeriod'. The client will think the server connection is bad when it sends a ping but doesn't receive response from the server for 'validatorPingTimeout'.

              So they should be properly configured according to you network configuration.

              • 4. Re: Problems after Migration from WebSphere to JBoss (Broken
                dahm

                Hi,

                here's the completx remoting-bisocket.xml:

                 <mbean code="org.jboss.remoting.transport.Connector" name="jboss.messaging:service=Connector,transport=bisocket"
                 display-name="Bisocket Transport Connector">
                 <attribute name="Configuration">
                 <config>
                 <invoker transport="bisocket">
                
                 <!--
                 There should be no reason to change these parameters - warning! Changing them may stop JBoss Messaging working
                 correctly
                 -->
                 <attribute name="marshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat
                 </attribute>
                 <attribute name="unmarshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat
                 </attribute>
                 <attribute name="dataType" isParam="true">jms</attribute>
                 <attribute name="socket.check_connection" isParam="true">false</attribute>
                 <attribute name="serverBindAddress">${jboss.bind.address}</attribute>
                 <attribute name="serverBindPort">14457</attribute>
                 <attribute name="clientSocketClass" isParam="true">org.jboss.jms.client.remoting.ClientSocketWrapper
                 </attribute>
                 <attribute name="serverSocketClass">org.jboss.jms.server.remoting.ServerSocketWrapper
                 </attribute>
                
                 <attribute name="numberOfCallRetries" isParam="true">1</attribute>
                
                 <!--
                 Siehe Posting von Alphonse: https://www.spree.de/wbb3/index.php?page=Thread&postID=184&highlight=readybank#post184
                 -->
                 <attribute name="pingFrequency" isParam="true">60000</attribute>
                 <attribute name="pingWindowFactor" isParam="true">2</attribute>
                 <attribute name="onewayThreadPool">org.jboss.jms.server.remoting.DirectThreadPool
                 </attribute>
                 <!-- End immutable parameters -->
                
                 <attribute name="stopLeaseOnFailure" isParam="true">true</attribute>
                
                 <!-- Periodicity of client pings. Server window by default is twice this figure -->
                 <attribute name="clientLeasePeriod" isParam="true">10000</attribute>
                 <attribute name="validatorPingPeriod" isParam="true">10000</attribute>
                 <attribute name="validatorPingTimeout" isParam="true">5000</attribute>
                 <attribute name="registerCallbackListener">false</attribute>
                
                 <attribute name="timeout" isParam="true">0</attribute>
                
                 <!--
                 Number of seconds to wait for a connection in the client pool to become free Offenbar nicht mehr in JBoss-Remoting
                 2.5.1: 11:19:50,109 WARN [MicroSocketClientInvoker] numberOfRetries is no longer used <attribute
                 name="numberOfRetries" isParam="true">10</attribute>
                 -->
                
                 <!--
                 Max Number of connections in client pool. This should be significantly higher than the max number of
                 sessions/consumers you expect
                 -->
                 <attribute name="JBM_clientMaxPoolSize" isParam="true">200</attribute>
                
                 <!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->
                 <attribute name="callbackTimeout">10000</attribute>
                
                 <!--
                 Use these parameters to specify values for binding and connecting control connections to work with your
                 firewall/NAT configuration <attribute name="secondaryBindPort">xyz</attribute> <attribute
                 name="secondaryConnectPort">abc</attribute>
                 -->
                
                 </invoker>
                 <handlers>
                 <handler subsystem="JMS">org.jboss.jms.server.remoting.JMSServerInvocationHandler
                 </handler>
                 </handlers>
                 </config>
                 </attribute>
                 </mbean>
                


                • 5. Re: Problems after Migration from WebSphere to JBoss (Broken
                  gaohoward

                  Hi, I think your configure are fine.

                  The client log shows that the connection is closed by server. But I didn't see the JBM server detected any client connection failure. Usually if a client lost its connection to the server, the server will detected it after a while (10 sec, as per clientLeasePeroid param), and clean up the resources. Then you will see some log like:

                  A problem has been detected with the connection to remote client xxxxxxx, jmsClientID=xxxxxxxx. It is possible the client has exited without closing its connection(s) or the network has failed. All associated connection resources will be cleaned up.

                  Can you check your server.log if it contains such log?

                  Also I would suggest you post it to remoting forum also, there you can get more configuration help on remoting tuning.

                  • 6. Re: Problems after Migration from WebSphere to JBoss (Broken
                    dahm

                    Yes, this happens, too:

                    2009-04-26 08:00:15,124 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] from registry
                    2009-04-26 08:00:15,125 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1g
                    2009-04-26 08:00:15,128 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474
                    083] disconnecting ...
                    2009-04-26 08:00:15,128 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] disconnectin
                    g ...
                    2009-04-26 08:00:15,148 TRACE [org.jboss.remoting.ConnectionNotifier] Client disconnected (session id = a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1j
                    2009-04-26 08:00:15,163 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,173 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.120,port=3426,localport=13385]
                    .14811481] closing socket
                    2009-04-26 08:00:15,173 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.120,port=3426,localport=13385]
                    .14811481] closed socket
                    2009-04-26 08:00:15,218 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[67006700, bisocket://10.93.41.120:2077474083] disconnected
                    2009-04-26 08:00:15,218 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1
                    j+a2l153c-xvcn8h-ftwnj949-1-ftyrxud0-1o] shut down
                    2009-04-26 08:00:15,229 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,229 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@8840884 from registry
                    2009-04-26 08:00:15,229 WARN [org.jboss.jms.server.connectionmanager.SimpleConnectionManager] A problem has been detected with the connection to remote c
                    lient a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1j, jmsClientID=f-d5jjnwtf-1-949jnwtf-h8ncvx-c351l2a. It is possible the client has exited without closing its co
                    nnection(s) or the network has failed. All associated connection resources will be cleaned up.
                    2009-04-26 08:00:15,233 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,256 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.ServerInvoker] ServerInvoker (SocketServerInvoker[192.168.231.17:173216532]) removing client callback ha
                    ndler with session id of 5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmns-1z6+5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmuk-1za.
                    2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] unrecognized listener ID: 5c4o6fh-iuij0f-ftwxi8xn-1-ftyrxmuk-1
                    za
                    2009-04-26 08:00:15,256 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@711c711c from registry
                    2009-04-26 08:00:15,265 TRACE [org.jboss.remoting.transport.Connector] org.jboss.remoting.transport.Connector@609b609b shutting down server invoker
                    2009-04-26 08:00:15,304 DEBUG [de.bwb.ubi.technik.JMSUtility] Nachricht versandt: delegator->JBossMessage[0]:NON-PERSISTENT, deliveryId=0
                    2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.InvokerRegistry] decremented org.jboss.remoting.transport.local.LocalClientInvoker@10581058's count, cur
                    rent count 2
                    2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] from registry
                    2009-04-26 08:00:15,304 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924
                    472] disconnecting ...
                    2009-04-26 08:00:15,304 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] disconnectin
                    g ...
                    2009-04-26 08:00:15,305 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2694,localport=13385]
                    .46604660] closing socket
                    2009-04-26 08:00:15,305 TRACE [org.jboss.remoting.transport.socket.SocketWrapper] ClientSocketWrapper[Socket[addr=/10.93.41.110,port=2694,localport=13385]
                    .46604660] closed socket
                    2009-04-26 08:00:15,322 DEBUG [de.bwb.ubi.kn.infrastruktur.service.ejbimpl.EjbJMSVersandServiceBean] Nachricht verschickt.
                    2009-04-26 08:00:15,322 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,343 TRACE [org.jboss.remoting.Lease] removed lease:a2l153c-xvcn8h-ftwnj949-1-ftyrxubq-1l
                    2009-04-26 08:00:15,343 TRACE [org.jboss.remoting.MicroRemoteClientInvoker] SocketClientInvoker[5ec35ec3, bisocket://10.93.41.110:1526924472] disconnected
                    2009-04-26 08:00:15,343 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[a2l1532-vedftv-ftx1lgyf-1-ftyrxy97-1
                    g+a2l1532-vedftv-ftx1lgyf-1-ftyrxyai-1l] shut down
                    2009-04-26 08:00:15,344 DEBUG [org.jboss.remoting.InvokerRegistry] decremented org.jboss.remoting.transport.local.LocalClientInvoker@21172117's count, cur
                    rent count 2
                    2009-04-26 08:00:15,565 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[SocketServerInvoker[ubitest:14457]] ig
                    noring the callback error
                    2009-04-26 08:00:15,565 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] Error handling callback
                    java.lang.Exception: Can not make remoting client invocation due to not being connected to server.
                     at org.jboss.remoting.Client.invoke(Client.java:1729)
                     at org.jboss.remoting.Client.invoke(Client.java:629)
                     at org.jboss.remoting.Client.invokeOneway(Client.java:678)
                     at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:835)
                     at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:708)
                     at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1454)
                     at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1366)
                     at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:327)
                     at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
                     at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:583)
                     at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
                     at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:638)
                     at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:506)
                     at org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:971)
                     at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
                     at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
                     at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionA
                    dvised.java:101)
                     at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
                    650789275322226.java)
                     at org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
                     at sun.reflect.GeneratedMethodAccessor131.invoke(Unknown Source)
                     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                     at java.lang.reflect.Method.invoke(Method.java:618)
                     at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
                     at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
                    650789275322226.java)
                     at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
                     at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268
                    650789275322226.java)
                     at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
                     at org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
                     at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
                     at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
                     at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:106)
                     at org.jboss.remoting.Client.invoke(Client.java:1724)
                     at org.jboss.remoting.Client.invoke(Client.java:629)
                     at org.jboss.remoting.Client.invoke(Client.java:617)
                     at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
                     at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnect
                    ionDelegate.java:221)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
                    _N3268650789275322226.java)
                     at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
                     at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
                    _N3268650789275322226.java)
                     at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
                    _N3268650789275322226.java)
                     at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
                     at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction
                    _N3268650789275322226.java)
                     at org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
                     at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:667)
                     at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:367)
                     at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:255)
                     at org.jboss.resource.adapter.jms.JmsXAResource.commit(JmsXAResource.java:98)
                     at org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
                     at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:636)
                     at com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2619)
                     at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1779)
                     at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
                     at com.arjuna.ats.arjuna.AtomicAction.end(AtomicAction.java:216)
                     at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commit(TransactionImple.java:240)
                     at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:501)
                     at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:361)
                     at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:181)
                     at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)
                     at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)
                     at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)
                     at org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)
                     at org.jboss.ejb.Container.invoke(Container.java:960)
                     at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
                     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                     at java.lang.reflect.Method.invoke(Method.java:618)
                     at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
                     at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
                     at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                     at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
                     at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
                     at org.jboss.invocation.unified.server.UnifiedInvoker.invoke(UnifiedInvoker.java:231)
                     at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
                     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                     at java.lang.reflect.Method.invoke(Method.java:618)
                     at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
                     at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
                     at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                     at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
                     at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
                     at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:216)
                     at $Proxy15.invoke(Unknown Source)
                     at org.jboss.remoting.ServerInvocationHandlerWrapper.invoke(ServerInvocationHandlerWrapper.java:77)
                     at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
                     at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
                     at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
                     at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:524)
                     at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
                    2009-04-26 08:00:15,884 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] did not find server socket factory configu
                    ration as mbean service or classname. Creating default server socket factory.
                     at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
                    2009-04-26 08:00:15,884 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] did not find server socket factory configu
                    ration as mbean service or classname. Creating default server socket factory.
                    2009-04-26 08:00:15,818 INFO [de.bwb.ubi.kn.boot.GISKeepAliveJob] Starte GIS-Anfragen
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] created server socket factory javax.net.De
                    faultServerSocketFactory@5bd65bd6
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] setting pingFre
                    quency to 60000
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.bisocket.BisocketServerInvoker] SocketServerInvoker[192.168.231.17:1452030437] setting pingWin
                    dowFactor to 2
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.transport.Connector] org.jboss.remoting.transport.Connector@786c786c started
                    2009-04-26 08:00:15,910 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.ServerInvoker] ServerInvoker (SocketServerInvoker[192.168.231.17:1452030437]) added client callback hand
                    ler CallbackManager[62476247] with session id of 5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6 and callback handle object
                    of null.
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.InvokerRegistry] removed org.jboss.remoting.transport.local.LocalClientInvoker@266f266f from registry
                    2009-04-26 08:00:15,910 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] Session id for callback handler is 5c4o6fh-iuij0f-ftwxi8xn-1-ftzb
                    zvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.ServerInvokerCallbackHandler] ServerInvokerCallbackHandler[5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2
                    kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6] using callbackTimeout value 10000
                    2009-04-26 08:00:15,910 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[UNITIALIZED] setting server invoker to
                     SocketServerInvoker[ubitest:14457]
                    2009-04-26 08:00:15,911 DEBUG [org.jboss.remoting.callback.DefaultCallbackErrorHandler] DefaultCallbackErrorHandler[SocketServerInvoker[ubitest:14457]] se
                    tting callback handler to ServerInvokerCallbackHandler[5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzvqq-2kz+5c4o6fh-iuij0f-ftwxi8xn-1-ftzbzw9y-2l6]
                    2009-04-26 08:00:15,911 TRACE [org.jboss.remoting.transport.local.LocalClientInvoker] Using local client invoker for invocation.
                    


                    • 7. Re: Problems after Migration from WebSphere to JBoss (Broken
                      gaohoward

                      Based on the log, I think the problem is that the server side closed a connection when ever it hasn't received a lease ping for more than 10 seconds. Maybe your network condition is not so good, or very busy. I suggest you enlarge the clientLeasePeriod parameter. also please set the pingFrequency like:

                      <attribute name="pingFrequency" isParam="true">214748364</attribute>
                      


                      JBM requires this to be very large to work.



                      • 8. Re: Problems after Migration from WebSphere to JBoss (Broken
                        dahm

                        We deliberately changed the pingFrequency since a value of 214748364 = 60 hours effectively disables the ping.