2 Replies Latest reply on Nov 13, 2008 1:46 PM by mclu

    Bridge stops working after IOException

    mclu

      I have 2 physical servers A and B (jboss 4.2.3 with jbm 1.4.0SP3).

      Messages are transfered with QOS 2 using a bridge between those servers.
      Bridge is deployed on A and messages are transfered from B to A.

      Sometimes it happens that the bridge stops working. I think is has to do with heavy machine load or short network failures.

      On B I can see the following stacktrace:

      10:38:48,750 ERROR [org.jboss.remoting.transport.socket.SocketClientInvoker] Got marshalling exception, exiting
      java.io.IOException: Connection reset by peer: socket write error
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(Unknown Source)
       at java.net.SocketOutputStream.write(Unknown Source)
       at com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(Unknown Source)
       at com.sun.net.ssl.internal.ssl.OutputRecord.write(Unknown Source)
       at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Unknown Source)
       at com.sun.net.ssl.internal.ssl.AppOutputStream.write(Unknown Source)
       at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
       at java.io.BufferedOutputStream.flush(Unknown Source)
       at java.io.DataOutputStream.flush(Unknown Source)
       at org.jboss.jms.wireformat.ClientDelivery.write(ClientDelivery.java:93)
       at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:945)
       at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:586)
       at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:418)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
       at org.jboss.remoting.Client.invoke(Client.java:1634)
       at org.jboss.remoting.Client.invoke(Client.java:548)
       at org.jboss.remoting.Client.invokeOneway(Client.java:598)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:826)
       at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:697)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1432)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1344)
       at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:322)
       at org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
       at org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:582)
       at org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
       at org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:606)
       at org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:505)
       at org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:935)
       at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
       at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
       at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
       at org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:795)
       at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
       at org.jboss.remoting.Client.invoke(Client.java:1634)
       at org.jboss.remoting.Client.invoke(Client.java:548)
       at org.jboss.remoting.Client.invoke(Client.java:536)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
       at org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
       at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:637)
       at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:393)
       at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:238)
       at org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
       at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:487)
       at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3107)
       at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3022)
       at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:2126)
       at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1819)
       at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
       at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
       at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1382)
       at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
       at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
       at org.jboss.aspects.tx.TxPolicy.endTransaction(TxPolicy.java:175)
       at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:87)
       at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:191)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:77)
       at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:106)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
       at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
       at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:214)
       at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:184)
       at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:81)
       at $Proxy1078.announceEmployee(Unknown Source)
      .... rest are my classes.....
      


      After that error messages are going in the Queue of server B but the bridge is not transferring them anymore. The JMXConsole on A says that the bridge is running. Restarting the bridge via server restart or via jmx solves the problem. Then Messages are bridged again.

      Questions:
      - can someone explain it?
      - is there a known Bug?
      - When does a new GA or SP Version comes out? Are there any timeplans?

      thx in advance


        • 1. Re: Bridge stops working after IOException
          timfox

          Have you tried a later version from SVN? Or looked in JIRA to see if this is a fixed issue?

          • 2. Re: Bridge stops working after IOException
            mclu

            Hi Tim!
            Not yet.

            How stable is this last version?
            We have a customer who used our beta software in production.
            We need to upgrade our software anyway so I plan to upgrade JBM also.

            Can I use the Version 1.4.0 SP3 CP03 with jboss 4.2.1 GA? I saw in jira that a lot of remoting and bridge stuff was fixed in cp02. Or are these releases for a special server/configuration?

            Or should I build from svn directly? Which Tag/Branch?

            thx in advance
            Markus