1 Reply Latest reply on Jan 8, 2010 11:47 AM by stlewis

    problem with ssh sesssions

    mtroscia

      Hi,

       

      I run my FuseESB in server mode and I manage my bundles by ssh client (Putty).

      I've got a bundle whitch consumes messages from queue (to get message I use Camel 1.6) and processes it. But from time to time it does't work - message consumer thred's can't to get message from queue (this thread is blocked).

       

      -


       

      Thread dump is following:

       

      "DefaultMessageListenerContainer-1" - Thread t@211

         java.lang.Thread.State: BLOCKED on org.apache.geronimo.gshell.io.SystemOutputHijacker$DelegateStream@a88273 owned by: pool-7-thread-1

           at java.io.PrintStream.println(PrintStream.java:756)

           at pl.com.marcin.TestClass.getInfo(TestClass.java:28)

           at pl.com.marcin.transformer.TransformerImpl.transform(Transformer.java:175)

           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

           at java.lang.reflect.Method.invoke(Method.java:597)

           at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:64)

           at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:78)

           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

           at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)

           at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)

           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

           at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:57)

           at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:40)

           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

           at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)

           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

           at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)

           at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)

           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)

           at $Proxy112.transformXSLT(Unknown Source)

           at pl.com.agora.services.element.generate.ElementProcessor.process(ElementProcessor.java:77)

           at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:80)

           at org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:189)

           at org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:133)

           at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)

           at org.apache.camel.processor.interceptor.StreamCachingInterceptor.proceed(StreamCachingInterceptor.java:87)

           at org.apache.camel.processor.interceptor.StreamCachingInterceptor.process(StreamCachingInterceptor.java:82)

           at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:52)

           at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)

           at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)

           at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:72)

           at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:543)

           at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:482)

           at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:451)

           at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)

           at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)

           at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:982)

           at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:974)

           at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:876)

           at java.lang.Thread.run(Thread.java:619)

       

      Locked ownable synchronizers:

           - None

       

      "pool-7-thread-1" - Thread t@11333

         java.lang.Thread.State: WAITING on org.apache.sshd.common.channel.Window@1f2b28d

           at java.lang.Object.wait(Native Method)

           at java.lang.Object.wait(Object.java:485)

           at org.apache.sshd.common.channel.Window.waitAndConsume(Window.java:122)

           at org.apache.sshd.common.channel.ChannelOutputStream.flush(ChannelOutputStream.java:89)

           - locked org.apache.sshd.common.channel.ChannelOutputStream@3873ec

           at org.apache.sshd.common.channel.ChannelOutputStream.write(ChannelOutputStream.java:65)

           - locked org.apache.sshd.common.channel.ChannelOutputStream@3873ec

           at org.apache.sshd.common.util.LoggingFilterOutputStream.write(LoggingFilterOutputStream.java:55)

           at java.io.PrintStream.write(PrintStream.java:430)

           - locked java.io.PrintStream@428d1e

           at java.io.PrintStream.write(PrintStream.java:430)

           - locked java.io.PrintStream@13eb7fc

           at org.apache.geronimo.gshell.io.SystemOutputHijacker$DelegateStream.write(SystemOutputHijacker.java:246)

           at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)

           at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)

           at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:85)

           - locked java.io.OutputStreamWriter@a13865

           at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:168)

           at java.io.PrintStream.write(PrintStream.java:477)

           - locked org.apache.geronimo.gshell.io.SystemOutputHijacker$DelegateStream@a88273

           at java.io.PrintStream.print(PrintStream.java:619)

           at java.io.PrintStream.println(PrintStream.java:756)

           - locked org.apache.geronimo.gshell.io.SystemOutputHijacker$DelegateStream@a88273

           at pl.com.agora.services.pub.MyProcessor$2.createMessage(MyProcessor.java:78)

           at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:570)

           at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:541)

           at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)

           at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)

           at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:531)

           at pl.com.agora.services.pub.MyProcessor.processMessage(MyProcessor.java:67)

           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

           at java.lang.reflect.Method.invoke(Method.java:597)

           at org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:163)

           at org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:92)

           at org.apache.camel.component.bean.BeanProcessor.process(BeanProcessor.java:118)

           at org.apache.camel.impl.ProcessorEndpoint.onExchange(ProcessorEndpoint.java:92)

           at org.apache.camel.impl.ProcessorEndpoint$1.process(ProcessorEndpoint.java:66)

           at org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)

           at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:84)

           at org.apache.camel.processor.interceptor.StreamCachingInterceptor.proceed(StreamCachingInterceptor.java:87)

           at org.apache.camel.processor.interceptor.StreamCachingInterceptor.process(StreamCachingInterceptor.java:82)

           at org.apache.camel.processor.Pipeline.process(Pipeline.java:115)

           at org.apache.camel.processor.Pipeline.process(Pipeline.java:89)

           at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)

           at org.apache.camel.processor.Pipeline.process(Pipeline.java:57)

           at org.apache.camel.processor.MulticastProcessor$ProcessCall.run(MulticastProcessor.java:138)

           at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

           at java.lang.Thread.run(Thread.java:619)

       

         Locked ownable synchronizers:

           - locked java.util.concurrent.locks.ReentrantLock$NonfairSync@270cd3

       

      Additionally my ssh session is closed.

       

      How can I resolve this problem.

       

      Cheers,

      Marcin

        • 1. Re: problem with ssh sesssions
          stlewis

          What I think is happening is that you're calling System.out.println from within your code  which is getting intercepted by this org.apache.geronimo.gshell.io.SystemOutputHijacker, looks like it's causing some kind of dead lock.  I believe the SystemOutputHijacker is pretty much there to catch all the calls to System.out.println that the JmsTemplate code makes and re-route them to the log file.

           

          You should be able to work around this quite easily though by switching over from System.out.println in your code to using log4j or slf4j instead, which is always a better idea in the long run anyways.