1 2 Previous Next 16 Replies Latest reply on Oct 17, 2005 2:04 PM by srinivas.chin

    JMS

    dalmo

      Hi, i'm getting this error when one of my node go down, anyone knows why this occurs? after the error clears and he reconnects to the JMS everything continues ok, the session is ok and i continue with the aplication on but the node delay a lot for getting up...

      2005-04-08 09:16:41,734 WARN [org.jboss.mq.Connection] Connection failure:
      org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketException: Connection reset)
       at org.jboss.mq.Connection.asynchFailure(Connection.java:436)
       at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:402)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:334)
       at java.lang.Thread.run(Thread.java:536)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
       at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
       at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
       at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2316)
       at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2383)
       at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2455)
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2604)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:281)
       ... 1 more
      2005-04-08 09:16:41,734 DEBUG [org.jboss.mq.il.uil2.SocketManager] End ReadTask.run
      2005-04-08 09:16:41,734 DEBUG [org.jboss.mq.il.uil2.SocketManager] End WriteTask.run
      2005-04-08 09:16:41,734 WARN [org.jboss.ejb.plugins.jms.JMSContainerInvoker] JMS provider failure detected:
      org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketException: Connection reset)
       at org.jboss.mq.Connection.asynchFailure(Connection.java:436)
       at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:402)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:334)
       at java.lang.Thread.run(Thread.java:536)
      Caused by: java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
       at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
       at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
       at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2316)
       at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2383)
       at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2455)
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2604)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:281)
       ... 1 more
      2005-04-08 09:16:41,765 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Trying to reconnect to JMS provider
      2005-04-08 09:16:51,765 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] unset exception listener
      2005-04-08 09:16:51,765 ERROR [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Could not stop JMS connection
      org.jboss.mq.SpyJMSException: Cannot disable the connection with the JMS server; - nested throwable: (java.io.IOException: Client is not connected)
       at org.jboss.mq.Connection.doStop(Connection.java:1289)
       at org.jboss.mq.Connection.stop(Connection.java:718)
       at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerStop(JMSContainerInvoker.java:832)
       at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl.onException(JMSContainerInvoker.java:1265)
       at org.jboss.mq.Connection.asynchFailure(Connection.java:446)
       at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:145)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:402)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:334)
       at java.lang.Thread.run(Thread.java:536)
      Caused by: java.io.IOException: Client is not connected
       at org.jboss.mq.il.uil2.SocketManager.internalSendMessage(SocketManager.java:229)
       at org.jboss.mq.il.uil2.SocketManager.sendMessage(SocketManager.java:197)
       at org.jboss.mq.il.uil2.UILServerIL.setEnabled(UILServerIL.java:189)
       at org.jboss.mq.Connection.doStop(Connection.java:1285)
       ... 8 more
      


        • 1. Re: JMS
          srinivas.chin

          Hi Dalmo,

          I am also facing the same problem. Please update this forum if you have got any solution.

          Thanks
          Srinivas

          • 2. Re: JMS
            srinivas.chin

            Does anyone from Jboss care to answer to this question pls. I am facing this problem in Production servers. And if the problem continues we are contemplating on moving to another application servers.

            Thanks
            Srinivas

            • 3. Re: JMS
              manik

              Hi, I'm having a look at this...

              • 4. Re: JMS
                manik

                1. This is probably best posted on the JMS forums, not here.

                2. The large delay when starting up the node can be caused by a HASingleton/JGroups misconfiguration. Perhaps some more info (JBoss version, JGroups version, config used, etc) would help.

                3. Regarding moving production onto another application server, production use is exactly what paid support is for. Relying on public forums support for production is not entirely recommended!

                Cheers,
                Manik

                • 5. Re: JMS
                  srinivas.chin

                  Hi manik,

                  Jboss version is 3.2.5. I am not sure about Jgroups version. I also used Mysql as Jms database to store messages in case of any trips.

                  Thanks
                  Srinivas

                  • 6. Re: JMS
                    srinivas.chin

                    To add further this is what is there in the deploy-hashsingleton-service.xml

                    <?xml version="1.0" encoding="UTF-8"?>
                    <!-- $Id: deploy-hasingleton-service.xml,v 1.1.2.2 2004/03/16 15:39:30 ejort Exp $ -->

                    <!-- ===================================================================== -->
                    <!-- -->
                    <!-- This service ensures that exactly one node in the cluster -->
                    <!-- deploys the services under "deploy-hasingleton" -->
                    <!-- -->
                    <!-- ===================================================================== -->



                    <!--
                    |
                    | Use the deployer service on the singleton node.
                    |
                    -->

                    jboss:service=DefaultPartition
                    <depends optional-attribute-name="TargetName">jboss.system:service=MainDeployer
                    deploy
                    ${jboss.server.home.url}/deploy-hasingleton
                    undeploy
                    ${jboss.server.home.url}/deploy-hasingleton



                    • 7. Re: JMS
                      srinivas.chin

                      Let me know how to get the JGroups version you wanted.

                      • 8. Re: JMS
                        manik

                        Hi,

                        java -classpath jgroups.jar org.jgroups.Version
                        java -classpath jboss-cache.jar org.jboss.cache.Version

                        Also, could you pls post your jboss cache config.

                        Thanks,
                        Manik

                        • 9. Re: JMS
                          srinivas.chin

                          C:\jboss-3.2.5\server\powersmarts\lib>java -classpath jgroups.jar org.jgroups.Version

                          Version: 2.2.4
                          CVS: $Id: Version.java,v 1.5 2004/04/28 18:44:58 belaban Exp $
                          History: (see doc/history.txt for details)


                          C:\jboss-3.2.5\server\powersmarts\lib>java -classpath jboss-cache.jar org.jboss.cache.Version
                          Exception in thread "main" java.lang.NoClassDefFoundError: org/jboss/cache/Version

                          C:\jboss-3.2.5\server\powersmarts\lib>java -classpath jboss-cache.jar org.jboss.cache.Version
                          Exception in thread "main" java.lang.NoClassDefFoundError: org/jboss/cache/Version

                          C:\jboss-3.2.5\server\powersmarts\lib>

                          • 10. Re: JMS
                            srinivas.chin

                            which location will this jboss cache config file located

                            • 11. Re: JMS
                              ovidiu.feodorov

                              JBoss 3.2.5 seems to be the problem in your case. A few critical HASingleton bugs have been fixed in 3.2.7. One of those bugs prevented the singleton to fail-over immediately, and it looks to me this is exactly what you're experiencing.

                              Before rushing to move to another application server, you should probably consider an upgrade first. The latest version is 4.0.3.

                              • 12. Re: JMS
                                srinivas.chin

                                Hi,

                                I already upgraded to 4.0.2RC on QA, and currently running my application on it, but I am facing another problem, when the server is stopped and my application still sends JMS messages to the JMS database queues. But when I restart the server I get the following exception and the messages are not consumed from the jms database. I have to stop the application and then remove the queues from the database and then run the server and application. the error is

                                20:58:11,557 INFO [UILServerILService] JBossMQ UIL service available at : /0.0.
                                0.0:8093
                                20:58:11,651 INFO [DLQ] Bound to JNDI name: queue/DLQ
                                20:58:11,714 INFO [RawDataQueue] Bound to JNDI name: queue/com.geps.gees.cs.pow
                                ersmarts.RawDataQueue
                                20:58:11,714 INFO [IR13Queue] Bound to JNDI name: queue/com.geps.gees.cs.powers
                                marts.IR13Queue
                                20:58:11,729 ERROR [IR49Queue] Starting failed jboss.mq.destination:service=Queu
                                e,name=com.geps.gees.cs.powersmarts.IR49Queue
                                org.jboss.mq.SpyJMSException: Could not restore messages to destination : QUEUE.
                                com.geps.gees.cs.powersmarts.IR49Queue; - nested throwable: (java.io.IOException
                                : Could not load the message: java.io.StreamCorruptedException: unexpected EOF i
                                n middle of data block)
                                at org.jboss.mq.pm.jdbc2.PersistenceManager.restoreQueue(PersistenceMana
                                ger.java:414)
                                at org.jboss.mq.server.JMSQueue.(JMSQueue.java:44)
                                at org.jboss.mq.server.jmx.Queue.startService(Queue.java:81)
                                at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanS
                                upport.java:271)
                                at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMB
                                eanSupport.java:221)
                                at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceControl
                                ler.java:837)
                                at $Proxy17.start(Unknown Source)
                                at org.jboss.system.ServiceController.start(ServiceController.java:367)
                                at org.jboss.system.ServiceController.start(ServiceController.java:389)
                                at org.jboss.system.ServiceController.start(ServiceController.java:389)
                                at org.jboss.system.ServiceController.start(ServiceController.java:389)
                                at org.jboss.system.ServiceController.start(ServiceController.java:389)
                                at org.jboss.system.ServiceController.start(ServiceController.java:389)
                                at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
                                at $Proxy4.start(Unknown Source)
                                at org.jboss.deployment.SARDeployer.start(SARDeployer.java:251)
                                at org.jboss.deployment.MainDeployer.start(MainDeployer.java:836)
                                at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:645)
                                at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:608)
                                at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
                                at $Proxy7.deploy(Unknown Source)
                                at org.jboss.deployment.scanner.URLDeploymentScanner.deploy(URLDeploymen
                                tScanner.java:304)
                                at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentS
                                canner.java:478)
                                at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.
                                doScan(AbstractDeploymentScanner.java:201)
                                at org.jboss.deployment.scanner.AbstractDeploymentScanner.startService(A
                                bstractDeploymentScanner.java:274)
                                at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanS
                                upport.java:271)
                                at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMB
                                eanSupport.java:221)
                                at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceControl
                                ler.java:837)
                                at $Proxy0.start(Unknown Source)
                                at org.jboss.system.ServiceController.start(ServiceController.java:367)
                                at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
                                at $Proxy4.start(Unknown Source)
                                at org.jboss.deployment.SARDeployer.start(SARDeployer.java:251)
                                at org.jboss.deployment.MainDeployer.start(MainDeployer.java:836)
                                at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:645)
                                at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:608)
                                at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:592)
                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
                                java:39)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
                                sorImpl.java:25)
                                at java.lang.reflect.Method.invoke(Method.java:324)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:60)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
                                at $Proxy5.deploy(Unknown Source)
                                at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:407)
                                at org.jboss.system.server.ServerImpl.start(ServerImpl.java:311)
                                at org.jboss.Main.boot(Main.java:145)
                                at org.jboss.Main$1.run(Main.java:399)
                                at java.lang.Thread.run(Thread.java:536)
                                Caused by: java.io.IOException: Could not load the message: java.io.StreamCorrup
                                tedException: unexpected EOF in middle of data block
                                at org.jboss.mq.pm.jdbc2.PersistenceManager.extractMessage(PersistenceMa
                                nager.java:495)
                                at org.jboss.mq.pm.jdbc2.PersistenceManager.restoreQueue(PersistenceMana
                                ger.java:401)
                                ... 98 more
                                20:58:11,854 WARN [ServiceController] Problem starting service jboss.mq.destina
                                tion:service=Queue,name=com.geps.gees.cs.powersmarts.IR49Queue
                                MBeanException: org.jboss.mq.SpyJMSException: Could not restore messages to dest
                                ination : QUEUE.com.geps.gees.cs.powersmarts.IR49Queue; - nested throwable: (jav
                                a.io.IOException: Could not load the message: java.io.StreamCorruptedException:
                                unexpected EOF in middle of data block)
                                Cause: org.jboss.mq.SpyJMSException: Could not restore messages to destination :
                                QUEUE.com.geps.gees.cs.powersmarts.IR49Queue; - nested throwable: (java.io.IOEx
                                ception: Could not load the message: java.io.StreamCorruptedException: unexpecte
                                d EOF in middle of data block)
                                at org.jboss.mx.server.ReflectedDispatcher.handleInvocationExceptions(Re
                                flectedDispatcher.java:91)
                                at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.
                                java:64)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:61)
                                at org.jboss.mx.server.Invocation.dispatch(Invocation.java:53)
                                at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
                                at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.
                                java:185)
                                at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
                                at org.j

                                • 13. Re: JMS
                                  mazz

                                  srinivas,

                                  > Does anyone from Jboss care to answer to this question pls.
                                  > I am facing this problem in Production servers

                                  As you can see, we'd love to help you out (manik and ovidieu are being true troopers here trying to help).

                                  But *please* keep in mind these forums are not actual support, provided for free and you may not always get as fast a reply as you'd like from JBoss folks. In fact, you may not get a JBoss reply at all.

                                  If you really need and desire JBoss to answer your support issues dealing with production servers in a timely manner (this seems to be what you want), may I suggest that you look into getting production support? I think you are only asking for trouble by relying on these free forums to help you work through production server problems. Again, they are free and you are not guaranteed to get a response that is timely and accurate.

                                  As much as we'd love to continue to help, we may have to go off and help our customers and you may be left without the quick help you need.

                                  • 14. Re: JMS
                                    darranl

                                    FYI 'RC' stands for release candidate - i.e. It might become a production release if no problems are found but then again it might not and a subsequent release candidate might be made.

                                    1 2 Previous Next