11 Replies Latest reply on Apr 21, 2015 2:23 PM by Jayashree Acharyya

    JBoss Shutdown Problems - Communication Exception

    Alex Grimnir Newbie

      I know this has been done to death, but I've read all the discussions mentioning the problem, and none of them have helped.

       

      I'm having a problem shutting down my jboss install (OpenKM). The VM runs fine and is started via init.d which runs run.sh -b 0.0.0.0. When I use the provided shutdown.sh (using ./shutdown.sh -s localhost:1099 -S), I get this error:

       

      [code]

      Exception in thread "main" javax.naming.CommunicationException: Could not obtain connection to any of these urls: localhost:1099 [Root exception is javax.naming.CommunicationException: Failed to connect to server localhost:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is java.net.SocketException: Network is unreachable]]]

              at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1562)

              at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:634)

              at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:627)

              at javax.naming.InitialContext.lookup(InitialContext.java:392)

              at org.jboss.Shutdown.main(Shutdown.java:214)

      Caused by: javax.naming.CommunicationException: Failed to connect to server localhost:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is java.net.SocketException: Network is unreachable]]

              at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:274)

              at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1533)

              ... 4 more

      Caused by: javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is java.net.SocketException: Network is unreachable]

              at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:248)

              ... 5 more

      Caused by: java.net.SocketException: Network is unreachable

              at java.net.PlainSocketImpl.socketConnect(Native Method)

              at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)

              at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)

              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)

              at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)

              at java.net.Socket.connect(Socket.java:529)

              at java.net.Socket.connect(Socket.java:478)

              at java.net.Socket.<init>(Socket.java:375)

              at java.net.Socket.<init>(Socket.java:276)

              at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:84)

              at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:77)

              at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:244)

              ... 5 more[/code]

       

      I've checked various files and the jmx console and made sure 1099 is the right port. Running lsof -i shows java as running 1098 BUT NOT 1099. netstat -an shows 0.0.0.0:1099 listening to 0.0.0.0:* (the same for 1098). jmx-console Naming service gives the port as 1099.

       

      On jboss 4.2.3.GA, Ubuntu 10.10 x64.

        • 1. Re: JBoss Shutdown Problems - Communication Exception
          Wolf-Dieter Fink Master

          Sounds as a network issue.

          Do you can ping localhost successful?

          • 2. Re: JBoss Shutdown Problems - Communication Exception
            Alex Grimnir Newbie

            PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=0.026 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=0.030 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=0.041 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=4 ttl=64 time=0.031 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=5 ttl=64 time=0.040 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=6 ttl=64 time=0.028 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=7 ttl=64 time=0.034 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=8 ttl=64 time=0.040 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=9 ttl=64 time=0.036 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=10 ttl=64 time=0.032 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=11 ttl=64 time=0.029 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=12 ttl=64 time=0.031 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=13 ttl=64 time=0.035 ms

            64 bytes from localhost.localdomain (127.0.0.1): icmp_req=14 ttl=64 time=0.027 ms

             

            --- localhost.localdomain ping statistics ---

            14 packets transmitted, 14 received, 0% packet loss, time 13002ms

            rtt min/avg/max/mdev = 0.026/0.032/0.041/0.008 ms

             

             

            Ping seems to work fine. That is from the terminal, and I can connect to the server from outside (I'm running OpenKM on jboss as we speak).

            • 3. Re: JBoss Shutdown Problems - Communication Exception
              Wolf-Dieter Fink Master

              does the script work without using "-s" oprion or use 127.0.0.1 or the real ip address?

              • 4. Re: JBoss Shutdown Problems - Communication Exception
                Alex Grimnir Newbie

                ./shutdown.sh -S does not work (same error), nor does ./shutdown.sh -s 127.0.0.1:1099 -S or ./shutdown.sh -s REALIP:1099 -S

                 

                All of those give the same error (just the reported server differs obviously).

                • 5. Re: JBoss Shutdown Problems - Communication Exception
                  Peter Johnson Master

                  Please post the results from running:

                  sudo netstat -anp | grep java

                  Also, try running this an post the response:

                  telnet localhost 1099

                  See this prior post on using telnet to debug hostname errors: https://community.jboss.org/thread/120995?tstart=0#409029

                  • 6. Re: JBoss Shutdown Problems - Communication Exception
                    Alex Grimnir Newbie

                    tcp   

                    0 0.0.0.0:3873        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:48360       0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:8009        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:1098        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:1099        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:8080        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:8083        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:60980       0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:44667       0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:4444        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:8093        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:4445        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 0.0.0.0:4446        0.0.0.0:*           LISTEN  5782/java
                    tcp    0 127.0.0.1:55279     127.0.0.1:2002      ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:48718     ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:54597     ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:36501     ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:36825     ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:44485     ESTABLISHED 5782/java
                    tcp    0 127.0.0.1:8009      127.0.0.1:42740     ESTABLISHED 5782/java
                    unix  2  [ ]     STREAM CONNECTED 872326002 5782/java

                     

                    telnet localhost 1099 -

                     

                    Trying ::1...

                    Trying 127.0.0.1...

                    Connected to localhost.

                    Escape character is '^]'.

                    ¬ísrjava.rmi.MarshalledObject|½ícü>IhashlocBytest[BobjBytesq~xpöuª¥ur[B¬óTàxp'¬íthttp://dplibrary:8083/q~q~uq~À¬ísr org.jnp.server.NamingServer_Stubxrjava.rmi.server.RemoteStubéþÜÉáexrjava.rmi.server.RemoteObjectÓa´

                    a3xpw4

                          UnicastRef2       dplibraryJ®¬ã3,´&8!8äÍxConnection closed by foreign host.

                     

                    I've checked that thread (dplibrary is the hostname of my server) and my /etc/hosts file looks very similar to the one you posted on 01NOV2006 -

                     

                    ::1 localhost.localdomain localhost

                    127.0.0.1 localhost.localdomain localhost

                    # Auto-generated hostname. Please do not remove this comment.

                    REALIP dplibrary

                    • 7. Re: JBoss Shutdown Problems - Communication Exception
                      Peter Johnson Master

                      Things I would try and look into:

                       

                      Try running: telnet dplibrary 1099

                      If that gets a connection error, then look at your firewall settings.

                       

                      Are you running SELinux? If so, try disabling it and try again. If it works after disabling SELinux, then you need to figure out which SE:Linux setting is preventing the connection.

                       

                      I have found this blog post to be helpful on configuring the hostname: http://jblevins.org/log/hostname

                      • 8. Re: JBoss Shutdown Problems - Communication Exception
                        Alex Grimnir Newbie

                        telnet dplibrary 1099 yields the same response as the previous post (garbled junk). I'm pretty sure I'm not running SELinux (it isn't marked as installed in aptitude).

                        • 9. Re: JBoss Shutdown Problems - Communication Exception
                          Peter Johnson Master

                          Well, I'm out of ideas. Looks like you are going to have to run shutdown within a debugger to get to the root cause. You'll need the sources for the JVM runtime since that is where the error is happening. If you downloaded the Oracle JDK, it came with the sources (look for src.zip). If you installed the JDK from Ubuntu package manager, you'll have to dig to find out if it came with the sources or if that is a separate package.

                          • 10. Re: JBoss Shutdown Problems - Communication Exception
                            Alex Grimnir Newbie

                            Hmmm, alrighty then. Thanks for all your help anyway guys; I'll let you know how that goes once I work out what the hell I'm doing (linux newbie).