1 2 3 Previous Next 33 Replies Latest reply on Jun 11, 2008 8:50 AM by Dimitris Andreadis

    HDScanner exception

    Emanuel Muckenhuber Master

      Using AS trunk (rev. 74306) the HDScanner fails after while (lets say between 2 and 5 minutes idle) with this Exception:

      2008-06-09 13:27:34,689 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
      java.lang.NullPointerException
       at java.util.Arrays$ArrayList.<init>(Arrays.java:2355)
       at java.util.Arrays.asList(Arrays.java:2341)
       at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:67)
       at sun.net.www.protocol.file.FileURLConnection.initializeHeaders(FileURLConnection.java:90)
       at sun.net.www.protocol.file.FileURLConnection.getLastModified(FileURLConnection.java:151)
       at org.jboss.virtual.plugins.context.AbstractURLHandler.exists(AbstractURLHandler.java:119)
       at org.jboss.virtual.VirtualFile.exists(VirtualFile.java:179)
       at org.jboss.system.server.profile.basic.ProfileImpl.getModifiedDeployments(ProfileImpl.java:308)
       at org.jboss.system.server.profileservice.hotdeploy.HDScanner.scan(HDScanner.java:260)
       at org.jboss.system.server.profileservice.hotdeploy.HDScanner.run(HDScanner.java:221)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
       at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
       at java.lang.Thread.run(Thread.java:595)
      


      Deployment does not seem to work after that anymore.
      Furthermore running the testsuite with one-test ends up with in the following:

      2008-06-09 13:27:44,429 ERROR [org.jboss.naming.Naming] (JBoss System Threads(1)-2) Naming accept handler stopping
      java.net.SocketException: Too many open files
       at java.net.PlainSocketImpl.socketAccept(Native Method)
       at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
       at java.net.ServerSocket.implAccept(ServerSocket.java:450)
       at java.net.ServerSocket.accept(ServerSocket.java:421)
       at org.jnp.server.Main$AcceptHandler.run(Main.java:482)
       at org.jboss.util.threadpool.RunnableTaskWrapper.run(RunnableTaskWrapper.java:148)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
       at java.lang.Thread.run(Thread.java:595)
      


      Someone else experiencing the same problem? Any hints?

      Thanks,
      Emanuel

        • 1. Re: HDScanner exception
          Dimitris Andreadis Master

          It must be due to the update to jboss-common-core 2.2.6.GA that includes this: http://jira.jboss.com/jira/browse/JBCOMMON-55

          We have dropped our org.jboss.net.protocol.file.FileURLConnection in favour of sun's implementation, but it seems it has problems?

          What OS/JDK are you on? We need to scan sun's bug database.

          • 2. Re: HDScanner exception
            Emanuel Muckenhuber Master

            Hmm my OS is Fedora 9.
            And i'm using: java version "1.5.0_15"
            Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
            Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_15-b04, mixed mode)

            But i also tried other sun jvm's on my box (the 1.5.0_15 32Bit, and 1.5.0_09 64Bit).
            Even tested it on a different machine running F9 and on Fedora 8 with 1.5.0_14-b03 64bit, all with the same result.

            I still can try it on my linux box at home - but that's why i wanted to ask if someone else is experiencing the same. I guess if you don't have the same error, it should work on windows :)





            • 3. Re: HDScanner exception
              Scott Stark Master

              I'm not seeing this on osx with the current trunk (rev 74311). I'm not finding the sun.net.www.protocol.file.FileURLConnection source on osx so I don't know what the NPE is coming from.

              • 4. Re: HDScanner exception
                Scott Stark Master

                The server has been running for a couple of hours now with no increase in file descriptors or ports. I suspect its the "Too many open files" causing the issues. What are the ulimits your running under?

                If you have an lsof output you can send it to me.

                • 5. Re: HDScanner exception
                  Anil Saldanha Master

                  Scott, this is an hindrance to development on F9 at the moment.

                  • 6. Re: HDScanner exception
                    Scott Stark Master

                    Well, I'll see if I can get f9 installed on a box here to see if I can reproduce it.

                    • 7. Re: HDScanner exception
                      Anil Saldanha Master

                      All you have to do is start AS5 and wait a minute or so.... So it is only in F9?

                      • 8. Re: HDScanner exception
                        Anil Saldanha Master

                        Use GParted (free from sf.net) to quickly create a partition your disk and do an install of F9 over the network (if possible). This may get things faster for you.

                        For now, we just bounce the AS frequently (a minute delay to recycle which is fine).

                        • 9. Re: HDScanner exception
                          Emanuel Muckenhuber Master

                           

                          "scott.stark@jboss.org" wrote:
                          The server has been running for a couple of hours now with no increase in file descriptors or ports. I suspect its the "Too many open files" causing the issues. What are the ulimits your running under?

                          ulimit is unlimited


                          If you have an lsof output you can send it to me.

                          on the way

                          • 10. Re: HDScanner exception
                            Emanuel Muckenhuber Master

                             

                            "anil.saldhana@jboss.com" wrote:
                            All you have to do is start AS5 and wait a minute or so.... So it is only in F9?

                            Hmm i also tried it on Fedora 8 with the same result

                            • 11. Re: HDScanner exception
                              Scott Stark Master

                               

                              "anil.saldhana@jboss.com" wrote:
                              All you have to do is start AS5 and wait a minute or so.... So it is only in F9?


                              Apparently. I have a default AS5 instance that has been running for ~6 hours under osx without any change in descriptor/port counts.


                              • 12. Re: HDScanner exception
                                Anil Saldanha Master

                                 

                                "scott.stark@jboss.org" wrote:
                                "anil.saldhana@jboss.com" wrote:
                                All you have to do is start AS5 and wait a minute or so.... So it is only in F9?


                                Apparently. I have a default AS5 instance that has been running for ~6 hours under osx without any change in descriptor/port counts.


                                No Scott. Emmanuel answered my Query. I was wondering about Fedora7/8 etc. So it is probably a bug in Sun JDK. OSX has its own impl. Let me try locate the class where the NPE is happening in JDK.

                                • 13. Re: HDScanner exception
                                  Scott Stark Master

                                  From Emanuel's lsof I see he is running the all config. I have problems with that starting up cleanly for some reason due to jboss messaging config issues. I'll start it up to see if its showing this problem over time as well.

                                  I see duplicate files open in his trace though from the all/deploy/cluster directory, for example the cluster-beans.xml:

                                  [522][valkyrie: /home/svn/JBossAS]$ grep cluster-beans.xml /tmp/lsof.output
                                  java 4422 emuckenh 98r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 114r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 153r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 185r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 217r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 249r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 281r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 313r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 345r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 377r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 409r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 441r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 473r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 505r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 537r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 569r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 601r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 633r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 665r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 697r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 729r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 761r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 793r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 827r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 859r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 891r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 923r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 955r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 987r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  java 4422 emuckenh 1019r REG 253,0 3202 124143 /home/emuckenh/workspace/as/trunk/build/output/jboss-5.0.0.CR1/server/all/deploy/cluster/cluster-beans.xml
                                  [523][valkyrie: /home/svn/JBossAS]$
                                  



                                  • 14. Re: HDScanner exception
                                    Anil Saldanha Master

                                    It is happening in default also.

                                    1 2 3 Previous Next