13 Replies Latest reply on Jul 26, 2012 10:29 AM by peterj

    Freeze and Server killed without any logs

    erache

      Hello everyone,

      I have a weird problem on my server. I tried an EAR on my computer, it works well and on a virtual machine, it doesn't work...

      So this is the configuration of my computer
      JBoss 5
      Windows
      Java 6

      And this is the configuration of the virtual machine
      JBoss 5 (I tried 5.0.0 and 5.0.1 and same result)
      Linux SMP 64 bits
      Java 6

      The EAR contains a jboss-seam project. So there is an EJB, a WAR and the jboss-seam.jar.

      On the linux machine, it deploys normally at the start... and just before deploying the WAR, it freezes and I have just this output :

      - KILLED

      .....
      12:12:21,634 INFO [SessionSpecContainer] Starting jboss.j2ee:ear=FPP-ear.ear,jar=jboss-seam-2.1.1.GA.jar,name=TimerServiceDispatcher,service=EJB3
      12:12:21,636 INFO [EJBContainer] STARTED EJB: org.jboss.seam.async.TimerServiceDispatcher ejbName: TimerServiceDispatcher
      12:12:21,647 INFO [JndiSessionRegistrarBase] Binding the following Entries in Global JNDI:
      
       FPP-ear/TimerServiceDispatcher/local - EJB3.x Default Local Business Interface
       FPP-ear/TimerServiceDispatcher/local-org.jboss.seam.async.LocalTimerServiceDispatcher - EJB3.x Local Business Interface
      
      12:12:21,911 INFO [SessionSpecContainer] Starting jboss.j2ee:ear=FPP-ear.ear,jar=jboss-seam-2.1.1.GA.jar,name=EjbSynchronizations,service=EJB3
      12:12:21,912 INFO [EJBContainer] STARTED EJB: org.jboss.seam.transaction.EjbSynchronizations ejbName: EjbSynchronizations
      12:12:21,925 INFO [JndiSessionRegistrarBase] Binding the following Entries in Global JNDI:
      
       FPP-ear/EjbSynchronizations/local - EJB3.x Default Local Business Interface
       FPP-ear/EjbSynchronizations/local-org.jboss.seam.transaction.LocalEjbSynchronizations - EJB3.x Local Business Interface
      
      12:12:40,871 INFO [TomcatDeployment] deploy, ctxPath=/fpp
      12:12:42,049 INFO [ContextConfig] WARNING: Security role name NONE used in an <auth-constraint> without being defined in a <security-role>
      12:12:49,487 INFO [ServletContextListener] Welcome to Seam 2.1.1.GA
      Killed
      ***@***:/apps/jboss-5.0.1.GA/bin#
      



      and nothing more, no stackTrace , no log...

      I don't really know what it can be ? what I can do ?
      If you any ideas, I would be grateful :). Thanks in advance :)

        • 1. Re: Freeze and Server killed without any logs
          peterj

          What is your virtual machines configuration - how much RAM are you allocating to it?

          What are your heap settings? Try setting heap min and max to the same value.

          What is Linux SMP? By SMP do you mean Symetric Multiprocessing? So are you saying that you configured your virtual machine with multiple processors? How many? How many processors does the host machine have?

          Which Linux distro are you using?

          What system metrics have you monitored so far? Memory usage? Processor utilization? Disk free space?

          • 2. Re: Freeze and Server killed without any logs
            erache

            I don't really know the configuration of the virtual machine. I just know the information by accessing by ssh:

            uname -a:

            Linux *** 2.6.27-11-server #1 SMP Thu Jan 29 20:13:12 UTC 2009 x86_64 GNU/Linux
            


            So yes it's a multiprocessing machine


            free:
             total used free shared buffers cached
            Mem: 1024224 175512 848712 0 3992 33020
            -/+ buffers/cache: 138500 885724
            Swap: 0 0 0
            


            So I think I don't really have a problem for memory usage

            And my JVM args are:
            JBoss Bootstrap Environment
            
             JBOSS_HOME: /apps/jboss-5.0.1.GA
            
             JAVA: /usr/lib/jvm/java-6-sun/bin/java
            
             JAVA_OPTS: -Dprogram.name=run.sh -server -Xms256m -Xmx1024m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true
            
             CLASSPATH: /apps/jboss-5.0.1.GA/bin/run.jar:/usr/lib/jvm/java-6-sun/lib/tools.jar
            


            Sorry if I didn't give much information first... but actually, I didn't know what to do exactly... Because I don't have much information... no logs... no trace :'(

            So I will have a look to the memory usage during launching... Do you know a good tool to watch the jvm memory usage ?!


            • 3. Re: Freeze and Server killed without any logs
              peterj

              Did you run 'free' with JBossAS running or stopped?

              Your max heap size is 1GB which end up using all of the RAM - how much swap space do you have?

              Based on your current free memory, assuming no swap, I would try reducing maxpermsize to 128m and set min and max heap to 512m - that should prevent all of the RAM from being used.

              Use 'top' to monitor java's memory usage as you deploy and run your app.

              • 4. Re: Freeze and Server killed without any logs
                erache

                I did what you told me... It goes further than last time but it still dies after a while... just annoying

                Here is what I get during deployment:

                top:

                Cpu0 : 1.2%us, 57.3%sy, 0.0%ni, 0.0%id, 39.5%wa, 2.0%hi, 0.0%si, 0.0%st
                Mem: 1024224k total, 1015156k used, 9068k free, 60k buffers
                Swap: 0k total, 0k used, 0k free, 3420k cached
                
                19656 root 20 0 1204m 840m 488 S 54.4 84.0 2:39.46 java
                ...
                


                The %MEM is not going up than 88%..



                SO you think it's a memory problem ? Why didn't it just say in the trace or log OutOfMemory...???

                Anyway thanks for helping me :)

                • 5. Re: Freeze and Server killed without any logs
                  peterj

                  Yes, it is memory. There is no OutOfMemoryException because it is not the Java heap running out of memory - it is the Linux VM. Looking again at free, I see that you have no swap space. I suspect that the OS is not letting you use that last 9MB of RAM because that is the buffer it needs to let administrators interact with the OS. Thus when the JVM asks for too much memory, the oS is killing it.

                  I already gave you the information you needed to "fix" this, so here is your pop quiz: what should you try next?

                  • 6. Re: Freeze and Server killed without any logs
                    erache

                    I would like to thank you for the explanation.

                    I finally made it with less memory ! So now I use these arguments:

                    -Xms256m
                    -Xmx256m
                    -XX:MaxPermSize=128m

                    And it works fine for the moment ^^

                    But the weird thing is that the memory used is still higher than the memory I allocated to the process:

                    total used free shared buffers cached
                    Mem: 1024224 1014552 9672 0 1740 191352
                    -/+ buffers/cache: 821460 202764
                    Swap: 0 0 0
                    


                    But it works !!

                    So thank you very much for your help :-)

                    • 7. Re: Freeze and Server killed without any logs
                      peterj

                       

                      But the weird thing is that the memory used is still higher than the memory I allocated to the process:


                      Where did you get the idea that the heap and the permgen are that only memory used by a Java process? There is also the JVM code, the Java stacks, and the C/C++ data structures used by the JVM.


                      • 8. Re: Freeze and Server killed without any logs
                        erache

                        I didn't know it could take this much space... ^^

                        I don't really know the allocation of memory in a jvm but I thought the most important memory allocation was by the heap and the permgen ...

                        I tried on a windows and when I specified 512 for heap space and 128 for permgen space, the process allocated ~650M but not more... On linux, it took almost ~800M.

                        • 9. Re: Freeze and Server killed without any logs
                          peterj

                          That is because Windows and Linux count memory usage differently, and both of them lie to some extent. Well, the numbers are correct but they often don't mean what you think they mean.

                          • 10. Re: Freeze and Server killed without any logs

                            Hi Peter, We are facing the same issue...Our linux configuration is as below

                             

                             

                            Untitled.png

                             

                            Pls help us thanks in advance..

                             

                            Ashutosh

                            • 11. Re: Freeze and Server killed without any logs
                              peterj

                              Please look back over this discussion - I asked the OP several questions, please answer those same questions. It would also help if you described in detail the issue you are having.

                              • 12. Re: Freeze and Server killed without any logs

                                Hi Peter

                                 

                                Below are the detail info of our system, Let me know if you need anything else for diagnosis the problem-

                                 

                                We have set the virtual memory system manage (automatic).

                                SMP is enable on this Host. We have given 2 processor & 8 GB RAM to this Server.

                                Linux distorts CentOS ,JDK1.5.13,JBOSS 5.0.1 GA

                                For System monitor we are using top command. Till 5.00 load average means normal & above 5.00 load average means abnormal.

                                 

                                top - 13:22:55 up 2 days, 17:04,  1 user,  load average: 0.00, 0.04, 0.06

                                Tasks: 145 total,   1 running, 144 sleeping,   0 stopped,   0 zombie

                                Cpu(s): 28.1%us,  0.7%sy,  0.0%ni, 71.0%id,  0.0%wa, 0.0%hi,  0.2%si,  0.0%st

                                Mem: 8055140k total,  4055984k used,  3999156k free, 283852k buffers

                                Swap:  5931000k total,        0k used,  5931000k free, 1424272k cached

                                 

                                And JAVA heap and other info below-

                                 

                                JAVA_OPTS="-Xms1536m -Xmx1536m -XX:NewSize=256m -XX:MaxNewSize=356m -XX:PermSize=256m -XX:MaxPermSize=356m -server

                                  -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:+UseParallelGC


                                • 13. Re: Freeze and Server killed without any logs
                                  peterj

                                  You never responded to the last sentence of my prior post.