2 Replies Latest reply on May 18, 2017 6:53 AM by vvoola

    JBoss EAP 6.0.1 domain mode deployment hangs

    kclair-rei
    kclair-rei May 29, 2013 4:53 PM

      I'm running JBoss EAP 6.0.1 in domain mode on RHEL.

       

      When I deploy an application through the web console in the domain controller and then try to start the application, these are the last INFO level logs I see (some info x'ed out):

       

      [Server:cluster1] 13:14:55,072 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:jboss/OnlineDataStore]

      [Server:cluster1] 13:14:55,073 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) JBAS010400: Bound data source [java:jboss/DataWarehouse]

      [Server:cluster1] 13:14:55,079 INFO  [org.jboss.as.remoting] (MSC service thread 1-7) JBAS017100: Listening on xx.xx.xx.xx:4447

      [Server:cluster1] 13:14:55,080 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-9) JBAS015876: Starting deployment of "online-web-99999-SNAPSHOT.war"

      [Server:cluster1] 13:15:01,445 INFO  [org.jboss.as.osgi] (MSC service thread 1-14) JBAS011907: Register module: Module "deployment.online-web-99999-SNAPSHOT.war:main" from Service Module Loader

      [Host Controller] 13:15:24,525 INFO  [org.jboss.as.host.controller] (proxy-threads - 1) JBAS010919: Registering server cluster1

       

      After that there is nothing - the context never registers, and the application is not accessible.

       

      The TRACE level logs after this show a lot of messaging activity.

       

      The DEBUG level logs after this show a lot of activity from org.jboss.osgi.resolver and after that consistent PeriodicRecovery activity.

       

      I'm not sure how to figure out what the application deployment is doing.

       

      When I try to get a thread dump using jstack, I need to use the -F option.   The thread dump can be viewed here: 

       

      http://pastie.org/7981108

       

      There are quite a few blocked threads that look like they are waiting on unix processes and file reads.  I can see that it created a deployment directory in $JBOSS_HOME/domain/servers/cluster1/tmp/vfs/ where the jars are unpacked, so it appears to be reading the war file without a problem.

       

      strace shows that the processes are either stuck on FUTEX() or wait4().

       

      I'd love some advice about where else to look to figure out what is going on.