8 Replies Latest reply on Oct 11, 2013 11:56 AM by razmikz

    JBOSS 7.0 Deployment failure

    razmikz

      Hi

      I'm new to Jboss. We have a Windows 2008 R2 server that I used to install Jboss 7.0. I have added the following to the servers Environment Variable C:\Program Files\Java\jdk1.7.0\bin

       

      Our developer has created a .war file that we have copied to the /standalone/deployments folder.

      When we start the instance (standalone.bat -b 0.0.0.0), within 2 seconds the deployment fails and in the /deployments folder we have a file  filename.war.failed.

      As soon as we delete this file the deployment succeeds. Everytime we have a new .WAR file, or everytime we stop the site and start it we see the same thing.

       

      I don't know much about the deployment process but is it possible that Jboss attempts to deploy before some conditions are met? And a few seconds later when we delete the filename.war.failed file the conditions are correct and deployment succeeds?

       

      Here is a typical log of the failed deployment. I would appreciate any help.

       

       

      D:\Test_Jboss\bin>standalone.bat -b 10.238.57.133
      Calling "D:\Test_Jboss\bin\standalone.conf.bat"
      '#' is not recognized as an internal or external command,
      operable program or batch file.
      JAVA_HOME is not set. Unexpected results may occur.
      Set JAVA_HOME to the directory of your local JDK to avoid this message.
      ===============================================================================

        JBoss Bootstrap Environment

        JBOSS_HOME: D:\Test_Jboss

        JAVA: java

        JAVA_OPTS: -XX:+TieredCompilation -Dprogram.name=standalone.bat -Xms64M -Xmx51
      2M -XX:MaxPermSize=256M -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.se
      rver.gcInterval=3600000 -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.war
      ning=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djboss.server.default.c
      onfig=standalone.xml -Djboss.bind.address.management=10.238.57.133 -Djboss.bind.
      address=10.238.57.133

      ===============================================================================

      16:14:19,875 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
      16:14:20,875 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
      16:14:20,937 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.1.Final "Brontes" sta
      rting
      16:14:24,359 INFO  [org.xnio] XNIO Version 3.0.3.GA
      16:14:24,375 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.3.GA
      16:14:24,406 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.3.GA
      16:14:24,359 INFO  [org.jboss.as.server] JBAS015888: Creating http management se
      rvice using socket-binding (management-http)
      16:14:24,641 INFO  [org.jboss.as.configadmin] JBAS016200: Activating ConfigAdmin
      Subsystem
      16:14:24,672 INFO  [org.jboss.as.clustering.infinispan] JBAS010280: Activating I
      nfinispan subsystem.
      16:14:24,656 INFO  [org.jboss.as.osgi] JBAS011940: Activating OSGi Subsystem
      16:14:24,703 INFO  [org.jboss.as.naming] JBAS011800: Activating Naming Subsystem

      16:14:24,688 INFO  [org.jboss.as.webservices] JBAS015537: Activating WebServices
      Extension
      16:14:24,688 INFO  [org.jboss.as.security] JBAS013101: Activating Security Subsy
      stem
      16:14:24,781 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap log han
      dlers
      16:14:24,844 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: St
      arting Naming Service
      16:14:24,875 INFO  [org.jboss.as.security] (MSC service thread 1-3) JBAS013100:
      Current PicketBox version=4.0.7.Final
      16:14:25,000 INFO  [org.jboss.as.connector] (MSC service thread 1-1) JBAS010408:
      Starting JCA Subsystem (JBoss IronJacamar 1.0.9.Final)
      16:14:25,219 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-3) JBAS01
      5400: Bound mail session [java:jboss/mail/Default]
      16:14:25,297 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerServic
      e Thread Pool -- 27) JBAS010403: Deploying JDBC-compliant driver class org.h2.Dr
      iver (version 1.3)
      16:14:25,891 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread
      1-4) Starting Coyote HTTP/1.1 on http--10.238.57.133-8180
      16:14:26,047 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service
      thread 1-2) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
      16:14:26,953 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC se
      rvice thread 1-1) JBoss Web Services - Stack CXF Server 4.0.2.GA
      16:14:27,407 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread
      1-1) JBAS015012: Started FileSystemDeploymentService for directory D:\Test_Jboss
      \standalone\deployments
      16:14:27,469 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS015014: Re-attempting failed deployment OrangeCountyDemo (2).war

      16:14:27,485 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS015014: Re-attempting failed deployment OrangeCountyDemo.war
      16:14:27,485 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100:
      Listening on /10.238.57.133:10000
      16:14:27,485 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS015014: Re-attempting failed deployment TestOrangeCountyDemo.war

      16:14:27,485 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100:
      Listening on /10.238.57.133:4448
      16:14:27,532 ERROR [org.jboss.as.controller.management-operation] (DeploymentSca
      nner-threads - 2) Operation ("composite") failed - address: (undefined) - failur
      e description: "JBAS014719: Invalid value specification OrangeCountyDemo (2).war
      "
      16:14:27,532 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS014719: Invalid value specification OrangeCountyDemo (2).war
      16:14:27,547 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) undefined
      16:14:27,547 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS015004: Caught exception writing deployment marker file D:\Test_
      Jboss\standalone\deployments\Archive\OrangeCountyDemo.war.failed: java.io.FileNo
      tFoundException: D:\Test_Jboss\standalone\deployments\Archive\OrangeCountyDemo.w
      ar.failed (Access is denied)
              at java.io.FileOutputStream.open(Native Method) [rt.jar:1.7.0_17]
              at java.io.FileOutputStream.<init>(Unknown Source) [rt.jar:1.7.0_17]
              at java.io.FileOutputStream.<init>(Unknown Source) [rt.jar:1.7.0_17]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.wr
      iteFailedMarker(FileSystemDeploymentService.java:916) [jboss-as-deployment-scann
      er-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.ac
      cess$3100(FileSystemDeploymentService.java:88) [jboss-as-deployment-scanner-7.1.
      1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$De
      ployTask.handleFailureResult(FileSystemDeploymentService.java:1056) [jboss-as-de
      ployment-scanner-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.sc
      an(FileSystemDeploymentService.java:447) [jboss-as-deployment-scanner-7.1.1.Fina
      l.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$De
      ploymentScanRunnable.run(FileSystemDeploymentService.java:149) [jboss-as-deploym
      ent-scanner-7.1.1.Final.jar:7.1.1.Final]
              at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [
      rt.jar:1.7.0_17]
              at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
      [rt.jar:1.7.0_17]
              at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [rt.jar:1
      .7.0_17]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
      access$301(Unknown Source) [rt.jar:1.7.0_17]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
      run(Unknown Source) [rt.jar:1.7.0_17]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [rt
      .jar:1.7.0_17]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [r
      t.jar:1.7.0_17]
              at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_17]
              at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-thread
      s-2.0.0.GA.jar:2.0.0.GA]

      16:14:27,563 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) undefined
      16:14:27,563 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) JBAS015004: Caught exception writing deployment marker file D:\Test_
      Jboss\standalone\deployments\Archive\TestOrangeCountyDemo.war.failed: java.io.Fi
      leNotFoundException: D:\Test_Jboss\standalone\deployments\Archive\TestOrangeCoun
      tyDemo.war.failed (Access is denied)
              at java.io.FileOutputStream.open(Native Method) [rt.jar:1.7.0_17]
              at java.io.FileOutputStream.<init>(Unknown Source) [rt.jar:1.7.0_17]
              at java.io.FileOutputStream.<init>(Unknown Source) [rt.jar:1.7.0_17]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.wr
      iteFailedMarker(FileSystemDeploymentService.java:916) [jboss-as-deployment-scann
      er-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.ac
      cess$3100(FileSystemDeploymentService.java:88) [jboss-as-deployment-scanner-7.1.
      1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$De
      ployTask.handleFailureResult(FileSystemDeploymentService.java:1056) [jboss-as-de
      ployment-scanner-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.sc
      an(FileSystemDeploymentService.java:447) [jboss-as-deployment-scanner-7.1.1.Fina
      l.jar:7.1.1.Final]
              at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$De
      ploymentScanRunnable.run(FileSystemDeploymentService.java:149) [jboss-as-deploym
      ent-scanner-7.1.1.Final.jar:7.1.1.Final]
              at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [
      rt.jar:1.7.0_17]
              at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
      [rt.jar:1.7.0_17]
              at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [rt.jar:1
      .7.0_17]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
      access$301(Unknown Source) [rt.jar:1.7.0_17]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
      run(Unknown Source) [rt.jar:1.7.0_17]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [rt
      .jar:1.7.0_17]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [r
      t.jar:1.7.0_17]
              at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_17]
              at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-thread
      s-2.0.0.GA.jar:2.0.0.GA]

      16:14:27,563 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-t
      hreads - 1) undefined
      16:14:27,907 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774
      : Service status report
      JBAS014777:   Services which failed to start:      service jboss.logging.handler
      .FILE: org.jboss.msc.service.StartException in service jboss.logging.handler.FIL
      E: java.io.FileNotFoundException: D:\Test_Jboss\standalone\log\server.log (Acces
      s is denied)

      16:14:27,922 ERROR [org.jboss.as.server] (Controller Boot Thread) JBAS015956: Ca
      ught exception during boot: java.lang.IllegalStateException: JBAS014643: Could n
      ot delete D:\Test_Jboss\standalone\configuration\standalone_xml_history\20130503
      -110307226\standalone.v1.xml
              at org.jboss.as.controller.persistence.ConfigurationFile.deleteRecursive
      (ConfigurationFile.java:441) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.controller.persistence.ConfigurationFile.deleteRecursive
      (ConfigurationFile.java:437) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.controller.persistence.ConfigurationFile.createHistoryDi
      rectory(ConfigurationFile.java:422) [jboss-as-controller-7.1.1.Final.jar:7.1.1.F
      inal]
              at org.jboss.as.controller.persistence.ConfigurationFile.successfulBoot(
      ConfigurationFile.java:273) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.controller.persistence.BackupXmlConfigurationPersister.s
      uccessfulBoot(BackupXmlConfigurationPersister.java:65) [jboss-as-controller-7.1.
      1.Final.jar:7.1.1.Final]
              at org.jboss.as.controller.AbstractControllerService.finishBoot(Abstract
      ControllerService.java:195) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.server.ServerService.boot(ServerService.java:268) [jboss
      -as-server-7.1.1.Final.jar:7.1.1.Final]
              at org.jboss.as.controller.AbstractControllerService$1.run(AbstractContr
      ollerService.java:155) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
              at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_17]

      16:14:27,938 FATAL [org.jboss.as.server] (Controller Boot Thread) JBAS015957: Se
      rver boot has failed in an unrecoverable manner; exiting. See previous messages
      for details.
      16:14:27,969 INFO  [org.jboss.as.osgi] JBAS011942: Stopping OSGi Framework
      16:14:27,969 INFO  [org.jboss.as.logging] JBAS011503: Restored bootstrap log han
      dlers
      16:14:28,079 INFO  [com.arjuna.ats.jbossatx] ARJUNA032018: Destroying Transactio
      nManagerService
      16:14:28,079 INFO  [com.arjuna.ats.jbossatx] ARJUNA032014: Stopping transaction
      recovery manager
      16:14:28,094 INFO  [org.apache.catalina.core.StandardContext] Container org.apac
      he.catalina.core.ContainerBase.[jboss.web].[default-host].[/] has not been start
      ed
      16:14:28,110 INFO  [org.jboss.as] JBAS015950: JBoss AS 7.1.1.Final "Brontes" sto
      pped in 145ms
      Press any key to continue . . .

        • 1. Re: JBOSS 7.0 Deployment failure
          ramsrib

          Things I noticed here is :

           

          You've trying to deploy two war files which is present in "D:\Test_Jboss\standalone\deployments"

          * OrangeCountyDemo (2).war

          * TestOrangeCountyDemo.war

           

          From the below log entries,

          >> 16:14:27,532 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 2) Operation ("composite") failed - address: (undefined) - failure description: "JBAS014719: Invalid value specification OrangeCountyDemo (2).war"

           

          Here deployment failed for "OrangeCountyDemo (2).war" file, please avoid white spaces in the deployment war file name. i.e., Rename the deployment file to "OrangeCountyDemo.war".

           

          After the above deployment failed, jboss will try to create a .failed marker file for that deployment module. But it seems that you don't have write access to those directories.

           

          All the below log entries says "Access is Denied",

          >> 16:14:27,547 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015004: Caught exception writing deployment marker file D:\Test_Jboss\standalone\deployments\Archive\OrangeCountyDemo.war.failed: java.io.FileNotFoundException: D:\Test_Jboss\standalone\deployments\Archive\OrangeCountyDemo.war.failed (Access is denied)

           

          >> 16:14:27,563 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015004: Caught exception writing deployment marker file D:\Test_Jboss\standalone\deployments\Archive\TestOrangeCountyDemo.war.failed: java.io.FileNotFoundException: D:\Test_Jboss\standalone\deployments\Archive\TestOrangeCountyDemo.war.failed (Access is denied)

           

          >> 16:14:27,907 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report JBAS014777:   Services which failed to start:  service jboss.logging.handler.FILE: org.jboss.msc.service.StartException in service jboss.logging.handler.FILE: java.io.FileNotFoundException: D:\Test_Jboss\standalone\log\server.log (Access is denied)

           

          Please fix the permissions issue by giving recursive write access to "D:\Test_Jboss\standalone" directory.

          • 2. Re: JBOSS 7.0 Deployment failure
            razmikz

            Hi Sri. Thank you for replying. Your notes were very helpful.

             

            1- The Windows Server has 3 instances of the Jboss (Development = C:\Jboss, TEST = D:\TEST_Jboss, Production = D:\PROD_Jboss).

            2- The example I had provided is the D:\TEST_Jboss\standalone\deployments

             

            As you noticed in the logs, I had created a folder called Archive under D:\TEST_Jboss\standalone\deployments to keep older versions of the WAR files. After reading your response, I realized that Jboss maybe scanning the /deployment folder and all subfolders. So I removed the Archive folder. Started this TEST instance and the deployment succeeded without errors (Thank you very much for pointing that our to me).

             

            Question 1- Why would the deployment succeed after deleting the ".failed" file in the D:\TEST_Jboss\standalone\deployments without touching the files in the Archive folder?

             

            More importantly, what happens now when I stop the TEST instance (The deployed file remains in the D:\TEST_Jboss\standalone\deployments folder) and attempt to restart the Jboss? I just tried this and noticed that the deployed war file extension changed to ".dodeploy" and a new file got created with ".isdeploying" extension. Nothing else happens. The site is not up.

             

            Question 2- What's the recommended method of stopping the Jboss instance so that the existing deployed WAR file is removed making it possible for the next Jboss Start-up to deploy the WAR file without errors and issues?

             

            I have not modified the security/permissions. I'll stay with one change for now and monitor the system for few days and see if I see other issues.

             

             

            • 3. Re: JBOSS 7.0 Deployment failure
              monsieursanta

              Hi Ramzik,

               

              @question 1

              filename.doDeploy tells the jboss to deploy the filename

               

              Once jboss starts it picks up this .dodeploy and renames to show that its deploying.

               

              Ideally if you app is successfully deployed, It should change to .deployed

               

              @question 2

              I usually go to the shell and do a ctrl+c to stop it and have not faced any issues so far

              • 4. Re: JBOSS 7.0 Deployment failure
                razmikz

                Hello Santosh. Thank you for your reply.

                 

                I do use the shell and command ctrl+C to stop an instance. No issues here.

                 

                The files .dodeploy and .isdeploying sat in the folder for 10 minutes and I finally deleted both of them manually, then restarted the Jboss and the war file was deployed without and issue.

                This doen't seem to be a good workaround/solution! I need to understand and resolve this so I can plan for automating the Jboss start-up process without worring about what happens to the deployment.

                 

                Any ideas as to why the .dodeploy and .isdeploying sat in the folder without a successful deployment? I would appreciate your assistance.

                 

                • 5. Re: JBOSS 7.0 Deployment failure
                  monsieursanta

                  @Razmik

                   

                  Did you see anything in the server logs . because there should a error etc something which stopped deployment.

                  • 6. Re: JBOSS 7.0 Deployment failure
                    ramsrib

                    >> Any ideas as to why the .dodeploy and .isdeploying sat in the folder without a successful deployment?

                    Might be both these files are not created by that jboss instance (while starting up). When Jboss starts up, it will check for the marker files. If '.isdeploying' already exists, then jboss will do nothing (just waits & exits).

                    • 7. Re: JBOSS 7.0 Deployment failure
                      ramsrib
                      • 8. Re: JBOSS 7.0 Deployment failure
                        razmikz

                        Thank you to all those who read and replied.

                        We didn't resolve the issues, however we installed the JBOSS as windows service and deployed WAR files using the JBOSS Administrator utility. The deployments succeed with errors.