EAR deployment fails due to ERROR [org.apache.catalina.core.StandardContext] (MSC service thread 1-2) Error listenerStart

[nilshan]

Hello

I am deploying EAR on JBOSS 7 AS. The Ear contains EJB Jars and WAR file. ( I am using Hibernate-3 and spring 3.1).

Below is the log for server startup and deployment of EAR. 

WARNING: -logmodule is deprecated. Please use the system property 'java.util.logging.manager' or the 'java.util.logging.LogManager' service loader.
23:45:44,590 INFO  [org.jboss.modules] JBoss Modules version 1.1.2.GA
23:45:44,740 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
23:45:44,785 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.2.0.Alpha1-SNAPSHOT "Steropes" starting
23:45:45,828 INFO  [org.jboss.as.server.deployment.scanner] JBAS015003: Found health.ear in deployment directory. To trigger deployment create a file called health.ear.dodeploy
23:45:45,847 INFO  [org.jboss.as.server] JBAS015888: Creating http management service using socket-binding (management-http)
23:45:45,848 INFO  [org.xnio] XNIO Version 3.0.4.GA
23:45:45,860 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.4.GA
23:45:45,869 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.8.GA
23:45:45,884 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap log handlers
23:45:45,978 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 31) JBAS010280: Activating Infinispan subsystem.
23:45:46,009 INFO  [org.jboss.as.configadmin] (ServerService Thread Pool -- 26) JBAS016200: Activating ConfigAdmin Subsystem
23:45:46,022 INFO  [org.jboss.as.osgi] (ServerService Thread Pool -- 39) JBAS011906: Activating OSGi Subsystem
23:45:46,037 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 48) JBAS015537: Activating WebServices Extension
23:45:46,068 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 38) JBAS011800: Activating Naming Subsystem
23:45:46,167 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 44) JBAS013171: Activating Security Subsystem
23:45:46,180 INFO  [org.jboss.as.naming] (MSC service thread 1-7) JBAS011802: Starting Naming Service
23:45:46,198 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-5) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.11.Final)
23:45:46,230 INFO  [org.jboss.as.security] (MSC service thread 1-3) JBAS013170: Current PicketBox version=4.0.9.Final
23:45:46,319 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
23:45:46,352 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-2) JBoss Web Services - Stack CXF Server 4.1.0.Beta1
23:45:46,534 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 27) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
23:45:46,544 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 27) JBAS010404: Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
23:45:46,550 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-8) Starting Coyote HTTP/1.1 on http-/0.0.0.0:8080
23:45:46,645 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) JBAS015012: Started FileSystemDeploymentService for directory E:\Development\jboss-as-7.2.0\standalone\deployments
23:45:46,649 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "health.ear"
23:45:46,665 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on 0.0.0.0:4447
23:45:46,665 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on 127.0.0.1:9999
23:45:46,807 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
23:45:46,825 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) JBAS010400: Bound data source [java:/WSCDS]
23:45:49,702 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015960: Class Path entry xbean.jar in /E:/Development/jboss-as-7.2.0/standalone/deployments/health.ear/lib/js.jar  does not point to a valid jar for a Class-Path reference.
23:45:49,721 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015876: Starting deployment of "comp1.jar"
23:45:49,723 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "comp2.jar"
23:45:49,723 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "comp3.jar"
23:45:49,723 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "comp4.jar"
23:45:49,723 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "comp5.jar"
23:45:49,724 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "web.war"
23:45:53,211 INFO  [org.jboss.as.osgi] (MSC service thread 1-2) JBAS011907: Register module: Module "deployment.health.ear:main" from Service Module Loader
23:45:53,259 INFO  [org.jboss.as.osgi] (MSC service thread 1-8) JBAS011907: Register module: Module "deployment.health.ear.comp1.jar:main" from Service Module Loader
23:45:53,214 INFO  [org.jboss.as.osgi] (MSC service thread 1-8) JBAS011907: Register module: Module "deployment.health.ear.comp2.jar:main" from Service Module Loader
23:45:53,216 INFO  [org.jboss.as.osgi] (MSC service thread 1-5) JBAS011907: Register module: Module "deployment.health.ear.comp3.jar:main" from Service Module Loader
23:45:53,257 INFO  [org.jboss.as.osgi] (MSC service thread 1-2) JBAS011907: Register module: Module "deployment.health.ear.comp4.jar:main" from Service Module Loader
23:45:53,265 INFO  [org.jboss.as.osgi] (MSC service thread 1-7) JBAS011907: Register module: Module "deployment.health.ear.comp5:main" from Service Module Loader
23:45:53,344 INFO  [org.jboss.as.osgi] (MSC service thread 1-4) JBAS011907: Register module: Module "deployment.health.ear.web.war:main" from Service Module Loader
23:46:03,994 ERROR [org.apache.catalina.core.StandardContext] (MSC service thread 1-2) Error listenerStart
23:46:03,994 ERROR [org.apache.catalina.core.StandardContext] (MSC service thread 1-2) Context [/health] startup failed due to previous errors
23:46:04,272 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC00001: Failed to start service jboss.web.deployment.default-host./health: org.jboss.msc.service.StartException in service jboss.web.deployment.default-host./health: JBAS018040: Failed to start context
    at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:95)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_13]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_13]
    at java.lang.Thread.run(Thread.java:619) [rt.jar:1.6.0_13]

23:46:04,486 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 28) JBAS018559: Deployed "health.ear"
23:46:04,488 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014777:   Services which failed to start:      service jboss.web.deployment.default-host./health: org.jboss.msc.service.StartException in service jboss.web.deployment.default-host./health: JBAS018040: Failed to start context

23:46:04,558 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9990/management
23:46:04,559 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990
23:46:04,559 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss AS 7.2.0.Alpha1-SNAPSHOT "Steropes" started (with errors) in 20180ms - Started 656 of 772 services (2 services failed or missing dependencies, 112 services are passive or on-demand)

The jboss-deployment-structure.xml I do use is as below...

<?xml version="1.0" encoding="UTF-8"?>
<jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1">
        <ear-subdeployments-isolated>false</ear-subdeployments-isolated>
        <deployment>
            <exclusions>
                <module name="org.hibernate" slot="main" />  
                <module name="org.jboss.as.jpa.hibernate" slot="3"/>   
                <module name="javax.servlet.api"/>                  
                <!-- <module name="org.dom4j" />
                <module name="org.slf4j" /> -->
            </exclusions>

            <dependencies>
                <module name="com.mysql" export="true"/>         
                <!-- Module to load the Hibernate 3 jars -->      
                <module name="org.hibernate.367" export="true"/>

                <!-- Module to load the Spring 3 jars -->
                <module name="org.springframework">
                    <imports>
                        <include path="META-INF**"/>
                        <include path="org**"/>
                    </imports>
                </module>
            </dependencies>
        </deployment>
        <sub-deployment name="comp1.jar"/>
        <sub-deployment name="comp2.jar"/>
        <sub-deployment name="comp3.jar"/>
        <sub-deployment name="comp4.jar"/>
        <sub-deployment name="comp5.jar"/>
        <sub-deployment name="web.war"/>
</jboss-deployment-structure>

This is sure for configuration issue and I am trying to dig out into detail but maybe sombody would have done/resolved this before me..

Can someone please point me the cause for the same ? 

Thanks  a tone. !

[nilshan]

The above problem was due to failure in registration of spring context listener registered in web.xml.

Also, my user-defined listener registed in web.xml for startup-task ( which in turn setting up hibernate configuration ) was causing the problem.

I tried to debug the failure of listener regisration and it is solved now.

NOTE:-

I was facing Spring AOP Proxies related issues which with reference knowledge from this link (http://cliffmeyers.com/blog/2006/12/29/spring-aop-cglib-or-jdk-dynamic-proxies.html) I resolved.

For Hibernate related issue I had to create new module for hibernate-3.

<deployment>

         <exclusions>

            <module name="org.hibernate" slot="main" />

        </exclusions>

         <dependencies>

            <module name="org.hibernate" slot="3" export="true"/> (This is the module I created for hibernate 3 under jboss_home_dir/modules/org/hibernate/3)

       </dependencies>

    </deployment>

Hope this helps to folks who might be facing the same problem.

[muthukumaran_m]

Hi Nilshan,

I am facing similar issue with Jboss EAP 6 and when I comment out the below listener from web.xml (I think this is what you have mentioned) the error goes

	<listener>
		<listener-class>com.liferay.portal.spring.context.PortalContextLoaderListener</listener-class>
	</listener>

I already have hibernated 3 under my lib folder of my application.

- First question I have is how did you find out that spring listener was cause of this issue. In the logs I don’t see any trace related to this. If you share the steps that you used to debug this it will be helpful of me

- What did you do to fix the spring context listener issue?

Thanks,

Muthu

[nilshan]

Hi Muthu,

I had the spring listener registration issue while deploying EAR on JBOSS AS 7, It was basically the class loading issue, which I resolved by providing and removing the dependencies in jboss-deployment-structure.xml. can you please provide me the error log , which may help us to find the root cause.

Thanks.

[muthukumaran_m]

Hi Nilshan,

Thanks for replying. I am using Jboss EAP 6 which is same as the community version Jboss As 7.

Jboss says "Error listenerStart". If I comment this listener my application starts fine. What all dependencies did you add/remove for this?

<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>

My biggest concern is that Jboss is not saying anything about what classes are missing, if it does then I will add them to the dependence. How did you debug your issue?

I have pasted the error from the console below

26,817 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-8) JBAS015893: Encountered invalid class name 'org.xmlpull.mxp1.MXParser,org.

l.mxp1_serializer.MXSerializer' for service type 'org.xmlpull.v1.XmlPullParserFactory'

28,003 INFO  [org.jboss.as.osgi] (MSC service thread 1-5) JBAS011907: Register module: Module "deployment.apmweb-customization-portlet.war:main"

ervice Module Loader

31,127 ERROR [org.apache.catalina.core.StandardContext] (MSC service thread 1-3) Error listenerStart

31,128 ERROR [org.apache.catalina.core.StandardContext] (MSC service thread 1-3) Context [/apmweb-customization-portlet] startup failed due to pr

errors

31,160 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-3) MSC000001: Failed to start service jboss.web.deployment.default-host./apmweb-c

zation-portlet: org.jboss.msc.service.StartException in service jboss.web.deployment.default-host./apmweb-customization-portlet: JBAS018040: Fail

start context

  at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:95)

  at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA-redhat-1.jar:1.0.2.G

at-1]

  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA-redhat-1.jar:1.0.2.GA-redhat-

  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_27]

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_27]

  at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_27]

31,637 INFO  [org.jboss.as.osgi] (MSC service thread 1-1) JBAS011907: Register module: Module "deployment.ROOT.war:main" from Service Module Load

31,708 INFO  [stdout] (MSC service thread 1-3) Loading vfs:/C:/jboss-eap-6.0/standalone/deployments/ROOT.war/WEB-INF/lib/portal-impl.jar/system.p

ies

31,943 INFO  [stdout] (MSC service thread 1-3) Loading vfs:/C:/jboss-eap-6.0/standalone/deployments/ROOT.war/WEB-INF/lib/portal-impl.jar/portal.p

ies

31,943 INFO  [stdout] (MSC service thread 1-3) Loading vfs:/C:/jboss-eap-6.0/standalone/deployments/ROOT.war/WEB-INF/classes/portal-ext.propertie

33,818 INFO  [stdout] (MSC service thread 1-3) 21:15:33,814 INFO  [MSC service thread 1-3][DialectDetector:71] Determine dialect for Microsoft SQ

er 10

33,833 INFO  [stdout] (MSC service thread 1-3) 21:15:33,832 INFO  [MSC service thread 1-3][DialectDetector:136] Found dialect com.liferay.portal.

m.hibernate.SQLServer2008Dialect

47,950 INFO  [stdout] (MSC service thread 1-3) Starting Liferay Portal Enterprise Edition 6.1.20 EE (Paton / Build 6120 / July 31, 2012)

48,632 INFO  [stdout] (MSC service thread 1-3) 21:15:48,632 INFO  [MSC service thread 1-3][BaseDB:455] Database does not support case sensitive q

50,651 INFO  [stdout] (MSC service thread 1-3) 21:15:50,651 INFO  [MSC service thread 1-3][LicenseManager:?] Portal Non-Production license valida

assed

50,653 INFO  [stdout] (MSC service thread 1-3) 21:15:50,653 INFO  [MSC service thread 1-3][ServerDetector:157] Server does not support hot deploy

50,658 INFO  [stdout] (MSC service thread 1-3) 21:15:50,658 INFO  [MSC service thread 1-3][PluginPackageUtil:1030] Reading plugin package for the

context

07,523 INFO  [stdout] (MSC service thread 1-3) 21:16:07,522 INFO  [MSC service thread 1-3][AutoDeployDir:106] Auto deploy scanner started for C:\

09,416 INFO  [stdout] (MSC service thread 1-3) 21:16:09,416 INFO  [MSC service thread 1-3][HotDeployImpl:178] Deploying apmweb-customization-port

om queue

09,418 INFO  [stdout] (MSC service thread 1-3) 21:16:09,417 INFO  [MSC service thread 1-3][PluginPackageUtil:1033] Reading plugin package for apm

stomization-portlet

09,419 INFO  [stdout] (MSC service thread 1-3) 21:16:09,419 WARN  [MSC service thread 1-3][PluginPackageUtil:1131] Plugin package on context apmw

tomization-portlet cannot be tracked because this WAR does not contain a liferay-plugin-package.xml file

11,675 INFO  [org.jboss.web] (MSC service thread 1-3) JBAS018210: Registering web context:

11,947 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 28) JBAS018559: Deployed "ROOT.war"

11,948 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 28) JBAS018559: Deployed "apmweb-customization-portlet.war"

11,950 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report

4777:   Services which failed to start:      service jboss.web.deployment.default-host./apmweb-customization-portlet: org.jboss.msc.service.Start

ion in service jboss.web.deployment.default-host./apmweb-customization-portlet: JBAS018040: Failed to start context

12,002 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990

12,004 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss EAP 6.0.0.GA (AS 7.1.2.Final-redhat-1) started (with errors) in 397728ms -

ed 1592 of 1679 services (2 services failed or missing dependencies, 83 services are passive or on-demand)

[nilshan]

Hi Muthu,

To find out the issue, I attached the spring source to my project and debugged from initWebApplicationContext(ServletContext) to entire flow which helped me to find out the issue related to my application. I hope, the same may help you.

Also, in JBOSS AS 7 the jboss-deployment-structure.xml helps to control the class loading dependencies. Below is the code snippet from my jboss-deployment-structure.xml which shows the exclusion of dependencies which JBOSS AS 7 adds and the dependencies from the application.

<?xml version="1.0" encoding="UTF-8"?>

<jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1">

    <ear-subdeployments-isolated>false</ear-subdeployments-isolated>

    <deployment>

        <!-- Exclude the dependecies the JBOSS AS 7 adds by loading the module.>

        <exclusions>

            <!-- excludes the hibernate 4 dependecy to be added by default >

            <module name="org.hibernate" slot="main" />

            <module name="org.slf4j" slot="main" />

            <module name="org.slf4j.impl" slot="main" />

            <module name="org.slf4j.ext" slot="main" />           

        </exclusions>

        <!-- Add the Custom defined module for hibernate 3>

        <dependencies>           

            <module name="org.hibernate" slot="3" export="true"/>            

        </dependencies>

    </deployment>

    <sub-deployment name="webtier.war" >       

        <resources>

              <resource-root path="myApp.jar" />

        </resources>       

        <dependencies>           

            <module name="deployment.wsc.ear.myApp.jar" export="true" services="export"/>

        </dependencies>       

    </sub-deployment>       

</jboss-deployment-structure>

I hope above helps. Meanwhile I will try to find more details to resolve your problem.

Thanks,