3 Replies Latest reply on Aug 12, 2019 7:01 AM by ouugiii

    JBoss fails to load class from WAR on service restart

    ouugiii

      Hi,

       

      We have an Icefaces JSF application running on JBoss 7.1 EAP. The application is deployed in an EAR that contains a WAR. Sometimes, when restarting (not redeploying) the service, the service starts without problems. However, when trying to access a page, we get the following exception:

       

           org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'dtoMapper' available

           ("dtoMapper" is a class is defined in the file WebContent/WEB-INF/classes/webContext.xml in the WAR)

       

      This seems to happen approximately 5-10% of the time the service is restarted but is very random. Restarting the service again usually solves the problem (sometimes multiple restarts are required). It seems as though the application sometimes ignores the webContext.xml file entirely.

       

      We have also noticed that the order in which JBoss loads modules (even its own) seems to change every time the service is started. Is this normal behaviour and could this be the reason for the random failure?

       

      Our environment:

      JBoss 7.1 EAP (installed as a Windows service)

      Icefaces 4.2.0

      Primefaces 5.3.0 (the application uses a single component from Primefaces)

      JSF 2.2.13

      Java EE 1.8u131

      spring-aop-4.3.20.RELEASE.jar

      spring-aspects-4.3.20.RELEASE.jar

      spring-beans-4.3.20.RELEASE.jar

      spring-context-4.3.20.RELEASE.jar

      spring-context-support-4.3.20.RELEASE.jar

      spring-core-4.3.20.RELEASE.jar

      spring-expression-4.3.20.RELEASE.jar

      spring-instrument-4.3.20.RELEASE.jar

      spring-instrument-tomcat-4.3.20.RELEASE.jar

      spring-jdbc-4.3.20.RELEASE.jar

      spring-jms-4.3.20.RELEASE.jar

      spring-ldap-core-1.3.1.RELEASE.jar

      spring-messaging-4.3.20.RELEASE.jar

      spring-orm-4.3.20.RELEASE.jar

      spring-oxm-4.3.20.RELEASE.jar

      spring-security-config-3.0.2.RELEASE.jar

      spring-security-core-3.0.2.RELEASE.jar

      spring-security-kerberos-core-1.0.0.M2.jar

      spring-security-taglibs-3.0.2.RELEASE.jar

      spring-security-web-3.0.2.RELEASE.jar

      spring-test-4.2.5.RELEASE.jar

      spring-tx-4.2.5.RELEASE.jar

      spring-web-4.2.5.RELEASE.jar

      spring-webmvc-4.2.5.RELEASE.jar

      spring-webmvc-portlet-4.2.5.RELEASE.jar

      spring-websocket-4.2.5.RELEASE.jar

      spring-ws-core-2.2.4.RELEASE.jar

      spring-ws-security-2.2.4.RELEASE.jar

      spring-xml-2.2.4.RELEASE.jar

       

      ----------------------------------------------------------------------------------------------------------------------------------------

      UPDATE 29.7.2019

       

      I'm able to reproduce the error on localhost by simply emptying or deleting the webContext.xml file.

      => Is the webContext file also empty or doesn't exist on the serverside 5-10% of the times the service is started?

       

      I've recognized the context files are loaded trough Springs ContextLoaderListener in the web.xml:

      <listener>

           <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>

      </listener>

       

      And the context files are loaded with this part:

      <context-param>

           <param-name>contextConfigLocation</param-name>

           <param-value>

                classpath*:context1.xml,

                classpath*:context2.xml,

                classpath*:context3.xml,

                classpath*:webContext.xml

           </param-value>

      </context-param>

       

      Does the order of the context files matter?

      Is there anything wrong with the way I try to load the context files?

       

      TESTING

      I've made a test by using different param-values for the parameter "contextConfigLocation" and logging the path to the context files.

       

      These are the options i've used for contextConfigLocation param-value:

           (1) classpath:CONTEXT_FILE_NAME

           (2) classpath:*CONTEXT_FILE_NAME

           (3) classpath*:CONTEXT_FILE_NAME

           (4) classpath:**/*CONTEXT_FILE_NAME

           (5) classpath*:*CONTEXT_FILE_NAME

           (6) classpath*:**/*CONTEXT_FILE_NAME

           (7) classpath*:*Context.xml

           (8) classpath*:**/*Context.xml

       

      I have used this method from StackOverflow: https://stackoverflow.com/a/9564301/6209399  with some slight modifications to log the paths of the context files.

       

      Results:

      - (2) and (4) fails because it doesn't find files on the Virtual File System (VFS) (Where all the context files are)

      - (1) and (3) fails with the same error approximately 5-10% of the time the service is restarted but is very random.

      - (7) Haven't still been able to reproduce the same error (Have restarted the application 100 times) (Seems to work, but don't understand why).

      - (5), (6) and (8) Haven't tested yet but they do find all the context files in my application.

       

      The logging method (resource.getDescription()) logs different thing based on the options [1-8] I put into the method:

      - (1) logs: "class path resource [CONTEXT_FILE_NAME]"

      - (3) logs:

      - "URL [vfs:/C:/PATH_TO_EAR/lib/MODULE1.jar/context1.xml]"

      - "URL [vfs:/C:/PATH_TO_EAR/PATH_TO_WAR/WEB-INF/classes/webContext.xml]"

      - (5-8) logs:

      - "VFS resource [/C:/PATH_TO_EAR/lib/MODULE1.jar/context1.xml]"

      - "VFS resource [/C:/PATH_TO_EAR/PATH_TO_WAR/WEB-INF/classes/webContext.xml]"

       

           - Why does the option (3) log 'URL ...' and the options (5-8) log 'VFS resource ...'? and does this affect how the context files are loaded?

       

      I've also recognized that when using option (7) and (8) to log the path to context files, it logs the context files in a specific order:

      - First it logs context files on the file system ("file [C:\PATH_TO_CONTEXT_FILE]")

      - Then it logs context files inside the WAR ("VFS resource [/C:/PATH_TO_EAR/PATH_TO_WAR/WEB-INF/classes/webContext.xml]")

      - Then it logs context files inside JARS ("VFS resource [/C:/PATH_TO_EAR/lib/MODULE1.jar/context1.xml]")

       

      - Does this order affect in which order the context files are loaded?

       

      ----------------------------------------------------------------------------------------------------------------------------------------

      UPDATE2 29.7.2019

      Option (7) has also produced the error! It was not a solution.

        • 1. Re: JBoss fails to load class from WAR on service restart
          jaikiran

          ouugiii  wrote:

           

          Hi,

           

          We have an Icefaces JSF application running on JBoss 7.1 EAP. The application is deployed in an EAR that contains a WAR. Sometimes, when restarting (not redeploying) the service, the service starts without problems.

          When you say service, which service do you mean? How are you restarting it?

           

          I also see references to WebContent directory in your post. Deployed applications don't have that directory (in standard deployment types). Is some IDE involved in managing these deployments and are you trying to deploy/restart the application from the IDE?

          • 2. Re: JBoss fails to load class from WAR on service restart
            ouugiii

            jaikiran  kirjoitti:

            When you say service, which service do you mean?

            When I say service, I mean a windows service, which is the same as the application as it is a standalone service.

             

            jaikiran  kirjoitti:

            How are you restarting it?

            Simply by navigating to the built-in application "services" on a remote windows computer and then by right-clicking the service we want to restart and choose the option "restart".

             

            jaikiran  kirjoitti:

            I also see references to WebContent directory in your post. Deployed applications don't have that directory (in standard deployment types).

            Which standards? could you please provide a link, haven't found anything about this. And why is this a problem?

             

            jaikiran  kirjoitti:

            Is some IDE involved in managing these deployments and are you trying to deploy/restart the application from the IDE?

            No IDE's are involved when deploying/restarting the application. Except in development mode (running the web application on localhost) where the web application is deployed and restarted from an IDE, but the problem never occurs in development mode. This means that when the web application is deployed/restarted from the IDE, the problem doesn't occur. But when the web application is deployed/restarted from windows built-in application "services", the problem might occur.

            • 3. Re: JBoss fails to load class from WAR on service restart
              ouugiii

              PROBLEM SOLVED


              Short description:

              This error was caused by:

              • race condition
              • project settings
              • use of different applicationContext-class-types in the web application and EJBs
              • A listener to update the applicationContext

                

              This problem was solved by forcing the ApplicationContext, before the web application starts, to be the ApplicationContext we wanted.

               

              Long description:

              So in our webApplication, we have a standard Spring ContextLoaderListener that listens for when a new application context is created and sets the whole web application context to be the newly created application context in class shared between the web application and EJBs

               

              We have the following setting in our web application:

                      <ear-subdeployments-isolated>false</ear-subdeployments-isolated>

              Which means that our web applications ear-modules classes can access other ear-modules and war-modules classes in the web application.

              More information can be found here: Class Loading in WildFly - WildFly 8 - Project Documentation Editor

               

              The applicationContext is set on two different places in our web application:

              • org.springframework.web.context.ContextLoaderListener
                • This sets the ApplicationContext to be an instance of class 'XmlWebApplicationContext'
                • no failures when this was set to the whole application
              • inside one of our EJB modules
                • But this one checks if an ApplicationContext already exists, if not, then it will create a new one.
                • This new ApplicationContext will be an instance of the class 'ClassPathXmlApplicationContext'
                • When this was set to the shared class, then the error was caused

                        

              If the EJB-module is loaded and run BEFORE the ContextLoaderListener, then the ApplicationContext of the whole web application will first be set to be an instance of the class 'ClassPathXmlApplicationContext' and then to 'XmlWebApplicationContext'. Which is ok!

              If the EJB-module is loaded and run AFTER the ContextLoaderListener, then the ApplicationContext of the whole web application will be set to be an instance of the class 'XmlWebApplicationContext'. Which is ok!

               

              When the error occurs, this is what has happened:

              1. The EJB-module is loaded and ran AT THE SAME TIME as the ContextLoaderListener
              2. ApplicationContext is not yet set
              3. The EJB-module checks if the ApplicationContext exist (Which does not exist)
              4. The ContextLoaderListener sets the ApplicationContext to be an instance of the class 'XmlWebApplicationContext'
              5. The listener updates the ApplicationContext for the whole application
              6. The EJB-module sets the applicationContext to be an instance of the class 'ClassPathXmlApplicationContext'
              7. The listener updates the ApplicationContext for the whole application

                  => Error occurres

               

              This problem was solved by adding another listener to the web.xml right after the ContextLoaderListener making sure the shared ApplicationContext will be the one ContextLoaderListener sets it to.