6 Replies Latest reply on Nov 30, 2012 10:26 AM by David Schlenk

    clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems

    David Schlenk Newbie

      Details of installation: Using MySQL for JCR & IDM in 3.5.0.Beta02 on JBoss AS 7.1.1.Final using the standalone-full-ha profile. Two nodes using mod_cluster on apache httpd to frontend.

       

      I've had success in the cluster using the default H2 database backend for JCR and IDM, but my attempt at moving to MySQL is still giving errors. Currently startup seems to indicate success and I can access the root JBoss AS welcome page OK. Attempting to access /portal results in a blank page and the following stack trace in the console:

       

      11:45:55,464 WARN  [org.exoplatform.web.CacheUserProfileFilter] (http-appserver-dev.pe.spanlink.com-172.17.13.153-8080-2) An error occured while cache user profile: javax.servlet.ServletException: java.lang.NullPointerException

                at org.exoplatform.portal.application.PortalController.onService(PortalController.java:129) [exo.portal.webui.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.container.web.AbstractHttpServlet.service(AbstractHttpServlet.java:132) [exo.kernel.container-2.4.0-CR2.jar:2.4.0-CR2]

                at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.web.filter.ExtensibleFilter$ExtensibleFilterChain.doFilter(ExtensibleFilter.java:110) [exo.portal.component.web.api-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.web.filter.ExtensibleFilter.doFilter(ExtensibleFilter.java:80) [exo.portal.component.web.api-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.web.filter.GenericFilter.doFilter(GenericFilter.java:79) [exo.portal.component.web.api-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.web.CacheUserProfileFilter.doFilter(CacheUserProfileFilter.java:72) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.frameworks.jcr.web.ThreadLocalSessionProviderInitializedFilter.doFilter(ThreadLocalSessionProviderInitializedFilter.java:122) [exo.jcr.framework.web-1.15.0-CR2.jar:1.15.0-CR2]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.web.login.ConversationStateUpdateFilter.doFilter(ConversationStateUpdateFilter.java:66) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.services.security.web.SetCurrentIdentityFilter.doFilter(SetCurrentIdentityFilter.java:88) [exo.core.component.security.core-2.5.0-CR2.jar:2.5.0-CR2]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.web.login.ClusteredSSOFilter.doFilter(ClusteredSSOFilter.java:62) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.web.login.RememberMeFilter.doFilter(RememberMeFilter.java:85) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.web.login.RememberMeFilter.doFilter(RememberMeFilter.java:47) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.gatein.sso.integration.SSODelegateFilter.doFilter(SSODelegateFilter.java:60) [sso-integration-1.3.0.CR03.jar:1.3.0.CR03]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.exoplatform.container.web.PortalContainerFilter.doFilter(PortalContainerFilter.java:78) [exo.kernel.container-2.4.0-CR2.jar:2.4.0-CR2]

                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]

                at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)

                at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)

                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:397) [jbossweb-7.0.13.Final.jar:]

                at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.gatein.sso.integration.SSODelegateValve.invoke(SSODelegateValve.java:159) [sso-integration-1.3.0.CR03.jar:1.3.0.CR03]

                at org.exoplatform.web.login.PortalClusteredSSOSupportValve.invoke(PortalClusteredSSOSupportValve.java:89) [exo.portal.component.web.security-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]

                at org.jboss.as.web.sso.ClusteredSingleSignOn.invoke(ClusteredSingleSignOn.java:339) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]

                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]

                at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]

                at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]

                at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]

                at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_34]

      Caused by: java.lang.NullPointerException

                at org.exoplatform.portal.pom.config.cache.PortalNamesCache.execute(PortalNamesCache.java:51) [exo.portal.component.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.portal.pom.config.POMSessionManager.execute(POMSessionManager.java:253) [exo.portal.component.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.portal.pom.config.POMDataStorage.getPortalConfig(POMDataStorage.java:133) [exo.portal.component.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.portal.config.DataStorageImpl.getPortalConfig(DataStorageImpl.java:304) [exo.portal.component.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.portal.application.PortalRequestHandler.execute(PortalRequestHandler.java:145) [exo.portal.webui.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.web.WebAppController.service(WebAppController.java:366) [exo.portal.component.web.controller-3.5.0.Beta02.jar:3.5.0.Beta02]

                at org.exoplatform.portal.application.PortalController.onService(PortalController.java:125) [exo.portal.webui.portal-3.5.0.Beta02.jar:3.5.0.Beta02]

                ... 55 more

       

       

      Attached is the startup log. Not sure if I need to do something more to move to MySQL? I created a module and then added it to the dependency list of gatein's module.xml. The schemas are created during startup properly as far as I can tell.

        • 1. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
          Michal Vanco Newbie

          Hi David,

          portal is deployed only at standalone.xml and standalone-ha.xml configurations. That's probably the main reason why you can't access the /portal at url.

           

          You can have a look at some more clustering details at https://github.com/gatein/gatein-portal/tree/master/packaging/jboss-as7 (part called "Clustered mode").

          Mysql backend should work for you without any issues.

           

          Hope this helps,

          Michal VanĨo

          • 2. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
            David Schlenk Newbie

            Ah, didn't realize it was limited to the non-full profiles. Is there a reason for that? I don't think I actually need the full profile - just curious

            • 3. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
              David Schlenk Newbie

              Actually, switching to standalone-ha.xml didn't solve the problem entirely. It does load /portal initally after the app server starts, but 3 minutes 44 seconds later I tried again and it threw the same stack trace with nothing to show for it but a blank page.

               

              Also I think I had previously propogated the required configuration to deploy portal in -full-ha.xml which is why the problem existed there as well. Any ideas? Seems to be about user profile cache?

              • 4. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
                David Schlenk Newbie

                Update: I've tried doing the minimal changes I think are required to switch to MySQL from a stock 3.5.0.Beta02 and the problem persists running with the normal standalone.xml profile. What I've done:

                 

                Replace the datasource definitions in standalone/configuration/standalone.xml with:

                 

                <datasource jndi-name="java:/jdbcidm_portal" pool-name="IDMPortalDS" enabled="true" use-java-context="true">

                     <connection-url>jdbc:mysql://db-dev.example.com:3306/jdbcidm_portal</connection-url>

                     <driver>com.mysql</driver>

                     <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>

                     <pool>

                          <min-pool-size>10</min-pool-size>

                          <max-pool-size>100</max-pool-size>

                          <prefill>true</prefill>

                     </pool>

                     <security>

                          <user-name>idmUser</user-name>

                          <password>XXXXXXXXX</password>

                     </security>

                     <statement>

                          <prepared-statement-cache-size>32</prepared-statement-cache-size>

                          <share-prepared-statements/>

                     </statement>    

                </datasource>

                <datasource jndi-name="java:/jdbcjcr_portal" pool-name="JCRPortalDS" enabled="true" use-java-context="true">

                     <connection-url>jdbc:mysql://db-dev.example.com:3306/jdbcjcr_portal</connection-url>

                     <driver>com.mysql</driver>

                     <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>

                     <pool>

                          <min-pool-size>10</min-pool-size>

                          <max-pool-size>100</max-pool-size>

                          <prefill>true</prefill>

                     </pool>

                     <security>

                          <user-name>jcrUser</user-name>

                          <password>XXXXXX</password>

                     </security>

                     <statement>

                          <prepared-statement-cache-size>32</prepared-statement-cache-size>

                          <share-prepared-statements/>

                     </statement>

                </datasource>

                 

                Also add a driver in that file:

                 

                <driver name="com.mysql" module="com.mysql">

                                        <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class>

                                    </driver>

                 

                Add a MySQL module by creating modules/com/mysql/main/module.xml with these contents:

                 

                <?xml version="1.0" encoding="UTF-8"?>

                <module xmlns="urn:jboss:module:1.0" name="com.mysql">

                  <resources>

                    <resource-root path="mysql-connector-java-5.1.22.jar"/>

                  </resources>

                  <dependencies>

                    <module name="javax.api"/>

                  </dependencies>

                </module>

                 

                Copied mysql-connector-java-5.1.22.jar to modules/com/mysql/main/

                 

                Added <module name="com.mysql" /> at line 190 after the h2database module dependency in modules/org/gatein/lib/main/module.xml

                 

                Am I missing a MySQL data source option? I based my datasource config on this article: https://community.jboss.org/wiki/DataSourceConfigurationInAS7

                • 5. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
                  Michal Vanco Newbie

                  Hi David,

                  your MySQL setup seems to be fine. You should try to add dependency on javax.transaction.api at mysql module.xml:

                   

                  <module xmlns="urn:jboss:module:1.0" name="com.mysql">

                     <resources>

                        <resource-root path="mysql-connector-java-5.1.22.jar"/>

                     </resources>

                     <dependencies>

                        <module name="javax.api"/>

                        <module name="javax.transaction.api"/>

                     </dependencies>

                  </module>

                   

                  But problem you hit wasn't likely related to DB setup, can you start from scratch (delete folders at standalone directory - data, log, tmp).

                  Use 3.5.0.Beta2, setup mysql as above and run with standalone.sh -c standalone-ha.xml.

                  Then share what you get.

                   

                  Best regards,

                  Michal

                  • 6. Re: clustered 3.5.0.Beta02 on AS 7.1.1 with MySQL backend problems
                    David Schlenk Newbie

                    OK I figured it out. I was using Chef to provision my MySQL server and the stock recipe for a MySQL server sets the wait_timeout server system variable to 180. The default as documented by MySQL is 28800. Perhaps the units changed since the MySQL recipe was written. In any event, overriding that value back to default solved all my problems.