4 Replies Latest reply on Nov 11, 2014 6:17 PM by Paul Ferraro

    Wildfly 8.1.0 + standard-full-ha in two node cluster: Infinispan: You have to start the service first!

    Jeroen van Vianen Newbie

      Hi,

       

      I have a two node cluster with standalone Wildfly 8.1.0 on JDK 8.0.25 started as follows:

       

      /opt/wildfly/bin/standalone.sh -c standalone-full-ha.xml -b=192.168.11.56 -bmanagement=192.168.11.56 -u 230.0.0.4 -Djboss.node.name=FW-PROD4-01

      /opt/wildfly/bin/standalone.sh -c standalone-full-ha.xml -b=192.168.11.57 -bmanagement=192.168.11.56 -u 230.0.0.4 -Djboss.node.name=FW-PROD4-02

       

      nginx is used as a loadbalancer / proxy.

       

      I have deployed four WAR files which contain Spring MVC and Hibernate. I want to use session replication and this works if I drop either one of above nodes from the nginx configuration.

       

      However, after a while I get a blank page in my browser and the following stacktrace:

       

      2014-10-31 13:50:33,273 ERROR [io.undertow.request] (default task-31) UT005023: Exception handling request to /: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException: You have to start the service first!

              at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

              at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

              at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:177) [spring-orm-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

              at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:113) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:61) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

              at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:240) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:73) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:146) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.server.Connectors.executeRootHandler(Connectors.java:177) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727) [undertow-core-1.0.15.Final.jar:1.0.15.Final]

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_25]

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_25]

              at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_25]

      Caused by: java.lang.IllegalStateException: You have to start the service first!

              at org.infinispan.affinity.KeyAffinityServiceImpl.getKeyForAddress(KeyAffinityServiceImpl.java:97)

              at org.wildfly.clustering.web.infinispan.AffinityIdentifierFactory.createIdentifier(AffinityIdentifierFactory.java:50)

              at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.createIdentifier(InfinispanSessionManager.java:139)

              at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.createIdentifier(InfinispanSessionManager.java:71)

              at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.createSession(DistributableSessionManager.java:90)

              at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:692) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:353) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:358) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]

              at com.mycompany.project.util.ShopSelectInterceptor.preHandle(ShopSelectInterceptor.java:37) [classes:1.0-SNAPSHOT]

              at org.springframework.web.servlet.HandlerExecutionChain.applyPreHandle(HandlerExecutionChain.java:130) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]

              ... 40 more

       

      In above stacktrace the last line in my code is bold and contains a call to HttpSession.getAttribute().

       

      The infinispan configuration in standard-full-ha.xml has not been changed from its default:

       

      <subsystem xmlns="urn:jboss:domain:infinispan:2.0">

          <cache-container name="server" default-cache="default" module="org.wildfly.clustering.server" aliases="singleton cluster">

              <transport lock-timeout="60000"/>

              <replicated-cache name="default" batching="true" mode="SYNC">

                  <locking isolation="REPEATABLE_READ"/>

              </replicated-cache>

          </cache-container>

          <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">

              <transport lock-timeout="60000"/>

              <distributed-cache name="dist" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">

                  <file-store/>

              </distributed-cache>

          </cache-container>

          <cache-container name="ejb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan" aliases="sfsb">

              <transport lock-timeout="60000"/>

              <distributed-cache name="dist" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">

                  <file-store/>

              </distributed-cache>

          </cache-container>

          <cache-container name="hibernate" default-cache="local-query" module="org.hibernate">

              <transport lock-timeout="60000"/>

              <local-cache name="local-query">

                  <transaction mode="NONE"/>

                  <eviction strategy="LRU" max-entries="10000"/>

                  <expiration max-idle="100000"/>

              </local-cache>

              <invalidation-cache name="entity" mode="SYNC">

                  <transaction mode="NON_XA"/>

                  <eviction strategy="LRU" max-entries="10000"/>

                  <expiration max-idle="100000"/>

              </invalidation-cache>

              <replicated-cache name="timestamps" mode="ASYNC">

                  <transaction mode="NONE"/>

                  <eviction strategy="NONE"/>

              </replicated-cache>

          </cache-container>

      </subsystem>

       

       

      I have the following questions:

      1. Why does this error occur?

      2. How do I start infinispan if it's not running?

      3. Why would this work from the start, but stop after a while?

       

      Thanks,

       

       

      Jeroen