Wildfly 8.1.0 + standard-full-ha in two node cluster: Infinispan: You have to start the service first!
jeroenvanvianen Oct 31, 2014 11:26 AMHi,
I have a two node cluster with standalone Wildfly 8.1.0 on JDK 8.0.25 started as follows:
/opt/wildfly/bin/standalone.sh -c standalone-full-ha.xml -b=192.168.11.56 -bmanagement=192.168.11.56 -u 230.0.0.4 -Djboss.node.name=FW-PROD4-01
/opt/wildfly/bin/standalone.sh -c standalone-full-ha.xml -b=192.168.11.57 -bmanagement=192.168.11.56 -u 230.0.0.4 -Djboss.node.name=FW-PROD4-02
nginx is used as a loadbalancer / proxy.
I have deployed four WAR files which contain Spring MVC and Hibernate. I want to use session replication and this works if I drop either one of above nodes from the nginx configuration.
However, after a while I get a blank page in my browser and the following stacktrace:
2014-10-31 13:50:33,273 ERROR [io.undertow.request] (default task-31) UT005023: Exception handling request to /: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException: You have to start the service first!
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:177) [spring-orm-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:113) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:61) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:25) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:240) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:73) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:146) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:177) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_25]
Caused by: java.lang.IllegalStateException: You have to start the service first!
at org.infinispan.affinity.KeyAffinityServiceImpl.getKeyForAddress(KeyAffinityServiceImpl.java:97)
at org.wildfly.clustering.web.infinispan.AffinityIdentifierFactory.createIdentifier(AffinityIdentifierFactory.java:50)
at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.createIdentifier(InfinispanSessionManager.java:139)
at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.createIdentifier(InfinispanSessionManager.java:71)
at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.createSession(DistributableSessionManager.java:90)
at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:692) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:353) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:358) [undertow-servlet-1.0.15.Final.jar:1.0.15.Final]
at com.mycompany.project.util.ShopSelectInterceptor.preHandle(ShopSelectInterceptor.java:37) [classes:1.0-SNAPSHOT]
at org.springframework.web.servlet.HandlerExecutionChain.applyPreHandle(HandlerExecutionChain.java:130) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) [spring-webmvc-4.1.1.RELEASE.jar:4.1.1.RELEASE]
... 40 more
In above stacktrace the last line in my code is bold and contains a call to HttpSession.getAttribute().
The infinispan configuration in standard-full-ha.xml has not been changed from its default:
<subsystem xmlns="urn:jboss:domain:infinispan:2.0">
<cache-container name="server" default-cache="default" module="org.wildfly.clustering.server" aliases="singleton cluster">
<transport lock-timeout="60000"/>
<replicated-cache name="default" batching="true" mode="SYNC">
<locking isolation="REPEATABLE_READ"/>
</replicated-cache>
</cache-container>
<cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="ejb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan" aliases="sfsb">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="hibernate" default-cache="local-query" module="org.hibernate">
<transport lock-timeout="60000"/>
<local-cache name="local-query">
<transaction mode="NONE"/>
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<invalidation-cache name="entity" mode="SYNC">
<transaction mode="NON_XA"/>
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</invalidation-cache>
<replicated-cache name="timestamps" mode="ASYNC">
<transaction mode="NONE"/>
<eviction strategy="NONE"/>
</replicated-cache>
</cache-container>
</subsystem>
I have the following questions:
1. Why does this error occur?
2. How do I start infinispan if it's not running?
3. Why would this work from the start, but stop after a while?
Thanks,
Jeroen