HttpSession Passivation/ Activation Design

- Hany Mesha (hmesha@novell.com)

JIRA Issues

JBAS-1623, JBCLUSTER-15

Design Forum Discussion

(Discuss this design on this forum thread)

Introduction

Session passivation is the ability to write http sessions which have been idle but not expired yet or active during the web application shutdown/ undeploy to a persistent store. Session activation is the ability load the previously passivated session into memory from the persistent store and be managed by the web application session manager to serve the user's http requests. This feature will help the cluster to scale better and overcome memory constraints to handle more sessions.

This feature will probably be production ready with the release of JBoss AS 5.0.

Dependencies

• JBossCache 1.3

• Tomcat 5 http session replication framework

Design Concepts

Http Session passivation consists of configurable passivation triggers to decide when to passivate sessions and passivation store to write the sessions to.

Passivation Triggers

Tomcat background process

The passivation process is triggered when the tomcat background process run, by default every 10 seconds. It tries to look for sessions to passivate if the session is still valid and one of the following conditions is met:

1. The current active sessions exceeds the max allowed sessions and exceeds the minmum idle time (configured in jboss-web.xml)

2. The idle time exceeds the max idle time (configured in jboss-web.xml)

Web application undeployment or shutdown

If passivation is enabled for the web application (configured in jboss-web.xml) During the process of clearing the sessions from the session manager before shutting down the server or undeploying the web application. If the session is valid, then it's passivated to allow for reconstruction of the session after a web application redeployment and user request to activate it.

Create new session

If there's no passivation When creating new session, the session is rejected by the cache manager if exceed the maximum allowed sessions (configured in jboss-web.xml). If passivation is enabled, then an attmept to passivate eligible sessions is made according the conditions mentioned above to free some memory and allow the server to handle more sessions.

Passivation Store

Has the responsiblity to write the evicted sessions to a persistent store. When a session is passivated by the session manager, The instance of JBoss Cache receives a call to evict that session. Which in turn is removed from in-memory cache and written to the persistent store which is instance of JBoss Cache cache loader. The passivation store is configurable in the file resources/tc5/tc5-cluster-service.xml.

Passivation Configuration

Tomcat Level Configuration

Passivation Store - is one per cache instance therefore, turning on the passivation on the cache loader turnes on the passivation for all the web applications running inside the web container. It's configured in tc5-cluster-service.xml under the deploy directory which essentially is the cache loader of the cache instance with passivation set to true and shared set to false.

NOTE: In later releases we will look for a way to configure the passivation store per web application. For the time being we will control this by allowing the user to set the passivation flag to false in jboss-web.xml to disable the passivation for a particular web application.

Example:

<!-- passivation store config using jboss cache 1.3 + --> <attribute name="CacheLoaderConfiguration">    <config>       <!-- if passivation is true, only the first cache loader is used; the rest are ignored -->       <passivation>true</passivation>       <preload>/</preload>       <shared>false</shared>       <!-- we can now have multiple cache loaders, which get chained -->       <cacheloader>          <class>org.jboss.cache.loader.FileCacheLoader</class>          <!-- same as the old CacheLoaderConfig attribute -->          <properties>             location=/tmp          </properties>          <!-- whether the cache loader writes are asynchronous -->          <async>true</async>          <!-- only one cache loader in the chain may set fetchPersistentState to true.               An exception is thrown if more than one cache loader sets this to true. -->          <fetchPersistentState>false</fetchPersistentState>          <!-- determines whether this cache loader ignores writes - defaults to false. -->          <ignoreModifications>false</ignoreModifications>       </cacheloader>    </config> </attribute>

JBoss Level Configuration

Passivation Triggers - is one per web application. It's configured in jboss-web.xml. Different web applications can have different passivation configuration according to their need.

NOTE: The user can disable the passivation for a web application by setting <session-passivation>false</session-passivation> in jboss-web.xml.

Example:

<max-active-sessions>100</max-active-sessions> <passivation-config>   <use-session-passivation>true</use-session-passivation>   <passivation-min-idle-time>5</passivation-min-idle-time>   <passivation-max-idle-time>10</passivation-max-idle-time> </passivation-config>

Implementation Overview

Passivation

Passivation is enabled when there's a configured passivation store with passivation enabled and share is disabled and The passivation flag <user-session-passivation> in jboss.xml is set to true. Otherwise, the session passivation for the web application is disabled.

When a session is passivated by the session manager according to one of the passivation triggers, the distributed store (JBoss Cache) receives a call to evict that session which is intercepted by the passivation interceptor to remove the session from the memory and write it to the configured passivation store (JBossCache Cache Loader).  The passivation interceptor sends a call back notification to the session manager's cache listener on every node in the cluster that the session is about to be passivated. The cache listener first check to see if the session is eligible for handling by its session manager then calls the session maanger to process the session passivation. Then passivation interceptor handles the call using JBossCache Cache loader to store the session to the configured cache loader.  The cache loader configuration must have passivation set to true and sharing set to false. The configuration can be set in the file resources/tc5/tc5-cluster-service.xml

Activation

When the node (session) is requested, it'll be loaded from JBossCacheManager.loadSession() which eventually calls cache.get(). It'll be intercepted by cache ActivationInterceptor loading it from the passivation store (filesystem or database) into the in-memory cache of this server. Then the session will be initialized by calling ClusteredSession.initAfterLoad() method. The result is loading the session attributes and its related nodes (children) and issuing post activation notification to its listeners.

Notification

The notification emitted by the TreeCache during passivation or activation will be broadcasted to the listeners that have registered with the TreeCache. In particluar, org.jboss.web.tomcat.tc5.session.CacheListener. Therefore its passivate or activate method will be invoked to notify the session listeners that the session is about to be passivated or has been activated. First, check if the passivation/ activation event is emitted from a non-session FQN or this server in the cluster, we skip it. Otherwise:

1. In passivate, we find the local session by calling JBossCacheManager.findLocalSession(). if the session found, call JBossCacheManager.processSessionPassivation method.

2. In activate, we do nothing since the cache activation doesn't mean the session has been activated on this particular server. Therefore we let the session manager to activate when the session is requested and its attributes have been loaded. That happens either via replication or user request to this session.

Passivation Management

• Session Expiration

JBossCacheManager.processExpires() is invoked from JBossManager.backgroundProcess() every 10 seconds (default) which expires idle session periodically. The logic of expire() is based on maxInactiveInterval (60 seconds by default) which is set in JBossCacheManager.createSession(). Therefore, the passivation time based configuration must not exceed this time otherwise the session will be expired rather than passivated.

• Session Removal

JBossCacheManager.processExpires() removes sessions from the cache by calling JBossCacheService local removal methods which in turn calls evict triggering passivation on removal. To overcome this problem, two new methods will be added to JBossCacheWrapper (removeLocal(Fqn) and removeLocalSubtree(Fqn)) to utilize the Option API in JBossCache 1.3 to remove the cache node locally without triggering passivation thus ensuring removal doesn't propagate. JBossCacheService.removeLocal methods will call those two new methods.

• Session Store Synchronization

When the session exceeds maxInactiveInterval or Logout via call to session.invalidate(), it is removed not evicted from the cache. that will ensure that it'll be removed from the passivation store which will keep the passivation store in sync with session manager local store at all time. Also, When the method JBossCacheManager.processSessionPassivation is called, it synchronizes on the session, removes the session from the actively managed sessions by this manager and adds the removed session ID to the list  of unloaded sessions with the host name that's is running the session maager as the owner and the session's last accessed time as the the time stamp.

• Session Manager Stop

JBossCacheManager.stop() is invoked during application server  shutdown or application undeploy. The first step in the stop process is to expire active sessions. When passivation is enabled and the session is valid, we will passivate those sessions by calling JBossCacheManager.processSessionPassivation() instead of expiring them in JBossCacheManager.clearSessions(). Otherwise, just let the expiration take place.

Implementation Details

Source Code Changes

• Package : org.jboss.web.tomcat.tc5.session

• Configuration Files: tc-cluster-service.xml, jboss-web_5_0.dtd

1. tc5-cluster-service.xml

• Add passivation store configuration using JBossCache 1.3 style for cache loader config

2. jboss-web.xml

• Add max-active-sessions optional elements to jboss-web_5_0.dtd

• Add passivation-config optional element to jboss-web_5_0.dtd

• Add use-session-passivation boolean optional element to jboss-web_5_0.dtd

• Add passivation-min-idle-time and passivation-max-idle-time optional elements to jboss-web_5_0.dtd

3. org.jboss.metadata.WebMetaData

• Add private boolean sessionPassivationMode - represents the passivation flag for this web application

• Add private int sessionPassivationMinIdleTime - represents the min time (SECS) the session has be idle since its last access time before passivating it for this web application. -1 means ignore it.

• Add private int sessionPassivationMaxIdleTime - represents the max time (SECS) the session is allowed to be idle since last access time before passivating it for this web application. -1 means session should not be forced out.

• Add private int maxActiveSessions - represents the max number of sessions allowed to be actively managed on this server by the session manager for this web application. -1 means unlimited.

• Add methods getSessionPassivationMode() and setSessionPassivationMode() - gets and sets the session passivation flag for this web application

• Add methods getSessionPassivationMinIdleTime() and setSessionPassivationMinIdleTime() - gets and sets the sessionPassivationMinIdleTime for this web application

• Add methods getSessionPassivationMaxIdleTime() and setSessionPassivationMaxIdleTime() - gets and sets the sessionPassivationMaxIdleTime for this web application

• Add methods getMaxActiveSessionsAllowed() and setMaxActiveSessionsAllowed() - gets and sets the maxActiveSessions allowed for this web application

• Change method importJBossWebXml() to parse optional elements from jboss-web.xml that represent the passivation flag and the passivation trigger elements

4. JBossManager

• Add protected boolean passivationMode_ - represents the passivation flag for this web application

• Add protected int passivationMinIdleTime_ - represents the passivation min idle time for this web application

• Add protected int passivationMaxIdleTime_ - represents the passivation max idle time for this web application

• Change method init() to assign to the passivationMode_, passivationMinIdleTime_, passivationMaxIdleTime, and maxActive_ to the values from WebMetaData that were set after parsing jboss-web.xml

5. JBossCacheWrapper:

• Add 2 methods ~removeLocal(Fqn) and ~removeLocalSubtree(Fqn), to serve session removal from distributed cache without invoking evict which in trigger passivation or trigger replication. It utilizes the Option API in JBossCache 1.3.

6. JBossCacheService

• Add ~evictSession(String) method that calls JBossCacheWrapper.evict(Fqn)

• Change the methods ~removeAttributeLocal(), ~removeSessionLocal(), and stop() to call JBossCacheWrapper.removeLocal(Fqn) instead of evict(Fqn)

• Change the methods ~removePojoLocal(), and ~removePojosLocal() to call ~removeLocalSubtree(Fqn)

• Add a new method ~isCachePassivationEnabled() that will return ture if the cache loader is not null, passivation is set to true and shared set to false in the cache loader config of this cache instance.

• Change start() method to check passivation status and writes to the log by calling log.info() method

7. JBossCacheManager

• Add public void ~processSessionPassivation(String, String) method that finds the local session based on the passed session id parameter. If exists, it synchronize on it, stops the local session activty to avoid broadcasting notification to the same server, call session.passivate() method, then ~proxy_.evictSession() to remove the session from the in-memory cache and write it to the passivation store. Finally, add the removed session id and host that owns it to the unloaded sessions map, remove it from active sessions map, and reduce the active counter by 1.

• Change ~clearSessions() method to check during active sessions processing. If passivation is enabled and the session is still valid, then passivate the session rather than expring it by calling ~processSessionPassivation() method.

• Change ~createSession(String) to check, when max active sessions is configured and less than the current active session, if passivation is enabled to try to passivate or invalidate eligiable sessions before attempting to reject the session by calling processExpires.

• Change ~processExpires() to process session passivation after determining that the session is valid and should not be expired. It checks first if session passivation is enabled, then get the idle time for the session and compare it to passivationMaxIdleTime, passivationMinIdleTime. If the session is eligiable for passivation then ~processSessionPassivation is called. Also it checks if the current active session exceeds the max allowed session for this web app and try to passivate sessions that have exceeded the min required time before passivation.

• Add ~isSessionPassivationEnabled() method that returns true if JBossManager.passivatioMode and JBossCacheService.isCachePassivationEnabled() otherwise false is returned.

• Modify CreateSession and loadSession methods instead of relying on the activeCounter_ protected field to get the number of active sessions managed by the manager, use sessions_.size() which is more accurate for concurrency support and will align better with the max active session implementation. In the past, max active sessions used to be set to -1 all the time so there was no implications of using activeCounter_ field. Now maxActive_ field is configurable and needs to rely on accurate represntation of the currenently active sessions managed by the manager. Therefore activeCounter_ will no longer suffice the purpose.

8. CacheListener

• Change the class extend to AbstractTreeCacheListener instead of implementing TreeCacheListener to allow adding the passivate and activate notification methods to it without having to inherit other notification that we're not interested in.

• In passivate, If we receive pre-passivation notification from the distrbuted store, we parse the fqn to see if it has buddy backup region in it. also check if we should handle this event and the size of Fqn is session size and if pre-passivation is true flag sent is true. call the session manager's ~processSessionPassivation()

• In activate, we do nothing.

NOTE: Cache Loader will always be unshared as we discussed this item on the forum and will be documented.

Test Cases

1. org.jboss.test.ScopedTestCase

• Add method ~testPassivationActivationMaxActive()

• Add method ~testPassivationActivationMaxIdle()

• Add method ~isSessionExistsInCache()

• Add method ~activateSession

2. org.jboss.test.UndeployTestCase

• Add method TestPassivationActivationonUndeploy()

• Add method ~activateSession

3. jboss-web.xml

• Add max active sessions and passivation configuration to the following xml files in testsuite/src/resources/cluster/http/http-field

• jboss-web-field-batch-false.xml

• jboss-web-field-scoped.xml

• jboss-web.field.xml

• Add max active sessions and passivation configuration to the following xml files in testsuite/src/resources/cluster/http/http-scoped

• jboss-web-attr-based.xml

• jboss-web-attr-set.xml

• jboss-web-set-trigger.xml

• jboss-web.xml