Http Session replication problems when NOT using sticky-sess

[jcprout]

I have a JBoss cluster, v4.0.3, with two nodes. Load balancing uses mod_jk, but without sticky-session.

I am having problems when a session is invalidated and then a new set of attributes set in a new session (i.e. someone logs out, then logs in again as a different user).

HttpSession sess = request.getSession(false);
if (sess != null)
{
 sess.invalidate();
}
sess = request.getSession(true);
sess.setAttribute(Constants.USER_KEY, userInfo);

My testing shows that the session that is invalidated isn't being removed on the other cluster node (not the node where this code is actually run). The result is that the new user who's logged in sometimes sees data from the previous user.

Searching the JBoss site, this is the document which has the most information about what I'm seeing: http://www.jboss.org/developers/projects/jboss/tc5-clustering.html

An extract follows:

Note that because of the dual use of in-memory and distributed store sessions and because JBossCache does yet not support the notion of modification event only from remote node(s), the current usage of session replication must use sticky session. That is, only one dedicated clustering node will handle the session request unless there is a failover. This should not pose as a severe restriction since it will be rare indeed that sticky session is not used in production today. We will relax this restriction in the future release.

My question is, does this mean that we have to use sticky-session, because otherwise, changes to a session on one cluster node won't necessarily be visible on the other cluster nodes?

I have "snapshot-mode" set to "instant" and "cache-mode" set to "REPL_SYNC"

Any help or suggestions much appreciated

John

[brian.stansberry]

We strongly encourage use of sticky sessions. But, that being said, if things are working properly what you're trying to do shoudn't give you the results your seeing

What's your replication-granularity setting (SESSION or ATTRIBUTE)?

[jcprout]

We are trying to avoid using sticky-session because that would mean that we also have to use it on the two hardware load-ballancers in front of the cluster and web servers. The Operations group doesn't want to make this change...

Replication-granularity is set in the jboss-web.xml file and we're not setting it, so the default must be used (is the default session?)

John

[brian.stansberry]

Yes, it's SESSION.

[jcprout]

I now have a simple test application which reliably shows that if I attempt to remove either a session (using HttpSession.invalidate()) or a session attribute (using HttpSession.removeAttribute(String)), the change is not immediately made on the remote cluster node. I am testing to find out how long it is before the session is removed on the remote node.

Should I open a bug report for this issue (supported by the test application) or is this not considered a bug?

Can anyone suggest any changes I can make to fix this issue, or to shorten the time between when a session is invalidated on one cluster node and when the same session is invalidated on other nodes in the cluster? Switching to using sticky-session is the obvious fix, but that will cause other problems in our environment

Thanks - John

[ben.wang]

If you do removeAttribute the effect should be immediate though because you are using REPL_SYNC mode. How do you test it? And how long does it take?

[jcprout]

My test application uses struts, but the session attribute I'm using as a marker is not used by struts.

I have 2 JSP pages. Each displays the name of the host that served it, and then lists the name/value of all session attributes, like so:

<%
 java.util.Enumeration enm = session.getAttributeNames();
 while(enm.hasMoreElements())
 {
 String attr = (String)enm.nextElement();
 out.println("Session variable: " + attr + " value: " + session.getAttribute(attr) + "<br />");
 }
%>

On one page I have a button that sets the marker session attribute, then another button (logout) that removes the attribute, then invalidates the session and forwards to the other page:

 HttpSession session = request.getSession(false);
 if(session != null)
 {
 session.removeAttribute("marker");
 session.invalidate();
 }

The second page allows me to make another request and forwards back to the first page. The first page reliably redisplays the marker session attribute, which was removed, the the session invalidated (on the other cluster server) by the previous request

Since I am running this on a two server cluster, using (mod_jk) equal weighted, round-robin load balancing, the requests are served first by one server, then by the other. Session is invalidated on one server, then displayed by a JSP page which executes on the other server. I always display the name of the server which ran each page just to confirm where the page was executed.

John

[jcprout]

I just found a parameter in jbossweb-tomcat55.sar/META-INF/jboss-service.xml which looked as though it should fix my problems:

 <!-- A flag indicating if the local vm session value should be used if
 it exists. When true, the existing vm local session values are used and
 updates are replicated, but updates to the same session on other nodes
 do not update the local session value. This mode is only useful for
 failover. When false, the session value is obtained from the distributed
 cache. This mode can be used with load balancing.
 -->
 <attribute name="UseLocalCache">true</attribute>

This looked like it should solve my problems, but I changed the value to "false" and it didn't make any difference :-(

John

[ben.wang]

Actually, we do have test cases to test removeAttribute and invalidate during cluster mode. In the test, we use httpclient to invalidate the session on one node and retrive it from the second one (using the same session id).

So I'd suggest you do a simpler test. Simply, have two node cluster with sticky session. invalidate/remove on one node, then shut it down, it will failover to the other one. Retrieve it again, and you will see whether it works or not.

[jcprout]

I'm sure that the scenario you describe will work, but the whole point of this thread is that I'm trying to make the cluster work when it's not using sticky-session. Is thre any way I can push the removeAttribute or invalidate to the other node without shutting down?

[ben.wang]

Actually our unit test case that uses httpclient does not assume sticky session at all, and it still works! So you should not need sticky session for this to work. I was just saying that so it is easier for you to troubleshoot your problem. You need to test your enviornment first.

[vpsangeetha]

I have a similar problem.

We have a 3 node jboss cluster that does not use sticky sessions. I tested the following cases:

Case 1:
1. Node 1 sets an attribute "Attribute1" in the session.
2. Node 2 gets "Attribute 1" - SUCCESS
3. Node 2 sets an attribute "Attribute 2" in the session.
4. Node 3 gets "Attribute 2" - SUCCESS

Case 2:
1. Node 1 sets an attribute "Attribute1" in the session.
2. Node 2 gets "Attribute 1" - SUCCESS
3. Node 2 sets an attribute "Attribute 2" in the session.
4. Node 1 gets "Attribute 2" - FAILS (It gets a null value)

Any idea why this is happening? I am using jboss 3.2.6. I have
"snapshot-mode" set to "instant", "cache-mode" set to "REPL_SYNC",
replication granularity is default(session) and "UseLocalCache" set to false.

[ben.wang]

I'd suggest you try out the latest AS 4.0.3P1 to see if it still works for you.

[brian.stansberry]

Even better, hold out for 4.0.4.CR2 (which is scheduled for the end of the week) as I've made a number of improvements.

Particularly, the session invalidation issue that was the start of this thread is fixed: see http://jira.jboss.com/jira/browse/JBAS-2927.

But, all that being said, use sticky sessions!!

[jcprout]

After some very careful testing of whether invalidate() and removeAttribute() are successfully replicated to cluster nodes other than the one where they were executed, we found that this reliably didn't work in a non-sticky-session configuration. The same tests worked correctly when we converted our test cluster to sticky-session, so we're in the process of converting our production cluster to use sticky-session.