> My cluster
> starts to slow down and process HTTP requests slower
> and slower, eventually causing the new requests to
> pile up in Apache.
I assume you are using Apache/mod_jk/Tomcat ?
> If I reboot the JBoss of the
> affected node, it takes forever to shutdown, minutes
> rather than seconds.
Can you take a stack trace next time this happens ? Also, a log would be helpful (check doc for how to turn tracing on).
> When the rebooted node starts
> up, it hangs at the start of the clustering right
> after if prints the GMS address and mentions Ctrl-C.
Again, take a stack trace to see where it's hanging.
Can you also post cluster-service.xml ?
> I assume you are using Apache/mod_jk/Tomcat ?
> Can you take a stack trace next time this happens ? Also, a log would be helpful (check doc for how to turn tracing on).
Ok, I will test turning tracing on now to make sure I know exactly how to do it. As for the stack trace, can you tell me what signal I can send to JBoss to prompt a stack trace. In windows, it is a Crtl-C, but I am running Linux and have it back grounded. I think it might be interrupt (INT or -2)?
I can post the cluster-service.xml if you wish, but I have not altered it from the default that ships with 3.0.6. If you still need to see it let me know.
It is signal 3 in Linux: kill -3 .
When this occurs, PLEASE generate a stacktrace dump (CTRL+BREAK in the console on Windows), without it, we won't be able to help you.
Just thought I would update this report.
This has not occurred again after I brought a corrupt cookie issue to the attention of the Tomcat people and they patched it. I suspect it was this issue that was causing the deadlock, perhaps by generating an unusually high number of sessions (that all needed to be replicated on nodes joining).
This patch is in Tomcat 4.1.24 so I would recommend anyone using Tomcat with Apache and experiencing this to upgrade to JBoss 3.0.7 to get the fix.