TimeoutException printing stacktraces of threads holding loc

[galder.zamarreno]

I think it'd be very good idea that whenever a TimeoutException is thrown, the stacktrace of the thread(s) holding locks is also printed out. The stacktrace of the thread requesting the lock is already being printed.

I can see this being very useful from a troubleshooting point of view, specially in situations when TRACE level logging can not be requested, i.e. production.

Such change would require changing the current lock implementations. We could take advantage of the existing boolean print_lock_details parameter to IdentiyLock.toString to and pass it on to the lock implementations, who contain the thread details.

Thoughts?

[manik]

The lock may not always be held by a Thread. It may be a GlobalTransaction. But I suppose it could be done for the instances where the lock owner is a Thread.

Could this get unnecessarily noisy for when a WL times out because of multiple RLs?

Also, these stacks would be printed to the log in the toString method, not appended to the return value of toString. Is this what you had in mind?

[galder.zamarreno]

Could this get unnecessarily noisy for when a WL times out because of multiple RLs?

Hmmm, good point. I guess we could log it at TRACE level. Adding TRACE level for the specific lock class in use could be part of the troubleshooting steps for TimeoutExceptions. This type of exception is quite likely the one reported most often.

Also, these stacks would be printed to the log in the toString method, not appended to the return value of toString. Is this what you had in mind?

Yeah but as long as the toString is called only when TimeoutException is thrown, otherwise it'd pollute other situations.

[jason.greene]

IMO adding deadlock detection would probably help reduce the need for all the logging.

[galder.zamarreno]

IMO adding deadlock detection would probably help reduce the need for all the logging.

Hmmm, I'm not sure whether we're in the same page here. Let me explain:

When you get a TE, you get a stacktrace of the Thread trying to acquire the lock. If you wanna find out what the thread(s) holding the lock are doing, you have to:

1.- Enable %t in the logging pattern so that you can figure out what each thread is logging.
2.- Add trace for cache and any other application that might be using the cache, i.e. portal, hibernate, ejb3....etc
3.- Ideally, you want the customer to provide a stacktrace so that you can see where the calls are coming from. This is specially useful in situations where projects hardly have any debug or trace logging, i.e. ejb3, portal.

1 and 2 are easy to get. 3 can be a bit trickier and depends on user timing. With this post, I'm trying to make 3 more predictable.

Jason, from what I understand, a TE might not be always related to a deadlock. It could just be that the other thread is being slow.

[galder.zamarreno]

FYI, I have created a thread in Clustering forum to continue this discussion, see http://www.jboss.com/index.html?module=bb&op=viewtopic&t=127721