Deadlock in BlockingCallbackStore

[jeremystone]

Can get deadlock when using JBoss Remoting (JBR 2.2.2.SP4) along with a JBoss Messaging (JBM 1.4.0.SP3) https connection factory.

I posted a report of a deadlock situation to the JBoss Messaging user forum (see http://www.jboss.com/index.html?module=bb&op=viewtopic&t=143024 for stack traces). I am posting it here also as it cuts across both projects.

To repeat my analysis from the JBM topic:

On the client
Timer-3 thread:
- Is polling for messages

On the application server
http-0.0.0.0-4459-5 thread (the server-side continuation of Client
Timer-3):
- Is waiting to lock the ArrayList of callbacks held by
the eventListenerContainer-10 thread.

eventListenerContainer-10 thread:
- Because the ServerInvokerCallbackHandler has decided that it needs to
persist the callback and is using a BlockingCallbackStore,
eventListenerContainer-10 is also waiting on an object that will be released
when someone calls BlockingCallbackStore getNext() (i.e. its waiting for
http-0.0.0.0-4459-5)

It occurs when BlockingCallbackStore gets used when memory is limited.

Any ideas? Shall I raise a bug?

[ron_sigal]

Hi Jeremy,

Sure, you could create a JIRA issue at https://jira.jboss.org/jira/browse/JBREM. I'll look into it as soon as I can.

Thanks for the information.

-Ron

[jeremystone]

Sorry, Ron, I tried but I don't seem to have bug sumit permissions.

[ron_sigal]

"JeremyStong" wrote:

Sorry, Ron, I tried but I don't seem to have bug sumit permissions.

Hmmm. Are you registered? I thought anyone that's registered could create a JIRA bug issue.

[ron_sigal]

Ok, I've created JBREM-1045 "ServerInvokerCallbackHandler can suffer deadlock when used with BlockingCallbackStore" (http://jira.jboss.org/jira/browse/JBREM-1045), applied a fix to branches 2.2 and 2.x, and attached a preview version of jboss-remoting.jar from the 2.2 branch.

The fix is to change ServerInvokerCallbackHandler.handleCallback() by taking the call to persistCallback(), which ultimately calls BlockingCallbackStore.add(), out of the synchronized(callbacks) block.

Jeremy, if you get a chance, let me know if the fix works for you. And thanks for pointing out the problem.

-Ron

[jeremystone]

Many thanks, Ron. I will give it a try soon.

[jeremystone]

That seems ok now. Thanks again.

[ron_sigal]

Glad to hear it.