Thread Deadlock when <distributable/> tag is present with jboss-eap-6.2/AS 7.3.0.Final-redhat-14

[juliusmensing]

Good Day to the JBoss Clustering community,

this is a web-application related question.

JBoss server: jboss-eap-6.2/AS 7.3.0.Final-redhat-14

I have issues when running a web-application which was deployed as an extracted war folder.

Since it is clustering enabled it has the <distributable/> tag in the web.xml file.

Somehow when this tag is present a thread deadlock (reproduced with VisualVM)

occurs quite often (rarely it works).

When the tag is removed the application works without any issues.

The same application runs fine on

• JBossAS [6.1.0.Final "Neo"]
• JBoss (Microcontainer) [5.1.0.GA (build: SVNTag=JBoss_5_1_0_GA date=200905221634)]

with the <distributable/> tag present.

Am I missing something in the code or a setting which is required for

jboss-eap-6.2/AS 7.3.0.Final-redhat-14 to work with a clustering enabled application?

Thank you for your help

[rhusar]

I am no aware of any problem that would be causing a deadlock. Can you provide a thread dump to see what is thread locked? (e.g. jstack)

[juliusmensing]

Hello Radoslav, here you go:

http://download.softpro-service.com/jme/JBossDeadlock.log

[rhusar]

The problem is introduced by 3rd party classes de.softpro.doc.SignDocDocument and de.softpro.sdweb.SessionDocument -- are these your classes or 3rd party libs? Looks like 3 separate client requests are locking on the same objects and this creating a deadlock..

[juliusmensing]

Good Morning Radoslav,

thank you for the analysis.

Yes, everything under de.softpro is from us (SOFTPRO GmbH).

How come that the same application works fine on JBoss 5 and 6 as well as Tomcat 6 and 7

without any issues and having the <distributable/> tag present.

Are there any significant differences in terms of clustering and this tag

when it comes to JBoss 7 ?

[juliusmensing]

Hello Radoslav,

I was in a discussion with our developers this morning

and the problem seems to be identified.

There is a inconsistent locking strategy between some of the

methods within our application.

I have uploaded three log files which demonstrate the thread lock scenario:

1. http://download.softpro-service.com/jme/JBossDeadlock.log
2. http://download.softpro-service.com/jme/JBossDeadlock2.log
3. http://download.softpro-service.com/jme/JBossDeadlock3.log

The getDocumentType method - which always appears in all three situations - is locking in this order: SignDocDocument -> SessionDocument

The other methods (e.g. getMetaDataString, getField, convPagePointToCanvasPoint) are locking in exactly the opposite order: SessionDocument -> SignDocDocument

Thus a dead lock is created.

We presume that this is the issue here.

[rhusar]

Are there any significant differences in terms of clustering and this tag when it comes to JBoss 7 ?

Sure. To begin with JBoss 7 was a major rewrite of many components, clustering did not change extremely, but there are definitely upgrades to major new versions of Infinispan, JGroups, Marshalling, etc.

IIUC you are accessing the session data concurrently, which needs to be dealt with very cautiously. Easy fix is to serialize all access to your session, possibly using a filter, but I am not sure if that's desirable.