Migrating from Wildfly9 to 10 cause issues with ThreadLocal

[abhinav.gupta01]

Hello,

Our application has been running successfully from past two years on widlfly9 , and we recently upgrade to wildfly10.

In our application BL we are keeping some properties on ThreadLocal.For BL we are taking threads from server pool.

In some occurances , we observe that new thread already has values populated on ThreadLocal. ( Seems like thread was not GC, before coming to application back )

Can we somehow solve this on WildFly  server configuration level ?

Attached standalone-full.xml.

Please suggest.

Thasks,

Abhinav

[jaikiran]

abhinav.gupta01  wrote:

Hello,

Our application has been running successfully from past two years on widlfly9 , and we recently upgrade to wildfly10.

WildFly 10 is already outdated, IMO. So if you are indeed upgrading it's better to upgrade to WildFly 12 (12.0.0.CR1 was just released today and Final will be released soon). If not WildFly 12, then at least WildFly 11 would be a better option.

abhinav.gupta01  wrote:

In our application BL we are keeping some properties on ThreadLocal.For BL we are taking threads from server pool.

What exactly is BL and what does the code look like which deals with this thread interaction?

abhinav.gupta01  wrote:

In some occurances , we observe that new thread already has values populated on ThreadLocal. ( Seems like thread was not GC, before coming to application back )

What exact values are those in the ThreadLocal? Something that your application sets?

[abhinav.gupta01]

Thanks Jaikiran.

>>What exactly is BL and what does the code look like which deals with this thread interaction?
I mean our application code.

>>What exact values are those in the ThreadLocal? Something that your application sets?

Yes.

private static ThreadLocal threadRequest = new ThreadLocal();

threadRequest.set(<MyObject>);

Now , with wildfly10 , I see that even on new Thread. In our logs , we also print thread id's. Now , I see that if same thread is return by thread pool , then in threadlocal my object is still available , while the thread life cycle was already completed once.

I think till wildfly9 , either the thread was always a new thread , or somehow it was cleaned before server supply it to application.

Thoughts ?

Thanks,

Abhinav

[jaikiran]

Server managed threads are pooled. Any application that sets thread locals is expected to clear it off once the "task" is completed. So the right thing to do here is to clear off those thread locals once you are done.

WildFly uses jboss-threads library for thread management and IIRC, it used to have (still has?) a thread local clearing executor. I don't remember if WildFly 9 was configured to use such an executor which would auto-clear all these thread locals and if later versions of WildFly intentionally (or accidentally) changed that behaviour. dmlloyd might know more.

Either way, the portable way to do this would be to clear off your application specific thread locals when you are done with your work in that thread.

[abhinav.gupta01]

Hello,

Our issue was solved by removing pool for SLSB from standalone-full.xml , I think it was not in wildlfy9 by default and added in defaults later for 10.

Thanks Jaikiran for pointers.

PS :  I will implement cleanup logic for my application and enable pooling again 

[jaikiran]

abhinav.gupta01  wrote:

Hello,

Our issue was solved by removing pool for SLSB from standalone-full.xml , I think it was not in wildlfy9 by default and added in defaults later for 10.

That's correct. As noted in release notes of WildFly 8 WildFly 8 Final is released! · WildFly  :

EJB SLSB pooling is disabled by default, which is a better performing configuration for most applications

That was then changed in WildFly 10 as noted in the release notes WildFly 10 Final is now available! · WildFly :

WildFly now pools stateless session beans by default ...

I would however still recommend that the application cleanup the threadlocal variables once it's done with the work, because it's brittle relying on the internal pooling details of the server.

[abhinav.gupta01]

Right. I am planning to have cleanup of threads once done with those.

Thanks Jaikiran for your immense support.