Using TransactionManager with Quasar Fibers (Lightweight Threads)

[nwhitehead]

TL;DR

I am trying to use the TransactionManager with Quasar Fibers. Essentially, in Quasar, fibers are lightweight threads which perform their execution through a fork-join pool. When a fiber blocks, quasar captures the stack in a continuation and suspends the fiber. On resumption, the fiber continues where it left off but will (in a likelihood) do so being executed by a different fork-join thread. This presents a problem when using fibers with Narayana because even though the fiber (and the transaction it started) is never executed by more than one thread at a time, several threads may be involved during the lifetime of one transaction. Quasar takes care of copying and clearing thread locals between executions of a fiber, but it seems the transaction manager takes careful note of each thread that touches a transaction. So while I was able to get this working, the transaction manager logs this at commit or rollback:

2016-05-14 06:31:41,762 WARN  [ForkJoinPool-default-fiber-pool-worker-5] arjuna: ARJUNA012094: Commit of action id 0:ffff7f000001:e88b:5736fe8b:10 invoked while multiple threads active within it.

2016-05-14 06:31:41,762 WARN  [ForkJoinPool-default-fiber-pool-worker-5] arjuna: ARJUNA012107: CheckedAction::check - atomic action 0:ffff7f000001:e88b:5736fe8b:10 commiting with 1 threads active!

I attempted to work around the issue by trying to get the fibers to suspend the transaction before the fiber is suspended (and resume it when the fiber resumes) but Quasar does not support this.

I realize this is a simplistic question for a very broad issue, but assuming there is no concurrent activity related to each transaction by multiple threads, is it safe to ignore these warnings ?

My inquiry to Quasar is here.

Thanks !

//Nicholas

[mmusgrov]

If multiple threads are associated with a transaction and one of them tries to commit it then the other threads may still be still be actively making updates to transactional resources and be unaware that the transaction is about to end.

Could you not arrange things such that the fibre that wants to end the transaction tells the other fibres to finish their work and only proceeds with transaction termination when it is knows they have all finished. From your question it sounds like you can determine when they have all finished their updates.

[tomjenkinson]

Unless I missed something I think that "the fiber is never executed by more than one thread at a time". Assuming ordering is also maintained I expect the general issue you are suggesting could happen (two threads active on one transaction simultaneously, commit mid update) is guaranteed not to happen with Quasar?

[tomjenkinson]

"before the fiber is suspended (and resume it when the fiber resumes) but Quasar does not support this" - does this mean there is no API in Quasar to hook into or that you had issues using some hooks that they may provide?

[nwhitehead]

Mchael & Tom;

Thanks for your replies.

The execution of a fiber is carried out by mutliple threads, but only one at any given time.

When the fiber is first scheduled, it is allocated a forkjoin thread which executes until the fiber is suspended (normally because it gets blocked).

At some point the fiber will be resumed and continue executing, but most likely by a different forkjoin thread (I just mean statistically speaking).

So no two threads would ever be executing in the fiber (or a fiber initiated transaction) at one time.

It looks like this  (my own poor rendering)

The Quasar API does supply a callback after the fiber has been suspended, but at a point when the thread's threadlocals have been removed. At that point a call to TransactionManager.getTransaction() returns null.

I have been discussing this with Quasar's developers. They are concerned about exposing the innards of the Fiber during suspension/resumption (admittedly, it is pretty hairy), but it seems they may support the idea of a callback that only exposes the executing thread (or executes in the same thread, I suppose) before the thread locals are cleared. In this case, I would be able to suspend the transaction. This would require an associated resumption callback that would occur after the thread locals have been restored and before the fiber started execution again. This is where I would call TransactionManager.resume().

In general, I am very interested in making the TM work with Quasar. It's a very powerful processing model and the potential benfits of lightweight threads is attracting a lot of interest. It does introduce challenges when trying to integrate Quasar with thread specific functionality in other libraries, and if you take a quick look at their concurrency primitives, they've reimplemented many of the indispensible classes like ReentrantLock and CountDownLatch to make them Fiber (or Thread) specific.

....but my immediate issue is..... if I know for a fact that multiple threads have partiicipated in a transaction, but sequentially, cooperatively and never concurrently, can I ignore the Arjuna warnings, or am I fundamentally breaking something ?

Thanks !

//Nicholas

[mmusgrov]

Another option, if you can get at each thread that has ran the fibre or continuation, could be to remember all the threads that have been involved and add them as thread locals. Then when it's time to commit/end the transaction you could manually update our view of which threads have been involved by calling narayana/ThreadActionData.java at master · jbosstm/narayana · GitHub

[nwhitehead]

Michael;

Sounds promising. I will look into it.

//Nicholas

[mmusgrov]

Nicholas Whitehead wrote:

The Quasar API does supply a callback after the fiber has been suspended, but at a point when the thread's threadlocals have been removed. At that point a call to TransactionManager.getTransaction() returns null.

I have been discussing this with Quasar's developers. They are concerned about exposing the innards of the Fiber during suspension/resumption (admittedly, it is pretty hairy), but it seems they may support the idea of a callback that only exposes the executing thread (or executes in the same thread, I suppose) before the thread locals are cleared. In this case, I would be able to suspend the transaction. This would require an associated resumption callback that would occur after the thread locals have been restored and before the fiber started execution again. This is where I would call TransactionManager.resume().

You could use the existing Quasar on suspend callback to call calling narayana/ThreadActionData.java at master · jbosstm/narayana · GitHub

But you would still need to resume the transaction either by calling resume() or by calling narayana/ThreadActionData.java at master · jbosstm/narayana · GitHub

[marklittle]

Another option would be to define your own CheckedAction implementation which doesn't issue the warning message if you are sure that is safe.

[tomjenkinson]

Nicholas Whitehead wrote:

....but my immediate issue is..... if I know for a fact that multiple threads have partiicipated in a transaction, but sequentially, cooperatively and never concurrently, can I ignore the Arjuna warnings, or am I fundamentally breaking something ?

If you are using JTA then this won't work without some extra code as a transaction won't automatically be removed from the thread it was associated with originally. You will need to suspend the committed transaction from the thread. This is a safety feature to ensure that a thread which completes the transaction on behalf of the original thread does not cause unexpected consequences in the original thread.

E.g. you can't have code that just looks like:

tm.begin

// do work

tm.begin

// do work

With the commits happening in different threads. You would need to correctly disassociate the thread before the second begin.

[nwhitehead]

Excellent point. I need to make sure that the forkjoin threads are not associated to a transaction that does not belong to a fiber that it is about to continue.

Thanks.

//Nicholas

[marklittle]

True, but if Nicholas is sure that these fibres are all essentially aspects of the same "virtual" thread then it's logically the same as a single thread being associated with the transaction. The CheckedAction code is based on some fairly fundamental work dating back to OTS and discussed http://www.cs.ncl.ac.uk/publications/trs/papers/628.pdf

But since one approach was never going to be suitable for all threading situations (even in OTS/JTS) that's why we make it configurable. I see two possible solutions using this: one, which I mentioned earlier, that simply ignores the thread associations and doesn't issue the warning (that could be risky if truly independent threads and these "fibre threads" are being used in the same transaction), and one that does the same thing which somehow knows how to distinguish the Java language threads being used by the "same" fibre and ignore them, but issue a warning if multiple independent threads are present during commit. But I don't know Quasar to be sure this is even possible.

[marklittle]

That's very risky. Really no one should be playing with ThreadActionData directly. Go through CheckedAction.

[nwhitehead]

Michael;

I think I see where I could [carefully] go with ThreadActionData

In this scenario, I would have a reference to the Transaction (BasicAction) and I want to execute a TransactionManager.suspend(), but I can't since the underlying threadlocal will not be accessible, and there's no such javax.transaction API call as myTransaction.suspend().

But ThreadActionData fills in the missing piece where I can call ThreadActionData.pushAction (myTransaction, false) which I think accomplishes the same thing.

By the same logic, on fiber continuation, I could call ThreadActionData.popAction().

This is a large simplification, and I take Mark's caution seriously, but to make this work, I need to be able to suspend a transaction (disassociate the transaction from the fork-join thread) and then resume it again.

Assuming this works as expected, I would not expect to see any warnings issued.

In case anyone is interested, this is the enhancement I requested from Quasar. I am trying to keep the two threads synchronized (no pun intended).

[mmusgrov]

I agree it is risky but CheckedAction will not help here since we print the warning before calling CheckedAction and then again in the default implementation of CheckedAction. We can fix that but it would mean Nicholas would have to wait for the next narayana release.

Nicholas are you okay with waiting for another release, the fix that marklittle refers to would be to override the default implementation (narayana/CheckedAction.java at master · jbosstm/narayana · GitHub) and suppress the warning there if you know it's safe?