Problem with storagemanager completions and syncNonTransacti

[timfox]

Currently the storageManager calls lineUp() on the operation context for *every* record added to the journal, even ones that were added not sync.

This doesn't work when syncNonTransactional = false.

Consider the case of sending a single persistent message with syncNonTransactional = false.

It's sent blocking.

The message reaches the server. The initial storeMessage() occurs with sync = false, and a line up is added to the context. The next storeReference() is called with sync = true, another line up is added to the context, but since syncNonTransactional is false the record is added to the journal with sync = false.

There is now two records in timed buffer with sync = false.

Since no more records arrive, these records never get flushed (which is ok).

Since the records never gets flushed, completion never occurs and the client never gets its response, so it hangs.

I assume there must be no tests with the new completion code and syncNonTransactional = false or this would have been caught.

What is the fix here?

Shouldn't we only be adding lineups for sync operations on the journal?

[timfox]

This seems to be a bug in the way operation context works.

For some operations e.g. non sync adds, updates or deletes to the journal- the callback should be executed immediately as long as there are no previous callbacks waiting in the queue for the operation context.

This means we need to distinguish between two types of lineups - one which is waiting for a completion and one that is not.

For non sync lineups there is also no need for a completion - we don't care about it.

Here are some examples of sequences of operations on a context

(S = sync operation, NS = non sync operation)


Case 1: Just non sync operations

NS1 - send response immediately since nothing lined up
NS2 - send response immediately since nothing lined up

Case 2: Sync followed by non sync operations

S1 - response won't be sent until completion comes back
NS1 - response won't be sent until the completion for S1 comes back
NS2 - response won't be sent until the completion for S1 comes back

[now the completion for S1 comes back, and this triggers...]

Response for S1 sent back
Response for NS1 sent back
Response for NS2 sent back