We stumbled upon a corner case where some queries like this:

SELECT .. WHERE [ns:property1] = 'A' AND [ns:property2] = 'B' ORDER  BY ['ns:property3']

randomly get stuck to an infinite loop on ModeShape 5.x.

After examining the issue we found out that in the query SELECT phase results may be filtered (NodeSequence.filter()) so that the first or first few batches returned are empty. Now, when BufferingSequence.loadAll() loads the rows in the buffer batchSize of zero is returned, which in turn causes the loop when SortingSequence tries to load batches of size zero:

modeshape/SortingSequence.java at master · ModeShape/modeshape · GitHub

Changing BufferingSequence as follows seems to correct (or at least workaround) the issue (not sure if this is necessarily a proper fix):

diff --git a/modeshape-jcr/src/main/java/org/modeshape/jcr/query/engine/process/BufferingSequence.java b/modeshape-jcr/src/main/java/org/modeshape/jcr/query/engine/process/BufferingSequence.java
index b779993..0a79cb4 100644
--- a/modeshape-jcr/src/main/java/org/modeshape/jcr/query/engine/process/BufferingSequence.java
+++ b/modeshape-jcr/src/main/java/org/modeshape/jcr/query/engine/process/BufferingSequence.java
@@ -137,7 +137,9 @@ public abstract class BufferingSequence extends DelegatingSequence {
                     ++batchSize;
                 }
             }
-            firstBatchCounted = true;
+            if (batchSize > 0) {
+               firstBatchCounted = true;
+            }
             batch = sequence.nextBatch();
         }