Why do you need to retrieve many processinstances at once in combination with the variables of those instances?
In trying to build this queue of 2000 process instances, I want to filter based on the values of these variables. So, say 1000 of these jobs are meant for someone that can see salary and one of the variables is salary...I want to show that person 1000 jobs and not 2000.
The only way I know to do this now is to loop the 2000 instances, pull back their vars one by one, and process them in code. Making 2000 calls, regardless of how lightweight they are, would seem to be less efficient than making say 10 pagination calls, pulling back a larger set of data, I would think? In any case, this is the basis of my original question...
Normally tasks are used to differentiate... If something with the salary needs to be done there is a task for this and you have taskslists...
But the tasklist is available once I know whether or not I have the job right? I want to filter on salary before i even present the user with the opportunity to work on the job, which is why I'm trying to snag the variables to look at.
I apologize for going around in a circle... Seems like I'm just missing a basic understanding of how JBPM is built to handle what I'm desiring to do "out of the box" or more efficiently/effectively. I'm potentially doing something (design wise) that is throwing things out of whack!?!?!
Perhaps you could use AssigmentHandler on task nodes to check process variables and assign given process instances to a different groups of users?! That should help showing only 1000 of process instances for people that should handle salary issues and so on... just a rough idea.
But you have processes, processes have tasks, tasks differ from eachother and tasks can be for different users. What you seem to describe is a generic task and based on the availability of as certain variable, that task is either for one user or another. Then the suggestion from Maciej is something you could do..