Scalability and Performance with Human Tasks

[newohybat]

Hello,

I need a piece of advice concerning following issue:

I thought about using human tasks for actions which can player do in a game. My idea was to start processes with allowed actions, while the first node of process would be human task node. Than player would choose to complete one task and process would proceed. But with this scenario I would end with hundred of thousands of pending human tasks. And this is what I am afraid of.

I have to do a showcase application with Drools and JBPM, which is MMO strategic game. And the main pro for the mentioned approach was that at time of creation of player, there would be a bunch of processes started and while completing these processes, other would be started as new opportunities emerge for the player. All this logic would be defined within JBPM processes, WorkItemHandlers and similar components of the suite.

On the other hand I could make a service layer, which would encapsulate the JBPM api for starting processes, etc. And this service wouldn't do anything else than simulating the human task in the preceding scenario. There wouldn't be any "idle" processes within session, which wouldn't have been completed even for weeks. But I loose the simplicity and elegancy of previous scenario, where everything would be based on processes itself.

So after short introduction at last to my question: How is it with scalability and performance with JBPM in my situation?

Best regards

Jan

[salaboy21]

Well it depends on how many players do you have and how many processes per player. As you mention, if you have a process with several human tasks the process will be stored in the database until the player completes the last standing task, so you don't need to be afraid of that.

In order to give you more thoughts you need to define:

1) When the process will be created

2) what kind of scalability do you need? Tasks are stored in the database.. so if the database scale your application will be able to scale too

3) does a process involves the interaction of several players?

The more info that you can give us the better that we will understand your scenario

[newohybat]

Hello Mauricio,

thank you for your time.

The application (game) would be suited for hundred of players, let's say 300.

To answer the need for further definition I'll try to describe my scenario.

If you're familiar with Civ series, this is very similar to that.

Map consists of Tiles, their numbers would be in tens of thousands, let's say 100 000.

Tiles can be improved for better properties. Let's say 10 improvements available.

Players have cities, they start with one, but their number could rise easily. Their count could raise to tenth of the number of tiles, so assume 10 000.

In these cities players can build buildings, there would be about 30 buildings available for building at max.

Each player conducts research of technologies, one technology reached enables others. Not all are available at once. Thus at most 10 advances are available at once. Advances form the technology tree.

Then I thought about candidate operations, that can be modeled via processes with human task.

• research of technologies - initially I would create processes which simulate technologies, which can be researched by a new player. As part of process there would be a node which enables technologie's successors in development tree. And so on until no successors of technology reached are available. So for 300 players and at most 10 advances available for research I would end with 3000 processes potentially waiting for user interaction.
• construction of buildings in cities: again I thought about modeling this as processes with human tasks as the actual trigger of action. These processes would be started in the advance processes mentioned above. For expected 10 000 cities with 30 buildings available to build (when all advances are reached and no building is in city yet - new city) it equals 300 000 of processes.
• improvement of tiles (irrigation, road, ...): for each tile there would be processes enabling their improvements. There are 100 000 tiles, 10 improvements, so 1000 000 processes would be needed (it seems sick :-D ).

That results with 330000 processes for the first two, million more with the third. Isn't this too much?

I know I could avoid the human tasks by creating a service layer which would start processes on demand, but this seemed as interesting alternative. The reason I wanted to implement these game operations by processes with human tasks is that after starting initial processes for new player, all others would be started automatically by the already started processes. I wouldn't need any major changes in code when altering or extending the game operations.

A process doesn't involve interaction of several players. But there is main process, which based on timer raises an event. That event should be intercepted by all others.

Well maybe I shouldn't use term scalability. I meant what are the limits of jbpm framework concerning the throughput, whether the amount of processes don't affect the performance significantly. You answered my unspoken question, that I shouldn't worry about storage of pending processes, but how is the performance when many processes are available to be run?

Jan

[salaboy21]

Hi Jan, and sorry for my delay in responding back.

Ok.. so you mention 10 000 tiles.. but in games is not so common to be processing the improvements for all the tiles at the same time, which makes me think.. a player can be improving 100 tiles max at the same time and then he will be attacked by another player which will stole 20.. so I don't see a problem with that.

You can use human tasks and processes to model the improvement researches. This will give you a lot of flexibility to scale in the sense that if you want to make it look more dynamic the only thing that you need to do is add more process definitions. If for some reason the human task service is too much for your needs, you can copy the same approach and implement something like a "Player Service" following the same principles of how human tasks interact with the process instances but making sure that it only include what you needs for the game. In doing so, you will decouple the storage of the player service in any persistent storage that scales better than a database.. (but this is only if you really need to).

The good thing about using this approach is that all the pieces are interchangeable, so if for some reason you notice a huge impact in performance you just create your own performant service (as the one that I mention before).

Wdyt?

[newohybat]

Hi,

again thanks for response.

As for the problem of improving tiles. Under influence of your observing I've realized, that only tiles managed by a city could be improved, thus the number of "improvement processes" wouldn't be so high initially. Moreover with improving of tile should be tied a unit, say worker, who enables the construction of an improvement. Thus "improvement process" would be only started for workers, who stand on a tile managed by a city. Starting and canceling of "improvement process" would be a reaction to move of a worker. With this I am quite satisfied. :-)

As for researches there I don't see any problem given the number of possibly simultaneously occuring processes. So I would consider research as closed issue.

Similar situation is with cities and its improvements(buildings). There won't be so many of them, I think.

I like your suggestion about the "Player Service". I think I'll start with the human task service, to get the game up and running at least in test mode. Then I'll consider the special service.

I'll ask different question. I have a project with entities, their dao, services, etc. I want to use this persistence project. Its interface only works with DTOs. But I want to persist changes made by drools and jbpm (but rather drools). I have a service for say PlayerDTO. I can call it, serve it altered object (fact). But where should I do this?

It seems to me unwise to call the service in RHS of each rule because I find it disturbing. I searched for possible listener based solution, but with no avail. Then I thought about drools rule with lowest priority (would be fired after all the alterations by other rules) where on the RHS I would either put them to a stack for later batch update, or update them directly. I like that this solution would update only altered facts, given I set @watch attribute in rule definitions for a propertyreactive facts. So is there a custom how to do that, or it is up to me to do it my own way?

I went back and examined your book JBPM5 Developer Guide once more. I came across the advanced section about multiple knowledge sessions. Then I thought I would split my knowledge into multiple sessions. One would contain all the facts I need in game (tiles,cities,players,units) together with drl definitions which do the computing of properties, production of cities, treasury, etc. Let's name it "Base Session". Then in seperate session I would have JBPM processes, that we talk about: research, tile improvement, city improvement....... well all the processes which would have a Human Task which trigger the action. Let's name it "Processes Session". The processes as I imagine them would not work over all the facts within session, it would work only with variables, so it seems to me that it is better to put them aside from the facts.

I would then need a few things:

• start and cancel a process in "Processes Session" by a rule in "Base Session"
• insert an event into "Processes Session" from the "Base Session"
• alter a fact in "Base Session" from within "Processes Session"

And here is my yet another question. Would be such a separation wise? And could be all my requests satisfied as for communication between sessions?

Thanks for your time and patience

Jan