Timer API

[porcherg]

In the last weeks we looked more deeply at the job executor and timer service of the PVM.

We see these use cases for the timer service:
- Timer service will be used in xpdl (deadlines) and in bpel (wait, pick...).

In all these cases,
- the task to execute on timer completion is to signal an execution.
- a timer needs to execute a task after a given delay or on a specific date.

In some cases, we would like to be able to have a task repeated at a specific rate.

In the extensions of the pvm, we want to be able to run with/without persistence, in standard/enterprise environment.
So we would like to create an interface for the TimerService.

Interface proposal:

This interface contains methods to schedule a task. A task can be scheduled at a specified date or after a specified delay.
A repeat delay can be specified.

A task is defined by an Activity and an Execution. When the timer is fired, the execute method of the activity is called, with the specified execution as parameter.
To allow more extensibility, activities are wrapped in an ObjectReference (just as ExternalActivities in Nodes).

As a result, the interface provides methods to schedule an activity:

public long schedule(ObjectReference<Activity> activityReference, Execution execution, Date date);
public long schedule(ObjectReference<Activity> activityReference, Execution execution, long delay);
public long schedule(ObjectReference<Activity> activityReference, Execution execution, Date date, long repeatDelay);
public long schedule(ObjectReference<Activity> activityReference, Execution execution, long delay, long repeatDelay);

These methods return an id associated with the scheduled task. This id can be used to cancel a task.
This interface contains other methods to query the timer service:

public boolean cancel(long id);
public Collection<ObjectReference<Activity>> getScheduledActivities();
public long getNextScheduledActivityTime();
public ObjectReference<Activity> getNextScheduledActivity();
public ObjectReference<Activity> getScheduledActivity(long id);

Implementation:
For our tests, Pascal has implemented this interface:
- in memory by using a JDK timer
- in db by using the pvm job executor

For the db implementation, we had to modify some parts of the pvm:
- the Timer class was not implemented (execute method was commented).
- the jobExecutor was executing all the tasks that were in the Job table in database.
So as to be able to execute a timer job only in a future time, we added a field "eligibleDate" to the job. A job cannot be executed before this eligible date.

- the jobExecutor wakes up when a new job is added. To wake up the jobExecutor when a task becomes eligible, we use a JDK timer that notifies the jobExecutor that a new job has been added. This way, we have a JDK timer that fire events, and a scheduled Job in db for persistence and crash recovery.
For crash recovery, a special method needs to be called. This method will recreate JDK timer for the persisted scheduled jobs.
Jobs that would have been executed during the crash have become eligible for the jobExecutor and are executed automatically on restart.

These two implementations work with basic tests.

While modifying the jobExecutor, we tried to see how to make it independent from the persistence.
We identified these issues:
- JobService is hibernate specific, we can create an interface for the service and have an in memory implementation
- JobExecutorThread obtains job Ids from the jobService, and creates a ExecuteJobCommand with this id. ExecuteJobCommand loads the job from the db. Maybe executeJobCommand can delegate the job loading to the jobService.
- JobExceptionHandler has the same behavior as ExecuteJobCommand.
- StandardMessageService stores the job directly in the db, maybe the job can be saved by the JobService.
- on completion, job are removing themselves from the db, maybe the job can call a remove method in the jobService.

Pascal tried to do these modifications, jobExecutor tests pass in db.
He tried to implement the memory version of JobService, but has not tested it yet.

What do you think of the proposed TimerAPI ? Do you see some use cases we cannot implement ?
What do you think about the modifications in the JobExecutor ?
Can this be a valid implementation for asynchronous jobs in memory ?


Regards,
Pascal and Guillaume

[tom.baeyens]

requirements are very much in sync with the requirements that I see.

api is close, but with one significant difference: in your proposal you have a number of schedule methods. i see a couple more parameters that are needed for creating timers. then the approach that you give, will result in one method for every combination of timer properties. if the number of properties gets above 5 the method-per-combination approach is not convenient any more.

therefor, i propose to introduce a value object that holds all the configuration parameters for a timer. the constructors of the timer value object could correspond to the method signatures that you give. but now, the timer has a number of other properties that can be set (with setter methods) before the timer is scheduled.

can you consider the following interface ?

interface TimerService {
 void schedule(Timer timer);
}

class Timer {
ObjectReference<Activity> activityReference;
Date dueDate;
Duration repeat;
int retries;

...some more properties...

... constructors, getters and setters ...

}

[tom.baeyens]

one more appendix: i don't think it is necessary to work with a generated return value to cancel the timer. instead, the timer object itself could be used (or modified).

For example, in the persistent scenario, hibernate assigns an id. When you later load that timer, then you can call cancel in the timer service with that timer object. Similarly, in a transient implementation of the tiemrservice, the TimerTask could be associated to the timer object. Since it is in memory, the TimerService can assume that it is the exact same timer object that will be passed in the timer service cancel method.


ps. i know that there are still a couple of open questions. i'll get to those after we discussed this value object approach in the interface.

[pascal.verdage]

After discussing it with Guillaume and Pierre, it appears that we could easily implement a Timer class if it extends Job. This class would be close from the original Timer class that existed in the pvm.

In that case, we can use the Timer object to cancel the timer.

Regards,
Pascal

[tom.baeyens]

what would be the problem then to go further with the Timer class as it is now ?

[pascal.verdage]

The difference between the Timer class as it is now and the one used in the tests is the deletion of two fields: 'processElement' and 'transitionName'. The corresponding actions are not the timer's job.

Firstly, taking a transition becomes a part of the execution of the activity. We could provide an TakeTransitionActivity class which would do that.

Secondly, the timer don't fire an event in the name of a ProcessElement, but the ProcessElement listens to the event fired by the timer then dispatches it.

Regards,
Pascal

[tom.baeyens]

"pascal.verdage" wrote:
Secondly, the timer don't fire an event in the name of a ProcessElement, but the ProcessElement listens to the event fired by the timer then dispatches it.

that puts things indeed in a whole different perspective as i was used to looking at it. let's keep exploring that path.

i think we would still need the process element as events can only be fired on a given process elements. no ?

the motivation for the transitionName was a performance optimisation. but let's not yet go into that as probably i will want to remove it myself as well given the new way on how we want to deal external activity instances.

[pascal.verdage]

Actually, the method fire requires an ObservableElement, which extends ProcessElement.
With this inheritance, we can't fire the event from the timer. But we could have an activity that fires a given event.

And in order not to have many timers when we want to do several actions, we could also provide an activity that contains a list of activities and execute them sequentially.

Regards,
Pascal

[mvaldes]

Guys,

As an intermediate conclusion I see that we are mostly agree on Timers requirements and the way to implement them.

Bull team has already a working state proposal on Timers which has been already updated with Tom's suggestions:

- To converge into one single schedule method leveraging a Value Object for properties
- To use the Timer object to cancel timers (so no need for return an id on schedule operation)
- Attributes relates to transitions and events were keept into Timer (for further discussion)

At Bull side we have to include support for timers in the next BPEL and XPDL extension milestones (to be released end of the next week) so I would propose the following actions to allow us to work on the Timers at PVM level and to ensure that we could continue Timers refactoring with Tom in the next days:

- Bull team to commit a stable version of the current Timer's implementation (based on the initial work made by Tom and including improvements).

- Tom could easily review those improvements. This also will simplify communication between both teams

Those actions will allows us to release the next M3 without divergence with the PVM source code.

From this commit we could continue the discussion. At Bull side we assume that refactoring will be required in the following days (after the M3) to achieve both (Jboss/Bull) requirements.

Main target would be to leverage job mechanism for both persistent and inMemory use cases.

Tom, are you agree with those actions ?

regards,
Miguel Valdes

[tom.baeyens]

Yes. Good plan.


The only thing that I would like is that properties are not removed without consulting me. They might be there for a reason that you guys don't see yet.

Then you can provide that feedback and I can remove it if I don't recall what it was for.

OK ?

[mvaldes]

Ok Tom, the idea is to keep those properties for further discussion (even if we do not leverage them yet).

I propose to add a comment on them to do not forget afterwards that they could be leveraged.

regards,
Miguel Valdes

[tom.baeyens]

ok

[pascal.verdage]

Guillaume committed the work on timers and JobExecutor I did in the last weeks (revision 691).

This commit allows Bull to use timers in Bonita and Orchestra M3 but all this is still arguable as well as the points that were still under discussion.

The changes are:
- addition of a field 'eligibleDate' to Job (date until which the job should not be executed) and modification of requests
- modification of the pvm timer (I commented in the transitionName and the processElement)
- renaming of the timer to PvmTimer to avoid naming conflicts in StandardTimerSession
- modification of the interface TimerSession (I renamed 'create' method in 'schedule' and added information getters methods)
- 2 implementations of TimerSession + tests
- modification of the different jobs to suppress the db dependency (the jobs now ask the JobService to delete them)
- creation of an interface JobSession based on the former JobService which becomes an implementation
- creation of a non-persisted implementation of JobSession

The tests of the TimerSession have not been added to the global test suite.

In the bindings, I kept the name job-service for StandardJobSession. It should be renamed but I preferred to wait until a decision is taken regarding the following question.

I created bindings for the implementations (TimerSession and JobSession) but we could have only one binding per interface with an attribute to choose the mode (persisted or in-memory). What do you think of it? Should we have one binding per interface or one binding per implementation?

Revision 692 updates names and configuration files in jobExecutor tests. There is an error in ContinuationTest (it is previous to r691) but I didn't identified it.

Regards,
Pascal

[tom.baeyens]

if you break something that is not in the overall test suite, it should not be a big problem. therefor it is good that you include your working tests in that suite. otherwise i might do a refactoring and not notice that i break your test.