jBPM 4 Emails Sync vs Async

[bradsdavis]

Alejandro and I were discussing the job executor and the sending of emails.

It is our theory that it would be best to always schedule emails to be sent rather than try to accomplish the task synchronously.

The benefit, which we believe is worth the cost of scheduling and executing asynchronously, is that emailing would then be participating in JTA transactions.

In other words... if the process failed after the email node was encountered, the email scheduling would be rolled back, and the email would not be sent -- which in essence is allowing JavaMail never to be called unless the transaction succeeds.

[kukeltje]

Yes, sounds like a good decision to me

[camunda]

+1

[jbarrez]

Agreed, conceptually this is the best choice.

Howerver, I would also like to have the synchronous option (as the non-default). Eg. for the sake of demos, this is much nicer to have ;-) But I'm sure there are other use cases.

[tom.baeyens]

sending emails direct or asynchronous should be a global setting.

also think about testing.

i used dumpster before and that worked great for testing emails.

if there is a global configuration property to send emails directly, then no special testing infrastructure should be provided.

[jbarrez]

Tom, I don't follow you with the global setting. I still think it should be able to be configured on the node itself.

Imo, there are cases in which you want the mail directly: eg for handling an exception or a timer times out when waiting for an external system -> send an email to the system admin directly.

In this case, scheduling it and waiting for the jobExecutor will cause a serious and unacceptable slow-down ... unless the jobExecutor is polling all the time (which is not good for performance).

For normal mails, I agree they are to be scheduled.

[tom.baeyens]

"jbarrez" wrote:
Tom, I don't follow you with the global setting. I still think it should be able to be configured on the node itself.

they don't exclude each other. ideally we have them both. in the configuraiton file, the default value can be specified and in the process, the default can be overwritten. but we need to prioritize which one we build first. my take on prioritization would be this:

1) direct sending
2) global configuration setting
3) configuration per usage in the process file

"jbarrez" wrote:

Imo, there are cases in which you want the mail directly: eg for handling an exception or a timer times out when waiting for an external system -> send an email to the system admin directly.

In this case, scheduling it and waiting for the jobExecutor will cause a serious and unacceptable slow-down ... unless the jobExecutor is polling all the time (which is not good for performance).

there is no delay. even inbetween polls. the job executor is notified of new jobs by means of a synchronization.

[jbarrez]

Ok, I'm following now, thanks for the overview (and also agree with the prioritization).

So that's a +1 for me too ;-)

[bradsdavis]

In testing, by using a database job scheduler, I would think that there would be no issue.

Seems like extra work when we all agree that in a real system, you would use an asynchronous [either through EJB timers or the database scheduler] to send emails so that they participate in the transaction.

I dont really see the value of making a component that is not real for testing.

[bradsdavis]

OK. I think I have a solution to this.

Both the synchronous and asynchronous emailer will implement the same interface, so they will be interchangeable. The asynchronous will use the synchronous emailler under the hood, it will just be called by the job executor.

[tom.baeyens]

"bradsdavis" wrote:
OK. I think I have a solution to this.

Both the synchronous and asynchronous emailer will implement the same interface, so they will be interchangeable. The asynchronous will use the synchronous emailler under the hood, it will just be called by the job executor.

Close. I discussed with Alejandro yesterday:

we need a MailSession interface for that. mail session would look something like this

interface MailSession {
 Mail createMail();
}

interface Mail {
 Mail setTo(String to);
 Mail setCC(String cc);
 Mail setBCC(String bcc);
 Mail setSubject(String subject);
 Mail setBody(String subject);
 Mail addAttachment(String name, InputStream attachment, String mimeType);

 void send(boolean sendAsync);
}

we thought it was easiest to develop something like the interface shown above.

Activies can get a MailSession from the current environment. But we don't need 2 different implementations.

Clients of this code (Activities) would need to determine if sending needs to be done asynchronous or not. this should eventually be based on a global process engine configuration that specifies the default and jpdl attributes that allow to overwrite this default.

Does this match with your expectations, Brad ?

[bradsdavis]

I personally think that in an enterprise environment, the majority of emails won't be synchronous. I think we should make it asynchronous by default and only send synchronously when defined as such.

Also, I think that the content [subject, body, attachments] should be separate from the recipients. The actual produced content may change per a client's need.

The implementation I was working towards allowed email producers to produce one or more base email type, which internally would be either:

1) Simple emails [think text]
2) Mime emails [think text with attachments]
3) Html emails [templated emails with images, etc]

By making it plugable, anyone can create a producer that extends our base type and adds attachments as needed.

I think an activity should produce one or more emails and send them to the mail service, which I think you are calling MailSession.

So, I think the interface should look more like:

package org.jbpm.pvm.internal.email.service;

import java.util.Collection;

import org.apache.commons.mail.Email;

public interface MailService {
 public void send(Collection<Email> emails);
}

Because the emails at this point already have their email addresses [to, cc, bcc], body, and attachments defined at the point they are sent to the mail service, there is no need for the mail service to care about:

 Mail setTo(String to);
 Mail setCC(String cc);
 Mail setBCC(String bcc);
 Mail setSubject(String subject);
 Mail setBody(String subject);
 Mail addAttachment(String name, InputStream attachment, String mimeType);

It would literally just be for servicing the sending of the base class Email.

[tom.baeyens]

"bradsdavis" wrote:
I personally think that in an enterprise environment, the majority of emails won't be synchronous. I think we should make it asynchronous by default and only send synchronously when defined as such.

i agree.

but in development we need to go step by step.

let's first develop the synchronous mail functionality. and in a subsequent iteration, add the ability to send them asynchronously.

and then let's work on the configuration options in the global jbpm.cfg.xml and in the jpdl process files.

"bradsdavis" wrote:
Also, I think that the content [subject, body, attachments] should be separate from the recipients. The actual produced content may change per a client's need.
...
By making it plugable, anyone can create a producer that extends our base type and adds attachments as needed.

such a framework is good and should be build before this interface. the base class should use this MailSession to send emails. Users should be able to configure their customized SendMail activity. The basic SendMail should be designed for inheritence and customization.

but again. let's go step by step. let's first get the basics working and then add the advanced features like this in subsequent iterations.

"bradsdavis" wrote:
I think an activity should produce one or more emails and send them to the mail service, which I think you are calling MailSession.

So, I think the interface should look more like:

package org.jbpm.pvm.internal.email.service;

import java.util.Collection;

import org.apache.commons.mail.Email;

public interface MailService {
 public void send(Collection<Email> emails);
}

Because the emails at this point already have their email addresses [to, cc, bcc], body, and attachments defined at the point they are sent to the mail service, there is no need for the mail service to care about:

 Mail setTo(String to);
 Mail setCC(String cc);
 Mail setBCC(String bcc);
 Mail setSubject(String subject);
 Mail setBody(String subject);
 Mail addAttachment(String name, InputStream attachment, String mimeType);

It would literally just be for servicing the sending of the base class Email.

i don't see why you want to send multiple emails and get rid of the destination properties. can you elaborate on that ?

[bradsdavis]

The email object itself contains the destination properties.

An email itself depending on its type [simple, mime, html] has different content types. The base email object always has to, cc, bcc.

See: http://commons.apache.org/email/userguide.html

The sending of multiple emails is to add flexibility. Generally you will want to produce one email with multiple to/cc/bccs defined. However, there is a business case where the email producer could produce multiple emails with one to address. An example would be a customer facing email.

Let's say a healthcare company needs to produce an email to all of its patients. They could create a custom email producer that produces multiple emails, one for each patient, personallized to that patient. Each email containing one to address, the patients. Maybe the business process is as follows.

Office closed business process.
1) Send email to each patient telling them their appointment has been cancelled, and to call us.
2) Send email to all practitioners to tell them not to come in.

For the patient emails, the email maybe should read "Brad Davis, your appointment for 10:00 AM has been canceled due to inclement weather." But this should be per patient. So, yours reads "Tom Baeyens, your appointment for 11:00 AM has been canceled due to inclement weather." and so on and so forth.

[tom.baeyens]

for multiple emails, what is wrong with calling

mailSession.createMail()....send();

multiple times ?

also, MailSession doesn't try to be a generic interface for sending emails. it should be limited in scope to only the jBPM process integration use cases.