7 Replies Latest reply on Mar 17, 2005 1:31 PM by acoliver

    What I am (not) Looking For In A Java-Based Enterprise E-mai

      I was thrilled when I came across the Apache James e-mail server a couple years ago, but after a bit of investigation, I decided not to embrace it. I really liked the Matcher / Mailet APIs, but honestly, I didn't like the idea that to use James, I had to embrace the Avalon server framework.

      Then, when JBoss announced its Mail Services project, I was even more excited at the prospect of using a mail server that was integrated into my favorite J2EE container. However, as time has gone on, it seems like I will again be asked to embrace some unrelated APIs in order to handle my e-mail processing; i.e. People are talking about using scripting languages like JavaScript and BeanShell for e-mail workflow processing. I think it would be a mistake to require non-Java languages and APIs as an exclusive means for processing e-mails.

      So I wanted to get on the record what *I'd* like to see in any Java-based Enterprise E-mail server:

      - It should be J2EE-based

      Honestly, I don't want to have to revisit pooling, resource and thread management, security, datasource access and transactions, and all the other services provided by the well-documented J2EE API and JBoss implementation.

      - It should provide a clear separation of e-mail server behaviors that change rarely and those we might change often

      The standard e-mail server behaviors (as documented in various RFCs) are well known and essentially static since the backward compatibility is essential to successful Internet e-mail processing. However, new processing behaviors seem to crop up weekly, like anti-Spam processing and dealing with new Internet security threats. When once we were only worried about viruses carried in exe payloads, then we needed to worry about worms and trojans deployed in e-mail attachments, now including image files, etc. New threats like spyware and phishing attacks are not viral in nature, but a robust e-mail server should be extensible enough to allow me to "plug-in" processing code for the detection and removal of these threats.

      The separation of concerns issue is very important to me. Consider the highly successful Servlet API. It was written so that the servlet container could handle all the handshaking of the incoming web request, but what was done with that request was up to developers. Given that the servlet container handles all the nasty socket-level details of thread spawning, HTTP handshaking, connection timeouts, etc., we can focus on the "business end" of the processing. This has resulted in several excellent frameworks, like Struts, Tapestry, and WebWork to deal with the HttpServletRequest and Response artifacts created by the servlet container.

      As it stands, neither the James project nor the JBoss project seem to cleanly separate the core e-mail server functionality from the post processing activity. (There's some concern in the James community about the idea of accepting all e-mails rather than bouncing rejected ones, but I'll leave that discussion for another time.)

      For the JBoss e-mail server, it seems like this separation could be achieved by "publishing" the e-mail from the SMTP server as a JMS message and calling a stateless SessionBean from the POP server. (To prevent being falsely identified as an open relay and getting bombarded with spam, there could be some 0th order check for invalid server domains before delivering the message to JMS.)

      Developers (including JBoss.org) could then develop the mail processing APIs separately from the server-centric code to handle socket connections, threads, etc. Perhaps some Struts-like framework would emerge to process the SMTP e-mail. Or, maybe the James Matcher/Mailet APIs would be adapted for post-process workflow. Or, maybe someone will implement a Rules Engine approach (JESS, Drools) for complex processing of rules.

      Thus, JBoss 4.x/5.x could be shipped with default mail services (SMPT server, POP server) and instructions on how an application can subscribe to receive SMTP e-mail messages or extend a stateless SessionBean to override a default, simple e-mail fetching strategy. Then we can focus on implementing the business processing of the SMTP e-mails and returning e-mails to authorized users.

      - User and mail repositories must support virtual hosting; i.e. the same server instance must be able to distinguish john.smith @ acme.com from john.smith @ jboss.org.

      - Server must allow runtime changes to user repository without server restart. Ideally, e-mail processing rules should be able to be changed and picked up by the e-mail processor without a server restart

      - Any post-processing must allow use of different mailstores any kind, file system, datastore, EIS/Connector. By maintaining a separation of concerns of e-mail server vs e-mail processing, e-mail received by the SMTP server should be able to be processed in any way the user desires

      In summary, in the beginning JBoss needn't develop and deliver every possible implementation of all business e-mail processes. Just exploit the existing J2EE APIs to provide us hooks to access the SMTP e-mail messages as JavaMail Messages and deliver JavaMail Messages to the POP/IMAP server. From there, we can develop, or use, mail processing APIs such as the James Mailet / Matcher API.

      Maybe this is the way JBoss is doing it now, but if so, that's not at all obvious to me. Instead, it looks like JBoss is trying to develop the "be all and end all" of e-mail servers. That's also what James is try to do. And Exchange. And MDaemon. And PostFix. And QMail. And all the others.

      I'm just suggesting we take a smaller bite at the apple and deliver a solid Java-based SMTP, POP, and IMAP server with hooks accessible from existing J2EE services.

      I'm sorry for the length of this post, but I wanted to plead for "less sooner" rather than "more later".

      Jack

        • 1. Re: What I am (not) Looking For In A Java-Based Enterprise E
          acoliver

          Please look here: http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-mail/src/java/org/jboss/mail/maillistener/JMSMailListenerMBean.java?rev=1.11&view=markup

          If you like, you can simply write a MDB today even with Milestone 2.

          And at the actual MailListener interface. I do not see what it is that you "can't" do.

          As for your concerned that we are biting off more than we can chew, we are very good developers. Especially Kabir, Mike and Dawie. I fake it pretty well hopefully no one realizes how bad I suck :-). We're so good that I think we can get the job done :-)

          Thanks,

          andy

          • 2. Re: What I am (not) Looking For In A Java-Based Enterprise E

            Thanks for the reply.

            The JMSMailListenerMBean interface is definitely along the lines of what I'm looking for in terms of decoupling the socket-level server stuff from the business code.

            I'll take a look at the code in more detail and try out M2.

            As for the MailListener interface, is there some reason the James Matcher / Mailet API isn't a good solution for mail processing? What are the pros and cons of the API?

            • 3. Re: What I am (not) Looking For In A Java-Based Enterprise E
              acoliver

              Yes. There are many reasons. Not the least of which is the mailet api is ill-conceived, and I won't even address the polito-technical issues involved (software is a competitive marketplace open source or not. Its just the way things are).

              Technically, many of the the IDEAs are good, the IMPLEMENTATION is bad or I'd have stolen the code (I originally stole their stream handling code but it was bad too -- a large customer ran it through a profiler and revealed it sucked 90% of our CPU footprint) so we'll be ditching it too. I'm not going to rattle off all of the reasons because I researched this over a year ago and I prefer to kill JAMES with code and not with words.

              The idea of doing a new mail server started before I joined JBoss, when I was still an Apache member. Needless to say, I'm intimately familiar with JAMES's source and if I feel pieces are useful, I'll use them.

              • 4. Re: What I am (not) Looking For In A Java-Based E-mail Serve

                Andy,

                Thanks for that reply. I am curious to learn more of your thoughts about the Mailet API.

                In considering James, I liked the idea of being able to chain standard and custom matchers to handle mail processing. It seems like a very flexible, almost aspect-oriented, approach to handling mail processing. Also, it's a 100% Java API, not JavaScript, BeanShell, php, perl, or some other non-Java solution.

                Jack

                • 5. Re: What I am (not) Looking For In A Java-Based Enterprise E
                  acoliver

                  Look, it sounds like you really like JAMES and want to use JAMES, so go use JAMES.

                  • 6. Re: What I am (not) Looking For In A Java-Based E-mail Serve

                    >> Look, it sounds like you really like JAMES and want to use JAMES, so go use JAMES <<

                    Tsk Tsk Andy.

                    Just because someone says he likes some design features of a competing piece of software is no reason to be so dismissive.

                    There are things I think I would like about using James, like its Mailet API. It seems a reasonable solution to the problem of what to do with an e-mail after it arrives. It's appears to be a flexible, AOP-like, all-Java way to build custom mail processing; i.e. scan it for viruses, scan it using Spam Assassin or other spam-detection methods, check for whether the recipient is a member of a hosted domain. store it or delete it, etc.

                    There are things I don't like about James, like its reliance on the now-defunct Avalon server framework instead of the well-known JMX and J2EE APIs.

                    However, I would caution against dismissing the James mail processing API (at least the design aspects like the interfaces) just because "it wasn't invented here".

                    If there are sound technical reasons why you don't like the Mailet API, I would still welcome a rational discussion about the benefits and flaws of the Mailet API as it might be applied to the JBoss Mail Server. Since that's what this discussion has evolved to, I'm going to start a new thread soliciting views on the Mailet API's applicability to the JBoss mail server.

                    • 7. Re: What I am (not) Looking For In A Java-Based Enterprise E
                      acoliver

                      If you impelement a mailet api adapter which is non-invasive to the rest of the server (suggest a maillistener), I'll commit it provided it has good unit tests and you make it optional and maintain the build targets associated with it.