Integrating jASEN

[wobbet]

Is someone already writing an anti-spam utility? I don't want to either duplicate effort or step on toes...

I'm looking at integrating jASEN(1) into the JBoss Mail Server. What would be required to do so? Is there a filter chain that I would need to plug into? My initial goal would be to use jASEN to assign a SPAM score and make that part of the Subject: line so that client-side filtering could be performed. After that, I don't really know what the goals would be.

If this is a worthy goal let me know and let me know how you think it should be implemented.

rjsjr

1 - http://jasen.sourceforge.net/ licensed LGPL

[mikezzz]

This is definitely worth doing. The main method for integrating with JBoss Mail is by implementing a MailListener. Start with looking at the org.jboss.mail.maillistener package. If you have any success submit a patch:

http://wiki.jboss.org/wiki/Wiki.jsp?page=MailServicesPatches

As far as I know, no one is working on Spam filtering, but it is the plan so go for it.

Mike.

[wobbet]

My first run will be a simple MailListener that would execute the jASEN code "inline". But for planning future work...

Another thought that I had was that it could be coded as an MBean/Servlet and listen on a port. That would allow it to work in the same fashion that SpamAssassin's daemon (spamd) works. Other services could then take advantage of the functionality in the same fashion they would spamd.

If I do that then I could go back and re-write the MailListener to open the port and send in the data and get the response back that way. Is that too much network overhead?

Or there's always the "give them lots of options to shoot themselves with" and allow the config to choose between the daemon or the inline style.

But the first thing is a simple inline implementation.

I hope my wife doesn't mind me staying up late this weekend...

[wobbet]

I have the source and I opened it up in Eclipse 3.1M6 (using MyEclipse(1) running Java 5 on WinXP Pro SP2.

Note 1 - I get a build error saying that an external builder is invalid. Is this an issue w/ my config or am I missing some config instructions?

Note 2 - After I remove the external builder from the project it will still not build saying "Build path entry is missing: org.jboss.ide.eclipse.jdt.core.classpath.junit-3.8.1" which implies I'm supposed to have the JBoss IDE plug-ins installed. Do I really need the JBoss IDE version of JUnit or is the standard JUnit good enough? I'll try the standard JUnit first and find out that way...

Note 3 - I get a large collection of "unbound classpath variable" errors for JBOSS_HOME. After actually setting JBOSS_HOME as a classpath variable those go away.

Note 4 - The Mail Services "Get The Source" page at http://wiki.jboss.org/wiki/Wiki.jsp?page=MailServicesGetTheSource is not up to date. The correct page describing how to get the source is at http://wiki.jboss.org/wiki/Wiki.jsp?page=CVSRepository.

I'm willing to do work to get up and running on this but having a document that describes what I need to do to get configured would be of great help. I'd rather spend the time coding than figurin' the configurin'.

rjsjr

1 - http://www.myeclipseide.com

[mikezzz]

Note 1 - Fixed, removed external builder.

Note 2 - Fixed, should use the junit.jar from lib directory.

Note 3 - This is standard for setting up eclipse, otherwise we would have to explicitly specify the jboss install location which will be different from machine. At the moment there is no "official JBoss Mail dev environment" the .project & .classpath files are simply mine as I happen to use eclipse.

Note 4 - Fixed.

Unfortunately there are only a few of us working on the project, and things like documenation and HOWTOs do tend to end up being secondary to coding (for better or for worse). Pointing these errors is helpful, as at times we are (blissfully) unware of what has gotten out of date. Any contributions that you would like to make would be useful, even a Wiki page describing "developing JBoss Mail using eclipse".

Please don't get disheartened by the state of play at the moment, we welcome all genuine interest and contributions.

Mike.

[wobbet]

Not disheartened at all. Mildly frustrated but not disheartened...

The simple fact that you made the changes is encouraging. I've seen other projects where people pointed out config errors and they were ignored. I ignore those projects :)

Is there a way that I can send you email to get some of this off of the forums?

I went ahead and downloaded the JBoss IDE and that cleaned up the Log4J errors so that wasn't a bad thing. I figured the IDE download would be a good thing anyway to allow direct debugging of my code.

On the Eclipse front, I've found that checking in the .project, .classpath, etc. files can cause difficulties. If you or I make even a minor change to one of those and commits then an update will cause conflicts. When I updated after you said "fixed" Eclipse told me that the .project and .classpath files were no longer valid.

My suggestion is that you provide a .tar.gz/.zip file containing the initial Eclipse config as part of the source distro rather than committing it. If someone is using Eclipse they can use that as a starting point and configure with the local changes they need.

I think I've found some defects in at least one of the XML files but I'm not 100% certain. Should I send a patch immediately or discuss w/ you first?

rjsjr

[mikezzz]

Re: E. mail. my address should be in the source code (Michael Barker) in one of the author comments.

Re: Eclipse. It is likely that you have modified the same line in the file that I fixed in CVS. Either hand edit the file to remove the conflict or delete the files (outside of eclipse) and do a cvs update to refresh them. I would still like to see the eclipse settings in CVS (as they generally need to versioned in line with the rest of the project), but perhaps they should go in a subdirectory and be copied in place if required. Anyway, I tend to find once you get over the initial dev. env. pain things tend to be reasonably smooth.

Re: Changes. I have found the best practice is to write the code first and submit patch via the wiki (see URL in my first post) and write a forum post to alert us to it. Even in the very short time that I have been on the project you get new people making suggestions and discussing ideas that never result in anything tangible. If you have gone to the effort of making a change and submitting a patch, your comments will be taken much more seriously. Please include a README file with detail of the change and JUnit tests if it is new functionality.

[starksm64]

Our jira site should be the vehicle for patches, not the wiki.
http://jira.jboss.com/jira/browse/JBMAIL

[acoliver]

Humm I have had working eclipse .project files for some time but never thought they were valuable to anyone else ;-) so I never committed them (at least not deliberately)

[pilhuhn]

"wobbet" wrote:
My initial goal would be to use jASEN to assign a SPAM score and make that part of the Subject: line so that client-side filtering could be performed.

Please don't munge with the Subject: line. Add another header line like X-SpamStatus or such.

[wobbet]

Comment - Based on feedback from Michael Barker and others I will initially be adding in a new header to a Mail object rather than munging the Subject line. I will call it "X-SpamScore" (X-SpamStatus was suggested in the forum but I think that "score" leaves room for end-user interpretation of the score better).

Question - jASEN requires that a message be input as MIME. Will a simple msg.toString give me MIME compliant text? Is there another method I need to call? Do I need to write one?

Thanks!

rjsjr

[acoliver]

ATM it should provided the original email was MIME. IIRC, we do not convert messages to MIME if they were simple text. Also you have to deal with bounce messages which are simple text (but if we generated the bounce message then no reason to check its spam nature). I actually favor a internal routing envelope myself.

[wobbet]

Annoyance #1 - toString() does NOT return a MIME string, it returns the simple object classname and instance id. What I really wanted is getRawStream() which appears to be working...

Status update
1 - Created the mail listener as a POJO and created an xbean descriptor for it (can I assume that the xbean/POJO combo is invoking AOP to make it behave as a real MBean?).

2 - Added the listener to the chain using the entries in jboss-service.xml

3 - launch debug within jboss ide and use telnet to invoke the mail

4 - hit the breakpoint in the listener

5 - run out of memory when initializing the jasen confg :(

The exception says I'm out of Java heap space.

Is this an issue w/ my debug config not allocating enough RAM to the JVM or is this an issue with not having enough allocated to Eclipse or is this an issue w/ jASEN just eating too much and needing to be put on a diet?

Yeah, yeah, yeah - you probably don't have enough info to tell me but I know that you've used the JBoss/Eclipse IDE and debugged within it so if there is some magic there then I know that you can help me with that.

Thanks!

rjsjr

[wobbet]

W00T!

I added

-XX:MaxPermSize=128m -Xmx512m

To the JVM start arguments in the debugger and it worked just fine!

I went into telnet on port 110 to retrieve the email that I had created and the X-SpamScore header was displayed with the same value that I saw presented during the debug session. Life is good...

Some config questions though...

First - I have my listener added in as a depends and as a listener in the org.jboss.mail.MailListenerChainService xmbean declaration in the jboss-service.xml file.

It appears that my listener is being invoked on the send instead of the receive but I wasn't paying enough attention to be sure. Is that really what I'm seeing? Is that important right now?

If it is on the send and it is important that I do it on the recieve, how do I config a listener chain for receive?

Second - jASEN has two different directories of data that it relies on - one for configuration information and one is the "database" it uses to determine scoring. I need to do a bit more research but from what I currently read the configuration can be read-only (if you never want to change the config of course) but the "database" should be read-write to allow training.

Where can/should these directories go for a production style deployment?

Third - Once I get this to the point where I can reliably deploy on a machine other than my dev machine I'll work on some jUnit tests. What sort of tests would be appropriate? I'm thinking that a collection of tests each with different email content that gives reasonable scores would be good. Should I add tests directly to my listener or should I add them in the listener chain tests (is there such a beast?) or should I do both or something completely different?

Finally - Someone (mikezzz???) suggested that I write up a page on how to write a listener along with info on getting stuff up and running in Eclipse. Do I need special permissions to edit Wiki pages and if so should I just submit something coherent to Mike or Andrew or ???

Many thanks!

rjsjr

[pilhuhn]

"wobbet" wrote:

1 - Created the mail listener as a POJO and created an xbean descriptor for it (can I assume that the xbean/POJO combo is invoking AOP to make it behave as a real MBean?).

Do you mean XMBeans? They have nothing to do with AOP and existed long before. In fact they are (iirc) implementations of DynamicMBeans. And they do really work very well in practice :-)