Unit Tests and XScripts

[mikezzz]

Hi,

I have fixed the JUnits that have broken as a result of the change to the HELO SMTP handler and JBMAIL-126 (I was doing very naugty things with the Velocity templates). They are working again with the exception the XScripts. I have disabled them for the time being as I can't locate the portion of code which is causing the scripts to hang (breaking the whole test suite). I know what's happening, but the XScripts obfuscate what the actual code and a impossible to debug and basically I can't be bothered. So these tests are going to remain disabled until someone is willing to set up the plate and maintain them.

When I get some time I will replace them with a Java based test which does basically the same thing that avoids the completely unecessary XML scripting language and requires much less code.

This would be a good task for someone willing to get into JBMail. There is a class called ProtocolHandler which provides this functionality. The TestSTARTTLSCmd class demonstrates it's use.

Cheers,
Mike.

[jeff87]

Sounds like that might be a good one for me possibly? I need to work on that new JUnit test for testing the size of an e-mail anyway. Would this be something you would like me to work on? This week will be tough and I'll be out of town this weekend so I don't think I could get anything to you this week.

Jeff

[acoliver]

I don't entirely agree with Mike. I think having some simple scripting test stuff would be good. I started out with something more textual and then Mike Andrews created this XScript thing. The usefulness of some kind of scripting thing is that I can grab the output from a log or a tool like ethereal and turn it into a test.

This would be good for you jeff...One thing that would make this especially good is once you fix the testing framework...document it so that the rest of us can figure out how to use it. That was a big problem, I never figured out how to use the XScript stuff so it kind of laid to waste. Having some kind of scripting thing that works, is simple to use and everyone knows how to use would be really great.

I understand the prefereence for Java, but Java isn't great to express the client/server interactions...at least not conceisely.

[mikezzz]

I don't really mind what the solution is however the two things I would like to see are:

- Simplicity (XScripts although not huge, is sufficently complex that when a test fails it is difficult to be certain that it is JBMail that is broken and not XScripts).
- Debugibility (is that a word? If a test fails I would like to be able to step through the conversation to see where if fails rather than having to write a second test to debug the problem).

I wrote the ProtocolHandler to meet these needs its quite simple (<100 lines of code).

public void testStartTls() {
 ProtocolHandler ph = new ProtocolHandler();
 ph.checkInput("220.*");
 ph.writeln("EHLO jboss.org");
 ph.checkInput("250-.*");
 ph.checkInput("250-STARTTLS.*");
 ph.checkInput("250 AUTH LOGIN PLAIN.*");
 ph.writeln("STARTTLS");
 ph.checkInput("220 Ready to start TLS.*");
}

And that is basically all of the functionality that is provided by the XScripts. However it could be seen as a bit messy to imeplement as you then need to wrap that in a test method. One option could be to link in JavaScript (via Rhino). Then all you would have to do is:

expect("220.*");
send("EHLO jboss.org");
expect("250-.*");
expect("250-STARTTLS.*");
expect("250 AUTH LOGIN PLAIN.*");
send("STARTTLS");
expect("220 Ready to start TLS.*");

And you could step though it with rhino's debugger if required. Generating the above from a log file or ethereal trace would be no more difficult than generating the XScript. "send" and "expect" are simply functions defined and added to the ScriptContext before running the script.

Mike

[acoliver]

Provided that these can be simply generated and I don't have to try real hard to dump something from ethereal I don't really care about the details frankly... I don't think the issue with debugging the script interpreter is a all that significant, but it sounds like a bikeshed to me. Right now it is a PITA to write script based unit tests. This needs to be remedied. No one liked my solution (there was an earlier interpreter by me that parsed literally:

C: bla bla
S: bla bla server response
C: bla bla
S: bla bla server response

but was going to get more complicated). One thing that I would like to see is conditionals (basically if/else/(assert/or)) where multiple responses can be accepted beyond "wildcards"

For instance accepting either that a mailbox is empty or has mails...

[mikezzz]

I was thinking more along the lines of regular expressions rather than just wildcards.

send("LIST");
expect("\+ OK [1-9][0-9]* messages .*");

99% of the logic that we need can be encapsulated using a regexp. However if we were to use an existing scripting language (JavaScript, Jython, JRuby, Groovy...) we would get all of the conditionals, looping etc. for free. I just think that rolling our own interpreter will be heavily time consuming and it won't be as functional or stable as something that already exists.

Being able to debug the scripts is very useful. When a script breaks, rather than throw an error they will generally hang as they are blocking while waiting for some input from the server that will never arrive (current reason why the XScripts are broken ATM). However I guess this could be solved by logging the conversation appropriately.

Mike.

[acoliver]

To me it is a matter of whatever gets us there the quickest with the least fuss. If ripping out the existing interpreter and rewriting all the scripts is the answer, cool. If fixing the bug that causes it to block rather than except after some timeout helps, cool. Either is fine with me.

[jeff87]

So should I be working on a JUnit class that is coded similar to TestSTARTTLSCmd or is there another way I should be working on the JUnit classes?

[acoliver]

Please make it work. How should it work? He who does, decides. The rest is just feedback (self included) :-)

-Andy