aop-mc-int tests with maven

[kabirkhan]

I'm seeing a load of failures here that worked before the maven integration (the output really, really sucks, and yes I have read the thread about the surefire plugin)

The test classes fall into 3 categories:
1) Tests that should be run without any special classloader/weaving set up
2) Tests that should be run with a special classloader/weaving set up
3) Tests that should be run both with and without special classloader/weaving set up

There were 4 sub-targets of test in the old ant build-test.xml

a) Run classes 1) and 3) with no security enabled, and no weaving
b) Run classes 1) and 3) with security enabled, and no weaving
c) Run classes 2) and 3) with no security enabled, and weaving
d) Run classes 2) and 3) with security enabled, and weaving

Can this be replicated sometime soon?

[kabirkhan]

The snapshot jar is missing jboss-aop_1_0.dtd which would explain some of the errors, it is also missing the jbossorg-eula.txt.

How do add these and update the snapshot?

While on that topic, what are the differences between the following folders in my local maven working copy?

Kabir@KabirDell /c/Documents and Settings/Kabir/.m2/repository/jboss/jboss-aop
$ ls -al
total 0
drwx------+ 5 Kabir None 0 Apr 23 13:21 .
drwx------+ 28 Kabir None 0 Apr 20 12:14 ..
drwx------+ 2 Kabir None 0 Apr 3 16:56 2.0.0-20070402.001441-3
drwx------+ 2 Kabir None 0 Apr 23 13:23 2.0.0-20070420.160953-1
drwx------+ 2 Kabir None 0 Apr 23 13:23 2.0.0-SNAPSHOT

[pgier]

I added the dtd and license to be included in the snapshot jar. It is in the resources section of the pom. To upload a new snapshot:

mvn deploy

But you will need to configure your settings.xml file with your svn account. You can see this for more information:
http://wiki.jboss.org/wiki/Wiki.jsp?page=MavenSnapshotRepository

[pgier]

There were 4 sub-targets of test in the old ant build-test.xml

a) Run classes 1) and 3) with no security enabled, and no weaving
b) Run classes 1) and 3) with security enabled, and no weaving
c) Run classes 2) and 3) with no security enabled, and weaving
d) Run classes 2) and 3) with security enabled, and weaving

Can this be replicated sometime soon?

I added a surefire plugin configuration to the pom to run the tests with four different configurations. I copied the settings from the old ant build scripts, but there are still some test failures.

Which tests are supposed to go in each group 1, 2, and 3?

[kabirkhan]

I'll try to dig out the old build-test.xml to add the tests, I'll let you know when I get stuck.

A few further questions:

I see we now have things like

 <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-surefire-plugin</artifactId>
 <executions>
 <execution>
 <id>weave</id>
 <phase>test</phase>
 <goals>
 <goal>test</goal>
 </goals>
....
 <execution>
 <id>weave-secure</id>
 <phase>test</phase>
 <goals>
 <goal>test</goal>
 </goals>
...

Is there a way to for example only run the weave-secure tests?

Some of the tests (those belonging to group 3) will be run 4 times, the others will be run twice. At the moment with the surefire plugin I assume that the last run will override the report for the previous run. With the old build they had a separate file generated for each security/weaving configuration, is that possible?
For example for the org.jboss.test.microcontainer.test.LifecycleTestCase test, you would get something like:

TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-weave.txt
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-weave.xml
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-weave-secure.txt
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-weave-secure.xml
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-no-weave.txt
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-no-weave.xml
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-no-weave-secure.txt
TEST-org.jboss.test.microcontainer.test.LifecycleTestCase-no-weave-secure.xml

The extension would also get passed into junit and get picked out by the org.jboss.ant.taskdefs.XMLJUnitMultipleResultFormatter and used as the title/identifier in the generated xml. An example of this can be found in the current aop cc runs
http://cruisecontrol.jboss.com/cc/artifacts/jboss-aop-testsuite/20070423105405/results/index.html, and then if you click pretty much any package name you will see that the same test has been run under different configurations. Pretty useful if a test fails in some configurations and not others - like for example the tests in the org.jboss.test.aop.jdk15annotated package :-)

Also, with the old build it was possible to just run one single test for a certain weaving mode, e.g

$ ant one-test-no-weave -Dtest=org.jboss.test.microcontainer.test.LifecycleTestCase

Is that still doable somehow?

[kabirkhan]

Hmmmm,

Looking at the old build-test.xml it does not look like we have the groups I mentioned, everything is run both with and without weaving, or in other words everything belongs to group 3)! I was sure we had a split between the tests, maybe I got confused somewhere. I'll double check and let you know if I find out something else.

In addition to the things I mentioned above, one thing that makes the console output impossible to read is that all the log messages/System.out.println()s from the tests appear in the console. With ant, the junit task would redirect these to to only appear in the log file. What maven-fu is needed to do that?

[kabirkhan]

In addition to the things I mentioned above, one thing that makes the console output impossible to read is that all the log messages/System.out.println()s from the tests appear in the console. With ant, the junit task would redirect these to to only appear in the log file. What maven-fu is needed to do that?

Here I mean running the tests.

Running org.jboss.test.microcontainer.test.CreatedBeansLifecycleCallbackTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
Running org.jboss.test.microcontainer.test.BeanCallingMethodInCtorTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec

is a lot easier to parse for errors/success than

Running org.jboss.test.microcontainer.test.CreatedBeansLifecycleCallbackTestCase
23859 DEBUG [CreatedBeansLifecycleCallbackTestCase] ==== setUp org.jboss.test.microcontainer.test.CreatedBeansLifecycleCallbackTestCase ====
23875 DEBUG [KernelFactory] Starting JBoss Kernel construction...
23875 DEBUG [KernelFactory] Completed JBoss Kernel construction. Duration: 0 milliseconds
23875 DEBUG [CreatedBeansLifecycleCallbackTestCase] No test specific deployment org/jboss/test/microcontainer/test/CreatedBeansLifecycleCallbackTestCase-aop.xml
23875 DEBUG [CreatedBeansLifecycleCallbackTestCase] No test specific deployment org/jboss/test/microcontainer/test/CreatedBeansLifecycleCallbackTestCase.xml
23875 DEBUG [CreatedBeansLifecycleCallbackTestCase] Security enabled: true
23875 DEBUG [CreatedBeansLifecycleCallbackTestCase] ==== Starting testLifecycleInterceptions ====
23875 DEBUG [CreatedBeansLifecycleCallbackTestCase] Deploying file:/C:/cygwin/home/Kabir/sourcecontrol/microcontainer-mvn/aop-mc-int/target/test-classes/org/jboss/test/microcontainer/test/CreatedBeansLifecycleCallbackTestCaseNotAutomatic.xml
=====> Adding lifecycle org.jboss.aop.microcontainer.lifecycle.LifecycleCallbackDefinition@f1770a:ControllerState@be6d81{Configured}
23937 DEBUG [CreatedBeansLifecycleCallbackTestCase] Undeploying file:/C:/cygwin/home/Kabir/sourcecontrol/microcontainer-mvn/aop-mc-int/target/test-classes/org/jboss/test/microcontainer/test/CreatedBeansLifecycleCallbackTestCaseNotAutomatic.xml
23937 DEBUG [CreatedBeansLifecycleCallbackTestCase] testLifecycleInterceptions took 62ms
23937 DEBUG [CreatedBeansLifecycleCallbackTestCase] ==== Stopping testLifecycleInterceptions ====
23937 DEBUG [CreatedBeansLifecycleCallbackTestCase] ==== tornDown org.jboss.test.microcontainer.test.CreatedBeansLifecycleCallbackTestCase ====
23937 DEBUG [CreatedBeansLifecycleCallbackTestCase] Undeploying []
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
Running org.jboss.test.microcontainer.test.BeanCallingMethodInCtorTestCase
23937 DEBUG [BeanCallingMethodInCtorTestCase] ==== setUp org.jboss.test.microcontainer.test.BeanCallingMethodInCtorTestCase ====
23953 DEBUG [KernelFactory] Starting JBoss Kernel construction...
23953 DEBUG [KernelFactory] Completed JBoss Kernel construction. Duration: 0 milliseconds
23953 DEBUG [BeanCallingMethodInCtorTestCase] No test specific deployment org/jboss/test/microcontainer/test/BeanCallingMethodInCtorTestCase-aop.xml
23953 DEBUG [BeanCallingMethodInCtorTestCase] Deploying file:/C:/cygwin/home/Kabir/sourcecontrol/microcontainer-mvn/aop-mc-int/target/test-classes/org/jboss/test/microcontainer/test/BeanCallingMethodInCtorTestCase.xml
23969 DEBUG [Aspect] Bound aspect InterceptedAdvice$Aspect; deployed:true
23984 DEBUG [AspectBinding] Bound binding ax6l-jgg29f-f0vbb4kk-1-f0vbblss-3e
23984 DEBUG [GenericBeanAspectFactory] Creating advice ax6l-jgg29f-f0vbb4kk-1-f0vbblsd-3d
public
private
private
24015 DEBUG [BeanCallingMethodInCtorTestCase] Security enabled: true
24015 DEBUG [BeanCallingMethodInCtorTestCase] ==== Starting testIntercepted ====
CalledInterceptor: [advisedMethod=public void org.jboss.test.microcontainer.support.BeanCallingMethodInCtor.publicMethod(), unadvisedMethod=public void org.jboss.test.microcontainer.support.BeanCallingMethodInCtor.publicMethod(), metadata=null, targetObject=org.jboss.test.microcontainer.support.BeanCallingMethodInCtor@1fc9484, arguments=null]
public
24015 DEBUG [BeanCallingMethodInCtorTestCase] testIntercepted took 0ms
24015 DEBUG [BeanCallingMethodInCtorTestCase] ==== Stopping testIntercepted ====
24015 DEBUG [BeanCallingMethodInCtorTestCase] ==== tornDown org.jboss.test.microcontainer.test.BeanCallingMethodInCtorTestCase ====
24015 DEBUG [BeanCallingMethodInCtorTestCase] Undeploying [file:/C:/cygwin/home/Kabir/sourcecontrol/microcontainer-mvn/aop-mc-int/target/test-classes/org/jboss/test/microcontainer/test/BeanCallingMethodInCtorTestCase.xml]
24015 DEBUG [Aspect] Unbinding aspect InterceptedAdvice$Aspect
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
/code]

[kabirkhan]

Also, with the old build it was possible to just run one single test for a certain weaving mode, e.g

$ ant one-test-no-weave -Dtest=org.jboss.test.microcontainer.test.LifecycleTestCase

Is that still doable somehow?

http://maven.apache.org/plugins/maven-surefire-plugin/examples/single-test.html

mvn test -Dtest=org.jboss.test.microcontainer.test.BeanCallingMethodInCtorTestCase

Seems to do the job, but runs it in all the configurations set up

[kabirkhan]

In addition to the things I mentioned above, one thing that makes the console output impossible to read is that all the log messages/System.out.println()s from the tests appear in the console. With ant, the junit task would redirect these to to only appear in the log file. What maven-fu is needed to do that?

Adding

 <redirectTestOutputToFile>true</redirectTestOutputToFile>
 </configuration>
 </execution>

to the pom helps. It has the problem though of (over)writing all tests for all modes to the surefire-reports directory, similarly to what I mentioned for the test log files. "reportsDirectory" could maybe be used to split where the tests run with different weaving/security settings. Paul, how would that work with the report plugin?

[pgier]

Is there a way to for example only run the weave-secure tests?

I was looking into a way to do this today, but couldn't find anything simple. One option is to set up several build profiles, and then activate certain profiles from the command line.
If I can't find an easier way to do this, I will set it up this way.

[kabirkhan]

"pgier" wrote:

Which tests are supposed to go in each group 1, 2, and 3?

It had been a while, so I was wrong.

All tests are run with weaving enabled

 <include name="org/jboss/test/**/*TestCase.class"/>

Some tests are run with no weaving

 <include name="org/jboss/test/microcontainer/test/**/*TestCase.class"/>

I will update the pom.xml and see how that goes

[kabirkhan]

A lot ot the weave tests fail in that the classes are not being woven. This

 <property>
 <name>java.system.class.loader</name>
 <value>org.jboss.aop.standalone.SystemClassLoader</value>
 </property>

should be telling the test to use the SystemClassLoader which does the weaving. Setting forkmode=always for each test execution configuration does not help, and it appears that weaving is not being done. As a sanity check, I ran some of the failing tests from eclipse with -Djava.system.class.loader=org.jboss.aop.standalone.SystemClassLoader and they work as expected

[pgier]

Maybe you can use something like this in the execution configuration instead:

<argLine>-Djava.system.class.loader=org.jboss.aop.standalone.SystemClassLoader</argLine>

[kabirkhan]

That does something, now I get

$ mvn
[INFO] Scanning for projects...
WAGON_VERSION: 1.0-beta-2
[INFO] ----------------------------------------------------------------------------
[INFO] Building JBoss Microcontainer AOP MC INT
[INFO] task-segment: [install]
[INFO] ----------------------------------------------------------------------------
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [jboss-retro:retro-compile {execution: default}]
[INFO] [retro] Compiling classes for jdk14
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test]
[INFO] Tests are skipped.
[INFO] [surefire:test {execution: weave}]
[INFO] Surefire report directory: C:\cygwin\home\Kabir\sourcecontrol\microcontainer-mvn\aop-mc-int\target\surefire-reports
Error occurred during initialization of VM
java.lang.Error: java.lang.ClassNotFoundException: org.jboss.aop.standalone.SystemClassLoader

org.jboss.aop.standalone.SystemClassLoader exists in jboss-aop-2.0.0-SNAPSHOT.jar.

[kabirkhan]

OK, I found the magic -X option

I am using

 <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-surefire-plugin</artifactId>
 <executions>
 <execution>
 <id>weave</id>
 <phase>test</phase>
 <goals>
 <goal>test</goal>
 </goals>
 <configuration>
 <skip>false</skip>
 <includes>
 <include>org/jboss/test/**/*TestCase.java</include>
 </includes>
 <argLine>-Djava.system.class.loader=org.jboss.aop.standalone.SystemClassLoader</argLine>
 <forkMode>always</forkMode>
 <systemProperties>
 <property>
 <name>jboss.aop.exclude</name>
 <value>org.jboss.,org.apache.</value>
 </property>
 <property>
 <name>jboss.aop.include</name>
 <value>org.jboss.test.</value>
 </property>
 <property>
 <name>jboss-junit-configuration</name>
 <value>weave</value>
 </property>
 <property>
 <name>build.testlog</name>
 <value>target/log</value>
 </property>
 <property>
 <name>jboss.aop.verbose</name>
 <value>true</value>
 </property>
 </systemProperties>
 <!--
 <redirectTestOutputToFile>true</redirectTestOutputToFile>
 -->
 </configuration>
 </execution>

$ mvn -X
+ Error stacktraces are turned on.
Maven version: 2.0.5
....
[DEBUG] Test Classpath :
[DEBUG] C:\cygwin\home\Kabir\sourcecontrol\microcontainer-mvn\aop-mc-int\target\classes
[DEBUG] C:\cygwin\home\Kabir\sourcecontrol\microcontainer-mvn\aop-mc-int\target\test-classes
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-vfs\2.0.4.snapshot\jboss-vfs-2.0.4.snapshot.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-common-core\2.0.4.GA\jboss-common-core-2.0.4.GA.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-common-logging-spi\2.0.4.GA\jboss-common-logging-spi-2.0.4.GA.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\org\apache\ant\ant\1.7.0\ant-1.7.0.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\javassist\javassist\3.5-SNAPSHOT\javassist-3.5-SNAPSHOT.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\junit\junit\3.8.1\junit-3.8.1.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\apache-xerces\xercesImpl\2.7.1\xercesImpl-2.7.1.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\ant\ant\1.6.5\ant-1.6.5.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-kernel\2.0.0-SNAPSHOT\jboss-kernel-2.0.0-SNAPSHOT.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-dependency\2.0.0-SNAPSHOT\jboss-dependency-2.0.0-SNAPSHOT.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\sun-jaf\activation\1.0.2\activation-1.0.2.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\wutka-dtdparser\dtdparser121\1.2.1\dtdparser121-1.2.1.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-logging-log4j\2.0.2.GA\jboss-logging-log4j-2.0.2.GA.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\org\apache\ant\ant-junit\1.7.0\ant-junit-1.7.0.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\qdox\qdox\1.6\qdox-1.6.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-kernel\2.0.0-SNAPSHOT\jboss-kernel-2.0.0-SNAPSHOT-tests.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-logging-spi\2.0.2.GA\jboss-logging-spi-2.0.2.GA.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\trove\trove\2.1.1\trove-2.1.1.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jbossxb\2.0.0.CR3\jbossxb-2.0.0.CR3.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\apache-log4j\log4j\1.2.8\log4j-1.2.8.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\apache-xerces\xml-apis\2.7.1\xml-apis-2.7.1.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-test\1.0.3.GA\jboss-test-1.0.3.GA.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\oswego-concurrent\concurrent\1.3.4\concurrent-1.3.4.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\profiler\jvmti\jboss-profiler-jvmti\1.0.0.CR5\jboss-profiler-jvmti-1.0.0.CR5.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\apache-slide\webdavlib\2.0\webdavlib-2.0.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\apache-httpclient\commons-httpclient\2.0.2\commons-httpclient-2.0.2.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-aop\2.0.0-SNAPSHOT\jboss-aop-2.0.0-SNAPSHOT.jar
[DEBUG] C:\Java\jdk\jdk1.5.0_10\jre\..\lib\tools.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\log4j\log4j\1.2.14\log4j-1.2.14.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\jboss\jboss-container\2.0.0-SNAPSHOT\jboss-container-2.0.0-SNAPSHOT.jar
[DEBUG] C:\Documents and Settings\Kabir\.m2\repository\org\apache\ant\ant-launcher\1.7.0\ant-launcher-1.7.0.jar
[DEBUG] Setting system property [jboss-junit-configuration]=[weave]
[DEBUG] Setting system property [build.testlog]=[target/log]
[DEBUG] Setting system property [jboss.aop.verbose]=[true]
[DEBUG] Setting system property [localRepository]=[C:\Documents and Settings\Kabir\.m2\repository]
[DEBUG] Setting system property [jboss.aop.include]=[org.jboss.test.]
[DEBUG] Setting system property [jboss.aop.exclude]=[org.jboss.,org.apache.]
[DEBUG] Setting system property [basedir]=[C:\cygwin\home\Kabir\sourcecontrol\microcontainer-mvn\aop-mc-int]
[DEBUG] Using JVM: C:\Java\jdk\jdk1.5.0_10\jre\bin\java
[INFO] Surefire report directory: C:\cygwin\home\Kabir\sourcecontrol\microcontainer-mvn\aop-mc-int\target\surefire-reports
Forking command line: C:\Java\jdk\jdk1.5.0_10\jre\bin\java -Djava.system.class.loader=org.jboss.aop.standalone.SystemClassLoader -classpath "C:\Documents and Settings\Kabir\.m2\repository\org\codehaus\plexus\plexus-archiver\1.0-alpha-7\plexus-archiver-1.0-alpha-7.jar;C:\Documents and Settings\Kabir\.m2\repository\junit\junit\3.8.1\junit-3.8.1.jar;C:\Documents and Settings\Kabir\.m2\repository\org\codehaus\plexus\plexus-container-default\1.0-alpha-8\plexus-container-default-1.0-alpha-8.jar;C:\Documents and Settings\Kabir\.m2\repository\org\apache\maven\surefire\surefire-api\2.3\surefire-api-2.3.jar;C:\Documents and Settings\Kabir\.m2\repository\classworlds\classworlds\1.1-alpha-2\classworlds-1.1-alpha-2.jar;C:\Documents and Settings\Kabir\.m2\repository\org\codehaus\plexus\plexus-utils\1.1\plexus-utils-1.1.jar;C:\Documents and Settings\Kabir\.m2\repository\commons-lang\commons-lang\2.1\commons-lang-2.1.jar;C:\Documents and Settings\Kabir\.m2\repository\org\apache\maven\surefire\surefire-booter\2.3\surefire-booter-2.3.jar" org.apache.maven.surefire.booter.SurefireBooter C:\DOCUME~1\Kabir\LOCALS~1\Temp\surefire3057tmp C:\DOCUME~1\Kabir\LOCALS~1\Temp\surefire3058tmp
Error occurred during initialization of VM
java.lang.Error: java.lang.ClassNotFoundException: org.jboss.aop.standalone.SystemClassLoader

Digging into this a bit, one of the tmp files contains the test classpath. So it seems that the surefire plugin is meant to instantiate the test with the reall classpath, and the-Djava.system.class.loader=org.jboss.aop.standalone.SystemClassLoader system property is being used "too early" (when creating the SureFireBooter rather than when instantiating the test)