-
1. Arquillian tests always passing
aslak Apr 23, 2011 4:24 PM (in response to toddpi314)Can you see the test actually execute?
e.g. System.out.println() before the Assert. and check the server log.
And your using the Assert object right, not the assert keyword? If the keyword, you have to activate assertion on the container jvm
-
2. Re: Arquillian tests always passing
toddpi314 Apr 24, 2011 5:46 PM (in response to aslak)Aslak, thanks for the advice.
Calling
assertTrue(false);
on TestNG 5.14 results in 'success', no logging output and no break-points. (this is with a solid mvn clean on the target too, very creepy).
Updating to 5.14.6 and modifying the code to
Assert.assertTrue(false);
resulted in correct failure and logging working just fine.
This is not sane, so I would like to conclude that 'My Environment was jacked'.
I am working just fine now. Thanks for the help!
-
3. Re: Arquillian tests always passing
toddpi314 Apr 24, 2011 6:55 PM (in response to toddpi314)Continuing on this subject...
Test class
public class SimpleRegistrationTest extends Arquillian { @Deployment public static Archive<?> createTestArchive() { return ShrinkWrap .create(WebArchive.class, "test.war") .addAsResource("test-persistence.xml", "META-INF/persistence.xml") .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml"); } @Inject Logger log; @Test public void dummyTest() { Assert.assertTrue(false); } @Produces public Logger produceLog(InjectionPoint injectionPoint) { return Logger.getLogger(injectionPoint.getMember().getDeclaringClass()); } }
Fails on profile
mvn clean test -Parq-jbossas-remote
with JBoss 6 Final container. As expected
Failed tests:
dummyTest(com.deepelement.membership.test.registration.SimpleRegistrationTest): java.lang.AssertionError : expected:<true> but was:<false> [Proxied because : Could not find suitable constructor]
But, when I inject a bean without defining prerequisites I get a success even though everything catastrophically should fail.
So, executing
public class SimpleRegistrationTest extends Arquillian { @Deployment public static Archive<?> createTestArchive() { return ShrinkWrap .create(WebArchive.class, "test.war") .addAsResource("test-persistence.xml", "META-INF/persistence.xml") .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml"); } @Inject Logger log; @Inject MyUberBean uberBean; @Test public void dummyTest() { Assert.assertTrue(false); } @Produces public Logger produceLog(InjectionPoint injectionPoint) { return Logger.getLogger(injectionPoint.getMember().getDeclaringClass()); } }
With the same container and profile results in
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
With no sign of the traditionally Deployment Dependency error on the JBoss Log (or test log)
But, when I define the correct Injection Depedencies in the ShrinkWrap, the results go back to 'failure'.
This is concerning because no JBoss/TestNG stack-trace and no test failure sure are a combination for false-quality.
Any advice?
-
4. Re: Arquillian tests always passing
toddpi314 Apr 24, 2011 7:53 PM (in response to toddpi314)This sounds allot like http://community.jboss.org/thread/164212
But, I am not using any fancy life-cycle annotations that would collide here.
-
5. Re: Arquillian tests always passing
toddpi314 Apr 24, 2011 8:11 PM (in response to toddpi314)Analysing the TestNG stacks for both the 'correct failure' case and the 'false success' case, it is clear that the only difference is about 25 instances of
DEBUG-SocketClientInvoker[2dc8b884, socket://127.0.0.1:4446] disconnecting ...
On the 'correct failure' case.
Jboss logs are all identical.
-
6. Re: Arquillian tests always passing
toddpi314 Apr 24, 2011 8:39 PM (in response to toddpi314)This bug occurs on all 5.14.* builds.
Not reproducible with JUnit
<dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.8.2</version> <scope>test</scope> </dependency>
Issue Resolved (sort of)
-
7. Re: Arquillian tests always passing
iphands Oct 7, 2013 5:38 PM (in response to toddpi314)Hi, I almost opened a new discussion but then I stumbled upon this one.
I am having and similar situation where apparently the @Test annotated method bodies are not being run, but all tests seem to pass.
Unlike the OP my generated war is free of all testng jars (and I am not trying to use testng).
Has anyone else experienced similar issues?
Here is the simplest example I can come up with:
package com.example; import java.io.File; import java.util.ArrayList; import java.util.Arrays; import java.util.List; import junit.framework.Assert; import org.jboss.arquillian.container.test.api.Deployment; import org.jboss.arquillian.junit.Arquillian; import org.jboss.seam.mock.JUnitSeamTest; import org.jboss.shrinkwrap.api.Archive; import org.jboss.shrinkwrap.api.ShrinkWrap; import org.jboss.shrinkwrap.api.exporter.ZipExporter; import org.jboss.shrinkwrap.api.spec.WebArchive; import org.jboss.shrinkwrap.resolver.api.maven.Maven; import org.jboss.shrinkwrap.resolver.api.maven.strategy.RejectDependenciesStrategy; import org.junit.Test; import org.junit.runner.RunWith; @RunWith(Arquillian.class) public class ExampleTest extends JUnitSeamTest { @Deployment(name = "ExampleTest") public static Archive<?> createDeployment() { final WebArchive war = ShrinkWrap.create(WebArchive.class, "portal-test.war").as(WebArchive.class); war.delete("/WEB-INF/web.xml"); war.addAsWebInfResource("./web.xml"); // List for libs final List<File> libs = new ArrayList<File>(); // Add seam libs libs.addAll(Arrays.asList(Maven.resolver().loadPomFromFile("pom.xml").resolve("org.jboss.seam:jboss-seam") .using(new RejectDependenciesStrategy(false, "org.testng:testng")).asFile())); // add test libs libs.addAll(Arrays.asList(Maven.resolver().loadPomFromFile("pom.xml").importTestDependencies().resolve().withTransitivity().asFile())); // add all libs war.addAsLibraries(libs.toArray(new File[libs.size()])); // export the war for fun! war.as(ZipExporter.class).exportTo(new File("/tmp/test.war"), true); return war; } @Test public void testFoo() throws Exception { System.out.println("Test\nTest\nTest\nTest\nTest\nTest\nTest\nTest\nTest\nTest\nTest\n"); Assert.assertTrue(false); Assert.assertEquals(1, 2); } }
Thanks!
-
8. Re: Arquillian tests always passing
nickhutt May 20, 2014 4:51 AM (in response to toddpi314)I had a similar issue with Arquillian.
I tried to instantiate an EJB that wasn't actually an EJB. Instead of reporting an error JUnit reported success for each test without running the tests.
It seems in this, and possibly other cases of serious error, all tests are incorrectly reporting success but are not run.
-
9. Re: Arquillian tests always passing
besolov May 20, 2014 11:33 AM (in response to toddpi314)Here is the answer: Injecting EJB into tests
casta_oh said:
When something goes bad with the injection, the test doesn`t work but the test results as "OK".
-
10. Re: Arquillian tests always passing
bmajsak Jul 11, 2014 9:41 AM (in response to besolov)This is strange. Did you try with latest core 1.1.5.Final?