Are Arquillian tests typically so slow?

[cheekynuss]

While the test results are as I expect, it takes a long time it takes to produce those results.

Here is an output of one of my testing runs.

cd C:\Users\EXPUNGED\git\agd\internal; "JAVA_HOME=C:\\Program Files\\Java\\jdk1.8.0_60" cmd /c "\"\"C:\\Program Files\\NetBeans 8.0.2\\java\\maven\\bin\\mvn.bat\" -Dmaven.ext.class.path=\"C:\\Program Files\\NetBeans 8.0.2\\java\\maven-nblib\\netbeans-eventspy.jar\" -Dfile.encoding=UTF-8 test\""
Scanning for projects...
                                                                        
------------------------------------------------------------------------
Building internal 0.1.0
------------------------------------------------------------------------


--- maven-dependency-plugin:2.6:copy (default) @ internal ---


--- maven-resources-plugin:2.5:resources (default-resources) @ internal ---
[debug] execute contextualize
Using 'UTF-8' encoding to copy filtered resources.
Copying 0 resource


--- maven-compiler-plugin:3.1:compile (default-compile) @ internal ---
Nothing to compile - all classes are up to date


--- maven-resources-plugin:2.5:testResources (default-testResources) @ internal ---
[debug] execute contextualize
Using 'UTF-8' encoding to copy filtered resources.
Copying 2 resources


--- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ internal ---
Nothing to compile - all classes are up to date


--- maven-surefire-plugin:2.10:test (default-test) @ internal ---
Surefire report directory: C:\Users\EXPUNGED\git\agd\internal\target\surefire-reports


-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.agd.internal.BTest
Nov 24, 2015 3:55:12 PM org.jboss.arquillian.container.glassfish.clientutils.GlassFishClientUtil getResponseMap
WARNING:  [status: CLIENT_ERROR reason: Not Found]
Nov 24, 2015 3:55:12 PM org.jboss.arquillian.container.glassfish.clientutils.GlassFishClientUtil getResponseMap
WARNING:  [status: CLIENT_ERROR reason: Not Found]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 164.598 sec


Results :


Tests run: 1, Failures: 0, Errors: 0, Skipped: 0


------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 2:48.023s
Finished at: Tue Nov 24 15:57:54 PST 2015
Final Memory: 12M/309M
------------------------------------------------------------------------

Here is the class I am testing:

LabRat.java

package com.agd.internal;


import javax.inject.Named;
import javax.enterprise.context.ApplicationScoped;

@Named(value = "labRat")
@ApplicationScoped
public class LabRat {
    boolean exist1 = true;
    boolean exist2 = true;
    boolean exist3 = true;
    boolean exist4 = true;
    boolean exist5 = true;
    boolean exist6 = true;
    boolean exist7 = true;
    boolean exist8 = true;
    boolean exist9 = true;
    boolean exist10 = true;
    /**
     * Creates a new instance of LabRat
     */
    public LabRat() {
    }


    public boolean isExist1() {
        return exist1;
    }


    public boolean isExist2() {
        return exist2;
    }


    public boolean isExist3() {
        return exist3;
    }


    public boolean isExist4() {
        return exist4;
    }


    public boolean isExist5() {
        return exist5;
    }


    public boolean isExist6() {
        return exist6;
    }


    public boolean isExist7() {
        return exist7;
    }


    public boolean isExist8() {
        return exist8;
    }


    public boolean isExist9() {
        return exist9;
    }


    public boolean isExist10() {
        return exist10;
    }
   
    public int sayOne() {
        return 1;
    }
   
}

This is my testcase class:

BTest.java

package com.agd.internal;


import javax.inject.Inject;
import org.jboss.arquillian.container.test.api.Deployment;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.asset.EmptyAsset;
import org.jboss.shrinkwrap.api.spec.WebArchive;
import org.junit.Test;
import org.junit.Assert;
import org.junit.runner.RunWith;

@RunWith(Arquillian.class)
public class BTest {
    @Deployment
    public static WebArchive createDeployment() {
        return ShrinkWrap.create(WebArchive.class)
            .addClass(LabRat.class)
            .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml");
    }
    
    @Inject
    LabRat rat;


    @Test
    public void is_exist() {
        Assert.assertTrue(rat.isExist1());
    }
}

I am currently using Payara 4.1 and I haven't adjust any JVM settings.

Operating system Windows 10. Processor Intel Core i5 760.

[raneves]

Yes the test with arquillian is very slow, but it is common because Arquillian needs create all context and builder the project.

You need create a Suite for test all test cases. Arquillian will support this feature, see this discussion

I created a workaround for suite all tests and references it here

[bmajsak]

Rafael Neves wrote:

Yes the test with arquillian is very slow, but it is common because Arquillian needs create all context and builder the project.

That's not really true. They are significantly slower than unit tests, but I would be far from calling them very slow. It all depends how your setup is looking like. In our projects we have few hundreds of Arquillian tests and the full-blown builds (including unit, integration and UI tests) takes around 5 minutes. It is a lot and can be optimized smartly, but if you look at the example from Howard, he runs 1 test only and it takes almost 3 minutes. I would say something is wrong with the setup or the container adapter he is using rather than Arquillian as such.

@Howard Can you share this sample project so I can have a chance to run it on my machine and get better picture?