4 Replies Latest reply on Dec 28, 2014 1:35 PM by Ivan St. Ivanov

    [forge-dev] Again on testing Forge commands

    Ivan St. Ivanov Novice

      Hi everybody,

       

      I resumed my Security addon development and reached my "favorite" point:

      writing and executing UI command tests. I have attached here the output of

      the test harness as well as the sample test that I wrote.

       

      Here are some observations:

        - It took one minute for Forge to run a simple UI test. And this is on

      Linux. From my experience, if I run the same test on Windows, it would take

      at least twice more

       

        - Even though Lincoln explained it to me at least twice, setting up

      @Deployment @AddonDependencies and AddonDependencyEntry's is still black

      magic to me. I usually copy those hoping that I didn't miss anything, but

      the result of this test proves that I missed something

       

        - For the most part the test was starting furnace, checking the missing

      dependencies, installing them one by one, but in the mean time it installed

      their transitive dependencies and for each of these operations, Forge was

      again shutting down and starting up furnace and weld. And then again

      calculating missing dependencies. Most of these operations take usually

      less than a second, but still there are so many of them that at the end it

      piles up to a whole minute

       

        - To be fair, some big chunks of this minute was taken by, what it seems

      to me, resolution of transitive dependencies:

      Dec 22, 2014 11:15:49 PM org.jboss.forge.furnace.impl.addons.AddonRunnable

      run

      INFO: >> Started container

      - 133ms

      Dec 22, 2014 11:15:58 PM

      org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

      INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT

      ....

      Dec 22, 2014 11:16:12 PM org.jboss.forge.furnace.impl.addons.AddonRunnable

      run

      INFO: >> Started container -

      1802ms

      Dec 22, 2014 11:16:24 PM

      org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

      INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT

       

        - The test failed with the following exception:

      java.lang.IllegalStateException: Test runner could not locate test class

      in any

      deployed Addon.

      at

      org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)

      at

      org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at

      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

      at

      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

      at

      org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

      ...

       

          However, the real reason was hidden in the massive console output a bit

      above it:

       

      Dec 22, 2014 11:16:25 PM

      org.jboss.weld.bootstrap.MissingDependenciesRegistry

      handleResourceLoadingException

      INFO: WELD-000119: Not generating any bean definitions from

      org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because

      of underlying class loading error: Type

      org.jboss.forge.addon.javaee.ProjectHelper from [Module

      "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"

      from AddonModuleLoader] not found.  If this is unexpected, enable DEBUG

      logging to see the full error.

       

      Enough with the observations. What can we do about it? Well, I see the

      following areas of improvement:

       

        - Fight the black magic. It shouldn't be so hard to setup a test. What I

      usually need is a UI test harness, project utilities, sometimes a parser

      and the addon that I am testing

        - Fight the slow startup time. So, we are using Arquillian. Imagine how

      would you feel if Arquillian was setting up from scratch Wildfly or (oh

      my!) WebLogic every time you run a Java EE test? Instead, it just relies on

      the fact that the target runtime is there

       

      So, can't we just create a composite test addon or something like that?

      That we use as kind of arquillian container and we just update the needed

      addons there. Instead of setting up everything from scratch. And in the

      @Deployment method we simply list the addons (or even at smaller

      granularity: files) that are changed and we want to be redeployed on top.

       

      This doesn't look too far away form the Arquillian model that we are all

      used to. And I believe that will be much faster to start (especially in the

      so called 'remote' arquillian mode).

       

      What do you think?

       

      Cheers,

      Ivan

       

        • 1. Re: [forge-dev] Again on testing Forge commands
          George Gastaldi Master

          I strongly agree and support that we should improve this area.

           

          Here is the test documentation about the

          @AddonDependencies/AddonDependencyEntry stuff:

          http://forge.jboss.org/document/test-your-addon

           

          I need to test this, but I think we could improve this by making the

          Maven Settings object offline (to avoid remote dependency lookups) .

          We already have a JIRA about this annoying error while running tests:

          https://issues.jboss.org/browse/FORGE-2125

           

          I think for the first option, we could assume that the test dependencies

          are the addons already declared in the project's pom.xml (or we could

          have some method in the ForgeArchive to assume that)

           

          Best Regards,

           

          George Gastaldi

           

          On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:

          Hi everybody,

           

          I resumed my Security addon development and reached my "favorite"

          point: writing and executing UI command tests. I have attached here

          the output of the test harness as well as the sample test that I wrote.

           

          Here are some observations:

            - It took one minute for Forge to run a simple UI test. And this is

          on Linux. From my experience, if I run the same test on Windows, it

          would take at least twice more

           

            - Even though Lincoln explained it to me at least twice, setting up

          @Deployment @AddonDependencies and AddonDependencyEntry's is still

          black magic to me. I usually copy those hoping that I didn't miss

          anything, but the result of this test proves that I missed something

           

            - For the most part the test was starting furnace, checking the

          missing dependencies, installing them one by one, but in the mean time

          it installed their transitive dependencies and for each of these

          operations, Forge was again shutting down and starting up furnace and

          weld. And then again calculating missing dependencies. Most of these

          operations take usually less than a second, but still there are so

          many of them that at the end it piles up to a whole minute

           

            - To be fair, some big chunks of this minute was taken by, what it

          seems to me, resolution of transitive dependencies:

          Dec 22, 2014 11:15:49 PM

          org.jboss.forge.furnace.impl.addons.AddonRunnable run

          INFO: >> Started container

          - 133ms

          Dec 22, 2014 11:15:58 PM

          org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

          INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT

          ....

          Dec 22, 2014 11:16:12 PM

          org.jboss.forge.furnace.impl.addons.AddonRunnable run

          INFO: >> Started container

          - 1802ms

          Dec 22, 2014 11:16:24 PM

          org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

          INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT

           

            - The test failed with the following exception:

          java.lang.IllegalStateException: Test runner could not locate test

          class

          in

          any deployed Addon.

          at

          org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)

          at

          org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)

          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

          at

          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

          at

          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

          at

          org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

          at

          org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

          ...

           

              However, the real reason was hidden in the massive console output

          a bit above it:

           

          Dec 22, 2014 11:16:25 PM

          org.jboss.weld.bootstrap.MissingDependenciesRegistry

          handleResourceLoadingException

          INFO: WELD-000119: Not generating any bean definitions from

          org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest

          because of underlying class loading error: Type

          org.jboss.forge.addon.javaee.ProjectHelper from [Module

          "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"

          from AddonModuleLoader] not found.  If this is unexpected, enable

          DEBUG logging to see the full error.

           

          Enough with the observations. What can we do about it? Well, I see the

          following areas of improvement:

           

            - Fight the black magic. It shouldn't be so hard to setup a test.

          What I usually need is a UI test harness, project utilities, sometimes

          a parser and the addon that I am testing

            - Fight the slow startup time. So, we are using Arquillian. Imagine

          how would you feel if Arquillian was setting up from scratch Wildfly

          or (oh my!) WebLogic every time you run a Java EE test? Instead, it

          just relies on the fact that the target runtime is there

           

          So, can't we just create a composite test addon or something like

          that? That we use as kind of arquillian container and we just update

          the needed addons there. Instead of setting up everything from

          scratch. And in the @Deployment method we simply list the addons (or

          even at smaller granularity: files) that are changed and we want to be

          redeployed on top.

           

          This doesn't look too far away form the Arquillian model that we are

          all used to. And I believe that will be much faster to start

          (especially in the so called 'remote' arquillian mode).

           

          What do you think?

           

          Cheers,

          Ivan

           

          >

           

          _______________________________________________

          forge-dev mailing list

          forge-dev@lists.jboss.org

          https://lists.jboss.org/mailman/listinfo/forge-dev

           

           

          • 2. Re: [forge-dev] Again on testing Forge commands
            Lincoln Baxter III Master

            Hey Ivan,

             

            Great observations, definitely things that we want to improve:

             

            Fight the black magic. It shouldn't be so hard to setup a test. What I

            usually need is a UI test harness, project utilities, sometimes a parser

            and the addon that I am testing

             

             

            Yes, this is an issue. We have a number of ideas for how to improve this,

            such as introducing a new @DeployAddon() or @AddonDeployment() annotation,

            which is really what our current @AddonDependency() annotation is doing -

            deploying an addon. The @AddonDependency() annotation would then deploy AND

            establish an addon dependency on the specified coordinate, eliminating the

            need to specify .withAddonDependency() in the @Deployment method itself.

            Essentially our current situation is one in which we must specify addon

            deployment and dependency metadata separately, which leads to quite a bit

            of duplicate information (boilerplate.) This should be easy to resolve.

            Unfortunately we have a problem with backwards compatability if we change

            the existing annotations, so we may need to deprecate them and introduce

            entirely new ones in a new package.

             

            I would actually like to solve this problem by refactoring the "Forge Test

            Harness" in Furnace and calling it the "Furnace Test Harness" which is

            really what it is.

             

            Fight the slow startup time. So, we are using Arquillian. Imagine how would

            you feel if Arquillian was setting up from scratch Wildfly or (oh my!)

            WebLogic every time you run a Java EE test? Instead, it just relies on the

            fact that the target runtime is there

             

             

            So one thing to note, is that the slowness here is not the fact that we are

            setting up Furnace/Forge from scratch. Just like Wildfly, the runtime

            environment is already there, but we still have to deploy our application

            (the addons.) There is a great deal of inefficiency, as George says, in the

            maven deployment of the addons, and this is (in my opinion) where our

            greatest opportunity for improvement lies.

             

            Very open to ideas here. It's been a while since i attached a profiler, but

            if nothing has changed, most of the time is spent in maven itself -

            resolving dependencies.

             

            On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <ggastald@redhat.com>

            wrote:

             

            I strongly agree and support that we should improve this area.

             

            Here is the test documentation about the

            @AddonDependencies/AddonDependencyEntry stuff:

            http://forge.jboss.org/document/test-your-addon

             

            I need to test this, but I think we could improve this by making the Maven

            Settings object offline (to avoid remote dependency lookups) .

            We already have a JIRA about this annoying error while running tests:

            https://issues.jboss.org/browse/FORGE-2125

             

            I think for the first option, we could assume that the test dependencies

            are the addons already declared in the project's pom.xml (or we could have

            some method in the ForgeArchive to assume that)

             

            Best Regards,

             

            George Gastaldi

             

            >

            On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:

             

            Hi everybody,

             

            I resumed my Security addon development and reached my "favorite" point:

            writing and executing UI command tests. I have attached here the output of

            the test harness as well as the sample test that I wrote.

             

            Here are some observations:

              - It took one minute for Forge to run a simple UI test. And this is on

            Linux. From my experience, if I run the same test on Windows, it would take

            at least twice more

             

               - Even though Lincoln explained it to me at least twice, setting up

            @Deployment @AddonDependencies and AddonDependencyEntry's is still black

            magic to me. I usually copy those hoping that I didn't miss anything, but

            the result of this test proves that I missed something

             

               - For the most part the test was starting furnace, checking the

            missing dependencies, installing them one by one, but in the mean time it

            installed their transitive dependencies and for each of these operations,

            Forge was again shutting down and starting up furnace and weld. And then

            again calculating missing dependencies. Most of these operations take

            usually less than a second, but still there are so many of them that at the

            end it piles up to a whole minute

             

               - To be fair, some big chunks of this minute was taken by, what it

            seems to me, resolution of transitive dependencies:

            Dec 22, 2014 11:15:49 PM org.jboss.forge.furnace.impl.addons.AddonRunnable

            run

            INFO: >> Started container

            - 133ms

            Dec 22, 2014 11:15:58 PM

            org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

            INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT

            ....

            Dec 22, 2014 11:16:12 PM

            org.jboss.forge.furnace.impl.addons.AddonRunnable run

            INFO: >> Started container

            - 1802ms

            Dec 22, 2014 11:16:24 PM

            org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

            INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT

             

               - The test failed with the following exception:

            java.lang.IllegalStateException: Test runner could not locate test class

            in any

            deployed Addon.

            at

            org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)

            at

            org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)

            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

            at

            sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

            at

            sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

            at

            org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

            at

            org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

            ...

             

                 However, the real reason was hidden in the massive console output a

            bit above it:

             

            Dec 22, 2014 11:16:25 PM

            org.jboss.weld.bootstrap.MissingDependenciesRegistry

            handleResourceLoadingException

            INFO: WELD-000119: Not generating any bean definitions from

            org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because

            of underlying class loading error: Type

            org.jboss.forge.addon.javaee.ProjectHelper from [Module

            "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"

            from AddonModuleLoader] not found.  If this is unexpected, enable DEBUG

            logging to see the full error.

             

            Enough with the observations. What can we do about it? Well, I see the

            following areas of improvement:

             

               - Fight the black magic. It shouldn't be so hard to setup a test. What

            I usually need is a UI test harness, project utilities, sometimes a parser

            and the addon that I am testing

              - Fight the slow startup time. So, we are using Arquillian. Imagine how

            would you feel if Arquillian was setting up from scratch Wildfly or (oh

            my!) WebLogic every time you run a Java EE test? Instead, it just relies on

            the fact that the target runtime is there

             

            So, can't we just create a composite test addon or something like that?

            That we use as kind of arquillian container and we just update the needed

            addons there. Instead of setting up everything from scratch. And in the

            @Deployment method we simply list the addons (or even at smaller

            granularity: files) that are changed and we want to be redeployed on top.

             

            This doesn't look too far away form the Arquillian model that we are all

            used to. And I believe that will be much faster to start (especially in the

            so called 'remote' arquillian mode).

             

            What do you think?

             

            Cheers,

            Ivan

             

            >

             

            _______________________________________________

            forge-dev mailing listforge-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev

             

            >

             

            _______________________________________________

            forge-dev mailing list

            forge-dev@lists.jboss.org

            https://lists.jboss.org/mailman/listinfo/forge-dev

             

             

             

             

            --

            Lincoln Baxter, III

            http://ocpsoft.org

            "Simpler is better."

             

            • 3. Re: [forge-dev] Again on testing Forge commands
              Lincoln Baxter III Master

              One possible solution for #2 is to provide a new "all-of-forge" dependency

              that is unzipped from the forge-distribution-offline artifact based on the

              version specified. This would need to be provided as an extension to the

              test-harness, however, because we can't assume that people are using

              Furnace to test Forge. (The Forge test harness could include this extension

              on top of the Furnace test harness.)

               

              Aren't all these F-names easy to keep track of?

               

              On Sat, Dec 27, 2014 at 10:09 AM, Lincoln Baxter, III <

              lincolnbaxter@gmail.com> wrote:

               

              Hey Ivan,

               

              Great observations, definitely things that we want to improve:

               

              Fight the black magic. It shouldn't be so hard to setup a test. What I

              >> usually need is a UI test harness, project utilities, sometimes a parser

              >> and the addon that I am testing

              >

              Yes, this is an issue. We have a number of ideas for how to improve this,

              such as introducing a new @DeployAddon() or @AddonDeployment() annotation,

              which is really what our current @AddonDependency() annotation is doing -

              deploying an addon. The @AddonDependency() annotation would then deploy AND

              establish an addon dependency on the specified coordinate, eliminating the

              need to specify .withAddonDependency() in the @Deployment method itself.

              Essentially our current situation is one in which we must specify addon

              deployment and dependency metadata separately, which leads to quite a bit

              of duplicate information (boilerplate.) This should be easy to resolve.

              Unfortunately we have a problem with backwards compatability if we change

              the existing annotations, so we may need to deprecate them and introduce

              entirely new ones in a new package.

               

              I would actually like to solve this problem by refactoring the "Forge Test

              Harness" in Furnace and calling it the "Furnace Test Harness" which is

              really what it is.

               

              Fight the slow startup time. So, we are using Arquillian. Imagine how

              >> would you feel if Arquillian was setting up from scratch Wildfly or (oh

              >> my!) WebLogic every time you run a Java EE test? Instead, it just relies on

              >> the fact that the target runtime is there

              >

              So one thing to note, is that the slowness here is not the fact that we

              are setting up Furnace/Forge from scratch. Just like Wildfly, the runtime

              environment is already there, but we still have to deploy our application

              (the addons.) There is a great deal of inefficiency, as George says, in the

              maven deployment of the addons, and this is (in my opinion) where our

              greatest opportunity for improvement lies.

               

              Very open to ideas here. It's been a while since i attached a profiler,

              but if nothing has changed, most of the time is spent in maven itself -

              resolving dependencies.

               

              On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <ggastald@redhat.com>

              wrote:

               

              >>  I strongly agree and support that we should improve this area.

              >>

              >> Here is the test documentation about the

              >> @AddonDependencies/AddonDependencyEntry stuff:

              >> http://forge.jboss.org/document/test-your-addon

              >>

              >> I need to test this, but I think we could improve this by making the

              >> Maven Settings object offline (to avoid remote dependency lookups) .

              >> We already have a JIRA about this annoying error while running tests:

              >> https://issues.jboss.org/browse/FORGE-2125

              >>

              >> I think for the first option, we could assume that the test dependencies

              >> are the addons already declared in the project's pom.xml (or we could have

              >> some method in the ForgeArchive to assume that)

              >>

              >> Best Regards,

              >>

              >> George Gastaldi

              >>

              >>

              >> On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:

              >>

              >> Hi everybody,

              >>

              >>  I resumed my Security addon development and reached my "favorite"

              >> point: writing and executing UI command tests. I have attached here the

              >> output of the test harness as well as the sample test that I wrote.

              >>

              >>  Here are some observations:

              >>   - It took one minute for Forge to run a simple UI test. And this is on

              >> Linux. From my experience, if I run the same test on Windows, it would take

              >> at least twice more

              >>

              >>    - Even though Lincoln explained it to me at least twice, setting up

              >> @Deployment @AddonDependencies and AddonDependencyEntry's is still black

              >> magic to me. I usually copy those hoping that I didn't miss anything, but

              >> the result of this test proves that I missed something

              >>

              >>    - For the most part the test was starting furnace, checking the

              >> missing dependencies, installing them one by one, but in the mean time it

              >> installed their transitive dependencies and for each of these operations,

              >> Forge was again shutting down and starting up furnace and weld. And then

              >> again calculating missing dependencies. Most of these operations take

              >> usually less than a second, but still there are so many of them that at the

              >> end it piles up to a whole minute

              >>

              >>    - To be fair, some big chunks of this minute was taken by, what it

              >> seems to me, resolution of transitive dependencies:

              >> Dec 22, 2014 11:15:49 PM

              >> org.jboss.forge.furnace.impl.addons.AddonRunnable run

              >> INFO: >> Started container

              >> - 133ms

              >> Dec 22, 2014 11:15:58 PM

              >> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

              >> INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT

              >> ....

              >>  Dec 22, 2014 11:16:12 PM

              >> org.jboss.forge.furnace.impl.addons.AddonRunnable run

              >> INFO: >> Started container

              >> - 1802ms

              >> Dec 22, 2014 11:16:24 PM

              >> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

              >> INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT

              >>

              >>    - The test failed with the following exception:

              >>  java.lang.IllegalStateException: Test runner could not locate test

              >> class

              >> in any deployed Addon.

              >>  at

              >> org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)

              >>  at

              >> org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)

              >>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              >>  at

              >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

              >>  at

              >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

              >>  at

              >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

              >>  at

              >> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

              >>  ...

              >>

              >>      However, the real reason was hidden in the massive console output a

              >> bit above it:

              >>

              >>  Dec 22, 2014 11:16:25 PM

              >> org.jboss.weld.bootstrap.MissingDependenciesRegistry

              >> handleResourceLoadingException

              >> INFO: WELD-000119: Not generating any bean definitions from

              >> org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because

              >> of underlying class loading error: Type

              >> org.jboss.forge.addon.javaee.ProjectHelper from [Module

              >> "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"

              >> from AddonModuleLoader] not found.  If this is unexpected, enable DEBUG

              >> logging to see the full error.

              >>

              >>  Enough with the observations. What can we do about it? Well, I see the

              >> following areas of improvement:

              >>

              >>    - Fight the black magic. It shouldn't be so hard to setup a test.

              >> What I usually need is a UI test harness, project utilities, sometimes a

              >> parser and the addon that I am testing

              >>   - Fight the slow startup time. So, we are using Arquillian. Imagine how

              >> would you feel if Arquillian was setting up from scratch Wildfly or (oh

              >> my!) WebLogic every time you run a Java EE test? Instead, it just relies on

              >> the fact that the target runtime is there

              >>

              >>  So, can't we just create a composite test addon or something like that?

              >> That we use as kind of arquillian container and we just update the needed

              >> addons there. Instead of setting up everything from scratch. And in the

              >> @Deployment method we simply list the addons (or even at smaller

              >> granularity: files) that are changed and we want to be redeployed on top.

              >>

              >>  This doesn't look too far away form the Arquillian model that we are

              >> all used to. And I believe that will be much faster to start (especially in

              >> the so called 'remote' arquillian mode).

              >>

              >>  What do you think?

              >>

              >>  Cheers,

              >> Ivan

              >>

              >>

              >>

              >> _______________________________________________

              >> forge-dev mailing listforge-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev

              >>

              >>

              >>

              >> _______________________________________________

              >> forge-dev mailing list

              >> forge-dev@lists.jboss.org

              >> https://lists.jboss.org/mailman/listinfo/forge-dev

              >>

              >

              >

              --

              Lincoln Baxter, III

              http://ocpsoft.org

              "Simpler is better."

               

               

               

               

              --

              Lincoln Baxter, III

              http://ocpsoft.org

              "Simpler is better."

               

              • 4. Re: [forge-dev] Again on testing Forge commands
                Ivan St. Ivanov Novice

                Hi everybody,

                 

                Thank you for the answers, guys!

                 

                Lincoln, you don't need a profiler here. You and George both are right that

                the reason for the slowness is resolving of Maven dependencies. So most

                probably, if we have Forge already setup and you just deploy your addon on

                top, maybe this could be fixed. I don't imagine too many addons at the

                moment that depend on anything besides what's in the core.

                 

                This would need to be provided as an extension to the test-harness,

                however, because we can't assume that people are using Furnace to test

                Forge.

                 

                Again, in 99% of the cases people will be using Furnace to test Forge. So

                it should be the default and whoever is not using that should do some

                configurations.

                 

                Again, what I really want is simplicity like the one I get with Arquillian

                tests: just tell Forge which is the addon under test (again, in 99% it is

                the addon of the test case) and whether I am relying on anything different

                than Furnace. So Forge should just take my addon, deploy it in an already

                prepared repository and run the test.

                 

                I know that it might sound very simplistic, but this is the role of people

                that do not work on the core: create impossible looking requirements for

                you, lol

                 

                Cheers,

                Ivan

                 

                On Sat, Dec 27, 2014 at 5:11 PM, Lincoln Baxter, III <

                lincolnbaxter@gmail.com> wrote:

                 

                One possible solution for #2 is to provide a new "all-of-forge" dependency

                that is unzipped from the forge-distribution-offline artifact based on the

                version specified. This would need to be provided as an extension to the

                test-harness, however, because we can't assume that people are using

                Furnace to test Forge. (The Forge test harness could include this extension

                on top of the Furnace test harness.)

                 

                Aren't all these F-names easy to keep track of?

                 

                On Sat, Dec 27, 2014 at 10:09 AM, Lincoln Baxter, III <

                lincolnbaxter@gmail.com> wrote:

                 

                >> Hey Ivan,

                >>

                >> Great observations, definitely things that we want to improve:

                >>

                >> Fight the black magic. It shouldn't be so hard to setup a test. What I

                >>> usually need is a UI test harness, project utilities, sometimes a parser

                >>> and the addon that I am testing

                >>

                >>

                >> Yes, this is an issue. We have a number of ideas for how to improve this,

                >> such as introducing a new @DeployAddon() or @AddonDeployment() annotation,

                >> which is really what our current @AddonDependency() annotation is doing -

                >> deploying an addon. The @AddonDependency() annotation would then deploy AND

                >> establish an addon dependency on the specified coordinate, eliminating the

                >> need to specify .withAddonDependency() in the @Deployment method itself.

                >> Essentially our current situation is one in which we must specify addon

                >> deployment and dependency metadata separately, which leads to quite a bit

                >> of duplicate information (boilerplate.) This should be easy to resolve.

                >> Unfortunately we have a problem with backwards compatability if we change

                >> the existing annotations, so we may need to deprecate them and introduce

                >> entirely new ones in a new package.

                >>

                >> I would actually like to solve this problem by refactoring the "Forge

                >> Test Harness" in Furnace and calling it the "Furnace Test Harness" which is

                >> really what it is.

                >>

                >> Fight the slow startup time. So, we are using Arquillian. Imagine how

                >>> would you feel if Arquillian was setting up from scratch Wildfly or (oh

                >>> my!) WebLogic every time you run a Java EE test? Instead, it just relies on

                >>> the fact that the target runtime is there

                >>

                >>

                >> So one thing to note, is that the slowness here is not the fact that we

                >> are setting up Furnace/Forge from scratch. Just like Wildfly, the runtime

                >> environment is already there, but we still have to deploy our application

                >> (the addons.) There is a great deal of inefficiency, as George says, in the

                >> maven deployment of the addons, and this is (in my opinion) where our

                >> greatest opportunity for improvement lies.

                >>

                >> Very open to ideas here. It's been a while since i attached a profiler,

                >> but if nothing has changed, most of the time is spent in maven itself -

                >> resolving dependencies.

                >>

                >> On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <ggastald@redhat.com>

                >> wrote:

                >>

                >>>  I strongly agree and support that we should improve this area.

                >>>

                >>> Here is the test documentation about the

                >>> @AddonDependencies/AddonDependencyEntry stuff:

                >>> http://forge.jboss.org/document/test-your-addon

                >>>

                >>> I need to test this, but I think we could improve this by making the

                >>> Maven Settings object offline (to avoid remote dependency lookups) .

                >>> We already have a JIRA about this annoying error while running tests:

                >>> https://issues.jboss.org/browse/FORGE-2125

                >>>

                >>> I think for the first option, we could assume that the test dependencies

                >>> are the addons already declared in the project's pom.xml (or we could have

                >>> some method in the ForgeArchive to assume that)

                >>>

                >>> Best Regards,

                >>>

                >>> George Gastaldi

                >>>

                >>>

                >>> On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:

                >>>

                >>> Hi everybody,

                >>>

                >>>  I resumed my Security addon development and reached my "favorite"

                >>> point: writing and executing UI command tests. I have attached here the

                >>> output of the test harness as well as the sample test that I wrote.

                >>>

                >>>  Here are some observations:

                >>>   - It took one minute for Forge to run a simple UI test. And this is on

                >>> Linux. From my experience, if I run the same test on Windows, it would take

                >>> at least twice more

                >>>

                >>>    - Even though Lincoln explained it to me at least twice, setting up

                >>> @Deployment @AddonDependencies and AddonDependencyEntry's is still black

                >>> magic to me. I usually copy those hoping that I didn't miss anything, but

                >>> the result of this test proves that I missed something

                >>>

                >>>    - For the most part the test was starting furnace, checking the

                >>> missing dependencies, installing them one by one, but in the mean time it

                >>> installed their transitive dependencies and for each of these operations,

                >>> Forge was again shutting down and starting up furnace and weld. And then

                >>> again calculating missing dependencies. Most of these operations take

                >>> usually less than a second, but still there are so many of them that at the

                >>> end it piles up to a whole minute

                >>>

                >>>    - To be fair, some big chunks of this minute was taken by, what it

                >>> seems to me, resolution of transitive dependencies:

                >>> Dec 22, 2014 11:15:49 PM

                >>> org.jboss.forge.furnace.impl.addons.AddonRunnable run

                >>> INFO: >> Started container

                >>> - 133ms

                >>> Dec 22, 2014 11:15:58 PM

                >>> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

                >>> INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT

                >>> ....

                >>>  Dec 22, 2014 11:16:12 PM

                >>> org.jboss.forge.furnace.impl.addons.AddonRunnable run

                >>> INFO: >> Started container

                >>> - 1802ms

                >>> Dec 22, 2014 11:16:24 PM

                >>> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy

                >>> INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT

                >>>

                >>>    - The test failed with the following exception:

                >>>  java.lang.IllegalStateException: Test runner could not locate test

                >>> class

                >>> in any deployed Addon.

                >>>  at

                >>> org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)

                >>>  at

                >>> org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)

                >>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                >>>  at

                >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                >>>  at

                >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                >>>  at

                >>> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)

                >>>  at

                >>> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)

                >>>  ...

                >>>

                >>>      However, the real reason was hidden in the massive console output

                >>> a bit above it:

                >>>

                >>>  Dec 22, 2014 11:16:25 PM

                >>> org.jboss.weld.bootstrap.MissingDependenciesRegistry

                >>> handleResourceLoadingException

                >>> INFO: WELD-000119: Not generating any bean definitions from

                >>> org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because

                >>> of underlying class loading error: Type

                >>> org.jboss.forge.addon.javaee.ProjectHelper from [Module

                >>> "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"

                >>> from AddonModuleLoader] not found.  If this is unexpected, enable DEBUG

                >>> logging to see the full error.

                >>>

                >>>  Enough with the observations. What can we do about it? Well, I see the

                >>> following areas of improvement:

                >>>

                >>>    - Fight the black magic. It shouldn't be so hard to setup a test.

                >>> What I usually need is a UI test harness, project utilities, sometimes a

                >>> parser and the addon that I am testing

                >>>   - Fight the slow startup time. So, we are using Arquillian. Imagine

                >>> how would you feel if Arquillian was setting up from scratch Wildfly or (oh

                >>> my!) WebLogic every time you run a Java EE test? Instead, it just relies on

                >>> the fact that the target runtime is there

                >>>

                >>>  So, can't we just create a composite test addon or something like

                >>> that? That we use as kind of arquillian container and we just update the

                >>> needed addons there. Instead of setting up everything from scratch. And in

                >>> the @Deployment method we simply list the addons (or even at smaller

                >>> granularity: files) that are changed and we want to be redeployed on top.

                >>>

                >>>  This doesn't look too far away form the Arquillian model that we are

                >>> all used to. And I believe that will be much faster to start (especially in

                >>> the so called 'remote' arquillian mode).

                >>>

                >>>  What do you think?

                >>>

                >>>  Cheers,

                >>> Ivan

                >>>

                >>>

                >>>

                >>> _______________________________________________

                >>> forge-dev mailing listforge-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev

                >>>

                >>>

                >>>

                >>> _______________________________________________

                >>> forge-dev mailing list

                >>> forge-dev@lists.jboss.org

                >>> https://lists.jboss.org/mailman/listinfo/forge-dev

                >>>

                >>

                >>

                >>

                >> --

                >> Lincoln Baxter, III

                >> http://ocpsoft.org

                >> "Simpler is better."

                >>

                >

                >

                --

                Lincoln Baxter, III

                http://ocpsoft.org

                "Simpler is better."

                 

                _______________________________________________

                forge-dev mailing list

                forge-dev@lists.jboss.org

                https://lists.jboss.org/mailman/listinfo/forge-dev