8 Replies Latest reply on Jan 16, 2014 3:41 AM by aslak

    Arquillian Native Platform extension

    kpiwko

      Hey Arquillians,

       

      for a long time we are struggling in using tools that are not they easily available from Java world. Things like Android SDK, PhantomJS driver binary, chromedriver server binary, containers downloaded via complicated Maven commands and similar stuff.

      While is it possible to download all the stuff manually, people still need to set paths and similar stuff. All that clutters the first experience with Arquillian.

       

      Therefore, I'm resurrecting proposal for native platform extension, that could be consumed by extension creators. The point is to create a solid platform that is able to handle all that stuff based on metadata model.

       

      Abilities

      • should be able to get an archive/binary from any external location
      • should be able to cache archive/binary using maven coordinates in local repository, acting as cache
      • should be able to handle different binaries per platform
      • should be able to execute code after installation and before installation
      • should be able to cache results - only first execution needs to do that
      • should have access to arquillian.xml descriptor
      • should be externally configurable (via arquillian.xml) - selected metadata could be overrided in arquillian.xml somehow
      • must log it's progress, so it does not look stuck
      • must be able to configure the workspace - the directory where platform related bits are extracted - so for instance Android SDK can be shared per multiple jobs

       

      Metadata model

      Metadata model is not yet available, but it could be based on YAML and look in similar way (suggestions welcomed):

       

      id: android-sdk
      version: 22.3
      supports:
           - linux
           - windows
      remoteUrl:
           linux: http://dl.google.com/android/android-sdk_r22.3-linux.tgz
           windows: http://dl.google.com/android/android-sdk_r22.3-windows.zip
      localFileName:
           linux: android-sdk_r22.3-linux.tgz
           windows: android-sdk_r22.3-windows.zip
      home:
           linux: android-sdk-linux
           windows: android-sdk-windows
      cacheCoordinates: org.arquillian.nativeplatforms.cache:android-sdk
      preInstallActions:
           any:
                - !!java org.arquillian.droidium.SayHelloToEarthlingsAction
      installActions:
           linux: # a sh script
                - tar xvzf ${localFileName} 
           windows: # a cmd script
                - unzip ${localFileName}
      postInstallActions:
           linux:
                - chmod -R a+x ${home}/tools
                - !!java org.arquillian.droidium.AndroidSdkInstallerAction
           windows:
                - !!java org.arquillian.droidium.AndroidSdkInstallerAction
      

      Workflow

       

      Based on metadata definition, any extension could use native platforms extensions in the following way (based on Android SDK example in Droidium)

      1. Check whether Android SDK is set and points to existing directory
        1. If yes, go to step 6
        2. If no, go to step 2.
      2. Get metadata for Android SDK
      3. Check whether an installation is available in local maven repository cache
        1. If yes, go to step 4
        2. If no, download it from external location and put it into cache
      4. Perform pre-install actions
      5. Perform install actions
      6. Perform post install actions - must be idempotent operation, for instance install API level 17 configured in Droidium
      7. Continue with the execution of Droidium

      The second execution will simply skip all steps, making the first execution run longer but having no effect on subsequent execution

       

      POC

       

      I have created a candidate with support for external process execution - based on current code in Drodium, I'd like to move it under Arquillian umbrella and release asap. It can immediately replace Process related code in Arquillian-Droidium and it would also be used in next Drone version to work with Appium.

       

      kpiwko/arquillian-native-platform · GitHub

       

      Feedback required :-)

       

      Thanks,

       

      Karel

       

      Additional materials

       

      Drafts (spoiler alert - contains some Czech/Slovak language notes):

      page01.png

      page02.png

        • 1. Re: Arquillian Native Platform extension
          smikloso

          Just to support and acknowledge Karel's words, this extension will be the first class citizen in upcomming Droidium releases. Metamodel looks good to me, maybe JSON support would be nice to have as well but it is highly optional and yaml could do its job as well.

           

          As a use case, by native extesion, in case of Droidium, user would not have to download all different apks on its own but this extension would do it. E.g. there is selendroid-server.apk needed and present every time (and other apks in upcoming selendroid releases) but since these would be downloaded / resolved dynamically via ShrinkWrap Maven resolver, user is good to have just plain test project not caring about all different selendroid apks at all nor finding all bits manually on the internet.

          • 2. Re: Arquillian Native Platform extension
            aslak

            I'm all for simplifying this part.

             

            Any thoughts on reusing the infrastructure here for downloading Containers dists as well?

            • 3. Re: Arquillian Native Platform extension
              kpiwko

              The same way. It would be up to container implementation to define metadata for the container installation and use it when appropriate, for example when container "home" is not defined or points to a non existing directory.

              If relying on other properties would be confusing, we can provide for example "autoinstall" property that would trigger the native platform extension. The point is that container extension will bring tested/certified version of the container installation itself to simplify usage but we need more experienced users to be able to supply their own container installations.

              • 4. Re: Arquillian Native Platform extension
                aslak

                Agree that we will still need the 'custom' option for medium/advanced usage, mostly looking at this for the 'first steps' option. Attempting to get the '5 min from zero to running' timeline nailed down with as few as possible ways of getting it wrong.

                • 5. Re: Arquillian Native Platform extension
                  kpiwko

                  Given recent feedback on twitter, this stuff needs a better name. Currently, the extension basically does:

                   

                  • it does process management
                  • it will do package management

                   

                  Given the topic and following Arquillian story, following ideas were gathered so far:

                  • Kryton
                  • Zed
                  • Spacelift
                  • Forklift
                  • Manager
                  • Packager
                  • Talking Toaster
                  • 6. Re: Arquillian Native Platform extension
                    smikloso

                    I am for "Boilerplate" as suggested on Twitter.

                     

                    Arquillian Boilerplate Extension

                     

                    /edit

                     

                    according to twitter discussion, spacelift is ok as well

                    • 7. Re: Arquillian Native Platform extension
                      kpiwko

                      So, I'm aiming for:

                       

                      groupId: org.arquillian.spacelift

                      artifactId: arquillian-spacelift

                      github: arquillian-spacelift

                      jira: Extension - Spacelift

                      jira versions: spacelift_1.0.0.Alpha1

                       

                      No extension in the name, same as Graphene, Warp or Droidium.aslak.aslak.conduct.no any concerns? I can refactor and release Alpha1 tomorrow.

                      • 8. Re: Arquillian Native Platform extension
                        aslak

                        Sounds good!