Arquillian Native Platform extension

[kpiwko]

Hey Arquillians,

for a long time we are struggling in using tools that are not they easily available from Java world. Things like Android SDK, PhantomJS driver binary, chromedriver server binary, containers downloaded via complicated Maven commands and similar stuff.

While is it possible to download all the stuff manually, people still need to set paths and similar stuff. All that clutters the first experience with Arquillian.

Therefore, I'm resurrecting proposal for native platform extension, that could be consumed by extension creators. The point is to create a solid platform that is able to handle all that stuff based on metadata model.

Abilities

• should be able to get an archive/binary from any external location
• should be able to cache archive/binary using maven coordinates in local repository, acting as cache
• should be able to handle different binaries per platform
• should be able to execute code after installation and before installation
• should be able to cache results - only first execution needs to do that
• should have access to arquillian.xml descriptor
• should be externally configurable (via arquillian.xml) - selected metadata could be overrided in arquillian.xml somehow
• must log it's progress, so it does not look stuck
• must be able to configure the workspace - the directory where platform related bits are extracted - so for instance Android SDK can be shared per multiple jobs

Metadata model

Metadata model is not yet available, but it could be based on YAML and look in similar way (suggestions welcomed):

id: android-sdk
version: 22.3
supports:
     - linux
     - windows
remoteUrl:
     linux: http://dl.google.com/android/android-sdk_r22.3-linux.tgz
     windows: http://dl.google.com/android/android-sdk_r22.3-windows.zip
localFileName:
     linux: android-sdk_r22.3-linux.tgz
     windows: android-sdk_r22.3-windows.zip
home:
     linux: android-sdk-linux
     windows: android-sdk-windows
cacheCoordinates: org.arquillian.nativeplatforms.cache:android-sdk
preInstallActions:
     any:
          - !!java org.arquillian.droidium.SayHelloToEarthlingsAction
installActions:
     linux: # a sh script
          - tar xvzf ${localFileName} 
     windows: # a cmd script
          - unzip ${localFileName}
postInstallActions:
     linux:
          - chmod -R a+x ${home}/tools
          - !!java org.arquillian.droidium.AndroidSdkInstallerAction
     windows:
          - !!java org.arquillian.droidium.AndroidSdkInstallerAction

Workflow

Based on metadata definition, any extension could use native platforms extensions in the following way (based on Android SDK example in Droidium)

1. Check whether Android SDK is set and points to existing directory
   1. If yes, go to step 6
   2. If no, go to step 2.
2. Get metadata for Android SDK
3. Check whether an installation is available in local maven repository cache
   1. If yes, go to step 4
   2. If no, download it from external location and put it into cache
4. Perform pre-install actions
5. Perform install actions
6. Perform post install actions - must be idempotent operation, for instance install API level 17 configured in Droidium
7. Continue with the execution of Droidium

The second execution will simply skip all steps, making the first execution run longer but having no effect on subsequent execution

POC

I have created a candidate with support for external process execution - based on current code in Drodium, I'd like to move it under Arquillian umbrella and release asap. It can immediately replace Process related code in Arquillian-Droidium and it would also be used in next Drone version to work with Appium.

kpiwko/arquillian-native-platform · GitHub

Feedback required :-)

Thanks,

Karel

Additional materials

Drafts (spoiler alert - contains some Czech/Slovak language notes):

[smikloso]

Just to support and acknowledge Karel's words, this extension will be the first class citizen in upcomming Droidium releases. Metamodel looks good to me, maybe JSON support would be nice to have as well but it is highly optional and yaml could do its job as well.

As a use case, by native extesion, in case of Droidium, user would not have to download all different apks on its own but this extension would do it. E.g. there is selendroid-server.apk needed and present every time (and other apks in upcoming selendroid releases) but since these would be downloaded / resolved dynamically via ShrinkWrap Maven resolver, user is good to have just plain test project not caring about all different selendroid apks at all nor finding all bits manually on the internet.

[aslak]

I'm all for simplifying this part.

Any thoughts on reusing the infrastructure here for downloading Containers dists as well?

[kpiwko]

The same way. It would be up to container implementation to define metadata for the container installation and use it when appropriate, for example when container "home" is not defined or points to a non existing directory.

If relying on other properties would be confusing, we can provide for example "autoinstall" property that would trigger the native platform extension. The point is that container extension will bring tested/certified version of the container installation itself to simplify usage but we need more experienced users to be able to supply their own container installations.

[aslak]

Agree that we will still need the 'custom' option for medium/advanced usage, mostly looking at this for the 'first steps' option. Attempting to get the '5 min from zero to running' timeline nailed down with as few as possible ways of getting it wrong.

[kpiwko]

Given recent feedback on twitter, this stuff needs a better name. Currently, the extension basically does:

• it does process management
• it will do package management

Given the topic and following Arquillian story, following ideas were gathered so far:

• Kryton
• Zed
• Spacelift
• Forklift
• Manager
• Packager
• Talking Toaster

[smikloso]

I am for "Boilerplate" as suggested on Twitter.

Arquillian Boilerplate Extension

/edit

according to twitter discussion, spacelift is ok as well

[kpiwko]

So, I'm aiming for:

groupId: org.arquillian.spacelift

artifactId: arquillian-spacelift

github: arquillian-spacelift

jira: Extension - Spacelift

jira versions: spacelift_1.0.0.Alpha1

No extension in the name, same as Graphene, Warp or Droidium.aslak.aslak.conduct.no any concerns? I can refactor and release Alpha1 tomorrow.

[aslak]

Sounds good!