Ideas how to merge Droidium for native and web tesing together?

[smikloso]

Hi,

Droidium now consists of three groups of artifacts:

1) Android container adapter

2) Native plugin for testing native APKs by Selendroid server

3) Web plugin for testing web apps which are deployed to AS and tested from Android point of view - it is using android-server.apk from Selenium.

Since Droidium depends on Selendroid heavily and Android server apk from Selenium project was officially deprecated (1) and they recommend Selendroid from now on (whoa, thats nice coincidence ) we are trying to drop web plugin altogether and simplify Droidium even more from user point of view.

Changes Selendroid team introduced are as follows:

They coded up very simple, almost dummy like, native android application which gets installed along Selendroid server and it is instrumented after that. To make long story short, from now on, when you want to test web apps deployed to AS, you have to do these things on Android side:

1) Install Selendroid server apk as you are used to for native testing

2) Install "android-driver.apk" from Selendroid as well to Android device

3) instrument android-driver.apk by Selendroid server as ordinary APK as you do when you test native way

4) Start WebKit view on Android (check (2))

And thats it - when you inject @Drone, you have WebDriver by which you are testing web pages on Android.

From Arquillian point of view, it was very simple and the only change needed to be introduced was this (technicalities Aslak and Karel would understand ):

arquillian-droidium/droidium-native/arquillian-droidium-native/src/main/java/org/arquillian/droidium/native_/deployment/…

So, when you get AfterDeploy, and that deployment is not being deployed against Android container, you check if it has @Instrumentable on it, and when it does, you fire custom AndroidDeploy event. This is the only change, since from Droidium point of view, whole deployment voodoo logic and instrumentation is the very same as for normal native deployment testing mode but in this case, AndroidDeploy event is not fired by Android container adapter itself from deploy method (because you are deploying wars which does not target Android) so you have to fire it on your own.

Questions:

1) When you look at test, it looks like this:

https://github.com/arquillian/arquillian-droidium/blob/41fdf33d8db95a635d4777f2d6102cc3633543f1/tests/droidium-web-02/src/test/java/org/arquillian/droidium/showcase/web/test01/DroidiumWebTestCase.java#L53

You just put @Instrumentable on deployment method as you would in case of native approach but right now, you are "instrumenting" war deployment which is not so intuitive. We are looking for other name of that annotation so it is totally clear that when you put it on deployment method, you know that something is going to be done on Android container side. What should be that annotation name like?

2) When you have test, you have to start activity from android driver apk from Selendroid you deployed automatically when @Instrumentable is put on deployment method, so you have to do this

https://github.com/arquillian/arquillian-droidium/blob/41fdf33d8db95a635d4777f2d6102cc3633543f1/tests/droidium-web-02/src/test/java/org/arquillian/droidium/showcase/web/test01/DroidiumWebTestCase.java#L67-L70

We are looking for a way how to "hide" it into Drone instantiation logic, because writing this all over again seems to be rather tedious. But the problem is that it is unclear how to connect injected Drone instance with knowing that this particular Drone instance is operating on Android deployment which is there for testing web pages so that switch to webview should be done (+ starting of activity).

When this is done, you get rid of web plugin totally and it will not be released anymore, having one solution with robust lifecycle eventing and logic, deleting any differences between testing of native android applications and web applications, you just inject webdriver and annotated deployments, thats it. Graphene fits into this very conveniently providing quite nice ecosystem which encapsulates it all. Additionally, with the existence of Spacelift extension, you would not need to bring all infrastructure apks with you manually, it would be just somehow resolved so the bootstrap of this is matter of minutes.

1)  https://code.google.com/p/selenium/wiki/AndroidDriver

2) https://github.com/selendroid/selendroid/blob/master/android-driver/src/main/java/io/selendroid/androiddriver/WebViewActivity.java#L28-L32

[smikloso]

Aslak was asking on twitter what about multiple containers.

I was trying to do that and I succeeded only for plain Android container (meaning no native nor web plugin). The problem with native plugin is that when I produce some instances via InstanceProducer, that production has some scope following Arquillian lifecycle but I need to make further difference like to which container it really depends.

For example, when I fire AndroidDeploy in some container context and that internally uses some other classes, for example AndroidDeploymentRegister where I put all deployments which were put into Android, I need to be aware of which container that deployment relates to. The same for SelendroidDeploymentRegister, Activity manager and so on, so I can switch between them internally. Other way is to implement it in "multiple containers" way so they would have some "tag" with android container qualifier which would make difference.

For example I am doing undeployment of Android apps not on propper undeployment logic but outside of it (somewhere AfterClass), this is known and discussed and it is because of how webdriver destroys itself in connection to Selendroid, it has its jira already Issue Navigator - JBoss Issue Tracker

In other words when I am out of container scope, I can not make difference where some particular deployment was deployed (against which container) so I can not unistall it since there needs to be some info where it was installed.

As briefly discussed with Karel, that could lead to some kind of custom scope or context but not sure about this.

[kpiwko]

Hey Stefan,

I'd say ability to get rid of -native and -web a have a single implementation is a great win w.r.t. functionality. So the more the code can be reduced, the better. You know, delete is my favorite programming key ;-).

Answering questions 1) and 2):

We need a better and more intuitive way how to define what should be done with the deployment. Here are possible scenarios I can think of:

• deploy APK to Android device/AVD and test it (either via shell or via Selendroid)
• deploy WAR/EAR to web container and test it (either via desktop browser or via Selendroid)
• deploying APK to Android and WAR/EAR to web container and test/interact with WAR via desktop/mobile browser and test APK via Selendroid

This means that we need to let user express these desires (now = Drodium 1.0.0.Alpha4):

1/ Deploy WAR/EAR to web container (AS)

     Now done via @Deployment method that @TargetsContainer

2/ Deploy APK to Android container (Droidium)

     Now done via @Deployment method that @TargetsContainer

3/ Deploy APK to Android container (Droidium) and instrument application

     Now done via @Deployment method that @TargetsContainer and @Instrumentable annotation

4/ Get desktop browser

     Now done via @Drone WebDriver, with configuration browser=one of firefox|chrome|...

5/ Get mobile browser

     Now done via @Drone WebDriver with configuration browser=android

6/ Get Selendroid browser

     Now done via @Drone WebDriver with configuration browser=android and @Instrumentable(port) and Drone remoteAddress matching the port.

All these fact should be doable independently. Correct me if I'm wrong. So, we really need?

a/ Tell apart desktop, mobile and Selendroid browser

     Proposal: create selendroid BrowserCapability in Droidium so Drone will understand and propertly setup browser=selendroid

                    browser=android will always be mobile android stock browser, but now handled via Droidium and Selendroid with automatic opening of WEB_VIEW

                    TODO: in short future, we might need a way how to identify mobile-firefox, mobile-chrome, etc. as well

b/ Figure out how to map @Deployment, WebDriver and Selendroid together

     Proposal: @Instrumentable on WAR does not make sense. @Instrumentable on APK is not clear in sense that it actually instruments Selendroid, not the APK itself. There is no use case to instrument APK itself (at this moment)

                    We could create @Droidium(enrichers= [{SelendroidEnricher.class}], port=int[14444]) annotation instead [denotes default values] that would actually even allow us to change Selendroid to different impl in future without breaking tests.

                    As both android and selendroid WebDriver BrowserCapabilities will be handled by Droidium, Droidium could automatically update remoteAddress to contain right value (hostName + port)

                    Drone could provide @Qualifier -> qualifer SPI, meaning that @Droidium can be marked as @Qualifier annotation for Drone and instead of class name, there could be a different qualifer model, such as port

How would it look like:

Test.java

@Deployment
@Droidium
@TargetsContainer("android")
public static Archive<?> createDeploymentAPK() {...}

@Deployment
@TargetsContainer("jbossas")
public static Archive<?> createDeploymentWAR() {...}

@Droidium
@Drone
WebDriver selendroid;

@Droidium(port=8888)
@Drone
WebDriver android;

arquillian.xml

<extension qualifer="webdriver-droidium">
   <property name="browser">selendroid</browser>
   <!-- remote url constructed based on default port 14444 + container URL  -->
</extension>

<extension qualifer="webdriver-droidium-8888">
   <property name="browser">android</browser>
   <!-- remote url constructed based on port 8888 + container URL  -->
</extension>

Alternatively, we can get better scheme, such as using name, e.g. @Droidium(name="foobar") would map to qualifer="webdriver-foobar". Also, there might be a better naming - @Droidium -> @DroidiumEnricher or such.

You get the idea.

Karel

[smikloso]

Hi,

1/ Deploy WAR/EAR to web container (AS)

     Now done via @Deployment method that @TargetsContainer

2/ Deploy APK to Android container (Droidium)

     Now done via @Deployment method that @TargetsContainer

3/ Deploy APK to Android container (Droidium) and instrument application

     Now done via @Deployment method that @TargetsContainer and @Instrumentable annotation

4/ Get desktop browser

     Now done via @Drone WebDriver, with configuration browser=one of firefox|chrome|...

5/ Get mobile browser

     Now done via @Drone WebDriver with configuration browser=android

6/ Get Selendroid browser

     Now done via @Drone WebDriver with configuration browser=android and @Instrumentable(port) and Drone remoteAddress matching the port.

This is correct, however note that 5) is right now done with web plugin extension we are willing to drop. There is installed android-server.apk from Selenium project, which was marked as deprecated. However it is possible to preserve backward compatibility with Selenium but I am for total dropping of deprecated things. It does not make sense to pull with you something which is going to be forgotten anyway.

a/ Tell apart desktop, mobile and Selendroid browser

     Proposal: create selendroid BrowserCapability in Droidium so Drone will understand and propertly setup browser=selendroid

                    browser=android will always be mobile android stock browser, but now handled via Droidium and Selendroid with automatic opening of WEB_VIEW

                    TODO: in short future, we might need a way how to identify mobile-firefox, mobile-chrome, etc. as well

Not sure what you mean by this. What would be "browser=selendroid" good for? Maybe I just did not get it.

b/ Figure out how to map @Deployment, WebDriver and Selendroid together

     Proposal: @Instrumentable on WAR does not make sense. @Instrumentable on APK is not clear in sense that it actually instruments Selendroid, not the APK itself. There is no use case to instrument APK itself (at this moment)

                    We could create @Droidium(enrichers= [{SelendroidEnricher.class}], port=int[14444]) annotation instead [denotes default values] that would actually even allow us to change Selendroid to different impl in future without breaking tests.

                    As both android and selendroid WebDriver BrowserCapabilities will be handled by Droidium, Droidium could automatically update remoteAddress to contain right value (hostName + port)

                    Drone could provide @Qualifier -> qualifer SPI, meaning that @Droidium can be marked as @Qualifier annotation for Drone and instead of class name, there could be a different qualifer model, such as port

@Instrumentable on WAR indeed does not make sense at all. Expression "instrumentation" is rather tricky. It is just a matter of how you look at it. Selendroid does instrument Android application, not other way around, so something which you can instrument _is_ "instrumentable". Of course you can not instrument APK itself. That is technically impossible. That has to do third party e.g. Selendroid server.

The argument about no breaking tests when you have the possiblity to use different enricher implementation is valid but quite improbable. I am not sure whether there will ever be any other implementation by which you could just replace Selendroid but nevermind, it can be there, just to be sure.

Annotating @Drone with @Droidium seems to be like lot of typing here. What about @Droidium annotation put on WebDriver while that @Droidium annotation is itself annotated with @Drone? When you take into account that you could have multiple Android browsers as you can have on desktop, which are identified with another custom annotation which lowercase version is appended to webdriver extension qualifier, in your case you would end up with @Drone @Droidium @First instead of mine @Droidium @First. So in the end you would not have qualifier as "webdriver-droidium-first" but just "webdriver-first" as now.

Automatic updating of remote address and port is welcome. I would remove port number form extension qualifier totally. Drone instance in arquillian.xml is recognized by qualifier anyway and address can be set up by port in @Droidium annotation itself. However I am not sure if it is enough to have port information in test - meaning it can be too late in order of execution logic. In that case, putting port in qualifier name make sense.

When you put @Droidium on @Deployment, according to your test, you could specify port argument there. I would leave @Droidium annotation only on deployment methods, however that contradicts with all @Droidium stuff mentioned above, @Droidium(name="foobar")  would be nice. However what about the case it is put on @Deployment is rather unknown.

[kpiwko]

Not sure what you mean by this. What would be "browser=selendroid" good for? Maybe I just did not get it.

browser=selendroid means application testing, browser=android means browser testing. Clearly distinguished from each other.

The argument about no breaking tests when you have the possiblity to use different enricher implementation is valid but quite improbable. I am not sure whether there will ever be any other implementation by which you could just replace Selendroid but nevermind, it can be there, just to be sure.

This would just annotation easier, but given the defaults, it's a nice idea to think about it. Not really saying it is required right now, it was just an idea

Annotating @Drone with @Droidium seems to be like lot of typing here. What about @Droidium annotation put on WebDriver while that @Droidium annotation is itself annotated with @Drone? When you take into account that you could have multiple Android browsers as you can have on desktop, which are identified with another custom annotation which lowercase version is appended to webdriver extension qualifier, in your case you would end up with @Drone @Droidium @First instead of mine @Droidium @First. So in the end you would not have qualifier as "webdriver-droidium-first" but just "webdriver-first" as now.

Basically, the idea was to be able to use @Droidium for both qualifying and Droidium related stuff for WebDriver. So, there is no @First, because first equals @Droidium. If there are more @Droidium, we can either use name or port to create a distinguished name for arquillian.xml.

Automatic updating of remote address and port is welcome. I would remove port number form extension qualifier totally. Drone instance in arquillian.xml is recognized by qualifier anyway and address can be set up by port in @Droidium annotation itself. However I am not sure if it is enough to have port information in test - meaning it can be too late in order of execution logic. In that case, putting port in qualifier name make sense.

I believe we can't unless we introduce name. Or some other way how to bind @Deployment with webdriver configuration in arquillian.xml.

When you put @Droidium on @Deployment, according to your test, you could specify port argument there. I would leave @Droidium annotation only on deployment methods, however that contradicts with all @Droidium stuff mentioned above, @Droidium(name="foobar")  would be nice. However what about the case it is put on @Deployment is rather unknown.

We can have two distinct annotations if that helps. But don't ask me for names.