Cube JAX-RS dependency

[nickarls]

I'm trying to setup a jenkins master that kicks off a SSH-communicating docker slave that runs arquillian cube for starting a postgresql-test-db and launching a WildFly through a remote connector.

I've tried mapping the docker socket to the slave container and using a unix:///var/run/docker.sock for dockr serverUri in arquillian.xml but apparently dockerjava still insists on calling the Docker REST endpoint since I get a stacktrace of

java.lang.RuntimeException: Could not create new instance of class org.jboss.arquillian.test.impl.EventTestRunnerAdaptor

        at org.jboss.arquillian.test.spi.SecurityActions.newInstance(SecurityActions.java:146)

        at org.jboss.arquillian.test.spi.SecurityActions.newInstance(SecurityActions.java:89)

        at org.jboss.arquillian.test.spi.TestRunnerAdaptorBuilder.build(TestRunnerAdaptorBuilder.java:49)

        at org.jboss.arquillian.junit.AdaptorManager.initializeAdaptor(AdaptorManager.java:21)

        at org.jboss.arquillian.junit.AdaptorManagerWithNotifier.initializeAdaptor(AdaptorManagerWithNotifier.java:19)

        at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:109)

        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)

        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)

        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)

        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)

        at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)

        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)

        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Caused by: java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

        at org.jboss.arquillian.test.spi.SecurityActions.newInstance(SecurityActions.java:144)

        ... 17 more

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException

        at javax.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:103)

        at com.github.dockerjava.jaxrs.JerseyDockerCmdExecFactory.init(JerseyDockerCmdExecFactory.java:229)

        at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:161)

        at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:47)

        at org.arquillian.cube.docker.impl.util.DefaultDocker.getDefaultDockerClient(DefaultDocker.java:13)

        at org.arquillian.cube.docker.impl.client.CubeDockerConfigurationResolver.resolveSystemDefaultSetup(CubeDockerConfigurationResolver.java:217)

        at org.arquillian.cube.docker.impl.client.CubeDockerConfigurationResolver.resolve(CubeDockerConfigurationResolver.java:70)

        at org.arquillian.cube.docker.impl.client.CubeDockerConfigurator.configure(CubeDockerConfigurator.java:82)

        at org.arquillian.cube.docker.impl.client.CubeDockerConfigurator.configure(CubeDockerConfigurator.java:63)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:86)

        at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)

        at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:90)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:133)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:105)

        at org.jboss.arquillian.core.impl.ManagerImpl.bindAndFire(ManagerImpl.java:232)

        at org.jboss.arquillian.core.impl.InstanceImpl.set(InstanceImpl.java:67)

        at org.arquillian.cube.impl.client.CubeConfigurator.configure(CubeConfigurator.java:23)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:86)

        at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)

        at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:90)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:133)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:105)

        at org.jboss.arquillian.core.impl.ManagerImpl.bindAndFire(ManagerImpl.java:232)

        at org.jboss.arquillian.core.impl.InstanceImpl.set(InstanceImpl.java:67)

        at org.jboss.arquillian.config.impl.extension.ConfigurationRegistrar.loadConfiguration(ConfigurationRegistrar.java:72)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:86)

        at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)

        at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:90)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:133)

        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:105)

        at org.jboss.arquillian.core.impl.ManagerImpl.start(ManagerImpl.java:253)

        at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.<init>(EventTestRunnerAdaptor.java:61)

        ... 22 more

Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException

        at javax.ws.rs.client.FactoryFinder.getModuleClassLoader(FactoryFinder.java:258)

        at javax.ws.rs.client.FactoryFinder.find(FactoryFinder.java:203)

        at javax.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:87)

        ... 65 more

Caused by: java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at javax.ws.rs.client.FactoryFinder.getModuleClassLoader(FactoryFinder.java:250)

        ... 67 more

Caused by: org.jboss.modules.ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api:main

        at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:240)

        ... 72 more

What I'm not getting is that it appears to be the WildFly module loader that doesn't pick up some module but where does it get that module from? In WildFly 12 there is no such module(?) so providing that dependency through jboss-deployment-structure.xml is not going to work anyway, right?

Am I using some wrong version of something(tm) and Cube is passing in something old into the dockerjava DockerClientBuilder?

[nickarls]

Ping aslak, is ARQ still alive? ;-)

The strange thing is that even if I have the serverURI defined as socket (and I can see it outputted when cube starts), dockerjava still insists on using the REST-integration. And the container has access to the socket, I've tested that by ssh:ing into the container and doing a curl against it. I've also modified the image to run a simple dockerjava test program and that works, too.

I tried including the RESTeasy client jars into the test archive but it started crying about not being able to handle the json-response and adding jersey too didn't help for some reason so I did a fallback to mounting the socket...

[nickarls]

Soooo, any theories why

1. Cube/docker-java doesn't pick up the unix:///var/run/docker.sock in the slave image even if it exists and is runnable?
2. Why it starts complaining about the jaxrs-api module?
3. What to do about it?

PS. How is docker-java even called? I tried exporting the ftest from cube/docker and I didn't see it merged into any jar but still it's there in the stacktrace.

Strange.

[nickarls]

Back to REST attempt. I tried the

<systemPropertyVariables>

     <!--  Force Jersey to be used when RestEasy is auto-discovered. Jersey is required by docker-java -->

     <javax.ws.rs.ext.RuntimeDelegate>org.glassfish.jersey.internal.RuntimeDelegateImpl</javax.ws.rs.ext.RuntimeDelegate>

</systemPropertyVariables>

mentioned in

Fix transitive deps with jaxrs issues · Issue #57 · arquillian/arquillian-cube · GitHub

but I *still* get the resteasy module reference!

[tomjenkinson]

I have a similar problem here with the module (that does not exist in WFLY 14) not being found, I think I must have a misconfiguration but I am not sure what would trigger needing that module. Tried to look in the code for Arquillian but can't find the reference in the repo I looked in.

Did you solve your problem yet? It might help me to modify my code to resolve mine hopefully

[zhfeng]

Hi Tom,

I think it could be an issue in jboss-jaxrs-api_spec/FactoryFinder.java at master · jboss/jboss-jaxrs-api_spec · GitHub

It has the reference of "org.jboss.resteasy.resteasy-jaxrs-api"

[tomjenkinson]

That is a good find Amos, it seems like it was changed two years ago: jboss-jaxrs-api_spec/src/main/java/javax/ws/rs/client/FactoryFinder.java at master · jboss/jboss-jaxrs-api_spec · GitHub

Maybe asoldano has some suggestion?

[tomjenkinson]

Actually, thanks to a suggestion from zhfeng we managed to solve my own problem, it might work for you too:

<dependency>

  <groupId>org.jboss.resteasy</groupId>

  <artifactId>resteasy-client</artifactId>

  <version>3.6.1.Final</version>

  <scope>test</scope>

</dependency>

[asoldano]

So, to shed some light on the issue here, the jboss fork of the jax-rs api spec jar is being used here. One of the minor changes it has compared to the vanilla version is that it adds a mechanism for using JBoss Modules to detect if RESTEasy is available when trying to build one of the jaxrs factories. That mechanism is triggered when no jax-rs implementation is found using 1) the context classloader, 2) $java.home/lib/jaxrs.properties, 3) the proper system property ("javax.ws.rs.client.ClientBuilder" in the case here). The vanilla api fallbacks on Jersey implementation, our api used to fallback to resteasy implementation before JBEE-169 was implemented and is now running the jboss modules mechanism above before fallbacking to Jersey implementation.

The problem is that there's a bug in JBEE-169 fix, if jboss-modules is available but the org.jboss.resteasy.resteasy-jaxrs-api module is not there, an unchecked exception is thrown and you get the failure above. I've just created [JBEE-197] ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api:main - JBoss Issue Tracker  and a PR to fix it [JBEE-197] Deal with scenario of missing resteasy module, but JBoss M… by asoldano · Pull Request #7 · jboss/jboss-jaxrs…

As to why the module is not there, that's another involved story related to the upgrade of WildFly to RESTEasy 3.1 series which was performed and then rollbacked (currently the jaxrs resolution is effectively controlled by the context classloader on the container if I remember properly, which is also why adding the resteasy-client artifact dependency as mentioned above fixes the problem).

Speaking of workarounds, you could also set the javax.ws.rs.client.ClientBuilder system property (it's not what you tried above).

Cheers

[nickarls]

Thanks, I'll look into it. Last time I was time-pressured for a POC so I had to ditch Cube for some utility-scripts against docker-java in the CI pipeline to start up the containers...

[nickarls]

OK. Back with a vengeance:

Scenario: dockerized jenkins with access to docker TCP REST endpoint

First attempt:

   setup: vanilla

   result: Caused by: org.jboss.modules.ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api

Second attempt:

   setup: add test-scoped resteasy-client to pom.xml

   result: javax.ws.rs.ProcessingException: RESTEASY003145: Unable to find a MessageBodyReader of content-type application/vnd.docker.raw-stream and type class java.io.InputStream

Third attempt:

   setup: same as last + <javax.ws.rs.ext.RuntimeDelegate>org.glassfish.jersey.internal.RuntimeDelegateImpl</javax.ws.rs.ext.RuntimeDelegate>

   result: javax.ws.rs.ProcessingException: RESTEASY003145: Unable to find a MessageBodyReader of content-type application/vnd.docker.raw-stream and type class java.io.InputStream

Fourth attempt:

   setup: just the <javax.ws.rs.ext.RuntimeDelegate>org.glassfish.jersey.internal.RuntimeDelegateImpl</javax.ws.rs.ext.RuntimeDelegate>

   result: Caused by: org.jboss.modules.ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api

Any more?

What should the javax.ws.rs.client.ClientBuilder be set to? I tried with <javax.ws.rs.client.ClientBuilder>org.glassfish.jersey.client.JerseyClientBuilder</javax.ws.rs.client.ClientBuilder> but not go.

Actually it's a bit confusing since If I use some dummy value it fails, proving the setting is picked up. But if I use the JerseyClientBuilder, I see no reference to it (or to j.w.r.c.ClientBuilder) in the stacktrace.

The stack is

Caused by: org.jboss.modules.ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api
at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:285)
at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:271)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.ws.rs.ext.FactoryFinder.getModuleClassLoader(FactoryFinder.java:252)
at javax.ws.rs.ext.FactoryFinder.find(FactoryFinder.java:205)
at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:136)
at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120)
at javax.ws.rs.core.UriBuilder.newInstance(UriBuilder.java:95)
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
at org.glassfish.jersey.client.JerseyWebTarget.<init>(JerseyWebTarget.java:71)
at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:290)
at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:76)
at com.github.dockerjava.jaxrs.JerseyDockerCmdExecFactory.init(JerseyDockerCmdExecFactory.java:237)
at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:161)
at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:47)

Apparently it is used but at some point still hits the module class loader from the jboss-jaxrs-api or something

[nickarls]

I can confirm that the fix works "in the wild". I compiled the change and threw it into the local repo of the build machine. It required upgrade to the EE8 API though so it would be nice to know of any other potential workarounds untils it hopefully makes the EE 8 Alpha2 spec API.

[asoldano]

Thanks for the verification Nicklas.

I've released jboss-jaxrs-api-2.1_1.0.2.Final which includes the fix. I've also opened up another PR (Misc fixes from master by asoldano · Pull Request #8 · jboss/jboss-jaxrs-api_spec · GitHub ) that would bring the fix to the jaxrs 2.0 api spec fork.

Regarding the workaround, I would have expected <javax.ws.rs.client.ClientBuilder>org.glassfish.jersey.client.JerseyClientBuilder</javax.ws.rs.client.ClientBuilder> to work, maybe the Jersey client builder does not really have that FQN ?

Cheers

[nickarls]

At least it looks correct: jersey/JerseyClientBuilder.java at master · jersey/jersey · GitHub

[esteve]

Hi,

I am getting the same error with Wildfly 15. I am using:

- Arquillian univers 1.2.0.1

- arquillian-cube-docker and arquillian-cube-requirement 1.18.2

Same two errors:

- org.jboss.modules.ModuleNotFoundException: org.jboss.resteasy.resteasy-jaxrs-api

If I add resteasy-client:

- RESTEASY003145: Unable to find a MessageBodyReader of content-type application/vnd.docker.raw-stream and type class java.io.InputStream

Do you have a way to solve it?

Thanks.