Remote EJB Invocation Error with HTTP Command as Method Argument

[tiburon]

Hi!

We found a strange problem with remote EJB calls to stateless EJB in Wildlfy 8.2.0. When a method argument contains a HTTP command, this command seems to be interpreted by the server and not treated as a regular string.

For Example we have this EJB method (which is also defined in the remote interface of our bean):

@Override
public void test(String parameter){
    ...
}

And we call this method remotely like this:

final Properties ejbProperties = new Properties();
ejbProperties.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");
ejbProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
ejbProperties.put("remote.connections", "default");
ejbProperties.put("remote.connection.default.host", ip);
ejbProperties.put("remote.connection.default.port", port);
ejbProperties.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");
ejbProperties.put("endpoint.name", "client-endpoint");

final EJBClientConfiguration ejbClientConfiguration = new PropertiesBasedEJBClientConfiguration(ejbProperties);
final ConfigBasedEJBClientContextSelector selector = new ConfigBasedEJBClientContextSelector(ejbClientConfiguration);
EJBClientContext.setSelector(selector);
StatelessEJBLocator<T> locator = new StatelessEJBLocator<T>(type, app, module, beanName, "");
T ejb = EJBClient.createProxy(locator);
ejb.test("GET");

This invocation does not reach the bean, but causes our client to freeze. I looked into the EJBClientInvocationHandler and found the line waiting for the response:

(jboss-ejb-client-2.0.1-Final, org.jboss.ejb.client.EJBInvocationHandler<T>, line 202)

...
final Object value = invocationContext.awaitResponse();
...

Is this a known bug did we miss an error in our code?

Tiburon

p.s. If we append a linefeed "\n" to the string, the invocation works as expected.

p.p.s We also tried jboss-ejb-client.2.1.1.Final with no success.

[jaikiran]

That's interesting. Same issues with "PUT", "POST" etc? What happens with:

ejb.test("GET http://localhost:8080/ HTTP/1.1");

(assuming localhost:8080 is where the server is up and running)

Is this an issue with the just released 9.0.0.CR2 too http://wildfly.org/downloads/?

[tiburon]

Hello Jaikiran, thanks for your quick response.

Yes this happens with all HTTP Commands I tried.

This is what is logged when I try the String you suggested. ( The client is also waiting for a response )

Jun 17, 2015 5:07:37 PM org.jboss.ejb.client.EJBClient <clinit>

INFO: JBoss EJB Client version 2.1.1.Final

GET http://localhost:8080/ HTTP/1.1

Jun 17, 2015 5:07:37 PM org.xnio.Xnio <clinit>

INFO: XNIO version 3.3.0.Final

Jun 17, 2015 5:07:38 PM org.xnio.nio.NioXnio <clinit>

INFO: XNIO NIO Implementation Version 3.3.0.Final

Jun 17, 2015 5:07:38 PM org.jboss.remoting3.EndpointImpl <clinit>

INFO: JBoss Remoting version 4.0.6.Final

Jun 17, 2015 5:07:40 PM org.jboss.ejb.client.remoting.VersionReceiver handleMessage

INFO: EJBCLIENT000017: Received server version 2 and marshalling strategies [river]

Jun 17, 2015 5:07:40 PM org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate

INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@694e1548, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@1c3a4799,channel=jboss.ejb,nodename=node1]} on channel Channel ID 9fbbdb20 (outbound) of Remoting connection 2781e022 to /127.0.0.1:8080

[jaikiran]

I haven't been able to reproduce this against 8.2.0.Final or even the latest 9.0.0.CR2, with the modified version of ejb-remote quickstart quickstart/ejb-remote at master · wildfly/quickstart · GitHub. Can you attach an application which reproduces this?

[tiburon]

Hello,

I verified this issue with a slightly modified version of the ejb-remote quickstart and clean installations of Wildfly 8.2.0 and Wildfly 9.0.0.CR2. I'll attach the project to this post. Could you please try that?

[jaikiran]

I gave your example a try locally, without any changes, against WildFly 8.1.0, 8.2.0 and 9.0.0.CR2 and all passed successfully without issues. Here's the logs:

Jun 18, 2015 4:51:38 PM org.jboss.ejb.client.EJBClient <clinit>

INFO: JBoss EJB Client version 2.0.1.Final

Obtained a remote stateless calculator for invocation

Adding 204 and 340 via the remote stateless calculator deployed on the server

Jun 18, 2015 4:51:38 PM org.xnio.Xnio <clinit>

INFO: XNIO version 3.2.2.Final

Jun 18, 2015 4:51:38 PM org.xnio.nio.NioXnio <clinit>

INFO: XNIO NIO Implementation Version 3.2.2.Final

Jun 18, 2015 4:51:38 PM org.jboss.remoting3.EndpointImpl <clinit>

INFO: JBoss Remoting version 4.0.3.Final

Jun 18, 2015 4:51:38 PM org.jboss.ejb.client.remoting.VersionReceiver handleMessage

INFO: EJBCLIENT000017: Received server version 2 and marshalling strategies [river]

Jun 18, 2015 4:51:38 PM org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate

INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@2ffd3e64, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@1b0b2fd9,channel=jboss.ejb,nodename=me]} on channel Channel ID 88919040 (outbound) of Remoting connection 585309d8 to localhost/127.0.0.1:8080

Remote calculator returned sum = 544

Subtracting 2332 from 3434 via the remote stateless calculator deployed on the server

Remote calculator returned difference = 1102

Obtained a remote stateful counter for invocation

Counter will now be incremented 5 times

Incrementing counter

Count after increment is 1

Incrementing counter

Count after increment is 2

Incrementing counter

Count after increment is 3

Incrementing counter

Count after increment is 4

Incrementing counter

Count after increment is 5

Counter will now be decremented 5 times

Decrementing counter

Count after decrement is 4

Decrementing counter

Count after decrement is 3

Decrementing counter

Count after decrement is 2

Decrementing counter

Count after decrement is 1

Decrementing counter

Count after decrement is 0

4

7

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

Is this reproducible on any machine within your infrastructure or is it just against a specific one? Can you get the thread dumps (2 or 3 of them 5 seconds apart) of the server and the client (both using jstack) when this hangs on your system?

[tiburon]

Hello,

after testing multiple systems which reproduce the issue, we found a machine on which the test does not fail. We are still investigating the cause of the problem, I hope I will be able to report here later what caused the issue.

Again, thanks a lot for your help!

[tiburon]

The problem was caused by our anti virus software, which interferes with the traffic on Port 8080. This was not very likely to us, because other HTTP Requests on 8080 work as expected.

Thank you very much for your support, sorry for blaming wildfly

[jaikiran]

Thanks for posting that information. It's indeed interesting - the way it blocks it and glad to know you narrowed it down.