Remote EJB invocations between two independent servers (programmatic authentication)

[ecimon]

I'm trying to invoke a remote EJB from another AS7 instance as described here (all nodes are running on EAP-6.1.0.Alpha). Is it possible to pass a username/password explicity during such call? (and reauthenticate later on a regular basis?)

standalone-full.xml:

<subsystem xmlns="urn:jboss:domain:remoting:1.1">
   <connector name="remoting-connector" socket-binding="remoting" security-realm="ApplicationRealm"/>
   <outbound-connections>
     <!-- Works with: username="myuser" and security-realm="MyEJBClientRealm" -->
     <remote-outbound-connection name="innerejb-outbound-connection" outbound-socket-binding-ref="innerejb-outbound-socket">
       <properties>
         <property name="SSL_ENABLED" value="false"/>
         <property name="SASL_POLICY_NOANONYMOUS" value="false"/>
       </properties>
     </remote-outbound-connection>
   </outbound-connections>
</subsystem>

jboss-ejb-client.xml:

<jboss-ejb-client xmlns="urn:jboss:ejb-client:1.0">
        <client-context>
                <ejb-receivers exclude-local-receiver="true">
                        <remoting-ejb-receiver outbound-connection-ref="innerejb-outbound-connection" />
                </ejb-receivers>
        </client-context>
</jboss-ejb-client>

Relevant test case (EJB client deployed on EAP-6.1.0.Alpha):

@MessageDriven(name = "TestMDB", activationConfig = { 
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "java:/jms/TestQueue"),
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"), 
        @ActivationConfigProperty(propertyName = "transactionTimeout", propertyValue = "500") 
})
public class TestMDB implements MessageListener {
        
    @Resource
    MessageDrivenContext messageDrivenContext;
        
    @Override
    public void onMessage(Message msg) {        
        logger.info("On message: principal={}", messageDrivenContext.getCallerPrincipal()); //anonymous      
        
        InitialContext ctx = null;
        try {            
            final Properties properties = new Properties();        
            //properties.put(Context.SECURITY_PRINCIPAL, username);
            //properties.put(Context.SECURITY_CREDENTIALS, password);

            //This seems to be ignored.
            properties.put("remote.connections", "default");
            properties.put("remote.connection.default.username", username);
            properties.put("remote.connection.default.password", password);
        
            properties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

            ctx = new InitialContext(properties);            

            InternalService internalService = (InternalService) ctx.lookup("ejb:/test/InternalServiceBean!com.example.service.InternalService");
            internalService.internalBusinessMethod();            
        } catch (Exception ex) {
           logger.error("Remote invocation failed!", ex);
        } finally {
            if (ctx != null) {
                try {
                    ctx.close();
                    //https://issues.jboss.org/browse/EJBCLIENT-85 
                    //https://issues.jboss.org/browse/EJBCLIENT-84
                } catch (NamingException ex) {}
            }
        }
    }   
}

Output:

11:18:15,923 INFO  [com.example.service.mdb.TestMDB] (Thread-8 (HornetQ-client-global-threads-1786467816)) On message: principal=anonymous
11:18:15,928 INFO  [org.jboss.as.naming] (Remoting "narnia" task-1) JBAS011806: Channel end notification received, closing channel Channel ID 746eeb79 (inbound) of Remoting connection 265cd23d to /127.0.0.1:35769
11:18:15,929 ERROR [org.jboss.remoting.remote.connection] (Remoting "narnia" read-1) JBREM000200: Remote connection failed: javax.security.sasl.SaslException: Cannot get userid/password [Caused by javax.security.auth.callback.UnsupportedCallbackException]
11:18:15,930 ERROR [com.example.service.mdb.TestMDB] (Thread-8 (HornetQ-client-global-threads-1786467816)) Remote invocation failed!: java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:test, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@22bfab1f
    at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:729) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:253) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:198) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:181) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:144) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]
    at com.sun.proxy.$Proxy118.internalBusinessMethod(Unknown Source)
    at com.example.service.mdb.TestMDB.onMessage(TestMDB.java:57) [testComponent-1.0.jar:]

Any input on this would be greatly appreciated.

Thanks,

Simon

[jaikiran]

Starting EAP 6.1 and WildFly 8, you have the option of scoped EJB client contexts. Take a look at https://docs.jboss.org/author/display/WFLY8/Scoped+EJB+client+contexts

[ecimon]

Thanks for the tip Jaikiran. Do you have a working example for this kind of scenario? I tried the following, but it doesn't work for me as expected.

1) I reverted 'org.jboss.ejb-client' to 1.0.16 (I tried with 1.0.23 at first). Also, I made it dependent on 'org.jboss.xnio' (see warning message below).

2) Adjusted test case:

    @Override

    public void onMessage(Message msg) {       

        logger.info("On message: principal={}", messageDrivenContext.getCallerPrincipal()); //anonymous     

        final Properties props = new Properties();

        props.put("remote.connections", "default");

        props.put("remote.connection.default.username", "myuser");

        props.put("remote.connection.default.password", "secret");

        props.put("remote.connection.default.host", "localhost");

        props.put("remote.connection.default.port", "4447");

        props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

        props.put("org.jboss.ejb.client.scoped.context","true");

        props.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");

        props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");              

        Context ejbRootNamingContext = null;

        try {

            ejbRootNamingContext = (Context) new InitialContext(props).lookup("ejb:");

            InternalService internalService = (InternalService) ejbRootNamingContext.lookup("test/InternalServiceBean!com.example.service.InternalService");

            internalService.internalBusinessMethod();

        } catch (Exception ex) {

           logger.error("Remote invocation failed!", ex);

        } finally {

            try {

                if (ejbRootNamingContext != null)

                    ejbRootNamingContext.close();

            } catch (Throwable t) {

                logger.warn("Unable to close context: {}", t.getMessage());

            }

        }

   }

3) Output:

16:21:12,662 INFO  [com.example.service.mdb.TestMDB] (Thread-12 (HornetQ-client-global-threads-120019578)) On message: principal=anonymous

16:21:12,662 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) endpoint.create.options. has the following options {}

16:21:12,663 WARN  [org.xnio.option.parse] (Thread-12 (HornetQ-client-global-threads-120019578)) Invalid option 'org.xnio.Options.SSL_ENABLED' in property 'remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED': java.lang.IllegalArgumentException: Class 'org.xnio.Options' not found

16:21:12,663 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) remote.connectionprovider.create.options. has the following options {}

16:21:12,663 WARN  [org.xnio.option.parse] (Thread-12 (HornetQ-client-global-threads-120019578)) Invalid option 'org.xnio.Options.SASL_POLICY_NOANONYMOUS' in property 'remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS': java.lang.IllegalArgumentException: Class 'org.xnio.Options' not found

16:21:12,663 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) remote.connection.default.connect.options. has the following options {}

16:21:12,663 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) remote.connection.default.channel.options. has the following options {}

16:21:12,663 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) Connection org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration$RemotingConnectionConfigurationImpl@1d88aa34 successfully created for connection named default

16:21:12,663 DEBUG [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration] (Thread-12 (HornetQ-client-global-threads-120019578)) No clusters configured in properties

16:21:12,672 ERROR [org.jboss.remoting.remote.connection] (Remoting "narnia" read-1) JBREM000200: Remote connection failed: java.io.IOException: Received an invalid message length of 369295616

16:21:12,672 INFO  [org.jboss.as.naming] (Remoting "narnia" task-2) JBAS011806: Channel end notification received, closing channel Channel ID 44f6b1cf (inbound) of Remoting connection 2968751b to /127.0.0.1:38230

16:21:12,689 ERROR [org.jboss.remoting.remote.connection] (Remoting "config-based-ejb-client-endpoint" read-1) JBREM000200: Remote connection failed: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

16:21:12,690 WARN  [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector] (Thread-12 (HornetQ-client-global-threads-120019578)) Could not register a EJB receiver for connection to localhost:4447: java.lang.RuntimeException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

     at org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:91) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

     at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:148) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

     at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.<init>(ConfigBasedEJBClientContextSelector.java:105) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

...

    at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:146) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

    ... 80 more

16:21:12,698 DEBUG [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector] (Thread-12 (HornetQ-client-global-threads-120019578)) Registered a reconnect handler in EJB client context org.jboss.ejb.client.EJBClientContext@646c6be6 for remote://localhost:4447

16:21:12,698 DEBUG [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector] (Thread-12 (HornetQ-client-global-threads-120019578)) Registered 0 remoting EJB receivers for EJB client context org.jboss.ejb.client.EJBClientContext@646c6be6

16:21:12,723 ERROR [com.example.service.mdb.TestMDB] (Thread-12 (HornetQ-client-global-threads-120019578)) Remote invocation failed!: java.lang.ClassCastException: org.jboss.ejb.client.naming.ejb.EjbNamingContext cannot be cast to com.example.service.InternalService

          at com.example.service.mdb.TestMDB.onMessage(TestMDB.java:58) [testComponent-1.0.jar:]

          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25]

          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25]

          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25]

          at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25]

          at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]

          at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]

          at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

Follow-up question: are jboss-ejb-client.xml and outbound-connections (standalone-full.xml) redundant in this case? (assuming everything is scoped)

[jaikiran]

Szymon Biliński wrote:

Follow-up question: are jboss-ejb-client.xml and outbound-connections (standalone-full.xml) redundant in this case? (assuming everything is scoped)

Yes, you won't need those for scoped EJB client contexts.

Szymon Biliński wrote:

Thanks for the tip Jaikiran. Do you have a working example for this kind of scenario? I tried the following, but it doesn't work for me as expected.

Here https://github.com/wildfly/wildfly/blob/7.2.0.Final/testsuite/integration/multinode/src/test/java/org/jboss/as/test/multinode/remotecall/scoped/context/DynamicJNDIContextEJBInvocationTestCase.java

[jaikiran]

ejbRootNamingContext = (Context) new InitialContext(props).lookup("ejb:");

            InternalService internalService = (InternalService) ejbRootNamingContext.lookup("test/InternalServiceBean!com.example.service.InternalService");

That JNDI name looks incorrect. It should be app-name/module-name/distinct-name/bean-name!bean-interface. So if "test" is your module-name (assuming you haven't packaged in a .ear) then that code should look like:

ejbRootNamingContext = (Context) new InitialContext(props).lookup("ejb:");

InternalService internalService = (InternalService) ejbRootNamingContext.lookup("/test//InternalServiceBean!com.example.service.InternalService");

[ecimon]

1) I was more concered about those SSL/XNIO warnings/errors, or is it expected? :-)

2) You're absolutely right about the JNDI name. My "test" component is deployed as a standalone JAR:

java:global/test/InternalServiceBean!com.example.service.InternalService

java:app/test/InternalServiceBean!com.example.service.InternalService

java:module/InternalServiceBean!com.example.service.InternalService

java:jboss/exported/test/InternalServiceBean!com.example.service.InternalService

java:global/test/InternalServiceBean

java:app/test/InternalServiceBean

java:module/InternalServiceBean

I've adjusted the JNDI:

ejbRootNamingContext = (Context) new InitialContext(props).lookup("ejb:");

//InternalService internalService = (InternalService) ejbRootNamingContext.lookup("test/InternalServiceBean!com.example.service.InternalService");

InternalService internalService = (InternalService) ejbRootNamingContext.lookup("/test//InternalServiceBean!com.example.service.InternalService");

internalService.internalBusinessMethod();

... but that leaves me with:

17:21:15,810 ERROR [com.example.service.mdb.TestMDB] (Thread-24 (HornetQ-client-global-threads-120019578)) Remote invocation failed!: java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:test, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@1ddddc95

          at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:693) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

          at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116) [jboss-ejb-client-1.0.16.Final.jar:1.0.16.Final]

(the final stacktrace is different; SSL/XNIO errors look exactly the same)

Any more tips? Thank you so much for your time by the way - I'm quite stuck on this one.

[jaikiran]

Are you sure you added the MANIFEST.MF (or even a jboss-deployment-structure.xml) containing the xnio dependency to the top level deployment of the "client" (the app which contains the MDB). What does it look like?

[ecimon]

jaikiran pai wrote:

Are you sure you added the MANIFEST.MF (or even a jboss-deployment-structure.xml) containing the xnio dependency to the top level deployment of the "client" (the app which contains the MDB). What does it look like?

No, I didn't - I assumed XNIO objects should be instantiated by the ejb-client library (see my previous comment). Adding "org.jboss.xnio" to MANIFEST.MF global-modules in standalone-full.xml fixed the problem for now (local dependency gives ClassCastExceptions in some cases). My current config, if anyone needs this:

final Properties props = new Properties();

props.put("remote.connections", "default");

props.put("remote.connection.default.username", "myuser");

props.put("remote.connection.default.password", "secret");

props.put("remote.connection.default.host", "localhost");

props.put("remote.connection.default.port", "4447");

//Implies "org.jboss.xnio" module dependency

props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");

props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_DISALLOWED_MECHANISMS", "JBOSS-LOCAL-USER");

props.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");

//https://docs.jboss.org/author/display/WFLY8/Scoped+EJB+client+contexts

props.put("org.jboss.ejb.client.scoped.context","true");

props.put("javax.security.sasl.policy.noplaintext", "false");

props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

However, I'm still facing a small issue with context closing (same MDB test-case). If I leave an intentional leak, than everything "works" as expected:

Context ejbRootNamingContext = null;

try {

   ejbRootNamingContext = (Context) new InitialContext(props).lookup("ejb:");

   InternalService internalService = (InternalService) ejbRootNamingContext.lookup("/test//InternalServiceBean!com.example.service.InternalService");

   internalService.internalBusinessMethod();

} catch (Exception ex) {

   logger.error("Remote invocation failed!", ex);

} finally {

   logger.info("Closing EJB context...");

   try {

         // Leaving an intentional leak "fixes" the problem.

         //if (ejbRootNamingContext != null)

         //    ejbRootNamingContext.close();

   } catch (Throwable t) {

         logger.warn("Unable to close context: {}", t.getMessage());

   }

   logger.info("Context closed.");

}

Otherwise, I get this:

12:55:55,416 INFO  [com.example.service.InternalServiceBean] (EJB default - 2) !! INTERNAL BUSINESS METHOD !!

12:55:55,432 INFO  [com.example.service.mdb.TestMDB] (Thread-1 (HornetQ-client-global-threads-1988690638)) Closing EJB context...

12:55:55,433 DEBUG [org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver] (Thread-1 (HornetQ-client-global-threads-1988690638)) Closing channelChannel ID e3ba16f7 (outbound) of Remoting connection 018d8cbf to localhost/127.0.0.1:4447

12:55:55,433 INFO  [org.jboss.ejb.client.remoting] (Remoting "config-based-ejb-client-endpoint" task-4) EJBCLIENT000016: Channel Channel ID e3ba16f7 (outbound) of Remoting connection 018d8cbf to localhost/127.0.0.1:4447 can no longer process messages

12:55:55,438 DEBUG [org.jboss.ejb.client.remoting.ChannelAssociation] (Thread-1 (HornetQ-client-global-threads-1988690638)) Closing channel Channel ID e3ba16f7 (outbound) of Remoting connection 018d8cbf to localhost/127.0.0.1:4447

12:55:55,438 DEBUG [org.jboss.ejb.client.remoting.ChannelAssociation] (Thread-1 (HornetQ-client-global-threads-1988690638)) Registering a re-connect handler org.jboss.ejb.client.remoting.EJBClientContextConnectionReconnectHandler@2c234cf8 for broken channel Channel ID e3ba16f7 (outbound) of Remoting connection 018d8cbf to localhost/127.0.0.1:4447 in EJB client context org.jboss.ejb.client.EJBClientContext@2edbc947

12:55:55,450 INFO  [com.example.service.mdb.TestMDB] (Thread-1 (HornetQ-client-global-threads-1988690638)) Context closed.

12:55:55,462 WARN  [com.arjuna.ats.jta] (Thread-1 (HornetQ-client-global-threads-1988690638)) ARJUNA016041: prepare on < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000101:-544235e9:52134a3e:5b, node_name=1, branch_uid=0:ffff7f000101:-544235e9:52134a3e:60, subordinatenodename=null, eis_name=unknown eis name > (ResourceImpl{transactionKey=0:ffff7f000101:-544235e9:52134a3e:5b, ejbClientContext=org.jboss.ejb.client.EJBClientContext@2edbc947, nodeName='narnia', state=null}) failed with exception -: java.lang.IllegalStateException: EJBCLIENT000027: No EJBReceiver available for node name narnia

          at org.jboss.ejb.client.EJBClientContext.requireNodeEJBReceiver(EJBClientContext.java:824) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at org.jboss.ejb.client.EJBClientContext.requireNodeEJBReceiverContext(EJBClientContext.java:865) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at org.jboss.ejb.client.EJBClientManagedTransactionContext$ResourceImpl.prepare(EJBClientManagedTransactionContext.java:230) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelPrepare(XAResourceRecord.java:213)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2516)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2483)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.prepare(BasicAction.java:2060)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1482)

          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:98)

          at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)

          at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)

          at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)

          at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:72) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25]

          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25]

          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25]

          at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25]

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at com.sun.proxy.$Proxy105.afterDelivery(Unknown Source)

          at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:322)

          at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1016)

          at org.hornetq.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:52)

          at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1161)

          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:106)

          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]

          at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

12:55:55,471 WARN  [com.arjuna.ats.arjuna] (Thread-1 (HornetQ-client-global-threads-1988690638)) ARJUNA012073: BasicAction.End() - prepare phase of action-id 0:ffff7f000101:-544235e9:52134a3e:5b failed.

12:55:55,471 WARN  [com.arjuna.ats.arjuna] (Thread-1 (HornetQ-client-global-threads-1988690638)) ARJUNA012075: Action Aborting

12:55:55,487 WARN  [com.arjuna.ats.jta] (Thread-1 (HornetQ-client-global-threads-1988690638)) ARJUNA016045: attempted rollback of < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000101:-544235e9:52134a3e:5b, node_name=1, branch_uid=0:ffff7f000101:-544235e9:52134a3e:60, subordinatenodename=null, eis_name=unknown eis name > (ResourceImpl{transactionKey=0:ffff7f000101:-544235e9:52134a3e:5b, ejbClientContext=org.jboss.ejb.client.EJBClientContext@2edbc947, nodeName='narnia', state=null}) failed with exception code -: java.lang.IllegalStateException: EJBCLIENT000027: No EJBReceiver available for node name narnia

          at org.jboss.ejb.client.EJBClientContext.requireNodeEJBReceiver(EJBClientContext.java:824) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at org.jboss.ejb.client.EJBClientContext.requireNodeEJBReceiverContext(EJBClientContext.java:865) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at org.jboss.ejb.client.EJBClientManagedTransactionContext$ResourceImpl.rollback(EJBClientManagedTransactionContext.java:251) [jboss-ejb-client-1.0.23.Final.jar:1.0.23.Final]

          at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:345)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2854)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2833)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Abort(BasicAction.java:1918)

          at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)

          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:98)

          at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)

          at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)

          at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)

          at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:72) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25]

          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25]

          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25]

          at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25]

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73) [jboss-as-ejb3-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]

          at com.sun.proxy.$Proxy105.afterDelivery(Unknown Source)

          at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:322)

          at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1016)

          at org.hornetq.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:52)

          at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1161)

          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:106)

          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]

          at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

12:55:55,491 WARN  [com.arjuna.ats.arjuna] (Thread-1 (HornetQ-client-global-threads-1988690638)) ARJUNA012091: Top-level abort of action 0:ffff7f000101:-544235e9:52134a3e:5b received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord

12:55:55,492 WARN  [org.hornetq.ra] (Thread-1 (HornetQ-client-global-threads-1988690638)) HQ152007: Unable to call after delivery: javax.resource.spi.LocalTransactionException: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:88)

          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25]

          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25]

          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25]

          at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25]

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)

          at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)

          at com.sun.proxy.$Proxy105.afterDelivery(Unknown Source)

          at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:322) [hornetq-ra-2.3.0.CR1.jar:]

          at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1016) [hornetq-core-client-2.3.0.CR1.jar:]

          at org.hornetq.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:52) [hornetq-core-client-2.3.0.CR1.jar:]

          at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1161) [hornetq-core-client-2.3.0.CR1.jar:]

          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:106) [hornetq-core-client-2.3.0.CR1.jar:]

          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]

          at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.

          at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177) [jbossjts-jacorb-4.17.3.Final.jar:4.17.3.Final (revision: 74343b48951c0fdab92316e56bfcaed605d620f6)]

          at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.3.Final.jar:4.17.3.Final (revision: 74343b48951c0fdab92316e56bfcaed605d620f6)]

          at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)

          at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:72)

          ... 16 more

12:55:55,588 INFO  [com.example.service.mdb.TestMDB] (Thread-4 (HornetQ-client-global-threads-1988690638)) On message: principal=anonymous

* No external datasource is involved.

* Setting durable=false on TestQueue doesn't change anything.

Any clues on this?

[ecimon]

I made a minimal project, that replicates this situation on WildFly-8.0.0.Alpha4 using jboss-ejb-client-2.0.0.Beta3:

https://github.com/sbilinski/jboss-ejb-client-sample

I would appreciate, if you or someone else could take a look on this.

Thanks.

[jaikiran]

Thanks, I'll take a look.