4 Replies Latest reply on Jun 28, 2013 1:29 PM by wdfink

    Jboss 7.2.0 server Hangs for Remote EJB communication

    harry009

      I have an issue with jboss-as-7.2.0.Alpha1-SNAPSHOT version.

       

      The setup have a Java client which communicates with Jboss server through Remote EJB communication. We call Remote EJB endpoint class with class A as a parameter through it. Class A has few instance fields one of which is byte[] byteArray.

       

      When request has large bytes array (20Kb or more ) in byteArray field, Java client hangs while communicating through Remote EJB to JBoss server. On server side, request never reaches to Remote EJB endpoint class and client wait for the response from Jboss server.

      But if request has no byte array in byteArray field, Java client successfully executes the request.

       

      Note : This issue is only replicable with Jboss AS 7.2.0.Alpha1-SNAPSHOT version and not Jboss AS 7 final version.

        • 1. Re: Jboss 7.2.0 server Hangs for Remote EJB communication
          jaikiran

          Can you attach a simple reproducible application (buildable source) please?

          • 2. Re: Jboss 7.2.0 server Hangs for Remote EJB communication
            mevans7

            We had the same problem.  I wrote a test EJB method that simply sent a byte buffer from the client to the server.  Sending less than 24,407 bytes succeeds.  Sending more than 24,409 bytes fails.  We are using JBoss AS 7.1.3 and jboss-remoting-3.2.14.GA.jar at the server.  We were using the jboss-client.jar file from the JBoss AS 7.1.3 bin/client directory.

             

            We did find a solution.  We unjared the jboss-client.jar file and inserted all the classes from the jboss-remoting-3.2.14.GA.jar file.  We re-jared the jboss-client.jar and used it with our client.  After this, we were able to transfer much larger buffers.

             

            Source for our test bean is as follows.

             

            BufferTest.ear:

            META-INF/

            META-INF/MANIFEST.MF

            BufferTest.jar

             

            BufferTest.jar:

            META-INF/MANIFEST.MF

            com/company/buffertest/ejb/BufferTest.class

            com/company/buffertest/ejb/BufferTestBean.class

            com/company/buffertest/ejb/BufferTestLocal.class

             

            BufferTest.java:

            package com.company.buffertest.ejb;

            import java.rmi.RemoteException;

             

            public interface BufferTest {

                public void stringToServer(String str) throws RemoteException;

                public String stringFromServer(int len) throws RemoteException;

                public void bytesToServer(byte [] bytes) throws RemoteException;

                public byte [] bytesFromServer(int len) throws RemoteException;

            }

             

            BufferTestLocal.java:

            package com.company.buffertest.ejb;

            public interface BufferTestLocal {

                public void stringToServer(String str);

                public String stringFromServer(int len);

                public void bytesToServer(byte [] bytes);

                public byte [] bytesFromServer(int len);

            }

             

            BufferTestBean.java:

            /**

            * Test problem sending 24,408 bytes from client to server via Remote EJB.

            */

            package com.company.buffertest.ejb;

             

            import javax.annotation.PostConstruct;

            import javax.annotation.Resource;

            import javax.ejb.Local;

            import javax.ejb.Remote;

            import javax.ejb.SessionContext;

            import javax.ejb.Stateless;

            import javax.ejb.TransactionAttribute;

            import javax.ejb.TransactionAttributeType;

             

            import org.apache.log4j.Logger;

             

             

            /**

            * Test to see if JBoss 7.1 can transfer large blocks of data over the remote interface.

            */

            @Stateless

            @Local ({BufferTestLocal.class})

            @Remote ({BufferTest.class})

            public class BufferTestBean implements BufferTestLocal, BufferTest {

                @Resource SessionContext ctx;

             

                private static final  Logger logger = Logger.getLogger(BufferTestBean.class);

             

                public BufferTestBean() {}

             

                @PostConstruct

                public void initialize() {

                    logger.debug("BufferTestBean initialized");

                }

             

                @TransactionAttribute(TransactionAttributeType.SUPPORTS)

                public String stringFromServer(int len) {

                    // Create a string of the requested length and return it.

                    char c = 'A';

                    StringBuffer sb = new StringBuffer(len);

                    for (int i = 0; i < len; i++) {

                        sb.append((char)(c + (i % 26)));

                    }

                    return sb.toString();

                }

             

                @TransactionAttribute(TransactionAttributeType.SUPPORTS)

                public void stringToServer(String str) {

                    logger.debug("Received a string of length: " + str.length());

                    logger.debug("String is: " + str);

                }

             

                @TransactionAttribute(TransactionAttributeType.SUPPORTS)

                public byte [] bytesFromServer(int len) {

                    // Create a string of the requested length and return it.

                    byte [] bytes = new byte[len];

             

                    for (int i = 0; i < len; i++) {

                        bytes[i] = (byte)(i % 255);

                    }

                    return bytes;

                }

             

                @TransactionAttribute(TransactionAttributeType.SUPPORTS)

                public void bytesToServer(byte [] bytes) {

                    logger.debug("Received a byte buffer of length: " + bytes.length);

                }

            }

             

            Client simply calls:

            byte [] bytes = new byte[25000];

            ejb.bytesToServer(bytes);

             

            With failing libraries, the client stack trace during the hang (using kill -3) is:


            "main" prio=10 tid=0x08746800 nid=0x6975 in Object.wait() [0xf7f0c000..0xf7f0d1f8]

            java.lang.Thread.State: WAITING (on object monitor)

            at java.lang.Object.wait(Native Method)

            - waiting on <0xee151608> (a org.xnio.streams.BufferPipeOutputStream)

            at java.lang.Object.wait(Object.java:485)

            at org.jboss.remoting3.remote.OutboundMessage$1.accept(OutboundMessage.java:99)

            at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:126)

            at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:114)

            at org.xnio.streams.BufferPipeOutputStream.flush(BufferPipeOutputStream.java:143)

            - locked <0xee151608> (a org.xnio.streams.BufferPipeOutputStream)

            at org.xnio.streams.BufferPipeOutputStream.close(BufferPipeOutputStream.java:161)

            - locked <0xee151608> (a org.xnio.streams.BufferPipeOutputStream)

            at org.jboss.remoting3.remote.OutboundMessage.close(OutboundMessage.java:193)

            - locked <0xee151608> (a org.xnio.streams.BufferPipeOutputStream)

            at org.jboss.ejb.client.remoting.ChannelAssociation.releaseChannelMessageOutputStream(ChannelAssociation.java:266)

            at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.processInvocation(RemotingConnectionEJBReceiver.java:203)

            at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:181)

            at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:42)

            at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183)

            at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:125)

            at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183)

            at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136)

            at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121)

            at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104)

            at $Proxy0.bytesToServer(Unknown Source)

            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

            at java.lang.reflect.Method.invoke(Method.java:597)

            at bsh.Reflect.invokeMethod(Unknown Source)

            at bsh.Reflect.invokeObjectMethod(Unknown Source)

            at bsh.Name.invokeMethod(Unknown Source)

            at bsh.BSHMethodInvocation.eval(Unknown Source)

            at bsh.BSHPrimaryExpression.eval(Unknown Source)

            at bsh.BSHPrimaryExpression.eval(Unknown Source)

            at bsh.BSHBlock.evalBlock(Unknown Source)

            at bsh.BSHBlock.eval(Unknown Source)

            at bsh.BSHBlock.eval(Unknown Source)

            at bsh.BSHIfStatement.eval(Unknown Source)

            at bsh.Interpreter.eval(Unknown Source)

            at bsh.Interpreter.source(Unknown Source)

            at bsh.Interpreter.main(Unknown Source)

             

            Stack trace at server (using kill -3) during hang:

             

            "Remoting "hostname -s" task-4" prio=10 tid=0x0a0e8800 nid=0x6d95 in Object.wait() [0xbd476000..0xbd476eb0]

            java.lang.Thread.State: WAITING (on object monitor)

            at java.lang.Object.wait(Native Method)

            - waiting on <0xf21427d0> (a org.xnio.streams.BufferPipeInputStream)

            at java.lang.Object.wait(Object.java:485)

            at org.xnio.streams.BufferPipeInputStream.read(BufferPipeInputStream.java:128)

            - locked <0xf21427d0> (a org.xnio.streams.BufferPipeInputStream)

            at org.jboss.remoting3.remote.InboundMessage$3.read(InboundMessage.java:122)

            - locked <0xf21427d0> (a org.xnio.streams.BufferPipeInputStream)

            at java.io.DataInputStream.readByte(DataInputStream.java:248)

            at org.jboss.as.ejb3.remote.protocol.versionone.AbstractMessageHandler$2.read(AbstractMessageHandler.java:249)

            at java.io.InputStream.read(InputStream.java:163)

            at java.io.FilterInputStream.read(FilterInputStream.java:116)

            at org.jboss.marshalling.SimpleDataInput.readFully(SimpleDataInput.java:150)

            at org.jboss.marshalling.river.RiverUnmarshaller.doReadByteArray(RiverUnmarshaller.java:1453)

            at org.jboss.marshalling.river.RiverUnmarshaller.doReadArray(RiverUnmarshaller.java:1521)

            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:334)

                   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

            at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

            at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.processMessage(MethodInvocationMessageHandler.java:162)

            at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.handleMessage(VersionOneProtocolChannelReceiver.java:180)

            at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:435)

            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

            at java.lang.Thread.run(Thread.java:619)

            • 3. Re: Jboss 7.2.0 server Hangs for Remote EJB communication
              frolovmx

              I had almost the same problem.
              The server was 7.1.3.Final and the client used org.jboss.remoting3:jboss-remoting:3.2.7.GA through dependency management of org.jboss.as:jboss-as-ejb-client-bom:7.1.2.Final.

              The problem occured if the client made 2 EJB calls simultanously, each EJB call in its own thread and the calls carried a big amount of data in both directions.

              I could not detect, which exact amount of data would cause the hang of the client. The range was about 17 - 18 K (length of String). The hang could not be reproduced on every machine.

               

              The solution was to replace org.jboss.remoting3:jboss-remoting:3.2.7.GA by org.jboss.remoting3:jboss-remoting:3.2.16.GA.

              • 4. Re: Jboss 7.2.0 server Hangs for Remote EJB communication
                wdfink

                Issue is solved in EAP6.1 versions and upstream WFLY