4 Replies Latest reply on May 13, 2014 11:40 AM by Nicolas Nicolas

    Problems in remote connection when returned packages are dropped

    Nicolas Nicolas Newbie

      Hello all,

       

      I just migrated JBoss servers from AS 5 to EAP 6 (6.2.1) and I detected, with my network administrator, some troubles in remote connections.

       

      There are three servers : one server (serverA) can call remote EJBs on the two others (serverB and serverC).

      ServerB and serverC contain the same ear with the same remote EJBs but, for the moment, I can't use cluster.

       

      I described all the remote connections in the serverA standalone.xml and jboss-ejb-client.xml files as described below :

       

      standalone.xml:

      <remote-outbound-connection name="remote-ejb-connectionB" outbound-socket-binding-ref="outbound-ejb-connectionB" username="me" security-realm="my-realm">

        <properties>

        <property name="SASL_POLICY_NOANONYMOUS" value="false" />

        <property name="SSL_ENABLED" value="false" />

        </properties>

      </remote-outbound-connection>

       

      <remote-outbound-connection name="remote-ejb-connectionC" outbound-socket-binding-ref="outbound-ejb-connectionC" username="me" security-realm="my-realm">

        <properties>

        <property name="SASL_POLICY_NOANONYMOUS" value="false" />

        <property name="SSL_ENABLED" value="false" />

        </properties>

      </remote-outbound-connection>

       

      jboss-ejb-client.xml:

      <ejb-receivers>

        <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connectionB" connect-timeout="3000" />

        <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connectionC" connect-timeout="3000"/>

      </ejb-receivers>

       

      When serverB and serverC are up : No problem.

      When serverB is up and serverC is down : No problem.

       

      But if one of this 2 servers is in an "intermediary state" (for example if the network administrator manually drops the packages sent by serverC to serverA or if he closes connections from serverA to serverC) : the application does not worked correctly!

      Indeed, the remote calls from serverA to serverC are blocked.

       

      I assume serverA tries to connect serverC to make the eligibleNode list but I would like to know if :

      - it's possible to configure serverA to limit the "checking time" to many milli-seconds,

      - if it's possible to only check serverC state if serverB is down or not gets the remote EJB called,

      - other solution?

       

      Optional question, if I use a cluster for serverB and serverC, and serverC is in an "intermediary state", the cluster will still works or not?

       

      Thanks in advance.

        • 1. Re: Problems in remote connection when returned packages are dropped
          jaikiran pai Master

          Can you post a thread dump showing what exactly the call is blocked on?

          • 2. Re: Problems in remote connection when returned packages are dropped
            Nicolas Nicolas Newbie

            Hello Jaikiran,

             

            As you can see in the logs below, when the package are dropped, serverA is blocked for a minute then it restarts all the calls.

             

            11:47:48 serverA: INFO  (AJP-38) - START

            11:47:48 serverA: INFO  (AJP-38) - Requesting [MyRemoteBean#getFirstEJB] on remote server

            11:47:48 serverA: INFO  (AJP-38) - [MyRemoteBean#getFirstEJB] of EJB ejb:XXXX/XXX/XX!foo.bar.MyRemoteBean succeeded in 12 ms

            11:47:48 serverA: INFO  (AJP-38) - Requesting [MyRemoteBean#getSecondEJB] on remote server

            11:48:50 serverA: INFO  (AJP-39) - START

            11:48:50 serverA: INFO  (AJP-39) - Requesting [MyRemoteBean#getFirstEJB] on remote server

            11:48:50 serverA: INFO  (AJP-39) - [MyRemoteBean#getFirstEJB] of EJB ejb:XXXX/XXX/XX!foo.bar.MyRemoteBean succeeded in 11 ms

            11:48:50 serverA: INFO  (AJP-39) - Requesting [MyRemoteBean#getSecondEJB] on remote server

            11:48:50 serverA: INFO  (AJP-39) - [MyRemoteBean#getSecondEJB] of EJB ejb:XXXX/XXX/XX!foo.bar.MyRemoteBean succeeded in 55 ms

            11:48:50 serverA: INFO  (AJP-39) - END

             

            We follow these steps :

             

            - Start serverA on a machine in standalone mode

            - Start serverC on another machine in standalone mode

            - Launch the test : it runs successfully in less than 5 seconds.

            - Dropped the package from serverC to serverA (sudo iptables -I FORWARD -p tcp --sport XXXX --dport 4447 -j DROP)

            - Launch the test : it runs in more than 1 minute!

             

            I presume serverA JBoss tries to use a blocked connection, has no response from serverC and has a timeout (1 minute?). Then it retries with another connection.

            If it's right, can I manage the timeout? I looked for the invocation-timeout property but, by default in 5 seconds and no 1 minute it isn't?

             

            We are going to continue test, by reject connection for instance.

            • 3. Re: Problems in remote connection when returned packages are dropped
              jaikiran pai Master

              Can you post the relevant code? Also please do take a thread dump when it hangs/waits for 1 minute ThreadDump. That will eliminate any guess work and will likely pinpoint what is taking that long to timeout. I don't remember there being a 1 minutes timeout somewhere.

              • 4. Re: Problems in remote connection when returned packages are dropped
                Nicolas Nicolas Newbie

                I am sorry but the jmx-console I have on the tests environment does not contains the module which allows to do the ThreadDump and we have only a JREinstalled, not a JDK...

                I'm going to see how I can do it.

                 

                But, the first need I have is to know if it's possible to parametrize serverA in the way to avoid an automatic ping of all the others servers when I do a remote call.

                Indeed, in my case the serverC site has only to be used if serverB does not work correctly. I know the best solution is to use a cluster but for the moment I can't...