Too many channels open

[akash_bansal]

Hi All,

While looking up EJB, I am facing following problem:

2013-09-12 04:15:57,210 <> ERROR RemotingConnectionEJBReceiver - Failed to open channel for context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@1e384de, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@395d601f,channel=jboss.ejb,nodename=swan-dev-host1:swan-dev-instance1]}

org.jboss.remoting3.ProtocolException: Too many channels open

    at org.jboss.remoting3.remote.RemoteConnectionHandler.handleOutboundChannelOpen(RemoteConnectionHandler.java:184)

    at org.jboss.remoting3.remote.RemoteConnectionHandler.open(RemoteConnectionHandler.java:314)

    at org.jboss.remoting3.ConnectionImpl.openChannel(ConnectionImpl.java:75)

    at org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection.openChannel(ConnectionPool.java:221)

    at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.associate(RemotingConnectionEJBReceiver.java:136)

    at org.jboss.ejb.client.EJBClientContext.registerEJBReceiver(EJBClientContext.java:375)

    at org.jboss.ejb.client.EJBClientContext.registerEJBReceiver(EJBClientContext.java:327)

    at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:134)

    at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.<init>(ConfigBasedEJBClientContextSelector.java:100)

    at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.<init>(ConfigBasedEJBClientContextSelector.java:68)

    at com.fds.util.EJBContextBuilder.buildEJBContext(EJBContextBuilder.java:66)

    at com.fds.util.EJBContextBuilder.createEJBContext(EJBContextBuilder.java:25)

Although, I am closing up the Context each time after performing operation on EJB. In my application I have lots of EJB call per request and as soon as request count exceeds 40, this exception starts coming. I request to all SMEs to help in this regard.

[jaikiran]

Which exact version of AS7/WildFly is this and what does the client code look like?

[ybxiang.china]

Although, I am closing up the Context each time after performing operation on EJB.

~~~~~~~~~~~Are you crazy? why not cache the EJB broker on client side, reuse it and close it when client exits?

[akash_bansal]

Hi Jaikiran,

We are using JBoss AS 7.2 version. Our client code is something like below:

With following lines of code, we are building Context to lookup EJB:

public Context  getInitialContext() throws NamingException

{

Properties prop = new Properties();

    prop.put(javax.naming.Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

    prop.put(javax.naming.Context.PROVIDER_URL, "remote://localhost:4447");

Context ctx = new InitialContext(prop);

    Properties env = new Properties();

env.put("jboss.naming.client.ejb.context", true);

    env.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");

    env.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");

    env.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

    env.put("remote.connections", "default");

    env.put("remote.connection.default.host", "localhost");

    env.put("remote.connection.default.port", "4447");

    env.put("remote.connection.default.username", "username");

    env.put("remote.connection.default.password", "passsword");

    EJBClientConfiguration clientConfig = new PropertiesBasedEJBClientConfiguration(env);

    ContextSelector<EJBClientContext> contexSelector = new ConfigBasedEJBClientContextSelector(clientConfig);

    EJBClientContext.setSelector(contexSelector);

    return ctx;

}

This function creates initial context and performs operations on EJB:

public void handleRequest() throws Exception{

     try{

     Context ctx = getInitialContext();

     EJB1 ejb1 = ctx.lookup("ejb:myear/mymodule/EJBBean1!com.EJB1");

     ejb1.performOP();

     EJB1 ejb2 = ctx.lookup("ejb:myear/mymodule/EJBBean2!com.EJB2");

     ejb2.performOP();

     ejb2.performDeleteOP();

     ejb1.performUpdateOP();

     }

     finally

     {

     ctx.close();

     }

}

The above function is called multiple times for different User requests and once Request count exceeds 40, Too many channels open problem starts coming. Your early help is really appreciated.

[akash_bansal]

Hi Jaikiran,

I observed one thing if I use ejb protocol while looking up EJB, that Outbound channels were not getting closed when I call close() function of InitialContext. When I sent around 40 requests to my Client, hardly 2 or 3 times Outbound channel got closed which I observed from the log when I enabled TRACE logging.

When I changed my code to use remote protocol rather then using ejb protocol, then for each call of close() function of InitialContext, I observed from the log that Outbound channels were getting closed.

[ybxiang.china]

I hope bellow code is right and useful to you.

public class ServerLink{

    private static final Logger log = Logger.getLogger(ServerLink.class);

    private static final String jndiName = "ejb:nms-server-ear/nms-server-ejb//SecuredRemoteSession!" + ISecuredRemoteSession.class.getName();

    private String serverIP;

    private String username;

    private String cachedHashedPassword;

    private ISecuredRemoteSession securedRemoteSessionProxy;

    private String sessionToken;

    /*****************************************************************

     *****************************************************************/

    private static ServerLink instance;

    private ServerLink(){

        m_propertyChangeListenersRegistry = new HashMap<String, List<PropertyChangeListener>>();

    }

    public static ServerLink getInstance(){

        if(instance==null){

            instance = new ServerLink();           

        }

        return instance;

    }

    /*****************************************************************

     *****************************************************************/   

    public void connectToServer(String serverIP, String username, String password) throws Exception{

        this.username = username;

        this.serverIP = serverIP;

        InitialContext context;

        try{

            Socket clientSocket = new Socket(serverIP,4447);

            clientSocket.close();

        }catch(Exception e){

            log.error("client socket could NOT be connected to server.",e);

            throw ConnectionToServerFailedException.INSTANCE;

        }

        try{           

            Properties p = new Properties();

            p.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "true");

            p.put("remote.connections", "default");

            p.put("remote.connection.default.host", serverIP);

            p.put("remote.connection.default.port", "4447");

            p.put("remote.connection.default.username", username);

            p.put("remote.connection.default.password", password);

            p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

            p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_DISALLOWED_MECHANISMS", "JBOSS-LOCAL-USER");

            p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");

            p.put("remote.connection.default.connect.options.org.xnio.Options.SSL_STARTTLS", "true");

            p.put("remote.connection.default.connect.timeout", "30000");//for xnio

            EJBClientConfiguration cc = new PropertiesBasedEJBClientConfiguration(p);

            ContextSelector<EJBClientContext> selector = new ConfigBasedEJBClientContextSelector(cc);

            EJBClientContext.setSelector(selector);

            EJBClientContext.getCurrent().registerInterceptor(0,new ClientSessionTokenInterceptor());

            EJBClientContext.getCurrent().registerInterceptor(1,new ClientExceptionInterceptor());

            Properties props = new Properties();

            props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

            context = new InitialContext(props);

            securedRemoteSessionProxy = (ISecuredRemoteSession)context.lookup(jndiName);

        }catch(Exception e){

            throw ConnectionToServerFailedException.INSTANCE;

        }

        shakeHands(username, password);

    }

    private void shakeHands(String username, String password) throws Exception{

        this.sessionToken = getSessionWithSSL().shakeHands();

        this.m_loggedIn = true;

        this.cachedHashedPassword = HashUtil.createPasswordHash(username, password);

        //

        fireLogginListeners(LOGGIN_STATES.LOGGEDIN);//loggin

    }

    public void heartBeat(){

        this.getSessionWithSSL().heartBeat();

    }

    /*****************************************************************

     *****************************************************************/

    public List<User> getAllUsers(){

        return this.getSessionWithSSL().getAllUsers();

    }

    ...

}

[wdfink]

Hi Akash,

you use a mix of remote-naming and ejb-client.

The URL "remote://....." is remote naming. If you follow this approach read this, but you have not the full feature stack for EJB's.

If you use the jboss-api (setSelector stuff) you have a programatical approach and you should initialize this only one time and use the IC with the property Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming" only.

Another option is to use the jboss-ejb-client.properties file to configure the connection, see here

I have some examples with different approaches in github

[akash_bansal]

Hi Wolf,

Thanks for your reply.

Even though we are creating One Selector for whole life of Application but I would like to know the Impact of Selector creation for each InitialContext. Will it create multiple channels that will cause "too many channels open" error.

[clichybi]

Hi Akash,

I just faced the same issue and try to solve by setting

p.put("endpoint.name", endPointName);

whereby endPointName is unique per created IC.

I can extend the limit, but get java.lang.OutOfMemoryError: unable to create new native thread while testing 1000 loops.

btw. reusing the IC is much faster - I just require to proceed to compare during performance testing.

[wdfink]

As you have a mix of remote-naming (the URL remote://) and the ejb-client API you may have issues as this is not supposed to work together.

The EJBClientContext.setSelector(...) is a static method and will change the selector for the whole JVM. If you create and set a new Selector each for each IC you may end in such "too many channel" as the replaced selector will not be closed automaticaly, it is done by GC.

You should read the documentation regading the different EJB client approaches and be sure that you don't mix it.

[akash_bansal]

Thanks Wolf for your valuable inputs.