6 Replies Latest reply on May 26, 2016 1:22 AM by jaikiran

JBOSS Issue: Failed to acquire a permit within 5 MINUTES

pbatiker May 24, 2016 11:57 AM

Description:
Users were unable to connect to application with an error message “JBAS014516: Failed to acquire a permit WITHIN 5 MINUTES”

We had the following configurations in place.

1. As per the product vendor recommendation, we have increased the max-pool-size from 20 (default) to 100 but still the error had occur.

2. Though we have increased to 100, it might not be sufficient for our user base, to derive this value vendor is suggesting us to monitor the EJB connection pool through JConsole but unfortunately we could not set up this monitor.

3. we have set the JVM memory to 4 GB but the same is not reflecting in task manager.

4. We have modified this parameter as follows

rem # Reduce the RMI GCs to once per hour for Sun JVMs.

set "JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv6Stack=false"

Moreover there additional configurations being suggest by Vendor & Red Hat,

1. Modify module.xml file to define a dependency that allows the JBOSS connector module to use JMX Mbeans - Vednor

2. EJBs can use different pool configurations to allow finer control of the total number of each EJB type, however to find the suitable value for your environment, we would recommend testing. The default EJB pool can be disabled and EJBs will not use a pool, instead a new EJB instance will be created when a given thread needs to invoke a method on an EJB. - Red Hat

Questions:

1. How to set up the JConsole to monitor a JBoss's (running as services instead of as a process via standalone.bat ) EJB connection pool? OR Is there any other free tools to monitor?

2. What are the parameters to be monitored in EJB connection pool?

3. Will it have an importance to reflect the JVM memory size in the task manager as 4 GB? If so, where to look in?

4. As per the documentation, JBoss JVM to prefer IPv4 but we have not enabled it. What will be the impact?

5. Any idea on the Vendor recommendation on module.xml file configuration changes?

6. As per the Red Hat, if we remove the bean-instance-pool-ref from the ejb3 subsystem, what will be the impact?

I hope, i have made myself clear if not please let me know if you have any other queries.

1. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

jaikiran May 25, 2016 1:43 AM (in response to pbatiker)

Please post the entire exception stacktrace, the exact version of JBoss AS you use and the pool configurations for your bean and what type of a bean it is.
Actions
2. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

pbatiker May 25, 2016 3:54 AM (in response to jaikiran)
Thank you for the replying back.

Please find the entire exceptions in sequence
12:14:45,806 ERROR [com.teamcenter.presentation.gateway.login] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-42) 172.31.26.200.43257.Anonymous.00001 - 2016/05/19-10:14:45,806 UTC - NLEIDM912 - Exception: accessing the value of a parameter from request com.teamcenter.presentation.gateway.login : com.teamcenter.presentation.wae.RequestManagerException: multiple members in array on String conversion
12:18:01,603 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-6) JBAS014134: EJB Invocation failed on component Gateway for method public abstract com.teamcenter.jeti.util.ResponsePayload com.teamcenter.jeti.ejb.Gateway.invokeMethod(java.lang.String,java.lang.String,com.teamcenter.jeti.util.RequestPayload,com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
12:18:01,666 ERROR [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) 2016/05/19-10:18:01,666 UTC - NLEIDM912 - Exception during invoke call Invocation failed; nested exception is: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
12:18:01,728 WARN [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) 2016/05/19-10:18:01,728 UTC - NLEIDM912 - RestResponseHandler: : ClientAbortException: java.net.SocketException: Connection reset by peer: socket write error

JBOSS AS Version: 7.1.0

EJB Configurations: As i am not from the JBOSS competency, here i am attaching the portion of the standalone.xml file

           <session-bean>
                <stateless>
                    <bean-instance-pool-ref pool-name="slsb-strict-max-pool"/>
                </stateless>
                <stateful default-access-timeout="5000" cache-ref="simple"/>
                <singleton default-access-timeout="5000"/>
            </session-bean>
            <mdb>
                <resource-adapter-ref resource-adapter-name="hornetq-ra"/>
                <bean-instance-pool-ref pool-name="mdb-strict-max-pool"/>
            </mdb>
            <pools>
                <bean-instance-pools>
                    <strict-max-pool name="slsb-strict-max-pool" max-pool-size="100" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
                    <strict-max-pool name="mdb-strict-max-pool" max-pool-size="100" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
                </bean-instance-pools>
            </pools>

I hope i have give clear information to you.
Actions
3. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

jaikiran May 25, 2016 4:11 AM (in response to pbatiker)

12:18:01,666 ERROR [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) 2016/05/19-10:18:01,666 UTC - NLEIDM912 - Exception during invoke call Invocation failed; nested exception is: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES

I am more interested in seeing the complete stacktrace of that exception, instead of just the error message.
Actions
4. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

pbatiker May 25, 2016 4:40 AM (in response to jaikiran)

Please find the stacktrace

12:14:45,806 ERROR [com.teamcenter.presentation.gateway.login] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-42) 172.31.26.200.43257.Anonymous.00001 - 2016/05/19-10:14:45,806 UTC - NLEIDM912 - Exception: accessing the value of a parameter from request com.teamcenter.presentation.gateway.login : com.teamcenter.presentation.wae.RequestManagerException: multiple members in array on String conversion
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.convertObjectToString(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.getRequestAttributeString(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.gateway.login.TcLoginManager.login(Unknown Source) [tcgateway.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.isLoggedIn(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source) [tcwae.jar:]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source) [tcgateway.jar:]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.ResponseTimeFilter.doFilter(Unknown Source) [tcgateway.jar:]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.10.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154) [jboss-as-web-7.1.0.Final.jar:7.1.0.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.10.Final.jar:]
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.10.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.10.Final.jar:]
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]

12:18:01,603 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-6) JBAS014134: EJB Invocation failed on component Gateway for method public abstract com.teamcenter.jeti.util.ResponsePayload com.teamcenter.jeti.ejb.Gateway.invokeMethod(java.lang.String,java.lang.String,com.teamcenter.jeti.util.RequestPayload,com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]

12:18:01,619 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) JBAS014134: EJB Invocation failed on component Gateway for method public abstract com.teamcenter.jeti.util.ResponsePayload com.teamcenter.jeti.ejb.Gateway.invokeMethod(java.lang.String,java.lang.String,com.teamcenter.jeti.util.RequestPayload,com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]

12:18:01,603 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-49) JBAS014134: EJB Invocation failed on component Gateway for method public abstract com.teamcenter.jeti.util.ResponsePayload com.teamcenter.jeti.ejb.Gateway.invokeMethod(java.lang.String,java.lang.String,com.teamcenter.jeti.util.RequestPayload,com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]

12:18:01,666 ERROR [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) 2016/05/19-10:18:01,666 UTC - NLEIDM912 - Exception during invoke call Invocation failed; nested exception is:
javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
JBAS014516: Failed to acquire a permit within 5 MINUTES com.teamcenter.soa.ptier : java.rmi.RemoteException: Invocation failed; nested exception is:
javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source) [tcwae.jar:]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source) [tcgateway.jar:]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.10.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154) [jboss-as-web-7.1.0.Final.jar:7.1.0.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.10.Final.jar:]
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.10.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.10.Final.jar:]
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]
Caused by: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
... 49 more
Caused by: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
... 63 more

12:18:01,634 ERROR [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-6) 2016/05/19-10:18:01,634 UTC - NLEIDM912 - Exception during invoke call Invocation failed; nested exception is:
javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
JBAS014516: Failed to acquire a permit within 5 MINUTES com.teamcenter.soa.ptier : java.rmi.RemoteException: Invocation failed; nested exception is:
javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source) [tcwae.jar:]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source) [tcgateway.jar:]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.10.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154) [jboss-as-web-7.1.0.Final.jar:7.1.0.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.10.Final.jar:]
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.10.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.10.Final.jar:]
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]
Caused by: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
... 49 more
Caused by: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
... 63 more

12:18:01,634 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-10) JBAS014134: EJB Invocation failed on component Gateway for method public abstract com.teamcenter.jeti.util.ResponsePayload com.teamcenter.jeti.ejb.Gateway.invokeMethod(java.lang.String,java.lang.String,com.teamcenter.jeti.util.RequestPayload,com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.invokeMethod(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]

12:18:01,728 WARN [com.teamcenter.soa.ptier] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-111) 2016/05/19-10:18:01,728 UTC - NLEIDM912 - RestResponseHandler: : ClientAbortException: java.net.SocketException: Connection reset by peer: socket write error
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:352) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:316) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:98) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.CompressionResponseStream.close(Unknown Source) [tcgateway.jar:]
at com.teamcenter.soa.ptier.RestResponseHandler.generateResponse(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseInterceptor.interceptRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source) [tcwae.jar:]
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source) [tcwae.jar:]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source) [tcgateway.jar:]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source) [TcSoaWeb.jar:10000.1.0, October 23, 2015]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.10.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154) [jboss-as-web-7.1.0.Final.jar:7.1.0.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.10.Final.jar:]
at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.10.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.10.Final.jar:]
at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_75]
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method) [rt.jar:1.7.0_75]
at java.net.SocketOutputStream.socketWrite(Unknown Source) [rt.jar:1.7.0_75]
at java.net.SocketOutputStream.write(Unknown Source) [rt.jar:1.7.0_75]
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:724) [jbossweb-7.0.10.Final.jar:]
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:449) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:299) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:1027) [jbossweb-7.0.10.Final.jar:]
at org.apache.coyote.Response.action(Response.java:190) [jbossweb-7.0.10.Final.jar:]
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:347) [jbossweb-7.0.10.Final.jar:]
... 32 more

12:18:01,728 ERROR [org.jboss.ejb3.invocation] (http--0%3A0%3A0%3A0%3A0%3A0%3A0%3A0-80-182) JBAS014134: EJB Invocation failed on component Gateway for method public abstract boolean com.teamcenter.jeti.ejb.Gateway.hasAssignedServer(com.teamcenter.jeti.util.ESessionContextInterface,java.lang.String) throws javax.resource.ResourceException,javax.naming.NamingException,com.teamcenter.jeti.util.JetiException,java.rmi.RemoteException: javax.ejb.EJBException: JBAS014516: Failed to acquire a permit within 5 MINUTES
at org.jboss.as.ejb3.pool.strictmax.StrictMaxPool.get(StrictMaxPool.java:113) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:47) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropogatingInterceptor.processInvocation(EJBRemoteTransactionPropogatingInterceptor.java:80) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ejb3.component.interceptors.EjbExceptionTransformingInterceptorFactories$1.processInvocation(EjbExceptionTransformingInterceptorFactories.java:65) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:179) [jboss-as-ejb3-7.1.0.Final.jar:7.1.0.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:173) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:43) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:92) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:175) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.2.Final.jar:1.0.2.Final]
at com.sun.proxy.$Proxy11.hasAssignedServer(Unknown Source)
at com.teamcenter.soa.ptier.GatewayEjbAction.handleAction(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionComponentInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.CallActionChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.node.ResponseChainInterceptor.interceptRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.request.RequestContainerImpl.traverseRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.RequestDispatcher.processRequestContainer(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.context.ControllerContextImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.injector.RequestInjectorImpl.processRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.handleRequest(Unknown Source)
at com.teamcenter.presentation.wae.internal.ctrl.Controller.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.presentation.gateway.filters.CompressionFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.teamcenter.soa.ptier.ResponseTimeFilter.doFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:154)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
Actions
5. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

pbatiker May 25, 2016 5:45 PM (in response to pbatiker)

Did you get chance to look into the issue?
Actions
6. Re: JBOSS Issue: Failed to acquire a permit within 5 MINUTES

jaikiran May 26, 2016 1:22 AM (in response to pbatiker)

So you have a pool of 100 stateless beans and yet when a request comes in to the SLSB, the request can't get hold of a bean instance even after waiting for 5 minutes. So clearly, the 100 beans are doing a operation which is taking longer than 5 minutes to complete. So unless you figure out what they are upto, there's no point trying to increase the bean instance pool size etc... Check the code of the stateless bean in question and see what it does. Also take thread dumps when this is a happening to see what the beans are doing. That will tell you what to fix and where.
Actions

Go to original post