[jbertram]

I would need to see the full stack trace of the error to comment further on that particular error.

[murthy516]

Thanks Justin,But I dont have Thread Dump.And My Stack Trace is

5:23:14,399 ERROR [treetlogs] (JBossWeb-threads - 330) Could not create a session: IJ000453: Unable to get managed connection for java:/JmsXA : javax.jms.JMSException: Could not create a session: IJ000453: Unable to get managed connection for java:/JmsXA

  at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:881)

  at org.hornetq.ra.HornetQRASessionFactoryImpl.createQueueSession(HornetQRASessionFactoryImpl.java:237)

  at com.enh.OrderPublisher.publishOrder(OrderPublisher.java:36)

  at com.par.PartyServicesImpl.completeDetails(MainImpl.java:11656)

  at sun.reflect.GeneratedMethodAccessor2736.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:601)

  at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)

  at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

  at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:226)

  at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:302)

  at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:188)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.security.AuthorizationInterceptor.processInvocation(AuthorizationInterceptor.java:122)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:76)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:42)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:32)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)

  at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)

  at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

  at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165)

  at org.jboss.as.webservices.invocation.AbstractInvocationHandlerEJB.invoke(AbstractInvocationHandlerEJB.java:112)

  at org.jboss.wsf.stack.cxf.JBossWSInvoker._invokeInternal(JBossWSInvoker.java:182)

  at org.jboss.wsf.stack.cxf.JBossWSInvoker.invoke(JBossWSInvoker.java:127)

  at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)

  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

  at java.util.concurrent.FutureTask.run(FutureTask.java:166)

  at org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)

  at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:107)

  at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)

  at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)

  at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)

  at org.jboss.wsf.stack.cxf.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:91)

  at org.jboss.wsf.stack.cxf.transport.ServletHelper.callRequestHandler(ServletHelper.java:169)

  at org.jboss.wsf.stack.cxf.CXFServletExt.invoke(CXFServletExt.java:87)

  at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)

  at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:108)

  at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)

  at org.jboss.wsf.stack.cxf.CXFServletExt.service(CXFServletExt.java:135)

  at org.jboss.wsf.spi.deployment.WSFServlet.service(WSFServlet.java:140)

  at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)

  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)

  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)

  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)

  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)

  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489)

  at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)

  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)

  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)

  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:372)

  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)

  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:679)

  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:519)

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

  at java.lang.Thread.run(Thread.java:722)

  at org.jboss.threads.JBossThread.run(JBossThread.java:122)

Is this is because of max-pool-size configuration in hornetq-server adapter which is in jms-connection-factories..How can I resolve this..Please help me out.

Thanks

kris

[jbertram]

To be clear, I wasn't asking for a thread dump.

I was expecting to see a "Caused by" at the bottom of the stack-trace.  I believe that should be logged by HornetQ.  Do you see anything like this in your log?

Perhaps it would be best if you simply attached the entire log where you're seeing this problem.

[murthy516]

Extension to the above log:

15:38:30,615 ERROR [org.hornetq.ra] (Thread-1283) HQ154002: Could not create session: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/JmsXA

  at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390)

  at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)

  at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)

  at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:832)

  at org.hornetq.ra.HornetQRASessionFactoryImpl.createSession(HornetQRASessionFactoryImpl.java:465)

  at com.hornetq.producers.SampleThread.run(SampleThread.java:35) [HornetQClientInServer.jar:]

Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])

  at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:383)

  at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397)

  at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365)

  at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329)

  ... 5 more

This is how my pubisher is behaving..

Help me out pls.

[jbertram]

Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])

This indicates the connection pool for the JmsXA <pooled-connection-factory> is exhausted.  I recommend you increase it with <max-pool-size>.

[murthy516]

Thank you Justin..Setting max-pool-size parameter in JBOSS solved out the problem..But,what can be the best parameter value to tune this max-pool-size..If I have more than 25 Queues running.

Thanks a Ton..Helped me out to get rid of this issue.

[jbertram]

It doesn't really matter how many queues you have running.  The real question is how many threads might concurrently call createConnection() on the JmsXA connection factory.

If, for example, you had MDBs consuming from each of those 25 queues, and all those MDBs were configured with the default "maxSession" (i.e. 15), and all of those MDBs also used the JmsXA connection factory then you would potentially have 375 ( i.e. 15 * 25) threads concurrently invoking createConnection().  Of course, it's highly unlikely that all 375 threads would create a connection concurrently but it is theoretically possible.  Therefore your safest max-pool-size would be 375.

[vatima2013]

Your recommendation to increase "max-pool-size" parameter for the JmsXA <pooled-connection-factory> "hornetq-ra" is absolutely correct (see How to increase the connection pool size for JmsXA in JBoss EAP 6 - Red Hat Customer Portal for details)

But your example calculation for the "max-pool-size" parameter is misleading:

1) there is no explicit correlation between above parameter and the number of MDBs: the number of MDBs must provide minimum throughput to process incoming messages some time (assuming default address full policy PAGE is used)

2) there is "max-pool-size" parameter in the "mdb-strict-max-pool" configuration. This value "overrides" "maxSession" value if it is less than the "maxSession" value. And changing this value doesn't solve the problem described in this thread

[jbertram]

there is no explicit correlation between above parameter and the number of MDBs: the number of MDBs must provide minimum throughput to process incoming messages some time (assuming default address full policy PAGE is used)

I don't understand what you're saying here.  Can you elaborate?

there is "max-pool-size" parameter in the "mdb-strict-max-pool" configuration. This value "overrides" "maxSession" value if it is less than the "maxSession" value. And changing this value doesn't solve the problem described in this thread

The "max-pool-size" parameter in the "mdb-strict-max-pool" configuration does not override the "maxSession" activation configuration property set on the MDB.  Remember, there are 2 pools involved here.  There is 1) an MDB instance pool controlled by the container and 2) a session pool controlled by the HornetQ JCA RA.

[vatima2013]

Justin Bertram schrieb:

there is no explicit correlation between above parameter and the number of MDBs: the number of MDBs must provide minimum throughput to process incoming messages some time (assuming default address full policy PAGE is used)

I don't understand what you're saying here.  Can you elaborate?

For example if message consumers are very slow and too expensive (e.g. using database connections and executing slow queries) and the number of messages sent is distributed extremely uneven in the time, then I'd tend to set max-pool-size in the hornetq-ra resource adapter big enough to allow almost "unlimited" sending of messages by clients. Full address policy PAGE would ensure writing of messages to the file system as long as there is enough space. As soon as the extreme sending load is gone, even a small number of MDB instances can process all messages in the queue sometime. Please correct me if HornetQ cannot not work this way.

there is "max-pool-size" parameter in the "mdb-strict-max-pool" configuration. This value "overrides" "maxSession" value if it is less than the "maxSession" value. And changing this value doesn't solve the problem described in this thread

The "max-pool-size" parameter in the "mdb-strict-max-pool" configuration does not override the "maxSession" activation configuration property set on the MDB.  Remember, there are 2 pools involved here.  There is 1) an MDB instance pool controlled by the container and 2) a session pool controlled by the HornetQ JCA RA.

Thank you for explaining the difference. I have misinterpreted this RH article: https://access.redhat.com/solutions/1197733

Should be "maxSession" kept lower (or equal) than the value of the "max-pool-size" parameter in the "mdb-strict-max-pool" to avoid "JBAS014516: Failed to acquire a permit within..." errors?

If not, what scenario could lead to JBAS014516 errors in case of MDBs?

I'm asking because we are going to throttle messages consumption using the parameters "maxSession" (throttle individually per MDB) and "max-pool-size" (throttle globally). We hope to prevent running out of database connections this way.

Thank you in advance,

Valerij

[jbertram]

For example if message consumers are very slow and too expensive (e.g. using database connections and executing slow queries) and the number of messages sent is distributed extremely uneven in the time, then I'd tend to set max-pool-size in the hornetq-ra resource adapter big enough to allow almost "unlimited" sending of messages by clients. Full address policy PAGE would ensure writing of messages to the file system as long as there is enough space. As soon as the extreme sending load is gone, even a small number of MDB instances can process all messages in the queue sometime. Please correct me if HornetQ cannot not work this way.

Yes, HornetQ can work that way, but that's not really related to the use-case I outlined in my previous comment.  I think you missed the fact that my calculation for max-pool-size for the pooled-connection-factory was based on the fact that the MDB itself would be the one using the connection factory.  That doesn't appear to be true in your case.

Should be "maxSession" kept lower (or equal) than the value of the "max-pool-size" parameter in the "mdb-strict-max-pool" to avoid "JBAS014516: Failed to acquire a permit within..." errors?

Yes.  Here's how it works...The HornetQ JCA RA maintains a pool of session for each MDB deployment.  The size of this session pool is determined by the "maxSession" activation configuration property.  When one of these sessions receives a message it essentially asks the container for an instance of the MDB so it can invoke the MDB's onMessage method and deliver the message.  The container maintains a pool of MDB instances.  By default, the size of this instance pool is determined by the "max-pool-size" of the "mdb-strict-max-pool".  If a session receives a message but is unable to acquire an instance of the MDB then you'll get an exception with a message like "JBAS014516".

One last point...

Previously I said that this scenario had 2 pools involved, but actually there are 3 pools involved in total:

• A pool of sessions for receiving inbound messages maintained by HornetQ JCA RA.  The size of this session pool is determined by the "maxSession" activation configuration property on the MDB.
• A pool of MDB instances maintained by the container.  By default, the size of this instance pool is determined by the "max-pool-size" of the "mdb-strict-max-pool".
• A pool of connections for sending outbound messages (i.e. the pooled-connection-factory) maintained by the HornetQ JCA RA.  The size of this connection pool is determined by the "max-pool-size" set on the pooled-connection-factory in use.

All of these should be tuned appropriately to avoid exceptions and/or resource starvation.