Receiving IllegalStateExceptions in client| JBoss.org Content Archive (Read Only)

15. Re: Receiving IllegalStateExceptions in client

timfox Feb 16, 2007 7:08 AM (in response to bander)

Also, the remoting client pool size by default is 50 concurrent connections, and these will timeout after a while (this is what you are seeing).

If your mySQL instance is slow, then you may have a lot of invocations backing up (remember you have 500 sessions - this is a lot) waiting to get one of those 50 from the pool, since they're taking significant time to execute.

You could try increasing the remoting client pool size to a value higher than 50.

In remoting-service.xml for the socket connector add an attribute and try a higher value.

<attribute name="clientMaxPoolSize">100</attribute>

I will also update the code so it clears up better after a failure so as not to give you the knock on IllegalStateException.

16. Re: Receiving IllegalStateExceptions in client

timfox Feb 16, 2007 10:40 AM (in response to bander)

JBoss remoting will try a certain number of times (default 30) with a pause of 1 second if the pool is full to wait for a connection to be returned.

This means by default invocations will wait 30 seconds for a connection to be returned before throwing an exception.

You can increase this period by setting the attribute "numberOfRetries" on the connector config.

This basically equates to the number of seconds to wait before barfing. You could set this to a high value (few minutes) in your case.

17. Re: Receiving IllegalStateExceptions in client

bander Feb 18, 2007 8:37 PM (in response to bander)

Ok - thanks. I'll have a play with these settings and report back.

18. Re: Receiving IllegalStateExceptions in client

bander Feb 18, 2007 10:35 PM (in response to bander)

The modifications I'm making to remoting-service.xml don't appear to be picked up by the client

I'm still seeing exceptions like:

ERROR 2007-02-19 14:23:24,687 [Manager Queue Six.1.1] org.jboss.jms.client.container.ExceptionInterceptor: Caught RuntimeException
org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://172.26.1.240:4457/?clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat]
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:510)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
at org.jboss.remoting.Client.invoke(Client.java:1414)
at org.jboss.remoting.Client.invoke(Client.java:511)
at org.jboss.jms.client.delegate.DelegateSupport.invoke(DelegateSupport.java:111)
at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledge(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:215)
at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect6.invoke(SessionAspect6.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at org.jboss.jms.client.remoting.MessageCallbackHandler.postDeliver(MessageCallbackHandler.java:162)
at org.jboss.jms.client.remoting.MessageCallbackHandler.callOnMessage(MessageCallbackHandler.java:137)
at org.jboss.jms.client.remoting.MessageCallbackHandler$ListenerRunner.run(MessageCallbackHandler.java:722)
at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(Unknown Source)
at java.lang.Thread.run(Thread.java:534)
Caused by: java.net.SocketException: Can not obtain client socket connection from pool. Have waited 30 seconds for available connection (50 in use)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:776)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:506)
... 29 more

It should not be waiting for only 30 seconds (i've set numberOfRetries to 600 i.e. 10 minutes) and there should be more than 50 connections in use (i've set clientMaxPoolSize to 100).

My remoting-service.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>

<!--
 Messaging Remoting service deployment descriptor.

 $Id: remoting-service.xml 1576 2006-11-17 04:22:42Z ron_sigal $
 -->

<server>

 <mbean code="org.jboss.remoting.transport.Connector"
 name="jboss.messaging:service=Connector,transport=socket"
 display-name="Socket transport Connector">
 <attribute name="Configuration">
 <config>
 <invoker transport="socket">
 <attribute name="marshaller" isParam="true">org.jboss.jms.server.remoting.JMSWireFormat</attribute>
 <attribute name="unmarshaller" isParam="true">org.jboss.jms.server.remoting.JMSWireFormat</attribute>
 <attribute name="serializationtype" isParam="true">jboss</attribute>
 <attribute name="dataType" isParam="true">jms</attribute>
 <attribute name="socket.check_connection" isParam="true">false</attribute>
 <attribute name="timeout">1800000</attribute>
 <attribute name="serverBindAddress">${jboss.bind.address}</attribute>
 <attribute name="serverBindPort">4457</attribute>
 <attribute name="leasePeriod">20000</attribute>
 <attribute name="clientSocketClass" isParam="true">org.jboss.jms.client.remoting.ClientSocketWrapper</attribute>
 <attribute name="serverSocketClass">org.jboss.jms.server.remoting.ServerSocketWrapper</attribute>
 <attribute name="numberOfRetries">600</attribute>
 <attribute name="clientMaxPoolSize">100</attribute>
 </invoker>
 <handlers>
 <handler subsystem="JMS">org.jboss.jms.server.remoting.JMSServerInvocationHandler</handler>
 </handlers>
 </config>
 </attribute>
 <depends>jboss.messaging:service=NetworkRegistry</depends>
 </mbean>
</server>

Is the jboss client aware of the server settings?

19. Re: Receiving IllegalStateExceptions in client

timfox Feb 19, 2007 5:56 AM (in response to bander)

Strange. The settings should certainly be picked up.

Are you using ServiceBindingManager? (I.e. are you mapping ports, ports01, ports02 etc?)

If so you need to make sure the SBM is using the exact same config as remoting-service.xml.

20. Re: Receiving IllegalStateExceptions in client

timfox Feb 19, 2007 12:06 PM (in response to bander)

I've just been informed by the remoting team that the attributes need "isParam=true" to ensure they're passed to the client side.

21. Re: Receiving IllegalStateExceptions in client

bander Feb 19, 2007 5:01 PM (in response to bander)

Thanks Tim - I found that bit of info in the JBoss 2.0.0 Remoting Guide (page 17) and got my test running without exceptions. I'll have to do some more testing before I'm convinced exceptions are being handled correctly by the client i.e. when things start to go wrong, they really go wrong (I got messages being delivered to the wrong queues again yesterday)

Could I suggest that the remoting-service.xml that ships with JBoss Messaging be updated to include the client side attributes? Even if they're just set to the defaults or commented out, it would be handy to see what can be configured.

22. Re: Receiving IllegalStateExceptions in client

craig2007 Sep 28, 2007 7:22 AM (in response to bander)

"timfox" wrote:
I've just been informed by the remoting team that the attributes need "isParam=true" to ensure they're passed to the client side.

Hi Everybody ...

We also got the same exception .We are using JBoss 1.0.1.GA (Prod Environment -not possible to upgrade to latest JBM).

There are around 30 machines which are communicating with Central machine thru JBoss Messaging. Every machine is working correctly ,but after sometime all machines hangs up and the exception i got is given below.....

Exception Cause --> org.jboss.jms.client.container.ExceptionInterceptor(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N6840474864420126404(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.container.ClientLogInterceptor(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N6840474864420126404(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.JBossConnectionFactory(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
Exception Cause --> org.jboss.jms.client.JBossConnectionFactory(javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat])
#####################################################################################################
Messaging Code Related Exceptions
Exception Occured
Exception Cause --> javax.jms.IllegalStateException: Can not get connection to server. Problem establishing socket connection for locator - InvokerLocator [socket://10.0.34.12:4457/?dataType=jms&marshaller=org.jboss.jms.server.remoting.JMSWireFormat&serializationtype=jboss&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat]
File Name : ConnectionForSender.java
Class Name : com.tcs.retail.framework.messaging.lookup.ConnectionForSender
Method Name : getConnection
Line No : 32

#####################################################################################################

To overcome this exception , i will change the setting for numberofretries and clientmaxpoolsize. In addtion to that there are some config details available in the specified quote attached with this message.

Tim , can you kindly tell me where this configuration changes needs to be done (file name and correct place)