The fact that the client side of a connection is seeing a 254 byte means that the ServerThread on the other side of the connection has closed its socket and returned itself to the threadpool. For example, it might have experienced a TimeoutException. It follows that creating a new connection is inevitable once the 254 byte shows up.
If the client invoker sees the 254 byte *before* it attempts to use the connection, then it will immediately close the connection and get another connection, which is somewhat faster. The latter optimization may or may not be possible, depending on the timing of the transmission of the 254 byte. I don't think there's much you can do about that.
If the ServerThread is closing the connection because of a SocketTimeoutException, then you could try increasing the timeout vaue.
By the way, this message hasn't been logged at INFO level for quite a while. You must be using a fairly old version of Remoting.
Yes, we have created a private branch with enhanced reliability. At the moment, there is no need to maintain the branch as we don't use NetworkRegistry anymore. So we will migrate to the latest release of remoting.
I just wanted to be sure, the messages do not indicate a serious problem in our code.
Thank you for the advice
I've you've found places where you were about to improve on Remoting's reliability, I'd be interested in hearing about them.
The only area I was needed to work on remoting improvement is NetworkRegistry. Our system is distributed among several DCs and we are unable to use multicast. Moreover, due specifics of our system, channels between DC are quite unreliable. So we tried to use JndiDetector.
However, the JndiDetector registered to many false server failure detections. The problem is described in detail here:
Also, it is hard to say when NetworkRegistry is coherent with overall system state. i.e. NetworkRegistry starts and shows invalid view of available system services until Detector runs for the first time. As our system structure is not very dynamic, I'd prefer to get a locator of service pointing onto non-existing server instead of "no such service" report.
Thanks for the information. I'll take a look at NetworkRegistry and JNDIDetector, although probably not for the next release.
By the way, back when I had time to read novels, I was a big fan of Dan Simmons. I still can't get the ocean planet bathroom out of my mind. :)