Any possibility there is a call failure when you invoke on one of the servers, such that the client proxy is failing over to the other server?
You could then see a pattern where the client sees there are 2 servers, contacts one, the call fails so the client removes the server from the target list, contacts the other server, call succeeds but server sees the client target list is out of sync with the server, so it reestablishes the 2 targets, etc.
Thanks for your response. I forgot to mention some application details. We have two SLSBs, ejb1 and ejb2, in a cluster of two server instances. In my test, I first lookup home objects and create one ejb1 instance and one ejb2 instance. Then an infinite loop is entered where a simple method is called on ejb1 and thenn on ejb2. At the same time, server one is stopped. My custom load balance policy for ejb2 is first called with 2 targets, then with 1, while custom load balance policy for ejb1 is called with 1 correctly.
In a second test, I created only ejb1, and my custom load balance policy is called with the right number of targets after server stop/restart.
Is there a way to avoid the wrong number of targets in custom load balance policy?
So in the first test, while one of the servers is down, ejb2 is cycling back and forth between 2 targets and one target?
ejb2 LBP was called with 2 then 1 and stayed with 1. This was the behavior for one server stop/restart. I have observed in our UAT env that some ejb LBP was cycling back and forth between 1 and 2 after restarting server instances in a round-robin fashion several times. I.e. the number of targets kept alternating between 1 and 2 when both server instances were up.