Still no success with this.
The wiki indicates that in jboss 4.x, that replication is handled via explicit RPC calls. I can only assume that the replication is not happening correctly, but I don't know when to start in attempting to determine what is at fault, any thoughts?
Are the queues deployed on both servers? A list call on HAJNDI will return references bound in HAJNDI directly, or, if nothing is bound in HAJNDI under the given name, the objects bound in the local JNDI tree on the server that handles the call. Because of this, if the queues are not bound in the global tree and are not deployed on both nodes, you will get inconsistent listings.
The queues are configured on both nodes. If one node goes down, the other will pick them up and deploy the queues. I can only see the queues in the jmx-console on one node at a time.
Is there a way to see what is bound to HAJNDI without doing the failover to local jndi? I'd really just like to confirm that the queues are getting bound there, at least then I think it would be clear that i have a configuration issue.
The stack trace indicates that the Global lookup fails, then the local one fails. (that is of course, *when* it fails, i can't tell where it was found when it is found).
You should be able to invoke list() or listBindings() to see what's stored in HA-JNDI. These calls don't access the local JNDI tree.
One observation I have after working with HA_JNDI in JBoss 5 Alpha is that even though you specify a provider url such as "jnp://nodeA:1100", the "smart stub" is distributing your lookups based upon a designated load balancing policy. In the case of HA-JNDI, the load balancing policy is currently hard-coded to round robin. So your lookup directed towards nodeA:1100 could be handled by nodeB:1100. This should only matter if the object is located in HA-JNDI and replication fails or if you're bound different objects into local JNDI.
I haven't tested this in JBoss 4 so I'm not sure how it's handled there. If you turn logging up and check the log, you can see which servers are handling your lookups and you should be able to determine whether this affects your results.