11 Replies Latest reply on Dec 1, 2005 10:54 PM by Max Cooper

    restricting EJB client to one cluster

    Max Cooper Newbie

      I have two jboss 4.0.1sp1 clusters in the same subnet running the same application. However, I have a problem where an EJB client from "cluster B" will automatically start making calls to "cluster A". It should not do that -- the client is intended to connect to cluster B only. The client uses the HA-JNDI from cluster B to do lookups; it should never make calls to cluster A.

      I have followed the instructions in the FAQ for having two clusters in the same subnet (http://wiki.jboss.org/wiki/Wiki.jsp?page=TwoClustersSameNetwork), including:
      - unique partition names (cluster A is "DefaultPartition", cluster B is different)
      - unique multicast IPs in cluster-service.xml for both the PartitionConfig and for the HA-JNDI AutoDiscoveryAddress (this wasn't covered in FAQ, but seems like it should be unique?)

      I can see in the jboss logging output that my clusters are separate -- each one has the right number and identification of it's members. The clusters are definitely separate, but it seems like the stub the client gets back from the HA-JNDI lookup will somehow go and find the other cluster on it's own. I have observed with a debugger that my client ends up with a reference to a node in the wrong cluster (by looking at the IP in the ref variable).

      My server configs started by copying "all" and haven't been changed much. And I am using JBoss AS 4.0.1sp1.

      What can I do to prevent my clients from calling EJBs on the wrong cluster?

      Thanks,
      -Max

        • 1. Re: restricting EJB client to one cluster
          Brian Stansberry Master

          What does your jndi.properties file on the client look like?

          • 2. Re: restricting EJB client to one cluster
            Max Cooper Newbie

            The client has the following JNDI properties:
            java.naming.factory.url.pkgs=jboss.naming:org.jnp.interfaces
            java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
            java.naming.provider.url=the_only_node_in_cluster_B:1100

            The cluster I want to connect to (B) has only one node, so the client does lookups using the HA-JNDI connected to port 1100 on that node.

            I know that I could probably just use port 1099 and forget about the clustering, but it is easier just to manage one kind of configuration for the servers (clustered), hence the one-node cluster.

            Also, we have more than one production cluster (and I think that some are in the same subnet). I need to get this working properly.

            Thanks,
            -Max

            • 3. Re: restricting EJB client to one cluster
              Brian Stansberry Master

              I suspect your client was failing to connect to the_only_node_in_cluster_B:1100 and was thus falling back on AutoDiscovery. By default, that uses port 1102, so it was finding the cluster A HA-JNDI.

              Try this:

              java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
              java.naming.provider.url=jnp://the_only_node_in_cluster_B:1100
              java.naming.factory.url.pkgs=org.jnp.interfaces
              jnp.discoveryPort=port_you_configured_for_discovery_on_cluster_B

              • 4. Re: restricting EJB client to one cluster
                Max Cooper Newbie

                Success!

                I ended up using the multicast IP rather than the port to differentiate the clusters. The "cluster B" HA-JNDI config in cluster-service.xml looks like this (note non-default multicast IP):

                <!-- Multicast Address and Group used for auto-discovery -->
                231.0.0.4
                1102

                ... and then I used these JNDI properties for the client:

                java.naming.factory.url.pkgs=jboss.naming:org.jnp.interfaces
                java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
                java.naming.provider.url=jnp://the_only_node_in_cluster_B:1100
                jnp.discoveryGroup=231.0.0.4

                I re-ran my test where I shutdown cluster B, and the client did NOT go find cluster A, which is precisely the behavior I wanted.

                I haven't tried this yet, but I think that I may set jnp.disableAutodiscovery=true on the client side. The clients are always configured with the complete list of cluster nodes (java.naming.provider.url=node1:1100,node2:1100,node3:1100). Disabling autodiscovery should be easier to manage since I won't need to put a unique IP or port in the client config (in addition to the list of nodes) for each cluster.

                Thank you very much for your help, bstansberry. I really appreciate it.

                -Max

                • 5. Re: restricting EJB client to one cluster
                  Max Cooper Newbie

                  Hmm... the forum ate my XML, trying again...

                  &lt;attribute name="AutoDiscoveryAddress"&gt;231.0.0.4&lt;/attribute&gt;
                  &lt;attribute name="AutoDiscoveryGroup"&gt;1102&lt;/attribute&gt;

                  -Max

                  • 6. Re: restricting EJB client to one cluster
                    Max Cooper Newbie

                    Bah... well you get the idea. :)

                    -Max

                    • 7. Re: restricting EJB client to one cluster
                      Brian Stansberry Master

                      Yeah, our forum loves that tasty XML ;-)

                      I'm getting good at guessing tags from their contents.

                      Glad it's working. Disabling discovery makes sense if you have a consistent static list of servers. You can disable it on the server side as well by setting the "DiscoveryDisabled" attribute to true in the HA-JNDI config. Just above where you configure the address/port. Leaves you with one less open port.

                      • 8. Re: restricting EJB client to one cluster
                        Brian Stansberry Master

                        One other point, in case you wanted to leave auto discovery on and not mess with ensuring the discovery addresses and ports are different. Including the following in jndi.properties will ensure it only connects to the desired partition:

                        jnp.partitionName=YourPartitionName

                        I originally suggested configuring the port on the client side as you'd already configured it on the server side.

                        • 9. Re: restricting EJB client to one cluster
                          Max Cooper Newbie

                          Brian,

                          I set jnp.disableAutodiscovery=true on the client and that worked for me.

                          I would now also like to disable HA-JNDI auto-discovery on the app server, but I'm having trouble doing that.

                          I started by specifying jnp.disableAutodiscovery=true in the properties file that my app's JNDI utility class uses, and also made sure to list out all the nodes in the java.naming.provider.url property value (was just localhost:1100). No problems here.

                          Next, I tried to disable auto-discovery in cluster-service.xml, but ran into trouble. The version of jboss that I am using does not support the "DiscoveryDisabled" attribute. The comments in DetachedHANamingServiceMBean.java and the code in DetachedHANamingService.java indicate that auto-discovery can be disabled by setting the AutoDiscoveryAddress to null. However, I can't figure out how to do that. So far, I've tried:
                          1. don't set it -- but it just uses the default
                          2. -- but setAutoDiscoveryAddress never gets called, so it uses the default
                          3. -- putting some spaces in there resulted in a call to setAutoDiscoveryAddress, but it complained that " " wasn't a multicast address
                          4. ${null} -- I was hoping ${null} would somehow evaluate to null, but it evaluated to the literal string "${null}" :(, so I just got an error that it couldn't find the address "${null}"

                          So, is there some syntax I can use in the xml file to get JBoss to call setAutoDiscoveryAddress() with a null argument?

                          Thanks,
                          -Max

                          • 10. Re: restricting EJB client to one cluster
                            Max Cooper Newbie

                            Ugh, more XML eating -- imagine the following:

                            2. I set the attribute with an empty value (no spaces).
                            3. I set the attribute with a few spaces as the value.
                            4. I set the attribute with ${null} as the value

                            -Max

                            • 11. Re: restricting EJB client to one cluster
                              Max Cooper Newbie

                              I also tried using an empty attribute element (one tag, rather than separate start and stop tags) for AutoDiscoveryAddress and setAutoDiscoveryAddress() did not get called, much like #2.

                              -Max