0 Replies Latest reply on Jan 27, 2010 4:00 PM by Rick Feldmann

    JBoss Clustering / JAAS

    Rick Feldmann Newbie
      Everyone,

      I have the following configuration and have been unable to have the  logons work properly.  Once properly logged in, I can terminate app2 on  either server and it will bounce to the other server.  It will even  bounce during normal execution between the 2 servers.  What am I missing?

      Jboss - 4.0.5
      Jboss cache - 1.4.0

      app1 is using a custom written sso module and is configured to have it's  own session cache that is shared with other servers running app1.
      app2 is configured to have it's on session cache that is shared with  other servers running app2.

      both app1 and app2 have a shared credential cache.

      our shared credential cache configuration file looks like and is the  same for both applications on all servers in the deploy directory:

      <?xml version="1.0" encoding="UTF-8" ?>
      <server>
      <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar" />

      <!--   ====================================================================  -->
      <!--  Defines TreeCache configuration for the Credential  Cache.             -->
      <!--   ====================================================================  -->


      <mbean code="org.jboss.cache.TreeCache"  name="jboss.cache:service=COTreeCache">
         <depends>jboss:service=Naming</depends>
         <depends>jboss:service=TransactionManager</depends>


         <!-- Configure the TransactionManager -->
         <!-- org.jboss.cache.DummyTransactionManagerLookup -->
         <attribute  name="TransactionManagerLookupClass">org.jboss.cache.BatchModeTransactionManagerLookup</attribute>

         <!--
           Node locking scheme :
           PESSIMISTIC (default)
           OPTIMISTIC
         -->
         <attribute name="NodeLockingScheme">PESSIMISTIC</attribute>        
         <!--
           Node locking isolation level :
           SERIALIZABLE
           REPEATABLE_READ (default)
           READ_COMMITTED
           READ_UNCOMMITTED
           NONE
          (ignored if NodeLockingScheme is OPTIMISTIC)
          -->
         <attribute name="IsolationLevel">REPEATABLE_READ</attribute>

         <!--          Valid modes are LOCAL
           REPL_ASYNC
           REPL_SYNC
           INVALIDATION_ASYNC
           INVALIDATION_SYNC
         -->
         <attribute name="CacheMode">REPL_SYNC</attribute>
            <!--       Whether each interceptor should have an mbean
           registered to capture and display its statistics.       <attribute name="UseInterceptorMbeans">true</attribute>
         -->

         <!--
           Name of cluster. Needs to be the same for all clusters,
           in order to find each other
         -->
         <attribute  name="ClusterName">Credential-${jboss.partition.name:DefaultPartition}</attribute>

         <attribute name="ClusterConfig">
           <config>

               <UDP mcast_addr="${jboss.partition.udpGroup:224.0.0.10}"  mcast_port="45568"
                    ip_ttl="${jgroups.mcast.ip_ttl:8}" ip_mcast="true"
                    mcast_recv_buf_size="2000000" mcast_send_buf_size="640000"
                    ucast_recv_buf_size="2000000" ucast_send_buf_size="640000"
                    enable_bundling="false"
                    loopback="false" />
                 <PING timeout="2000" num_initial_members="4" />
                 <MERGE2 min_interval="10000" max_interval="20000" />
                 <FD_SOCK  />
                 <FD shun="true" timeout="10000" max_tries="5"/>
                 <VERIFY_SUSPECT timeout="3000" num_msgs="3" />
                 <pbcast.NAKACK gc_lag="50"  retransmit_timeout="300,600,1200,2400,4800"
                    max_xmit_size="8192" />
                 <UNICAST timeout="300,600,1200,2400,4800" window_size="100"  min_threshold="10" />
                 <pbcast.STABLE desired_avg_gossip="20000" max_bytes="400000" />
                 <FRAG frag_size="8192" />
                 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
                    shun="true" print_local_addr="true"/>
                 <pbcast.STATE_TRANSFER />


           </config>
         </attribute>

         <!--         The max amount of time (in milliseconds) we wait until the
           initial state (ie. the contents of the cache) are retrieved from
           existing members in a clustered environment
         -->
         <attribute name="InitialStateRetrievalTimeout">5000</attribute>

         <!--         Number of milliseconds to wait until all responses for a
           synchronous call have been received.
         -->
         <attribute name="SyncReplTimeout">10000</attribute>

         <!--  Max number of milliseconds to wait for a lock acquisition -->
         <attribute name="LockAcquisitionTimeout">15000</attribute>

      </mbean>
      </server>     


      the jboss-service.xml file in tc5-cluster-sar/META-INF directory for  app1 is:

      <?xml version="1.0" encoding="UTF-8"?>

      <!--  ===================================================================== -->
      <!--                                                                        -->
      <!--  Customized TreeCache Service Configuration for Tomcat 5  Clustering   -->
      <!--                                                                        -->
      <!--  ===================================================================== -->

      <server>

         <!--  ==================================================================== -->
         <!-- Defines TreeCache  configuration                                      -->
         <!--  ==================================================================== -->

          <!-- Note we are using TreeCacheAop -->
         <mbean code="org.jboss.cache.aop.TreeCacheAop"
             name="jboss.cache:service=TomcatClusteringCache">

             <depends>jboss:service=Naming</depends>
             <depends>jboss:service=TransactionManager</depends>
             <!-- We need the AspectDeployer to deploy our FIELD granularity  aspects -->
             <depends>jboss.aop:service=AspectDeployer</depends>

             <!-- Name of cluster. Needs to be the same for all nodes in the
                  cluster, in order to find each other
             -->
             <attribute  name="ClusterName">APP1-Session-${jboss.partition.name:Cluster}</attribute>
                    <!--
                 Isolation level : SERIALIZABLE
                                   REPEATABLE_READ (default)
                                   READ_COMMITTED
                                   READ_UNCOMMITTED
                                   NONE
             -->
             <attribute name="IsolationLevel">REPEATABLE_READ</attribute>

             <!-- Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC
                              If you use REPL_SYNC and a UDP-based ClusterConfig
                  we recommend you comment out the FC (flow control)
                  protocol in the ClusterConfig section below.
             -->
             <attribute name="CacheMode">REPL_SYNC</attribute>

             <!--
               Indicates whether to the cache should unmarshall objects  replicated
               from other cluster nodes, or store them internally as a byte[]
               until a web app requests them.  Must be "true" if session  replication
               granularity "FIELD" is used in any webapp, otherwise "false" is
               recommended.
             -->
             <attribute name="UseRegionBasedMarshalling">false</attribute>
                    <!--  Whether or not the entire tree is inactive upon startup, only
               responding to replication messages after activateRegion() is
               called to activate one or more parts of the tree when a webapp is
               deployed.  Must have the same value as  "UseRegionBasedMarshalling".
             -->
             <attribute name="InactiveOnStartup">false</attribute>
                      <!--  Make sure to specify BatchModeTransactionManager only! -->
             <attribute  name="TransactionManagerLookupClass">org.jboss.cache.BatchModeTransactionManagerLookup</attribute>

             <!-- Configures binary format of messages sent between cluster  nodes.
                  Changing this allows a later version of JBoss Cache to  interoperate
                  with an earlier version. You might, for example, change this
                  if you are integrating a 4.0.4 server into a cluster with
                  servers running an earlier AS version.
                              Possible values:
                              1.2.3     JBC 1.2.3 or earlier; bundled with AS 4.0.3.SP1  and earlier
                  1.2.4     JBC 1.2.4
                  1.2.4.SP1 JBC 1.2.4.SP1
                  1.2.4.SP2 JBC 1.2.4.SP2; bundled with AS 4.0.4
                              For version 1.3.0.GA and later, use the version name.
                         If left blank or commented out, JBoss Cache will use the  default
                  for that release (e.g. 1.4.0 for releases in the 1.4.0 series.
                               The binary format of replication version 1.4.0 is much more  efficient
                  than earlier releases, so there is a significant  performance penalty
                  to trying to interoperate 1.4.0 with earlier releases vs. a  pure
                  1.4.0 cluster.
             -->
                    <attribute name="ReplicationVersion">1.4.0.GA</attribute>
                    <!-- JGroups protocol stack properties. Can also be a URL,
                  e.g. file:/home/bela/default.xml
             <attribute name="ClusterProperties"></attribute>
             -->

             <attribute name="ClusterConfig">
                 <!--
                 The default UDP stack:
                 - If you have a multihomed machine, set the UDP protocol's  bind_addr attribute to the
                 appropriate NIC IP address, e.g bind_addr="192.168.0.2".
                 - On Windows machines, because of the media sense feature  being broken with multicast
                 (even after disabling media sense) set the UDP protocol's  loopback attribute to true
                            - If your CacheMode is set to REPL_SYNC we recommend you  comment
                 out the FC (flow control) protocol
                 -->
                 <config>
                     <UDP mcast_addr="${jboss.partition.udpGroup:224.0.0.30}"
                          mcast_port="45577"
                          ucast_recv_buf_size="20000000"
                          ucast_send_buf_size="640000"
                          mcast_recv_buf_size="25000000"
                          mcast_send_buf_size="640000"
                          loopback="false"
                          max_bundle_size="64000"
                          max_bundle_timeout="30"
                          use_incoming_packet_handler="true"
                          use_outgoing_packet_handler="true"
                          ip_ttl="${jgroups.mcast.ip_ttl:2}"
                          enable_bundling="false"/>
                     <PING timeout="2000" num_initial_members="3"/>
                     <MERGE2 max_interval="100000" min_interval="20000"/>
                     <FD_SOCK />
                     <FD shun="true" timeout="20000" max_tries="5"/>
                     <VERIFY_SUSPECT timeout="1500" />
                     <pbcast.NAKACK max_xmit_size="60000"
                                    use_mcast_xmit="false" gc_lag="50"
                                    retransmit_timeout="300,600,1200,2400,4800"
                                    discard_delivered_msgs="true"/>
                     <UNICAST timeout="300,600,1200,2400,3600" />
                     <pbcast.STABLE stability_delay="1000"  desired_avg_gossip="50000"
                                    max_bytes="400000"/>
                     <pbcast.GMS print_local_addr="true" join_timeout="3000"
                                 join_retry_timeout="2000" shun="true"/>
                   <!--  commented out based on recommendation above about  using REPL_SYNC instead of REPL_ASYNC <FC max_credits="2000000"  min_threshold="0.10"/> -->
                     <FRAG2 frag_size="60000" />
                     <pbcast.STATE_TRANSFER />
                </config>

                <!-- Alternate TCP stack: customize it for your environment,  change bind_addr and initial_hosts -->
                <!--
                <config>
                   <TCP bind_addr="thishost" start_port="7810" loopback="true"
                        tcp_nodelay="false" down_thread="false"  up_thread="false"/>
                   <TCPPING initial_hosts="thishost[7810],otherhost[7810]"  port_range="3" timeout="3500"
                      num_initial_members="3" up_thread="false"  down_thread="false"/>
                   <MERGE2 min_interval="5000" max_interval="10000"
                      up_thread="false" down_thread="false"/>
                   <FD_SOCK down_thread="false" up_thread="false"/>
                   <FD shun="true" up_thread="false" down_thread="false"
                      timeout="10000" max_tries="5"/>
                   <VERIFY_SUSPECT timeout="1500" down_thread="false"  up_thread="false" />
                   <pbcast.NAKACK down_thread="false" up_thread="false"  gc_lag="100"
                      retransmit_timeout="3000"/>
                   <pbcast.STABLE desired_avg_gossip="20000"  down_thread="false" up_thread="false" />
                   <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"  shun="true"
                      print_local_addr="true" down_thread="false"  up_thread="false"/>
                   <FC max_credits="2000000" down_thread="false"  up_thread="false"
                      min_threshold="0.10"/>
                   <FRAG2 frag_size="60000" down_thread="false"  up_thread="false"/>
                   <pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
                </config>
                -->

             </attribute>

             <!--
                 Number of milliseconds to wait until all responses for a
                 synchronous call have been received.
             -->
             <attribute name="SyncReplTimeout">20000</attribute>

             <!-- Max number of milliseconds to wait for a lock acquisition -->
             <attribute name="LockAcquisitionTimeout">15000</attribute>

             <!-- Buddy Replication config.
                         See  http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCacheBuddyReplicationDesign
                  and the JBoss Cache docs for more on buddy replication.
                              By default, buddy replication is disabled.
                              Following are the configuration elements likely to be changed:
                              buddyReplicationEnabled  true if you want buddy  replication; false if data
                                           should be replicated to all nodes  in the cluster
                                                       numBuddies               to how many backup nodes should  each node replicate
                                           its state
                                                       buddyPoolName            allows logical subgrouping of  nodes within the cluster;
                                           if possible, buddies will be  chosen from nodes in the
                                           same buddy pool
                                                       Do not change the data gravitation related  options.                    -->
             <attribute name="BuddyReplicationConfig">
                 <config>
                     <buddyReplicationEnabled>false</buddyReplicationEnabled>
                      <buddyLocatorClass>org.jboss.cache.buddyreplication.NextMemberBuddyLocator</buddyLocatorClass>
                     <buddyLocatorProperties>
                         numBuddies = 1
                         ignoreColocatedBuddies = true
                     </buddyLocatorProperties>

                     <buddyPoolName>default</buddyPoolName>
                     <buddyCommunicationTimeout>2000</buddyCommunicationTimeout>

                     <autoDataGravitation>false</autoDataGravitation>
                      <dataGravitationRemoveOnFind>true</dataGravitationRemoveOnFind>
                      <dataGravitationSearchBackupTrees>true</dataGravitationSearchBackupTrees>

                 </config>
             </attribute>
                   </mbean>

      </server>




      the jboss-service.xml file in tc5-cluster-sar/META-INF directory for  app2 is:

      <?xml version="1.0" encoding="UTF-8"?>

      <!--  ===================================================================== -->
      <!--                                                                        -->
      <!--  Customized TreeCache Service Configuration for Tomcat 5  Clustering   -->
      <!--                                                                        -->
      <!--  ===================================================================== -->

      <server>

         <!--  ==================================================================== -->
         <!-- Defines TreeCache  configuration                                      -->
         <!--  ==================================================================== -->

          <!-- Note we are using TreeCacheAop -->
         <mbean code="org.jboss.cache.aop.TreeCacheAop"
             name="jboss.cache:service=TomcatClusteringCache">

             <depends>jboss:service=Naming</depends>
             <depends>jboss:service=TransactionManager</depends>
             <!-- We need the AspectDeployer to deploy our FIELD granularity  aspects -->
             <depends>jboss.aop:service=AspectDeployer</depends>

             <!-- Name of cluster. Needs to be the same for all nodes in the
                  cluster, in order to find each other
             -->
             <attribute  name="ClusterName">APP2-Session-${jboss.partition.name:Cluster}</attribute>
                    <!--
                 Isolation level : SERIALIZABLE
                                   REPEATABLE_READ (default)
                                   READ_COMMITTED
                                   READ_UNCOMMITTED
                                   NONE
             -->
             <attribute name="IsolationLevel">REPEATABLE_READ</attribute>

             <!-- Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC
                              If you use REPL_SYNC and a UDP-based ClusterConfig
                  we recommend you comment out the FC (flow control)
                  protocol in the ClusterConfig section below.
             -->
             <attribute name="CacheMode">REPL_SYNC</attribute>

             <!--
               Indicates whether to the cache should unmarshall objects  replicated
               from other cluster nodes, or store them internally as a byte[]
               until a web app requests them.  Must be "true" if session  replication
               granularity "FIELD" is used in any webapp, otherwise "false" is
               recommended.
             -->
             <attribute name="UseRegionBasedMarshalling">false</attribute>
                    <!--  Whether or not the entire tree is inactive upon startup, only
               responding to replication messages after activateRegion() is
               called to activate one or more parts of the tree when a webapp is
               deployed.  Must have the same value as  "UseRegionBasedMarshalling".
             -->
             <attribute name="InactiveOnStartup">false</attribute>
                      <!--  Make sure to specify BatchModeTransactionManager only! -->
             <attribute  name="TransactionManagerLookupClass">org.jboss.cache.BatchModeTransactionManagerLookup</attribute>

             <!-- Configures binary format of messages sent between cluster  nodes.
                  Changing this allows a later version of JBoss Cache to  interoperate
                  with an earlier version. You might, for example, change this
                  if you are integrating a 4.0.4 server into a cluster with
                  servers running an earlier AS version.
                              Possible values:
                              1.2.3     JBC 1.2.3 or earlier; bundled with AS 4.0.3.SP1  and earlier
                  1.2.4     JBC 1.2.4
                  1.2.4.SP1 JBC 1.2.4.SP1
                  1.2.4.SP2 JBC 1.2.4.SP2; bundled with AS 4.0.4
                              For version 1.3.0.GA and later, use the version name.
                         If left blank or commented out, JBoss Cache will use the  default
                  for that release (e.g. 1.4.0 for releases in the 1.4.0 series.
                               The binary format of replication version 1.4.0 is much more  efficient
                  than earlier releases, so there is a significant  performance penalty
                  to trying to interoperate 1.4.0 with earlier releases vs. a  pure
                  1.4.0 cluster.
             -->
                    <attribute name="ReplicationVersion">1.4.0.GA</attribute>
                    <!-- JGroups protocol stack properties. Can also be a URL,
                  e.g. file:/home/bela/default.xml
             <attribute name="ClusterProperties"></attribute>
             -->

             <attribute name="ClusterConfig">
                 <!--
                 The default UDP stack:
                 - If you have a multihomed machine, set the UDP protocol's  bind_addr attribute to the
                 appropriate NIC IP address, e.g bind_addr="192.168.0.2".
                 - On Windows machines, because of the media sense feature  being broken with multicast
                 (even after disabling media sense) set the UDP protocol's  loopback attribute to true
                            - If your CacheMode is set to REPL_SYNC we recommend you  comment
                 out the FC (flow control) protocol
                 -->
                 <config>
                     <UDP mcast_addr="${jboss.partition.udpGroup:224.0.0.20}"
                          mcast_port="45577"
                          ucast_recv_buf_size="20000000"
                          ucast_send_buf_size="640000"
                          mcast_recv_buf_size="25000000"
                          mcast_send_buf_size="640000"
                          loopback="false"
                          max_bundle_size="64000"
                          max_bundle_timeout="30"
                          use_incoming_packet_handler="true"
                          use_outgoing_packet_handler="true"
                          ip_ttl="${jgroups.mcast.ip_ttl:2}"
                          enable_bundling="false"/>
                     <PING timeout="2000" num_initial_members="3"/>
                     <MERGE2 max_interval="100000" min_interval="20000"/>
                     <FD_SOCK />
                     <FD shun="true" timeout="20000" max_tries="5"/>
                     <VERIFY_SUSPECT timeout="1500" />
                     <pbcast.NAKACK max_xmit_size="60000"
                                    use_mcast_xmit="false" gc_lag="50"
                                    retransmit_timeout="300,600,1200,2400,4800"
                                    discard_delivered_msgs="true"/>
                     <UNICAST timeout="300,600,1200,2400,3600" />
                     <pbcast.STABLE stability_delay="1000"  desired_avg_gossip="50000"
                                    max_bytes="400000"/>
                     <pbcast.GMS print_local_addr="true" join_timeout="3000"
                                 join_retry_timeout="2000" shun="true"/>
                   <!--  commented out based on recommendation above about  using REPL_SYNC instead of REPL_ASYNC <FC max_credits="2000000"  min_threshold="0.10"/> -->
                     <FRAG2 frag_size="60000" />
                     <pbcast.STATE_TRANSFER />
                </config>

                <!-- Alternate TCP stack: customize it for your environment,  change bind_addr and initial_hosts -->
                <!--
                <config>
                   <TCP bind_addr="thishost" start_port="7810" loopback="true"
                        tcp_nodelay="false" down_thread="false"  up_thread="false"/>
                   <TCPPING initial_hosts="thishost[7810],otherhost[7810]"  port_range="3" timeout="3500"
                      num_initial_members="3" up_thread="false"  down_thread="false"/>
                   <MERGE2 min_interval="5000" max_interval="10000"
                      up_thread="false" down_thread="false"/>
                   <FD_SOCK down_thread="false" up_thread="false"/>
                   <FD shun="true" up_thread="false" down_thread="false"
                      timeout="10000" max_tries="5"/>
                   <VERIFY_SUSPECT timeout="1500" down_thread="false"  up_thread="false" />
                   <pbcast.NAKACK down_thread="false" up_thread="false"  gc_lag="100"
                      retransmit_timeout="3000"/>
                   <pbcast.STABLE desired_avg_gossip="20000"  down_thread="false" up_thread="false" />
                   <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"  shun="true"
                      print_local_addr="true" down_thread="false"  up_thread="false"/>
                   <FC max_credits="2000000" down_thread="false"  up_thread="false"
                      min_threshold="0.10"/>
                   <FRAG2 frag_size="60000" down_thread="false"  up_thread="false"/>
                   <pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
                </config>
                -->

             </attribute>

             <!--
                 Number of milliseconds to wait until all responses for a
                 synchronous call have been received.
             -->
             <attribute name="SyncReplTimeout">20000</attribute>

             <!-- Max number of milliseconds to wait for a lock acquisition -->
             <attribute name="LockAcquisitionTimeout">15000</attribute>

             <!-- Buddy Replication config.
                         See  http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCacheBuddyReplicationDesign
                  and the JBoss Cache docs for more on buddy replication.
                              By default, buddy replication is disabled.
                              Following are the configuration elements likely to be changed:
                              buddyReplicationEnabled  true if you want buddy  replication; false if data
                                           should be replicated to all nodes  in the cluster
                                                       numBuddies               to how many backup nodes should  each node replicate
                                           its state
                                                       buddyPoolName            allows logical subgrouping of  nodes within the cluster;
                                           if possible, buddies will be  chosen from nodes in the
                                           same buddy pool
                                                       Do not change the data gravitation related  options.                    -->
             <attribute name="BuddyReplicationConfig">
                 <config>
                     <buddyReplicationEnabled>false</buddyReplicationEnabled>
                      <buddyLocatorClass>org.jboss.cache.buddyreplication.NextMemberBuddyLocator</buddyLocatorClass>
                     <buddyLocatorProperties>
                         numBuddies = 1
                         ignoreColocatedBuddies = true
                     </buddyLocatorProperties>

                     <buddyPoolName>default</buddyPoolName>
                     <buddyCommunicationTimeout>2000</buddyCommunicationTimeout>

                     <autoDataGravitation>false</autoDataGravitation>
                      <dataGravitationRemoveOnFind>true</dataGravitationRemoveOnFind>
                      <dataGravitationSearchBackupTrees>true</dataGravitationSearchBackupTrees>

                 </config>
             </attribute>
                   </mbean>

      </server>

      the server.xml files for both apps contains the following valve:

             <Valve  className="org.jboss.web.tomcat.tc5.sso.ClusteredSingleSignOn"  cacheConfig="Session-${jboss.partition.name:Cluster}"/>


      We are behind physical load balancers performing round robin load balancing.

      The sequence of events is as follow:

      User opens browers and goes to url for app1.

      app1 displays the login page and the user hits the submit button after  entering a valid user id and password

      app1 validates the user id and password

      app1 logs into app2

      app2 validates the user id and password

      app2 returns successful login to app1

      app1 returns a redirect (302) with the url of app2 to the browser

      the browser then connects to app2.

      if the browser hits app2 on the same server that app1 used for the  login, the user is successfuly logged into app2 and shown the home page  for app2.  the user can then move around app2 without any knowledge of  what server they are on and app2 can be stopped on either server and the  user does not encounter any dropped connections.

      if the browser hits app2 on a different server than app1 used for the  login, the user is returned back to the app1 login page.

      am I missing something in the configuration files?
      yes, this is older release of jboss, but we are unable to upgrade the to  a newer version at this time.

      Thanks In Advance,

      Rick Feldmann
      work@rf-consulting.com