2 Replies Latest reply on Dec 10, 2019 4:21 AM by joderran2

    Infinispan Jgroups Problems Over JBoss EAP 7.1 (Web Cache)

    joderran2

      I have this Jboss Configuration: 2 nodes instances of Jboss eap 7.1.6 on differente machine clustering (udp) each other on standalone mode. The first consideration that i'll do is the above similar configuration with cluster on a single instance of jboss is working fine, but in a case of 2 different jboss on different machines does not work correctly.

      I start any Jboss with a canonical:

      Start of Jboss Instances

      ./standalone.sh -c standalone-full-ha.xml -bmanagement=first_ip -b 0.0.0.0 -u 230.0.0.4

      ./standalone.sh -c standalone-full-ha.xml -bmanagement=second_ip -b 0.0.0.0 -u 230.0.0.4

      In the log i see the cluster seems started correctly

       

      server.log (node2)
      2019-11-21 10:34:41,259 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000078: Starting JGroups channel ee
      2019-11-21 10:34:41,259 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000078: Starting JGroups channel ee
      2019-11-21 10:34:41,260 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel ee
      2019-11-21 10:34:41,260 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel ee
      2019-11-21 10:34:41,261 INFO [org.jboss.weld.Version] (MSC service thread 1-7) WELD-000900: 2.4.7 (redhat)
      2019-11-21 10:34:41,279 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel ee: [node1|1] (2) [node1, node2]
      2019-11-21 10:34:41,279 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel ee: [node1|1] (2) [node1, node2]
      2019-11-21 10:34:41,279 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000094: Received new cluster view for channel ee: [node1|1] (2) [node1, node2]
      2019-11-21 10:34:41,282 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ee: [node1|1] (2) [node1, node2]
      2019-11-21 10:34:41,284 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel ee local address is node2, physical addresses are [second_ip:55200]
      2019-11-21 10:34:41,286 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel ee local address is node2, physical addresses are [second_ip:55200]
      2019-11-21 10:34:41,287 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000079: Channel ee local address is node2, physical addresses are [second_ip:55200]
      2019-11-21 10:34:41,287 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000079: Channel ee local address is node2, physical addresses are [second_ip:55200]

       

      This is the first anomaly: why in the log i have 4 similar rows for each channel named ee? In the Jboss eap 7.0, the log show 4 differents rows each to any protocol: server, ejb, hibernate, web. But the problems is not here. When i try to test a web session shared, i see that each node preserve it's own session: it seems that every standalone/node work separately. Why this?

      Above other configuration for every standalone node (standalone-full-ha.xml):

      standalone-full-ha.xml (node2) - interesting parts
      <server name="node2" xmlns="urn:jboss:domain:5.0">
      ...
      <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
        <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server">
        <transport lock-timeout="60000"/>
        <replicated-cache name="default">
        <transaction mode="BATCH"/>
        </replicated-cache>
        </cache-container>
        <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
        <transport lock-timeout="60000"/>
        <distributed-cache name="dist">
        <locking isolation="REPEATABLE_READ"/>
        <transaction mode="BATCH"/>
        <file-store/>
        </distributed-cache>
        </cache-container>
        <cache-container name="ejb" aliases="sfsb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan">
        <transport lock-timeout="60000"/>
        <distributed-cache name="dist">
        <locking isolation="REPEATABLE_READ"/>
        <transaction mode="BATCH"/>
        <file-store/>
        </distributed-cache>
        </cache-container>
        <cache-container name="hibernate" default-cache="local-query" module="org.hibernate.infinispan">
        <transport lock-timeout="60000"/>
        <local-cache name="local-query">
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
        </local-cache>
        <invalidation-cache name="entity">
        <transaction mode="NON_XA"/>
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
        </invalidation-cache>
        <replicated-cache name="timestamps" mode="ASYNC"/>
        </cache-container>
        </subsystem>
        ....
        <subsystem xmlns="urn:jboss:domain:jgroups:5.0">
        <channels default="ee">
        <channel name="ee" stack="udp"/>
        </channels>
        <stacks>
        <stack name="udp">
        <transport type="UDP" socket-binding="jgroups-udp"/>
        <protocol type="PING"/>
        <protocol type="MERGE3"/>
        <protocol type="FD_SOCK"/>
        <protocol type="FD_ALL"/>
        <protocol type="VERIFY_SUSPECT"/>
        <protocol type="pbcast.NAKACK2"/>
        <protocol type="UNICAST3"/>
        <protocol type="pbcast.STABLE"/>
        <protocol type="pbcast.GMS"/>
        <protocol type="UFC"/>
        <protocol type="MFC"/>
        <protocol type="FRAG2"/>
        </stack>
        <stack name="tcp">
        <transport type="TCP" socket-binding="jgroups-tcp"/>
        <socket-protocol type="MPING" socket-binding="jgroups-mping"/>
        <protocol type="MERGE3"/>
        <protocol type="FD_SOCK"/>
        <protocol type="FD_ALL"/>
        <protocol type="VERIFY_SUSPECT"/>
        <protocol type="pbcast.NAKACK2"/>
        <protocol type="UNICAST3"/>
        <protocol type="pbcast.STABLE"/>
        <protocol type="pbcast.GMS"/>
        <protocol type="MFC"/>
        <protocol type="FRAG2"/>
        </stack>
        </stacks>
        </subsystem>
        .....
        <interfaces>
        <interface name="management">
        <inet-address value="${jboss.bind.address.management:127.0.0.1}"/>
        </interface>
        <interface name="public">
        <inet-address value="${jboss.bind.address:127.0.0.1}"/>
        </interface>
        <interface name="private">
        <inet-address value="${jboss.bind.address.private:second_ip}"/>
        </interface>
        <interface name="unsecure">
        <inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>
        </interface>
      </interfaces>
      <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
        <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
        <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
        <socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
        <socket-binding name="http" port="${jboss.http.port:8080}"/>
        <socket-binding name="https" port="${jboss.https.port:8443}"/>
        <socket-binding name="iiop" interface="unsecure" port="3528"/>
        <socket-binding name="iiop-ssl" interface="unsecure" port="3529"/>
        <socket-binding name="jgroups-mping" interface="private" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
        <socket-binding name="jgroups-tcp" interface="private" port="7600"/>
        <socket-binding name="jgroups-udp" interface="private" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
        <socket-binding name="modcluster" port="0" multicast-address="${jboss.modcluster.multicast.address:224.0.1.105}" multicast-port="23364"/>
        <socket-binding name="txn-recovery-environment" port="4712"/>
        <socket-binding name="txn-status-manager" port="4713"/>
        <outbound-socket-binding name="mail-smtp">
        <remote-destination host="localhost" port="25"/>
        </outbound-socket-binding>
      </socket-binding-group>

       

      In practice: when i try to, via a simple counter visit web application, navigate on first instance i read a value of this counter; after i switch on second instance, this counter restart to first value and not continue from previous value (in this war i have put <distributable/> tag in web.xml)

      any suggestions? Thanks

        • 1. Re: Infinispan Jgroups Problems Over JBoss EAP 7.1 (Web Cache)
          pferraro

          joderran2  wrote:

           

          This is the first anomaly: why in the log i have 4 similar rows for each channel named ee? In the Jboss eap 7.0, the log show 4 differents rows each to any protocol: server, ejb, hibernate, web.

           

          Those log messages are emitted by each Infinispan cache-container in your configuration.  Each cache-container uses a fork of the "ee" channel.

           

          joderran2  wrote:

           

          When i try to test a web session shared, i see that each node preserve it's own session: it seems that every standalone/node work separately. Why this?

          In practice: when i try to, via a simple counter visit web application, navigate on first instance i read a value of this counter; after i switch on second instance, this counter restart to first value and not continue from previous value (in this war i have put <distributable/> tag in web.xml)

          any suggestions? Thanks

           

          Can you paste the session logic from this "simple counter visit web application"?  Please also paste your web.xml and any jboss-web.xml.

          • 2. Re: Infinispan Jgroups Problems Over JBoss EAP 7.1 (Web Cache)
            joderran2

            Hi Paul,

             

            In fact I performed an improper test. From the same browser I invoked this web app without going through a single access point: once I called the node1, then the node2 and I didn't realize that in reality the web sessions, despite being the clustered nodes, are still separate and each one works for own account. Verifying the functioning of the cluster is possible only when the cluster works behind a single access point (Radware or Apache etc.)

             

            Regarding the naming of the channels in the log, you're right it's just a question of the label, however I have verified by raising the log level that actually all 4 channels start.

            Thanks so much