1 2 Previous Next 15 Replies Latest reply on Mar 22, 2016 1:44 PM by Paul Ferraro

    How to disable Clustering between different instances in the same machine

    Giacomo Genovese Newbie

      Hi,

      I'm running different CapeDwarf instances in the same VM using the Standalone-capedwarf-module.xml file for both instances.

      I use to set different IPs, node names, and folders for each instance.

       

      Using such settings, Capedwarf automatically starts clustering between nodes.

      I would like disable clustering but I get confused with settings.

       

      How can I disable the clustering between instances? 

       

      Thanks in advance for the support.

       

      Regrads,

      Giacomo.

      P.S. I don't care about High Availability.

       

        • 1. Re: How to disable Clustering between different instances in the same machine
          Ales Justin Master

          Hmmm, I guess there should be a way to disable this ... let me check and get back to you.

          • 3. Re: How to disable Clustering between different instances in the same machine
            Paul Ferraro Master

            In WildFly, we ship a separate HA and non-HA profiles (e.g.standalone.xml vs standalone-ha.xml) to allow users to easily disable clustering.  In general, I would suggest CapeDwarf do something similar.

             

            As I see it, there are 2 ways to solve your problem:

            1. Isolate your instances such that they don't cluster with each other.
            2. Modify the configuration to remove the need for JGroups.

             

            #1 is fairly simple.  Just start each server instance with distinct multicast addresses via the "jboss.default.multicast.address" system property.

            #2 will require a number of changes:

            1. Remove your jgroups subsystem/extension
            2. Remove any <transport/> elements from your infinispan subsystem configuration
            3. Replace all the <distributed-cache/> and <replicated-cache/> elements in your infinispan subsystem configuration with <local-cache/>.
            • 4. Re: How to disable Clustering between different instances in the same machine
              Giacomo Genovese Newbie

              Hi Paul,

               

              Thanks for your help.

              I need to use the second way you suggested.

              I tried to modify my standalone-capedwarf-modules.xml according to your hint. I can load "Add" the application but I cannot Deploy it.

              When I try to do the deploy I receive these errors back.

               

              18:33:42,275 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-4) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,279 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) JBAS010281: Started memcache_vo-1 cache from capedwarf container

              18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-1) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 46) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-7) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-2) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 46) JBAS010281: Started dist cache from capedwarf container

              18:33:42,647 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-5) ISPN000031: MBeans were successfully registered to the platform MBean server.

              18:33:42,645 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) JBAS010281: Started metadata_vo-1 cache from capedwarf container

              18:33:42,651 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-5) JBAS010281: Started datastore_versions_vo-1 cache from capedwarf container

              18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) JBAS010281: Started dist_vo-1 cache from capedwarf container

              18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) JBAS010281: Started data_vo-1 cache from capedwarf container

              16:14:57,679 ERROR [org.jboss.as.controller.management-operation] (XNIO-1 task-1) JBAS014613: Operation ("deploy") failed - address: ([("deployment" => "1_VO.war")]) - failure description: {"JBAS014771: Services with missing/unavailable dependencies" => [

                  "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing [jboss.jgroups.channel.capedwarf]"

              ]}

              16:14:57,682 ERROR [org.jboss.as.server] (XNIO-1 task-1) JBAS015870: Deploy of deployment "1_VO.war" was rolled back with the following failure message:

              {"JBAS014771: Services with missing/unavailable dependencies" => [

                  "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                  "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing [jboss.jgroups.channel.capedwarf]"

              ]}

              ............

              ...........

              JBAS014775:    New missing/unsatisfied dependencies:

                    service jboss.capedwarf.cache-lifecycle.channel.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.data.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.datastore_versions.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.default.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.dist.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.logs.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.memcache.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.metadata.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.search.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.cache-lifecycle.tasks.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                    service jboss.capedwarf.mux-gen.vo-1 (missing) dependents: [service jboss.capedwarf.cache-lifecycle.logs.vo-1, service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService, service jboss.capedwarf.cache-lifecycle.tasks.vo-1, service jboss.capedwarf.cache-lifecycle.search.vo-1, JBAS014799: ... and 2 more ]

              Do you have any other suggestion?

              I copy below my Infinispan setting in the standalone-capedwarf-modules.xml. It could be useful to detect the error I'm doing.

              Thanks again!

              <subsystem xmlns="urn:jboss:domain:infinispan:2.0">

                          <cache-container name="capedwarf" default-cache="default" module="org.infinispan.query" statistics-enabled="true">

                              <local-cache name="default">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="100000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="datastore_versions">

                                  <locking isolation="REPEATABLE_READ"/>

                                  <transaction mode="NON_XA" locking="OPTIMISTIC"/>

                                  <eviction strategy="LIRS" max-entries="100000"/>

                                  <file-store preload="false" purge="false"/>

                              </local-cache>

                              <local-cache name="data">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                              </local-cache>

                              <local-cache name="metadata">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                              </local-cache>

                              <local-cache name="memcache" statistics-enabled="true">

                                  <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                              </local-cache>

                              <local-cache name="dist">

                                  <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                                  <eviction strategy="LIRS" max-entries="100000"/>

                                  <file-store preload="false" purge="false"/>

                              </local-cache>

                              <local-cache name="tasks">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.block_waiting_ack">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="search">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.block_waiting_ack">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="prospective_search">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.block_waiting_ack">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="logs">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.block_waiting_ack">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="channel">

                                  <transaction mode="NON_XA"/>

                                  <eviction strategy="LIRS" max-entries="10000"/>

                                  <file-store preload="false" purge="false"/>

                                  <indexing index="LOCAL">

                                      <property name="hibernate.search.default.block_waiting_ack">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.exclusive_index_use">

                                          false

                                      </property>

                                      <property name="hibernate.search.default.directory_provider">

                                          infinispan

                                      </property>

                                      <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                          java:jboss/infinispan/container/capedwarf

                                      </property>

                                  </indexing>

                              </local-cache>

                              <local-cache name="LuceneIndexesMetadata">

                                  <transaction mode="NONE"/>

                                  <eviction strategy="NONE"/>

                                  <file-store preload="true" purge="false"/>

                              </local-cache>

                              <local-cache name="LuceneIndexesData">

                                  <transaction mode="NONE"/>

                                  <eviction strategy="NONE"/>

                                  <file-store preload="true" purge="false"/>

                              </local-cache>

                              <local-cache name="LuceneIndexesLocking">

                                  <transaction mode="NONE"/>

                                  <eviction strategy="NONE"/>

                              </local-cache>

                          </cache-container>

                      </subsystem>

              • 5. Re: How to disable Clustering between different instances in the same machine
                Giacomo Genovese Newbie

                I did some tests and I get that probably changing the standalone-capedwarf-modules.xml is not enough to reach the goal.

                Any help?

                 

                Thanks.

                • 6. Re: How to disable Clustering between different instances in the same machine
                  Paul Ferraro Master

                  Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                      "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                      "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing

                   

                  In the meantime, you should probably look into the cluster isolation approach.  If CapeDwarf is indeed based on WF10, the simplest way to do this is to bind your jgroups traffic to localhost.  To do that, modify the jgroups socket bindings to use a separate interface.

                  e.g.

                      <interface name="private">
                          <inet-address value="${jboss.bind.address.private:127.0.0.1}"/>
                      </interface>
                  

                  and modify your jgroups socket bindings to use it.

                  e.g.

                      <socket-binding name="jgroups-mping" interface="private" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
                      <socket-binding name="jgroups-tcp" interface="private" port="7600"/>
                      <socket-binding name="jgroups-tcp-fd" interface="private" port="57600"/>
                      <socket-binding name="jgroups-udp" interface="private" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
                      <socket-binding name="jgroups-udp-fd" interface="private" port="54200"/>
                  
                  • 7. Re: How to disable Clustering between different instances in the same machine
                    Giacomo Genovese Newbie

                    Thanks Paul for your answer.

                    I already tested the isolation approach and it works. I used some solution I find in some books.

                    I didn't try to bind the addresses to my localhost, but I switched all the addresses related to the cluster, "mod-cluster" and "messaging-group" included.

                     

                    I look forward for the kind reply of the CapeDwarf team.

                     

                    Thanks Again.

                    • 8. Re: How to disable Clustering between different instances in the same machine
                      Ales Justin Master

                      We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                      Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                      • 9. Re: How to disable Clustering between different instances in the same machine
                        Ales Justin Master

                        We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                        Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                         

                        And this HA-like cron services are hard coded -- they do expect a channel to be present.

                        • 11. Re: How to disable Clustering between different instances in the same machine
                          Giacomo Genovese Newbie

                          Ales Justin ha scritto:

                           

                          We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                          Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                           

                          And this HA-like cron services are hard coded -- they do expect a channel to be present.

                          So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                          This because GAE needs HA-like services. is it right?

                           

                          Thanks for the reply.

                          • 12. Re: How to disable Clustering between different instances in the same machine
                            Ales Justin Master

                            So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                            This because GAE needs HA-like services. is it right?

                             

                            No, this is because we implemented GAE's cron job like a HA.

                             

                            I could eventually re-write this part, to fallback on simple singleton / in-memory cron, if there is no channel.

                            (but unfortunately I don't have the time atm ...)

                             

                            So yes, do use isolation atm.

                            • 13. Re: How to disable Clustering between different instances in the same machine
                              Giacomo Genovese Newbie

                              Ales Justin ha scritto:

                               

                              So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                              This because GAE needs HA-like services. is it right?

                               

                              No, this is because we implemented GAE's cron job like a HA.

                              Well, unfortunately for me it's the same

                               

                              Do you think that migrate the JAVA code of the App to Jboss (WildFly) could be a good idea? How much effort could it require?

                               

                              Thanks a lot for supporting me.

                              • 14. Re: How to disable Clustering between different instances in the same machine
                                Ales Justin Master

                                Do you think that migrate the JAVA code of the App to Jboss (WildFly) could be a good idea? How much effort could it require?

                                 

                                It depends on what the app does, how it's implemented, are you gonna migrate it ever to GAE, etc ...

                                 

                                e.g. I have apps that run purely on GAE (Appspot), and I test them on CapeDwarf, and vice-versa

                                e.g.2 I like the GAE API a lot, as it's quite simple and intuitive, yet still quite powerful

                                1 2 Previous Next