1 2 Previous Next 15 Replies Latest reply on Mar 22, 2016 1:44 PM by pferraro

    How to disable Clustering between different instances in the same machine

    giacomo.genovese

      Hi,

      I'm running different CapeDwarf instances in the same VM using the Standalone-capedwarf-module.xml file for both instances.

      I use to set different IPs, node names, and folders for each instance.

       

      Using such settings, Capedwarf automatically starts clustering between nodes.

      I would like disable clustering but I get confused with settings.

       

      How can I disable the clustering between instances? 

       

      Thanks in advance for the support.

       

      Regrads,

      Giacomo.

      P.S. I don't care about High Availability.

       

        • 1. Re: How to disable Clustering between different instances in the same machine
          alesj

          Hmmm, I guess there should be a way to disable this ... let me check and get back to you.

          • 2. Re: How to disable Clustering between different instances in the same machine
            giacomo.genovese

            Ok...thanks!

            • 3. Re: How to disable Clustering between different instances in the same machine
              pferraro

              In WildFly, we ship a separate HA and non-HA profiles (e.g.standalone.xml vs standalone-ha.xml) to allow users to easily disable clustering.  In general, I would suggest CapeDwarf do something similar.

               

              As I see it, there are 2 ways to solve your problem:

              1. Isolate your instances such that they don't cluster with each other.
              2. Modify the configuration to remove the need for JGroups.

               

              #1 is fairly simple.  Just start each server instance with distinct multicast addresses via the "jboss.default.multicast.address" system property.

              #2 will require a number of changes:

              1. Remove your jgroups subsystem/extension
              2. Remove any <transport/> elements from your infinispan subsystem configuration
              3. Replace all the <distributed-cache/> and <replicated-cache/> elements in your infinispan subsystem configuration with <local-cache/>.
              • 4. Re: How to disable Clustering between different instances in the same machine
                giacomo.genovese

                Hi Paul,

                 

                Thanks for your help.

                I need to use the second way you suggested.

                I tried to modify my standalone-capedwarf-modules.xml according to your hint. I can load "Add" the application but I cannot Deploy it.

                When I try to do the deploy I receive these errors back.

                 

                18:33:42,275 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-4) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,279 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) JBAS010281: Started memcache_vo-1 cache from capedwarf container

                18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-1) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 46) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-7) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,645 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-2) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 46) JBAS010281: Started dist cache from capedwarf container

                18:33:42,647 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-5) ISPN000031: MBeans were successfully registered to the platform MBean server.

                18:33:42,645 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) JBAS010281: Started metadata_vo-1 cache from capedwarf container

                18:33:42,651 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-5) JBAS010281: Started datastore_versions_vo-1 cache from capedwarf container

                18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) JBAS010281: Started dist_vo-1 cache from capedwarf container

                18:33:42,647 INFO  [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) JBAS010281: Started data_vo-1 cache from capedwarf container

                16:14:57,679 ERROR [org.jboss.as.controller.management-operation] (XNIO-1 task-1) JBAS014613: Operation ("deploy") failed - address: ([("deployment" => "1_VO.war")]) - failure description: {"JBAS014771: Services with missing/unavailable dependencies" => [

                    "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing [jboss.jgroups.channel.capedwarf]"

                ]}

                16:14:57,682 ERROR [org.jboss.as.server] (XNIO-1 task-1) JBAS015870: Deploy of deployment "1_VO.war" was rolled back with the following failure message:

                {"JBAS014771: Services with missing/unavailable dependencies" => [

                    "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                    "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing [jboss.jgroups.channel.capedwarf]"

                ]}

                ............

                ...........

                JBAS014775:    New missing/unsatisfied dependencies:

                      service jboss.capedwarf.cache-lifecycle.channel.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.data.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.datastore_versions.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.default.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.dist.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.logs.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.memcache.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.metadata.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.search.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.cache-lifecycle.tasks.vo-1 (missing) dependents: [service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService]

                      service jboss.capedwarf.mux-gen.vo-1 (missing) dependents: [service jboss.capedwarf.cache-lifecycle.logs.vo-1, service jboss.undertow.deployment.default-server.default-host./1_VO.UndertowDeploymentInfoService, service jboss.capedwarf.cache-lifecycle.tasks.vo-1, service jboss.capedwarf.cache-lifecycle.search.vo-1, JBAS014799: ... and 2 more ]

                Do you have any other suggestion?

                I copy below my Infinispan setting in the standalone-capedwarf-modules.xml. It could be useful to detect the error I'm doing.

                Thanks again!

                <subsystem xmlns="urn:jboss:domain:infinispan:2.0">

                            <cache-container name="capedwarf" default-cache="default" module="org.infinispan.query" statistics-enabled="true">

                                <local-cache name="default">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="100000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="datastore_versions">

                                    <locking isolation="REPEATABLE_READ"/>

                                    <transaction mode="NON_XA" locking="OPTIMISTIC"/>

                                    <eviction strategy="LIRS" max-entries="100000"/>

                                    <file-store preload="false" purge="false"/>

                                </local-cache>

                                <local-cache name="data">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                </local-cache>

                                <local-cache name="metadata">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                </local-cache>

                                <local-cache name="memcache" statistics-enabled="true">

                                    <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                </local-cache>

                                <local-cache name="dist">

                                    <transaction mode="NON_XA" locking="PESSIMISTIC"/>

                                    <eviction strategy="LIRS" max-entries="100000"/>

                                    <file-store preload="false" purge="false"/>

                                </local-cache>

                                <local-cache name="tasks">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.block_waiting_ack">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="search">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.block_waiting_ack">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="prospective_search">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.block_waiting_ack">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="logs">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.block_waiting_ack">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="channel">

                                    <transaction mode="NON_XA"/>

                                    <eviction strategy="LIRS" max-entries="10000"/>

                                    <file-store preload="false" purge="false"/>

                                    <indexing index="LOCAL">

                                        <property name="hibernate.search.default.block_waiting_ack">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.exclusive_index_use">

                                            false

                                        </property>

                                        <property name="hibernate.search.default.directory_provider">

                                            infinispan

                                        </property>

                                        <property name="hibernate.search.infinispan.cachemanager_jndiname">

                                            java:jboss/infinispan/container/capedwarf

                                        </property>

                                    </indexing>

                                </local-cache>

                                <local-cache name="LuceneIndexesMetadata">

                                    <transaction mode="NONE"/>

                                    <eviction strategy="NONE"/>

                                    <file-store preload="true" purge="false"/>

                                </local-cache>

                                <local-cache name="LuceneIndexesData">

                                    <transaction mode="NONE"/>

                                    <eviction strategy="NONE"/>

                                    <file-store preload="true" purge="false"/>

                                </local-cache>

                                <local-cache name="LuceneIndexesLocking">

                                    <transaction mode="NONE"/>

                                    <eviction strategy="NONE"/>

                                </local-cache>

                            </cache-container>

                        </subsystem>

                • 5. Re: How to disable Clustering between different instances in the same machine
                  giacomo.genovese

                  I did some tests and I get that probably changing the standalone-capedwarf-modules.xml is not enough to reach the goal.

                  Any help?

                   

                  Thanks.

                  • 6. Re: How to disable Clustering between different instances in the same machine
                    pferraro

                    Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                        "jboss.capedwarf.mux-gen.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.default.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.tasks.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.channel.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.search.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.logs.vo-1 is missing [jboss.jgroups.channel.capedwarf]",

                        "jboss.capedwarf.cache-lifecycle.prospective_search.vo-1 is missing

                     

                    In the meantime, you should probably look into the cluster isolation approach.  If CapeDwarf is indeed based on WF10, the simplest way to do this is to bind your jgroups traffic to localhost.  To do that, modify the jgroups socket bindings to use a separate interface.

                    e.g.

                        <interface name="private">
                            <inet-address value="${jboss.bind.address.private:127.0.0.1}"/>
                        </interface>
                    

                    and modify your jgroups socket bindings to use it.

                    e.g.

                        <socket-binding name="jgroups-mping" interface="private" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
                        <socket-binding name="jgroups-tcp" interface="private" port="7600"/>
                        <socket-binding name="jgroups-tcp-fd" interface="private" port="57600"/>
                        <socket-binding name="jgroups-udp" interface="private" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
                        <socket-binding name="jgroups-udp-fd" interface="private" port="54200"/>
                    
                    • 7. Re: How to disable Clustering between different instances in the same machine
                      giacomo.genovese

                      Thanks Paul for your answer.

                      I already tested the isolation approach and it works. I used some solution I find in some books.

                      I didn't try to bind the addresses to my localhost, but I switched all the addresses related to the cluster, "mod-cluster" and "messaging-group" included.

                       

                      I look forward for the kind reply of the CapeDwarf team.

                       

                      Thanks Again.

                      • 8. Re: How to disable Clustering between different instances in the same machine
                        alesj

                        We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                        Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                        • 9. Re: How to disable Clustering between different instances in the same machine
                          alesj

                          We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                          Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                           

                          And this HA-like cron services are hard coded -- they do expect a channel to be present.

                          • 10. Re: How to disable Clustering between different instances in the same machine
                            alesj

                            CapeDwarf is based off WF 8.2.1.Final

                            * capedwarf-jboss-as/pom.xml at master · capedwarf/capedwarf-jboss-as · GitHub

                            If CapeDwarf is indeed based on WF10

                            • 11. Re: How to disable Clustering between different instances in the same machine
                              giacomo.genovese

                              Ales Justin ha scritto:

                               

                              We use the existing JGroups channel to handle custom HA-like scheduling aka cron jobs.

                              Someone from the CapeDwarf team is going to have to answer that one, as I don't know these services do, nor why they require a JGroups channel:

                               

                              And this HA-like cron services are hard coded -- they do expect a channel to be present.

                              So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                              This because GAE needs HA-like services. is it right?

                               

                              Thanks for the reply.

                              • 12. Re: How to disable Clustering between different instances in the same machine
                                alesj

                                So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                                This because GAE needs HA-like services. is it right?

                                 

                                No, this is because we implemented GAE's cron job like a HA.

                                 

                                I could eventually re-write this part, to fallback on simple singleton / in-memory cron, if there is no channel.

                                (but unfortunately I don't have the time atm ...)

                                 

                                So yes, do use isolation atm.

                                • 13. Re: How to disable Clustering between different instances in the same machine
                                  giacomo.genovese

                                  Ales Justin ha scritto:

                                   

                                  So, If I have understood well, It means that I cannot disable Clustering in Capedwarf but only isolate it. Isn't it?

                                  This because GAE needs HA-like services. is it right?

                                   

                                  No, this is because we implemented GAE's cron job like a HA.

                                  Well, unfortunately for me it's the same

                                   

                                  Do you think that migrate the JAVA code of the App to Jboss (WildFly) could be a good idea? How much effort could it require?

                                   

                                  Thanks a lot for supporting me.

                                  • 14. Re: How to disable Clustering between different instances in the same machine
                                    alesj

                                    Do you think that migrate the JAVA code of the App to Jboss (WildFly) could be a good idea? How much effort could it require?

                                     

                                    It depends on what the app does, how it's implemented, are you gonna migrate it ever to GAE, etc ...

                                     

                                    e.g. I have apps that run purely on GAE (Appspot), and I test them on CapeDwarf, and vice-versa

                                    e.g.2 I like the GAE API a lot, as it's quite simple and intuitive, yet still quite powerful

                                    1 2 Previous Next