1 2 Previous Next 16 Replies Latest reply on Jan 30, 2015 3:19 AM by rvansa

    Which ports does infinispan use?

    nguyenbkcse

      Hi, I'm using Wildfly 8 for my server, I don't want to disable my Linux firewall and just want to open some ports that Infinispan uses for cluster. As I known, Infinispan use Jgroups, so I opened all ports that Jgroups uses follow the configuration in domain.xml of Wildfly:

       

      <socket-binding-group name="full-ha-sockets" default-interface="public">
         <socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
         <socket-binding name="http" port="${jboss.http.port:8080}"/>
         <socket-binding name="https" port="${jboss.https.port:8443}"/>
         <socket-binding name="jacorb" interface="unsecure" port="3528"/>
         <socket-binding name="jacorb-ssl" interface="unsecure" port="3529"/>
         <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
         <socket-binding name="jgroups-tcp" port="7600"/>
         <socket-binding name="jgroups-tcp-fd" port="57600"/>
         <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
         <socket-binding name="jgroups-udp-fd" port="54200"/>
         <socket-binding name="messaging-group" port="0" multicast-address="${jboss.messaging.group.address:231.7.7.7}" multicast-port="${jboss.messaging.group.port:9876}"/>
         <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>
         <socket-binding name="txn-recovery-environment" port="4712"/>
         <socket-binding name="txn-status-manager" port="4713"/>
         <outbound-socket-binding name="mail-smtp">
            <remote-destination host="localhost" port="25"/>
         </outbound-socket-binding>
      </socket-binding-group>
      

       

      But when I started my cluster of two nodes, Infinispan cache is not sync between 2 nodes, I don't know which port I missed. Can anyone help me to know which exactly port Infinispan/Jgroups uses?

      Thanks for all.

        • 1. Re: Which ports does infinispan use?
          arun2arunraj

          Hi Nguyen,

           

          But when I started my cluster of two nodes, Infinispan cache is not sync between 2 nodes


          Can you please explain How did you ensured Infinispan cache is not synced ?

           

          Thanks & Regards,
          ArunRaj. R

          • 2. Re: Which ports does infinispan use?
            nguyenbkcse

            I used two clients for two server nodes (client A connected to server A, client B connected to server B). When I used client A to add an entry to the cache of server A (call entry X), it's expected that the cache of server B must have entry X (because of the synchronization of Infinispan cache of 2 nodes). But when I used client B to get cache content of server B, I didn't see entry X. That's my simple explanation of my situation.

            Thanks.

            • 3. Re: Which ports does infinispan use?
              nadirx

              By default jgroups will use the udp stack from the wildfly configuration, so from the above the ports that matter are:

               

              jgroups-mping 45700 (multicast, used for discovery)

              jgroups-udp 55200 (unicast), 45688 (multicast) (used for data transfer)

              jgroups-udp-fd 54200 (unicast) (used for failure detection)


              Unicast traffic will use the default server bind address, while multicast traffic will use a dedicated multicast address (230.0.0.4 by default in your config)


              In order to see if the cluster has formed, look at the logs of both nodes. It will show clustering views forming. Attach them here so that we can look at them



              • 4. Re: Re: Which ports does infinispan use?
                nguyenbkcse

                Hi Tristan,

                I have open all ports you mentioned, but my server port-offset is 250 as in my host.xml configuration:

                <server name="server-three" group="other-server-group" auto-start="true">
                    <!-- server-three avoids port conflicts by incrementing the ports in
                        the default socket-group declared in the server-group -->
                    <socket-bindings port-offset="250"/>
                    <jvm name="default">
                      <jvm-options>
                        <option value="-agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n"/>
                      </jvm-options>
                  </jvm>
                </server>
                

                so I added 250 for each port, 55200 => 55450, 54200 => 54450, other multicast ports are remained same value (45700, 45688). But when I tested 2 servers, their cache still not synced. I attached here my 2 server logs:

                • 5. Re: Re: Which ports does infinispan use?
                  rvansa

                  Do you have correctly set multicast routing in your cluster? Could you test that it works using netcat (nc) on those machines? Especially the multicasts.

                  Do those machines have the same IPv4/v6 settings?

                  Beyond that, logs with TRACE level on org.jgroups package provides us with more clues.

                  • 6. Re: Re: Re: Which ports does infinispan use?
                    nguyenbkcse

                    Hi Radim, I have tested multicast on my machine using McastReceiverTest and McastSenderTest of Jgroups, multicast worked correctly. But for my 2 server nodes, it seems to be not working. Here are my logs file of Jgroups of 2 servers, can you take a look and tell me if there are any trouble.

                    Thank you

                    • 7. Re: Re: Re: Which ports does infinispan use?
                      rvansa

                      From the logs it seems that those nodes are correctly clustered:

                       

                      2015-01-26 13:36:01,423 DEBUG [org.jgroups.protocols.pbcast.GMS] (Incoming-1,shared=udp) master:server-three1/web: installing view [master:server-three1/web|1] (2) [master:server-three1/web, slave:server-three-slave/web]

                       

                      2015-01-26 11:36:01,485 DEBUG [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool -- 58) slave:server-three-slave/web: installing view [master:server-three1/web|1] (2) [master:server-three1/web, slave:server-three-slave/web]

                      • 8. Re: Re: Re: Which ports does infinispan use?
                        nguyenbkcse

                        But inspite of the logs, cache was not synced betwwen two nodes.

                        • 9. Re: Re: Re: Which ports does infinispan use?
                          nadirx

                          Can you post your domain.xml ?

                          • 10. Re: Re: Re: Which ports does infinispan use?
                            rvansa

                            Okay, so we need the TRACE logs on both org.jgroups and org.infinispan to find what's wrong.

                            • 11. Re: Re: Re: Re: Which ports does infinispan use?
                              nguyenbkcse

                              Hi Tristan and Radim, I posted my domain.xml and Infinispan logs, can you check if there's any problems. Thank all.

                              • 12. Re: Re: Re: Which ports does infinispan use?
                                nadirx

                                Sorry Nguyen, but something is escaping me: from the domain.xml configuration I don't see any declaration of custom caches (in addition to the "system" ones used by clustering internals). Above you claim you are writing data to the cache from one node and reading it from another. Which cache is this ? A custom one ? And how are you initializing/retrieving it ?

                                • 13. Re: Re: Re: Which ports does infinispan use?
                                  rvansa

                                  When have you executed cache.get()? I don't see any execution of GetKeyValueCommand in the logs. I can see that the value slave:server-three-slave=null was replicated to master.

                                  • 14. Re: Re: Re: Which ports does infinispan use?
                                    nguyenbkcse

                                    Yes, I used a custom Infinispan cache and declared it in my pom.xml. Cache run normally when I disable the firewall.

                                    1 2 Previous Next