1 2 3 4 Previous Next 51 Replies Latest reply on Jul 18, 2013 12:23 PM by kasik Go to original post
      • 45. Re: Infinispan on EC2
        benze

        Hi,

         

        I've been using Tomcat for years, but I have been put on a new project which will be using both JBoss 6 and Infinispan on EC2.  I am new to both JBoss 6 and configuring it in a cluster.  After having read around a lot, I am having a lot of difficulty finding good documentation explaining how / where exactly to configure JBoss/Infinispan to work in a cluster.

         

        I understand that both JB and IS use JGroups for the cluster communication.  I am very much interested in using JDBC_PING protocol, as it sounds ideal for my needs.  But my problem is that I just do not understand where/how to start configuring JBoss to use it (ie: which config files must be modified/etc).

         

        Can anyone please point me in the right direction?  I've already read through the JBoss AS 5.1 Clustering Guide (http://docs.jboss.org/jbossclustering/cluster_guide/5.1/html/index.html) but I am no further ahead understanding how to properly configure this.  Any help and/or advice would be greatly appreciated!

         

        Thanks!


        Eric

         

         




        • 46. Re: Infinispan on EC2
          belaban

          I gave a presentation on running AS6 on EC2 at JBossWorld, check out the presentation at http://www.redhat.com/summit/2011/presentations/jbossworld/whats_new/wednesday/ban_w_310_running_in_the_cloud.pdf.

          Cheers,

          • 47. Re: Infinispan on EC2
            benze

            Thanks for the link!  Definitely helped clear up a couple of critical (yet strangely difficult to find) information points.  But I am still seriously struggling to get this to work.

             

            I have tried to update my jgroups-channelfactory-stacks.xml file as follows:

             

            {code:xml}

                <stack name="tcp"

                       description="TCP based stack, with flow control and message bundling.                                                                                                         

                                    TCP stacks are usually used when IP multicasting cannot                                                                                                          

                                    be used in a network, e.g. because it is disabled (e.g.                                                                                                          

                                    routers discard multicast)">

                    <config>

                        <TCP

                             singleton_name="tcp"

                             start_port="${jboss.jgroups.tcp.tcp_port:7600}"

                           ....

                           ..../>

                       <JDBC_PING datasource_jndi_name="java:jgroups-DS"/>

                        <MERGE2 max_interval="100000" min_interval="20000"/>

                        <FD_SOCK start_port="${jboss.jgroups.tcp.fd_sock_port:57600}"/>

                      ....

                      ....

            {code:xml}

             

            And my mysql-ds.xml file as follows:

             

            {code:xml}

            <datasources>

              <local-tx-datasource>

                <jndi-name>jgroups-DS</jndi-name>

                <connection-url>jdbc:mysql://ec2-50-17-01-01.compute-1.amazonaws.com:3306/jgroups</connection-url>

                <driver-class>com.mysql.jdbc.Driver</driver-class>

                <user-name>jboss</user-name>

                <password>jboss</password>

                <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>

                <!-- should only be used on drivers after 3.22.1 with "ping" support    -->

                <valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLValidConnectionChecker</valid-connection-checker-class-name>

                <metadata>

                  <type-mapping>mySQL</type-mapping>

                </metadata>

              </local-tx-datasource>

            </datasources>

            {code:xml}

             

            Finally, I start the server with the following command:

            ./run.sh -c all -Djboss.default.jgroups.stack=tcp

             

            After a few seconds, I get the following output in my log.

            {noformat}

            14:14:20,266 ERROR [JGroupsTransport] Errors instantiating [org.jboss.ha.ispn.DefaultCacheContainerFactory$ChannelFactoryChannelLookup]!  Not using a channel lookup.

            14:14:20,266 INFO  [JGroupsTransport] Unable to use any JGroups configuration mechanisms provided in properties {channelLookup=org.jboss.ha.ispn.DefaultCacheContainerFactory$ChannelFactoryChannelLookup, stack=tcp, channelFactory=org.jboss.ha.framework.server.JChannelFactory@1943fd4d, channelId=DefaultPartition-HAPartition}.  Using default JGroups configuration!

            14:14:20,290 INFO  [JChannel] JGroups version: 2.11.0.GA

            14:14:20,766 WARN  [UDP] send buffer of socket java.net.DatagramSocket@787b217 was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)

            {noformat}

             

            I've gone ahead and tried to enable DEBUG logging during the boot sequence and get a little more info, but nothing there seems to help me out either (I don't necessarily understand what I see nor what I am looking for):

            {noformat}

            16:10:06,954 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Starting JGroups Channel

            16:10:06,956 DEBUG [org.jboss.ha.core.channelfactory.JChannelFactory] [org.jboss.ha.core.channelfactory.JChannelFactory] Stack config after adding singleton_name is TCP(oob_thread_pool.max_threads=200;oob_thread_pool.keep_alive_time=1000;max_bundle_size=64000;tcp_nodelay=true;diagnostics_addr=224.0.75.75;timer.num_threads=12;loopback=true;skip_suspected_members=true;thread_pool.min_threads=20;oob_thread_pool.rejection_policy=discard;thread_pool.max_threads=200;enable_diagnostics=true;thread_pool.enabled=true;start_port=7600;use_incoming_packet_handler=true;oob_thread_pool.enabled=true;send_buf_size=640000;use_send_queues=false;enable_bundling=false;thread_pool.rejection_policy=discard;discard_incompatible_packets=true;thread_pool.keep_alive_time=5000;diagnostics_port=7500;thread_pool.queue_enabled=true;singleton_name=tcp;max_bundle_timeout=30;oob_thread_pool.queue_enabled=false;sock_conn_timeout=300;oob_thread_pool.min_threads=20;recv_buf_size=20000000;thread_pool.queue_max_size=1000):JDBC_PING(datasource_jndi_name=java:jgroups-DS):MERGE2(min_interval=20000;max_interval=100000):FD_SOCK(start_port=57600):FD(max_tries=5;timeout=6000):VERIFY_SUSPECT(timeout=1500):BARRIER:pbcast.NAKACK(gc_lag=0;use_mcast_xmit=false;retransmit_timeout=300,600,1200,2400,4800;discard_delivered_msgs=true):UNICAST(timeout=300,600,1200,2400,3600):pbcast.STABLE(desired_avg_gossip=50000;max_bytes=400000;stability_delay=1000):VIEW_SYNC(avg_send_interval=10000):pbcast.GMS(print_local_addr=true;view_bundling=true;join_timeout=3000;view_ack_collection_timeout=5000;resume_task_timeout=7500):UFC(max_credits=2000000;ignore_synchronous_response=true):MFC(max_credits=2000000;ignore_synchronous_response=true):FRAG2(frag_size=60000):pbcast.STREAMING_STATE_TRANSFER:pbcast.FLUSH(start_flush_timeout=10000;timeout=0)

            16:10:06,974 INFO  [org.jgroups.JChannel] [org.jgroups.JChannel] JGroups version: 2.11.0.GA

            16:10:06,993 DEBUG [org.jgroups.conf.ClassConfigurator] [org.jgroups.conf.ClassConfigurator] Using jg-magic-map.xml as magic number file and jg-protocol-ids.xml for protocol IDs

            16:10:07,206 WARN  [org.jgroups.stack.Configurator] [org.jgroups.stack.Configurator] TCP property skip_suspected_members was deprecated and is ignored

            16:10:07,207 WARN  [org.jgroups.stack.Configurator] [org.jgroups.stack.Configurator] TCP property use_incoming_packet_handler was deprecated and is ignored

            16:10:07,208 WARN  [org.jgroups.stack.Configurator] [org.jgroups.stack.Configurator] TCP property start_port was deprecated and is ignored

            16:10:07,208 WARN  [org.jgroups.stack.Configurator] [org.jgroups.stack.Configurator] TCP property timer.num_threads was deprecated and is ignored

            16:10:07,252 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Errors instantiating [org.jboss.ha.ispn.DefaultCacheContainerFactory$ChannelFactoryChannelLookup]!  Not using a channel lookup.

            16:10:07,254 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] [org.infinispan.util.logging.AbstractLogImpl] Unable to use any JGroups configuration mechanisms provided in properties {channelLookup=org.jboss.ha.ispn.DefaultCacheContainerFactory$ChannelFactoryChannelLookup, stack=tcp, channelFactory=org.jboss.ha.framework.server.JChannelFactory@72f264c6, channelId=DefaultPartition-HAPartition}.  Using default JGroups configuration!

            16:10:07,265 INFO  [org.jgroups.JChannel] [org.jgroups.JChannel] JGroups version: 2.11.0.GA

            16:10:07,397 DEBUG [org.jgroups.protocols.pbcast.GMS] [org.jgroups.protocols.pbcast.GMS] changed role to org.jgroups.protocols.pbcast.ClientGmsImpl

            16:10:07,451 DEBUG [org.jgroups.stack.Configurator] [org.jgroups.stack.Configurator] set property UDP.diagnostics_addr to default value /224.0.75.75

            16:10:07,476 DEBUG [org.jgroups.protocols.FRAG2] [org.jgroups.protocols.FRAG2] received CONFIG event: {bind_addr=/127.0.0.1}

            16:10:07,516 DEBUG [org.jgroups.protocols.FRAG2] [org.jgroups.protocols.FRAG2] received CONFIG event: {flush_supported=true}

            16:10:07,519 DEBUG [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] creating sockets

            16:10:07,520 DEBUG [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] sockets will use interface 127.0.0.1

            16:10:07,524 WARN  [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] send buffer of socket java.net.DatagramSocket@71f3668b was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)

            16:10:07,525 WARN  [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] receive buffer of socket java.net.DatagramSocket@71f3668b was set to 20MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)

            16:10:07,526 WARN  [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] send buffer of socket java.net.MulticastSocket@22d16a1f was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)

            16:10:07,526 WARN  [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] receive buffer of socket java.net.MulticastSocket@22d16a1f was set to 25MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)

            16:10:07,527 DEBUG [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP] socket information:

            , mcast_addr=228.6.7.8:46655, bind_addr=/127.0.0.1, ttl=2

            sock: bound to 127.0.0.1:46536, receive buffer size=131071, send buffer size=131071

            mcast_sock: bound to 127.0.0.1:46655, send buffer size=131071, receive buffer size=131071

            16:10:07,548 DEBUG [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP$PacketReceiver] created unicast receiver thread

            16:10:07,549 DEBUG [org.jgroups.protocols.UDP] [org.jgroups.protocols.UDP$PacketReceiver] created multicast receiver thread

            16:10:10,562 DEBUG [org.jgroups.protocols.pbcast.GMS] [org.jgroups.protocols.pbcast.ClientGmsImpl] initial_mbrs are []

            16:10:10,563 DEBUG [org.jgroups.protocols.pbcast.GMS] [org.jgroups.protocols.pbcast.ClientGmsImpl] no initial members discovered: creating group as first member

            {noformat}

             

            So it would seem to me as though something is not working.  However, I am not quite sure what, nor do I have any understanding what the errors mean.

             

            Is there anyway I can get further debug information to try and understand why this configuration is non functional?  It shouldn't be the mysql connection as I am able to connect from the command line using the same parameters and credentials.  According to the JDBC_PING wiki doc page, the connector is supposed to automatically create the needed table if it doens't already exist.  But I check my database, and it is empty.

             

            Any help would be greatly appreciated!


            Thanks,

             

            Eric

             

            Message was edited by: Eric B Added debug log info

            • 48. Re: Infinispan on EC2
              benze

              Hi Sanne,


              I am having a lot of difficulty configuring for JDBC_PING.  I read the wiki page, but I am obviously missing something.  Is there any additional documentation on how to configure JB AS6 for cluster mode using JDBC_PING?  I tried reading

              http://www.redhat.com/summit/2011/presentations/jbossworld/whats_new/wednesday/ban_w_310_running_in_the_cloud.pdf,but that didn't help either.  Am still getting strange error logs...


              Thanks,


              Eric

              • 49. Re: Infinispan on EC2
                galder.zamarreno

                Eric, I'd suggest you open a new discussion with your JDBC_PING issues and mark it as a question.

                • 50. Re: Infinispan on EC2
                  sannegrinovero

                  Eric, sorry for noticing this so late. From the JGroups mailing list I'm understanding that you solved the problem by opening the firewall, correct?

                  If you have more issues please open new threads so I might actually see them quicker.

                   

                  thanks,

                  Sanne

                  • 51. Re: Infinispan on EC2
                    kasik

                    Hi!

                     

                    I want to run my application in AWS EC2. For that I created two instances with Tomcat 7 servers. Previously I used Infinispan in LAN with UDP. I attached jgroups.xml with previous configuration. What I need to change in jgroups.xml for using it in EC2? I tried to use TCP with S3_PING, but it doesn't work.

                     

                    Thanks in advance!

                    1 2 3 4 Previous Next