3 Replies Latest reply on Apr 17, 2007 9:24 AM by canty28

    Horizontal Clustering not working _ need help

    canty28

      Hi Everyone,

      I am currently running JBOSS 4.0.3 SP1. Currently i am configuring UAT environment. I am trying to acheive horizontal clustering. Box A has 3 jvm's -bduat_server1, bduat_server2,bduat_server3. Box B has 3 JVM's - bduat2_server1,bduat2_server2,bduat2_server3. Currently vertical clustering works fine. When i start up all instances on Box A, all the instances pick up the ear file from the farm. But when box B is started it is being recognized but the ear file doesnt get pushed on from Box A to Box B.

      I am currently pasting the logs of Box B .

      =========================================================================

      JBoss Bootstrap Environment

      JBOSS_HOME: /servers/jboss

      JAVA: /usr/java/j2sdk1.4.2_11/bin/java

      JAVA_OPTS: -Djava.net.preferIPv4Stack=true -server -Xms512m -Xmx768m -Dprogram.name=run.sh

      CLASSPATH: /servers/jboss/bin/run.jar:/usr/java/j2sdk1.4.2_11/lib/tools.jar

      =========================================================================

      17:50:20,127 INFO [Server] Starting JBoss (MX MicroKernel)...
      17:50:20,129 INFO [Server] Release ID: JBoss [Zion] 4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231054)
      17:50:20,130 INFO [Server] Home Dir: /servers/jboss
      17:50:20,131 INFO [Server] Home URL: file:/servers/jboss/
      17:50:20,131 INFO [Server] Patch URL: null
      17:50:20,132 INFO [Server] Server Name: bdgkmuat2_server1
      17:50:20,132 INFO [Server] Server Home Dir: /servers/jboss/server/bdgkmuat2_server1
      17:50:20,133 INFO [Server] Server Home URL: file:/servers/jboss/server/bdgkmuat2_server1/
      17:50:20,133 INFO [Server] Server Temp Dir: /servers/jboss/server/bdgkmuat2_server1/tmp
      17:50:20,133 INFO [Server] Root Deployment Filename: jboss-service.xml
      17:50:20,615 INFO [ServerInfo] Java version: 1.4.2_11,Sun Microsystems Inc.
      17:50:20,616 INFO [ServerInfo] Java VM: Java HotSpot(TM) Server VM 1.4.2_11-b06,Sun Microsystems Inc.
      17:50:20,616 INFO [ServerInfo] OS-System: Linux 2.6.5-7.282-smp,i386
      17:50:21,224 INFO [Server] Core system initialized
      17:50:23,976 INFO [ServiceBindingManager] Initializing store
      17:50:23,976 INFO [ServiceBindingManager] Using StoreURL: file:/servers/jboss/docs/examples/binding-manager/sample-bindings.xml
      17:50:24,711 INFO [WebService] Using RMI server codebase: http://ss1kmbat02.usa.wachovia.net:8183/
      17:50:24,792 INFO [Log4jService$URLWatchTimerTask] Configuring from URL: resource:log4j.xml
      17:50:25,223 INFO [NamingService] Started jndi bootstrap jnpPort=1199, rmiPort=1198, backlog=50, bindAddress=/0.0.0.0, Client SocketFactory=null, Server SocketFactory=org.jboss.net.sockets.DefaultSocketFactory@ad093076
      17:50:31,077 INFO [SnmpAgentService] SNMP agent going active
      17:50:32,055 INFO [DefaultPartition] Initializing
      17:50:32,120 INFO [STDOUT]
      -------------------------------------------------------
      GMS: address is 0.0.0.0:32969 (additional data: 17 bytes)
      -------------------------------------------------------
      17:50:34,168 INFO [DefaultPartition] Number of cluster members: 1
      17:50:34,168 INFO [DefaultPartition] Other members: 0
      17:50:34,171 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
      17:50:34,178 INFO [DefaultPartition] New cluster view for partition DefaultPartition (id: 0, delta: 0) : [170.13.57.43:1199]
      17:50:34,187 INFO [DefaultPartition] I am (170.13.57.43:1199) received membershipChanged event:
      17:50:34,187 INFO [DefaultPartition] Dead members: 0 ([])
      17:50:34,188 INFO [DefaultPartition] New Members : 0 ([])
      17:50:34,188 INFO [DefaultPartition] All Members : 1 ([170.13.57.43:1199])
      17:50:34,260 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1200, backlog=50, bindAddress=/0.0.0.0
      17:50:34,279 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /0.0.0.0:1102, group=230.0.0.4, HA-JNDI address=170.13.57.43:1200
      17:50:35,715 INFO [orb] ORB run
      17:50:35,849 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000D8000102000000000D3137302E31332E35372E343300000E2C000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C00000000000100010000000105010001000101090000000105010001000000210000005000000000000000010000000000000024000000200000007E00000000000000010000000D3137302E31332E35372E343300000DC9000000000000000000000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
      17:50:35,994 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000D8000102000000000D3137302E31332E35372E343300000E2C000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C00000000000100010000000105010001000101090000000105010001000000210000005000000000000000010000000000000024000000200000007E00000000000000010000000D3137302E31332E35372E343300000DC9000000000000000000000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
      17:50:37,080 INFO [MailService] Mail Service bound to java:/Mail
      17:50:37,614 INFO [TreeCache] setting cluster properties from xml to: UDP(ip_mcast=true;ip_ttl=8;loopback=false;mcast_addr=230.1.2.7;mcast_port=45577;mcast_recv_buf_size=80000;mcast_send_buf_size=150000;ucast_recv_buf_size=80000;ucast_send_buf_size=150000):PING(down_thread=false;num_initial_members=3;timeout=2000;up_thread=false):MERGE2(max_interval=20000;min_interval=10000):FD_SOCK:VERIFY_SUSPECT(down_thread=false;timeout=1500;up_thread=false):pbcast.NAKACK(down_thread=false;gc_lag=50;max_xmit_size=8192;retransmit_timeout=600,1200,2400,4800;up_thread=false):UNICAST(down_thread=false;min_threshold=10;timeout=600,1200,2400;window_size=100):pbcast.STABLE(desired_avg_gossip=20000;down_thread=false;up_thread=false):FRAG(down_thread=false;frag_size=8192;up_thread=false):pbcast.GMS(join_retry_timeout=2000;join_timeout=5000;print_local_addr=true;shun=true):pbcast.STATE_TRANSFER(down_thread=true;up_thread=true)
      17:50:37,699 INFO [TreeCache] interceptor chain is:
      class org.jboss.cache.interceptors.CallInterceptor
      class org.jboss.cache.interceptors.LockInterceptor
      class org.jboss.cache.interceptors.UnlockInterceptor
      class org.jboss.cache.interceptors.ReplicationInterceptor
      17:50:37,699 INFO [TreeCache] cache mode is REPL_ASYNC
      17:50:37,733 INFO [STDOUT]
      -------------------------------------------------------
      GMS: address is ss1kmbat02:32972
      -------------------------------------------------------
      17:50:39,859 INFO [TreeCache] viewAccepted(): new members: [msilsh0012v010:32945, ss1kmbat02:32972]
      17:50:42,796 ERROR [FD_SOCK] received null cache; retrying
      17:50:44,799 INFO [TreeCache] state could not be retrieved (must be first member in group)
      17:50:44,991 INFO [Embedded] Catalina naming disabled
      17:50:45,568 INFO [Http11Protocol] Initializing Coyote HTTP/1.1 on http-0.0.0.0-8180
      17:50:45,570 INFO [Catalina] Initialization processed in 512 ms
      17:50:45,572 INFO [StandardService] Starting service jboss.web
      17:50:45,575 INFO [StandardEngine] Starting Servlet Engine: Apache Tomcat/5.5
      17:50:45,596 INFO [StandardHost] XML validation disabled
      17:50:45,618 INFO [Catalina] Server startup in 48 ms
      17:50:45,779 INFO [TomcatDeployer] deploy, ctxPath=/invoker, warUrl=.../deploy/httpha-invoker.sar/invoker.war/
      17:50:46,097 INFO [WebappLoader] Dual registration of jndi stream handler: factory already defined
      17:50:46,311 ERROR [FD_SOCK] received null cache; retrying
      17:50:46,750 INFO [TomcatDeployer] deploy, ctxPath=/ws4ee, warUrl=.../tmp/deploy/tmp30698jboss-ws4ee.war/
      17:50:46,905 INFO [TomcatDeployer] deploy, ctxPath=/, warUrl=.../deploy/jbossweb-tomcat55.sar/ROOT.war/
      17:50:47,015 INFO [TomcatDeployer] deploy, ctxPath=/juddi, warUrl=.../deploy/juddi-service.sar/juddiws.war/
      17:50:47,204 INFO [RegistryServlet] Loading jUDDI configuration.
      17:50:47,207 INFO [RegistryServlet] Resources loaded from: /WEB-INF/juddi.properties
      17:50:47,207 INFO [RegistryServlet] Initializing jUDDI components.
      17:50:47,530 INFO [TomcatDeployer] deploy, ctxPath=/web-console, warUrl=.../deploy/management/console-mgr.sar/web-console.war/
      17:50:48,050 INFO [TomcatDeployer] deploy, ctxPath=/jbossmq-httpil, warUrl=.../deploy-hasingleton/jms/jbossmq-httpil.sar/jbossmq-httpil.war/
      17:50:48,668 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-local-jdbc.rar
      17:50:48,960 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-xa-jdbc.rar
      17:50:49,119 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-local-jdbc.rar
      17:50:49,337 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-xa-jdbc.rar
      17:50:49,649 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jms/jms-ra.rar
      17:50:49,722 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/mail-ra.rar
      17:50:49,831 ERROR [FD_SOCK] received null cache; retrying
      17:50:50,484 INFO [TreeCache] received the state (size=192 bytes)
      17:50:50,537 INFO [TreeCache] transient state: 140 bytes
      17:50:50,540 INFO [TreeCache] setting transient state
      17:50:50,598 INFO [TreeCache] locking the old tree
      17:50:50,601 INFO [TreeCache] locking the old tree was successful
      17:50:50,601 INFO [TreeCache] setting the transient state was successful
      17:50:50,601 INFO [TreeCache] forcing release of all locks in old tree
      17:50:50,991 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:name=DefaultDS,service=DataSourceBinding' to JNDI name 'java:DefaultDS'
      17:50:51,411 INFO [A] Bound to JNDI name: queue/A
      17:50:51,416 INFO [B] Bound to JNDI name: queue/B
      17:50:51,421 INFO [C] Bound to JNDI name: queue/C
      17:50:51,424 INFO [D] Bound to JNDI name: queue/D
      17:50:51,428 INFO [ex] Bound to JNDI name: queue/ex
      17:50:51,467 INFO [testTopic] Bound to JNDI name: topic/testTopic
      17:50:51,469 INFO [securedTopic] Bound to JNDI name: topic/securedTopic
      17:50:51,469 INFO [testDurableTopic] Bound to JNDI name: topic/testDurableTopic
      17:50:51,470 INFO [testQueue] Bound to JNDI name: queue/testQueue
      17:50:51,539 INFO [UILServerILService] JBossMQ UIL service available at : /0.0.0.0:8193
      17:50:51,601 INFO [DLQ] Bound to JNDI name: queue/DLQ
      17:50:51,832 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:name=JmsXA,service=ConnectionFactoryBinding' to JNDI name 'java:JmsXA'
      17:50:51,862 INFO [TomcatDeployer] deploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
      17:50:52,305 INFO [Http11Protocol] Starting Coyote HTTP/1.1 on http-0.0.0.0-8180
      17:50:52,416 INFO [ChannelSocket] JK: ajp13 listening on /0.0.0.0:8109
      17:50:52,429 INFO [JkMain] Jk running ID=0 time=0/22 config=null
      17:50:52,435 INFO [Server] JBoss (MX MicroKernel) [4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231054)] Started in 32s:299ms


      These were the logs from box A

      2007-04-16 17:52:18,995 INFO [org.apache.coyote.http11.Http11Protocol] Starting Coyote HTTP/1.1 on http-0.0.0.0-8180
      2007-04-16 17:52:19,142 INFO [org.apache.jk.common.ChannelSocket] JK: ajp13 listening on /0.0.0.0:8109
      2007-04-16 17:52:19,155 INFO [org.apache.jk.server.JkMain] Jk running ID=0 time=1/33 config=null
      2007-04-16 17:52:19,163 INFO [org.jboss.system.server.Server] JBoss (MX MicroKernel) [4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231054)] Started in 2m:1s:694ms
      2007-04-16 17:52:54,955 INFO [org.jboss.cache.TreeCache] viewAccepted(): new members: [ss1kmbat01:32945, msilsh0012v011:32972]
      2007-04-16 17:53:05,412 INFO [org.jboss.cache.TreeCache] locking the tree to obtain transient state
      2007-04-16 17:53:05,412 DEBUG [org.jboss.cache.Node] acquiring RL: fqn=/, caller=Thread[MessageDispatcher up processing thread,5,jboss], lock=
      2007-04-16 17:53:05,413 DEBUG [org.jboss.cache.Node] acquired RL: fqn=/, caller=Thread[MessageDispatcher up processing thread,5,jboss], lock=read owners=[Thread[MessageDispatcher up processing thread,5,jboss]]

      Can someone help me in this issue. Your help would be greatly appreciated.

      Thanks
      SK

        • 1. Re: Horizontal Clustering not working _ need help
          canty28

          Here is my tc5-Cluster.xml


          <!--
          The default UDP stack:
          - If you have a multihomed machine, set the UDP protocol's bind_addr attribute to the
          appropriate NIC IP address, e.g bind_addr="192.168.0.2".
          - On Windows machines, because of the media sense feature being broken with multicast
          (even after disabling media sense) set the UDP protocol's loopback attribute to true
          -->

          <UDP mcast_addr="${jboss.partition.udpGroup:230.1.2.7}" mcast_port="45577"
          ip_ttl="8" ip_mcast="true"
          mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
          ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
          loopback="false" />
          <PING timeout="2000" num_initial_members="3"
          up_thread="false" down_thread="false"/>
          <MERGE2 min_interval="10000" max_interval="20000"/>
          <FD_SOCK/>
          <VERIFY_SUSPECT timeout="1500"
          up_thread="false" down_thread="false"/>
          <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
          max_xmit_size="8192" up_thread="false" down_thread="false"/>
          <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10"
          down_thread="false"/>
          <pbcast.STABLE desired_avg_gossip="20000"
          up_thread="false" down_thread="false"/>
          <FRAG frag_size="8192"
          down_thread="false" up_thread="false"/>
          <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
          shun="true" print_local_addr="true"/>
          <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>


          • 2. Re: Horizontal Clustering not working _ need help
            canty28

            Please also find my cluster-service.xml


            <UDP mcast_addr="${jboss.partition.udpGroup:228.1.2.3}" mcast_port="45566"
            ip_ttl="8" ip_mcast="true"
            mcast_send_buf_size="800000000" mcast_recv_buf_size="150000000"
            ucast_send_buf_size="800000000" ucast_recv_buf_size="150000000"
            loopback="false" bind_addr="${jboss.bind.address}" />
            <PING timeout="2000" num_initial_members="3"
            up_thread="true" down_thread="true"/>
            <MERGE2 min_interval="10000" max_interval="20000"/>
            <FD shun="true" up_thread="true" down_thread="true"
            timeout="2500" max_tries="5"/>
            <VERIFY_SUSPECT timeout="3000" num_msgs="3"
            up_thread="true" down_thread="true"/>
            <pbcast.NAKACK gc_lag="50" retransmit_timeout="300,600,1200,2400,4800"
            max_xmit_size="8192"
            up_thread="true" down_thread="true"/>
            <UNICAST timeout="300,600,1200,2400,4800" window_size="100" min_threshold="10"
            down_thread="true"/>
            <pbcast.STABLE desired_avg_gossip="20000"
            up_thread="true" down_thread="true"/>
            <FRAG frag_size="8192"
            down_thread="true" up_thread="true"/>
            <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
            shun="true" print_local_addr="true"/>
            <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>



            The reason why we had to increase the mcast and ucast buffer sizes was mainly because the war file which we were deploying is around 107 M. Please help me in this matter. I have been following the forums. I have also tried adding route and also adding bind address to the cluster-service.xml but that eventually throws an error.We are running JBOSS 4.0.3SP1 on SLES9. Any help in this matter or previous experiences would be helpful.

            • 3. Re: Horizontal Clustering not working _ need help
              canty28

              Here is the error i receive when i actually put the binding address in the cluster-service.xml file

              09:18:17,523 ERROR [JChannel] exception: java.lang.Exception: exception caused by UDP.start(): java.lang.Exception: UDP.createSockets(): cannot list on any port in range 0-1
              09:18:17,528 WARN [ServiceController] Problem starting service jboss:service=DefaultPartition
              ChannelException: java.lang.Exception: exception caused by UDP.start(): java.lang.Exception: UDP.createSockets(): cannot list on any port in range 0-1
               at org.jgroups.JChannel.connect(JChannel.java:328)
               at org.jboss.ha.framework.server.ClusterPartition.startService(ClusterPartition.java:298)
               at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:274)
               at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:230)
               at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:943)
               at $Proxy0.start(Unknown Source)
               at org.jboss.system.ServiceController.start(ServiceController.java:428)
               at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
               at $Proxy4.start(Unknown Source)
               at org.jboss.deployment.SARDeployer.start(SARDeployer.java:285)
               at org.jboss.deployment.MainDeployer.start(MainDeployer.java:989)
               at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:790)
               at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:753)
               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
               at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:118)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
               at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:127)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
               at $Proxy10.deploy(Unknown Source)
               at org.jboss.deployment.scanner.URLDeploymentScanner.deploy(URLDeploymentScanner.java:319)
               at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:507)
               at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:192)
               at org.jboss.deployment.scanner.AbstractDeploymentScanner.startService(AbstractDeploymentScanner.java:265)
               at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:274)
               at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:230)
               at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:943)
               at $Proxy0.start(Unknown Source)
               at org.jboss.system.ServiceController.start(ServiceController.java:428)
               at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
               at $Proxy4.start(Unknown Source)
               at org.jboss.deployment.SARDeployer.start(SARDeployer.java:285)
               at org.jboss.deployment.MainDeployer.start(MainDeployer.java:989)
               at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:790)
               at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:753)
               at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:737)
               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
               at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
               at java.lang.reflect.Method.invoke(Method.java:324)
               at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
               at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
               at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:118)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
               at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:127)
               at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
               at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
               at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
               at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
               at $Proxy5.deploy(Unknown Source)
               at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:453)
               at org.jboss.system.server.ServerImpl.start(ServerImpl.java:330)
               at org.jboss.Main.boot(Main.java:187)
               at org.jboss.Main$1.run(Main.java:438)
               at java.lang.Thread.run(Thread.java:534)