12 Replies Latest reply on May 7, 2008 8:03 AM by praveena chalamcharla

    Unable to run multiple cluster nodes.

    praveena chalamcharla Newbie

      Hi All,
      Im running JBoss 4.2.2.GA, want to cluster on two machines for each holding multiple JBoss instances.
      I took httpd 2.0.63 Apache server as load balancer.
      I followed JBoss 4 clustering.pdf document as reference.
      In order to add session replication features, I am unable to find the related and specified XML file.
      According to the document.
      Kindly please let me know the solution for my problem or url or link for a related and exact document.
      Please respond me at the earliest.

        • 1. Re: Unable to run multiple cluster nodes.
          Brian Stansberry Master

          Not sure which PDF you are talking about. For 4.2.2 make sure you use http://labs.jboss.com/file-access/default/members/jbossas/freezone/docs/Clustering_Guide/beta422/pdf/Clustering_Guide.pdf .

          Also not sure what file you are talking about. A jboss-web.xml? That's one you create yourself in place in your war's WEB-INF directory.

          • 2. Re: Unable to run multiple cluster nodes.
            praveena chalamcharla Newbie

            Hi,

            Thank you for your document.

            As I have mentioned I want to cluster nodes belonging to two different machines each of which have different jboss servers running seperately on different ports. For this, I have copied the Jboss home directory and named seperately as Jboss1, Jboss2, Jboss3,.. I have also changed different port numbers where necessary, like bind address. Now, I have used the httpd 2.0.63 apache load balancer for clustering all these nodes. Initially, all the nodes seeming working well. But, when one node drops, I found the browser displaying the "Session expiry". sometimes, I am even experiencing "Service unavailable 503 " error with load balancer.

            This is my workers.properties file I have attached. (Please check the AJP 1.3 Connector port values too.)
            # Define list of workers that will be used
            # for mapping requests
            # The configuration directives are valid
            # for the mod_jk version 1.2.18 and later
            #
            worker.list=loadbalancer,status

            # Define Node1
            # modify the host as your host IP or DNS name.
            worker.node1.port=18009
            worker.node1.host=10.20.220.3
            worker.node1.type=ajp13
            worker.node1.lbfactor=1
            # worker.node1.connection_pool_size=10 (1)

            # Define Node2
            # modify the host as your host IP or DNS name.
            worker.node2.port=28009
            worker.node2.host= 10.20.220.3
            worker.node2.type=ajp13
            worker.node2.lbfactor=1
            # worker.node2.connection_pool_size=10 (1)

            # Define Node3
            # modify the host as your host IP or DNS name.
            worker.node3.port=38009
            worker.node3.host=10.20.220.3
            worker.node3.type=ajp13
            worker.node3.lbfactor=1
            # worker.node3.connection_pool_size=10 (1)

            # Define Node4
            # modify the host as your host IP or DNS name.
            worker.node4.port=48009
            worker.node4.host=10.20.220.3
            worker.node4.type=ajp13
            worker.node4.lbfactor=1
            # worker.node4.connection_pool_size=10 (1)

            # Define Node5
            # modify the host as your host IP or DNS name.
            worker.node5.port=58009
            worker.node5.host=10.20.220.15
            worker.node5.type=ajp13
            worker.node5.lbfactor=1
            # worker.node5.connection_pool_size=10 (1)

            # Define Node6
            # modify the host as your host IP or DNS name.
            worker.node6.port=68009
            worker.node6.host=10.20.220.15
            worker.node6.type=ajp13
            worker.node6.lbfactor=1
            # worker.node6.connection_pool_size=10 (1)


            # Load-balancing behaviour
            worker.loadbalancer.type=lb
            worker.loadbalancer.balance_workers=node1,node2,node3,node4,node5,node6
            worker.loadbalancer.sticky_session=1
            #worker.list=loadbalancer

            # Status worker for managing load balancer
            worker.status.type=status



            My uriworkermap.properties file.
            # Simple worker configuration file
            #

            # Mount the Servlet context to the ajp13 worker

            /casper-2.1=loadbalancer

            /casper-2.1**/*=loadbalancer

            /web-console=loadbalancer
            /web-console/*=loadbalancer




            These are my issues after reading the document you have sent. (http://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Clustering_Guide/beta422/pdf/Clustering_Guide.pdf)

            1. Under the Http Session clustering, the 7th sub topic, Enabling session replication in your application, you have asked to configure session replication using the replication-config element
            in the jboss-web.xml. But, I found 6 different files with that name under "all" configuration. Please let me know where I need to add this attribute.

            Thank you.

            Praveena Chalamcharla,
            Securview.

            • 3. Re: Unable to run multiple cluster nodes.
              Brian Stansberry Master

              A jboss-web.xml file is the JBoss-specific deployment descriptor you create yourself and place in your war's WEB-INF directory.

              • 4. Re: Unable to run multiple cluster nodes.
                praveena chalamcharla Newbie


                As a continution message, I am getting that session expiration message. And until, I reset the cookie, I could not get back the load balancer again. But, this is violating the clustering completely.

                PS: As my office timings are entirely reverse to yours, I could not contact you online. If there is any alternative to this, please let me know.


                Thank you.

                • 5. Re: Unable to run multiple cluster nodes.
                  praveena chalamcharla Newbie

                  Hi,

                  Could not solve the issue. Please find some time to make out this issue.


                  When I have included attribute in my application and added jboss-web.xml as follows in the WEB-INF of my application as:

                  <jboss-web>

                  <replication-config>
                  <replication-trigger>SET_AND_NON_PRIMITIVE_GET
                  <replication-granularity>SESSION</replication-granularity>
                  <replication-field-batch-mode>true</replication-field-batch-mode>
                  </replication-config>

                  </jboss-web>


                  But, I am finding http 404 status when running the Load balancer to cluster the servers. http://10.20.220.135/casper-2.1/app

                  And when I am trying to run the servers independently like http://10.20.220.15:9999/casper-2.1/app, also I could not find the application running and also got the similar error. Prior to my adding of attribute and jboss-web.xml, the server independently could run well.

                  Please let me know what might have gone wrong with this.

                  I have got the server.log file, when I am running both the servers and the load balancer.

                  2008-04-25 18:41:59,632 INFO [org.jboss.cache.statetransfer.StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
                  2008-04-25 18:42:00,650 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...
                  2008-04-25 18:42:04,333 INFO [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.casper] New cluster view for partition casper (id: 5, delta: 1) : [10.20.220.15:1099, 10.20.220.135:1099]
                  2008-04-25 18:42:04,334 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] membership changed from 1 to 2
                  2008-04-25 18:42:04,334 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] Begin notifyListeners, viewID: 5
                  2008-04-25 18:42:04,334 INFO [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.casper] I am (10.20.220.15:1099) received membershipChanged event:
                  2008-04-25 18:42:04,334 INFO [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.casper] Dead members: 0 ([])
                  2008-04-25 18:42:04,334 INFO [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.casper] New Members : 1 ([10.20.220.135:1099])
                  2008-04-25 18:42:04,334 INFO [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.casper] All Members : 2 ([10.20.220.15:1099, 10.20.220.135:1099])
                  2008-04-25 18:42:04,334 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] End notifyListeners, viewID: 5
                  2008-04-25 18:42:04,343 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] getState called.
                  2008-04-25 18:42:04,343 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] getState for DistributedState
                  2008-04-25 18:42:04,343 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] getState for HASessionStateTransfer
                  2008-04-25 18:42:04,343 DEBUG [org.jboss.ha.hasessionstate.server.HASessionStateImpl./HASessionState/Default] Building and returning state of HASessionState
                  2008-04-25 18:42:04,344 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] getState for HAJNDI
                  2008-04-25 18:42:04,344 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.casper] getState for DistributedReplicantManager
                  2008-04-25 18:42:04,458 DEBUG [org.jboss.ha.framework.server.HARMIServerImpl$RefreshProxiesHATarget] replicantsChanged 'HAJNDI' to 2 (intra-view id: -1270392965)
                  2008-04-25 18:42:04,713 DEBUG [org.jboss.cache.invalidation.bridges.JGCacheInvalidationBridge] The list of replicant for the JG bridge has changed, computing and updating local info...
                  2008-04-25 18:42:04,713 DEBUG [org.jboss.cache.invalidation.bridges.JGCacheInvalidationBridge] ... No bridge info was associated to this node
                  2008-04-25 18:42:04,715 DEBUG [org.jboss.cache.invalidation.bridges.JGCacheInvalidationBridge] Updating list of invalidation groups that are bridged...
                  2008-04-25 18:42:04,715 DEBUG [org.jboss.cache.invalidation.bridges.JGCacheInvalidationBridge] ... nothing needs to be bridged.
                  2008-04-25 18:42:07,016 INFO [org.jboss.cache.TreeCache] viewAccepted(): [10.20.220.15:33117|5] [10.20.220.15:33117, 10.20.220.135:32833]
                  2008-04-25 18:42:09,180 INFO [org.jboss.cache.TreeCache] viewAccepted(): [10.20.220.15:33119|5] [10.20.220.15:33119, 10.20.220.135:32835]
                  2008-04-25 18:42:10,650 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...
                  2008-04-25 18:42:20,651 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...


                  And the server console is showing a message like

                  ObjectName: jboss:service=FarmMember,partition=DefaultPartition
                  State: CONFIGURED
                  I Depend On:
                  jboss:service=DefaultPartition
                  jboss.web:service=WebServer
                  jboss.system:service=MainDeployer

                  --- MBEANS THAT ARE THE ROOT CAUSE OF THE PROBLEM ---
                  ObjectName: jboss:service=DefaultPartition
                  State: NOTYETINSTALLED
                  Depends On Me:
                  jboss.ha:service=HASingletonDeployer
                  jboss:service=FarmMember,partition=DefaultPartition


                  18:40:42,374 INFO [Http11Protocol] Starting Coyote HTTP/1.1 on http-10.20.220.135-9999
                  18:40:42,392 INFO [AjpProtocol] Starting Coyote AJP/1.3 on ajp-10.20.220.135-8009
                  18:40:42,403 INFO [Server] JBoss (MX MicroKernel) [4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA date=200710221139)] Started in 23s:486ms


                  Note: I have changed the DefaultPartition as "casper" in the cluster-service.xml file as

                  I am also finding an error message in the error log of the apache server as:

                  [Fri Apr 25 17:47:50 2008] [error] [client 10.20.220.15] File does not exist: /usr/local/apache2/htdocs/favicon.ico


                  I have followed the same procedure given in the doc.


                  Thank you..

                  Praveena,
                  Securview

                  • 6. Re: Unable to run multiple cluster nodes.
                    Brian Stansberry Master

                     

                    /casper-2.1**/*=loadbalancer


                    should be

                    /casper-2.1/*=loadbalancer


                    No **

                    To get rid of the startup failures you posted, don't change the partition name in cluster-service.xml. Restore it to the original value of ${jboss.partition.name:DefaultPartition}. You can then start JBoss like this to have your partition named 'casper'

                    ./run.sh -Djboss.partition.name=casper -c myConfig

                    As a convenience there is a switch that does that exact same thing:

                    ./run.sh -g casper -c myConfig

                    As to why browsing to http://10.20.220.15:9999/casper-2.1/app no longer works, I can't say for certain based on what you've posted. I assume 9999 is the port the Tomcat HttpConnector is listening on; i.e. you've changed it from 8080.

                    Only thing I see is that your posted "replication-trigger" element is missing the closing tag.

                    • 7. Re: Unable to run multiple cluster nodes.
                      praveena chalamcharla Newbie

                      Hi,
                      As it was a weekend, I could not respond to your suggestion. I have downloaded a new jboss server and load balancer again and made the necessary changes. But I am very happy with your response. Even then, I got many issues. Finally,

                      I have done the necessary modifications you have asked me to do for the "uriworkermap.properties" file. I have run the servers with the command, ./run.sh -Djboss.partition.name=casper -c all -b 10.20.220.135, as I am running directly from "all" config.
                      Please find below the console of the server instance, when I have run 2 server instances running with different IP's and httpd 2.0.63 apache load balancer.



                      [root@localhost bin]# ./run.sh -Djboss.partition.name=casper -c all -b 10.20.220.135
                      =========================================================================

                      JBoss Bootstrap Environment

                      JBOSS_HOME: /home/karsh/jboss-4.2.2.GA

                      JAVA: java

                      JAVA_OPTS: -Dprogram.name=run.sh -Xms128m -Xmx512m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true

                      CLASSPATH: /home/karsh/jboss-4.2.2.GA/bin/run.jar

                      =========================================================================

                      15:09:58,886 INFO [Server] Starting JBoss (MX MicroKernel)...
                      15:09:58,887 INFO [Server] Release ID: JBoss [Trinity] 4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA date=200710221139)
                      15:09:58,888 INFO [Server] Home Dir: /home/karsh/jboss-4.2.2.GA
                      15:09:58,888 INFO [Server] Home URL: file:/home/karsh/jboss-4.2.2.GA/
                      15:09:58,889 INFO [Server] Patch URL: null
                      15:09:58,889 INFO [Server] Server Name: all
                      15:09:58,889 INFO [Server] Server Home Dir: /home/karsh/jboss-4.2.2.GA/server/all
                      15:09:58,889 INFO [Server] Server Home URL: file:/home/karsh/jboss-4.2.2.GA/server/all/
                      15:09:58,890 INFO [Server] Server Log Dir: /home/karsh/jboss-4.2.2.GA/server/all/log
                      15:09:58,890 INFO [Server] Server Temp Dir: /home/karsh/jboss-4.2.2.GA/server/all/tmp
                      15:09:58,890 INFO [Server] Root Deployment Filename: jboss-service.xml
                      15:09:59,159 INFO [ServerInfo] Java version: 1.7.0,Sun Microsystems Inc.
                      15:09:59,159 INFO [ServerInfo] Java VM: IcedTea Client VM 1.7.0-b21,Sun Microsystems Inc.
                      15:09:59,159 INFO [ServerInfo] OS-System: Linux 2.6.23.1-42.fc8,i386
                      15:09:59,472 INFO [Server] Core system initialized
                      15:10:01,942 INFO [WebService] Using RMI server codebase: http://10.20.220.135:8083/
                      15:10:01,944 INFO [Log4jService$URLWatchTimerTask] Configuring from URL: resource:jboss-log4j.xml
                      15:10:02,350 INFO [TransactionManagerService] JBossTS Transaction Service (JTA version) - JBoss Inc.
                      15:10:02,350 INFO [TransactionManagerService] Setting up property manager MBean and JMX layer
                      15:10:02,504 INFO [TransactionManagerService] Starting recovery manager
                      15:10:02,588 INFO [TransactionManagerService] Recovery manager started
                      15:10:02,588 INFO [TransactionManagerService] Binding TransactionManager JNDI Reference
                      15:10:04,706 INFO [EJB3Deployer] Starting java:comp multiplexer
                      15:10:05,692 INFO [STDOUT]
                      -------------------------------------------------------
                      GMS: address is 10.20.220.135:32880
                      -------------------------------------------------------
                      15:10:07,834 INFO [TreeCache] viewAccepted(): [10.20.220.15:33215|1] [10.20.220.15:33215, 10.20.220.135:32880]
                      15:10:07,856 INFO [TreeCache] TreeCache local address is 10.20.220.135:32880
                      15:10:07,911 INFO [TreeCache] received the state (size=1024 bytes)
                      15:10:07,934 INFO [TreeCache] state was retrieved successfully (in 78 milliseconds)
                      15:10:07,935 INFO [TreeCache] parseConfig(): PojoCacheConfig is empty
                      15:10:08,015 INFO [STDOUT] no object for null
                      15:10:08,019 INFO [STDOUT] no object for null
                      15:10:08,043 INFO [STDOUT] no object for null
                      15:10:08,060 INFO [STDOUT] no object for {urn:jboss:bean-deployer}supplyType
                      15:10:08,069 INFO [STDOUT] no object for {urn:jboss:bean-deployer}dependsType
                      15:10:09,673 INFO [NativeServerConfig] JBoss Web Services - Native
                      15:10:09,673 INFO [NativeServerConfig] jbossws-native-2.0.1.SP2 (build=200710210837)
                      15:10:10,298 INFO [SnmpAgentService] SNMP agent going active
                      15:10:10,548 INFO [casper] Initializing
                      15:10:10,582 INFO [STDOUT]
                      -------------------------------------------------------
                      GMS: address is 10.20.220.135:32883
                      -------------------------------------------------------
                      15:10:12,650 INFO [casper] Number of cluster members: 2
                      15:10:12,650 INFO [casper] Other members: 1
                      15:10:12,650 INFO [casper] Fetching state (will wait for 30000 milliseconds):
                      15:10:12,725 INFO [casper] state was retrieved successfully (in 75 milliseconds)
                      15:10:13,317 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/10.20.220.135
                      15:10:13,323 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /10.20.220.135:1102, group=230.0.0.4, HA-JNDI address=10.20.220.135:1100
                      15:10:13,492 INFO [TreeCache] No transaction manager lookup class has been defined. Transactions cannot be used
                      15:10:13,610 INFO [STDOUT]
                      -------------------------------------------------------
                      GMS: address is 10.20.220.135:32885
                      -------------------------------------------------------
                      15:10:15,689 INFO [TreeCache] viewAccepted(): [10.20.220.15:33221|1] [10.20.220.15:33221, 10.20.220.135:32885]
                      15:10:15,690 INFO [TreeCache] TreeCache local address is 10.20.220.135:32885
                      15:10:15,791 INFO [STDOUT]
                      -------------------------------------------------------
                      GMS: address is 10.20.220.135:32888
                      -------------------------------------------------------
                      15:10:17,850 INFO [TreeCache] viewAccepted(): [10.20.220.15:33223|1] [10.20.220.15:33223, 10.20.220.135:32888]
                      15:10:17,852 INFO [TreeCache] TreeCache local address is 10.20.220.135:32888
                      15:10:18,387 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000F4000102000000000E31302E32302E3232302E313335000DC8000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C00000000050100010000000105010001000101090000000105010001000000210000006C000000000000000100000000000000240000001C0000007E00000000000000010000000A3132372E302E302E31000DC900400000000000080606678102010101000000170401000806066781020101010000000764656661756C7400000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
                      15:10:18,505 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000F4000102000000000E31302E32302E3232302E313335000DC8000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C00000000050100010000000105010001000101090000000105010001000000210000006C000000000000000100000000000000240000001C0000007E00000000000000010000000A3132372E302E302E31000DC900400000000000080606678102010101000000170401000806066781020101010000000764656661756C7400000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
                      15:10:19,104 INFO [Embedded] Catalina naming disabled
                      15:10:19,251 INFO [AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0/jre/lib/i386/client:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0/jre/lib/i386:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
                      15:10:19,312 INFO [Http11Protocol] Initializing Coyote HTTP/1.1 on http-10.20.220.135-8080
                      15:10:19,313 INFO [AjpProtocol] Initializing Coyote AJP/1.3 on ajp-10.20.220.135-8009
                      15:10:19,313 INFO [Catalina] Initialization processed in 208 ms
                      15:10:19,313 INFO [StandardService] Starting service jboss.web
                      15:10:19,315 INFO [StandardEngine] Starting Servlet Engine: JBossWeb/2.0.1.GA
                      15:10:19,357 INFO [Catalina] Server startup in 44 ms
                      15:10:19,452 INFO [TomcatDeployer] deploy, ctxPath=/, warUrl=.../deploy/jboss-web.deployer/ROOT.war/
                      15:10:20,009 INFO [TomcatDeployer] deploy, ctxPath=/invoker, warUrl=.../deploy/httpha-invoker.sar/invoker.war/
                      15:10:20,142 INFO [TomcatDeployer] deploy, ctxPath=/jbossws, warUrl=.../deploy/jbossws.sar/jbossws-context.war/
                      15:10:20,213 INFO [TomcatDeployer] deploy, ctxPath=/juddi, warUrl=.../deploy/juddi-service.sar/juddi.war/
                      15:10:20,285 INFO [RegistryServlet] Loading jUDDI configuration.
                      15:10:20,285 INFO [RegistryServlet] Resources loaded from: /WEB-INF/juddi.properties
                      15:10:20,285 INFO [RegistryServlet] Initializing jUDDI components.
                      15:10:20,639 INFO [TomcatDeployer] deploy, ctxPath=/web-console, warUrl=.../deploy/management/console-mgr.sar/web-console.war/
                      15:10:21,318 INFO [MailService] Mail Service bound to java:/Mail
                      15:10:21,420 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-local-jdbc.rar
                      15:10:21,461 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-xa-jdbc.rar
                      15:10:21,484 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-local-jdbc.rar
                      15:10:21,509 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-xa-jdbc.rar
                      15:10:21,554 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jms/jms-ra.rar
                      15:10:21,578 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/mail-ra.rar
                      15:10:21,626 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/quartz-ra.rar
                      15:10:21,639 INFO [QuartzResourceAdapter] start quartz!!!
                      15:10:21,692 INFO [SimpleThreadPool] Job execution threads will use class loader of thread: main
                      15:10:21,718 INFO [QuartzScheduler] Quartz Scheduler v.1.5.2 created.
                      15:10:21,722 INFO [RAMJobStore] RAMJobStore initialized.
                      15:10:21,722 INFO [StdSchedulerFactory] Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
                      15:10:21,722 INFO [StdSchedulerFactory] Quartz scheduler version: 1.5.2
                      15:10:21,722 INFO [QuartzScheduler] Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
                      15:10:22,299 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=DefaultDS' to JNDI name 'java:DefaultDS'
                      15:10:22,589 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' to JNDI name 'java:JmsXA'
                      15:10:22,652 WARN [JAXWSDeployerHookPreJSE] Cannot load servlet class: com.casper.db.kb.fw.LuceneServlet
                      15:10:22,653 WARN [JAXWSDeployerHookPreJSE] Cannot load servlet class: com.casper.web.portal.servlets.CasperReportPrint
                      15:10:22,656 INFO [TomcatDeployer] deploy, ctxPath=/casper-2.1, warUrl=.../deploy/casper-2.1.war/
                      15:10:22,701 INFO [WebappClassLoader] validateJarFile(/home/karsh/jboss-4.2.2.GA/server/all/./deploy/casper-2.1.war/WEB-INF/lib/servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
                      15:10:22,800 INFO [STDOUT] DBName---------------------casperdb
                      15:10:22,807 INFO [STDOUT] Schedular Started..........................
                      15:10:22,830 INFO [STDOUT] yesterday time2008-04-28 15:10:22.829
                      15:10:22,831 INFO [STDOUT] Date of download:2008-04-28
                      15:10:22,889 INFO [JBossCacheManager] Starting JBossManager
                      15:10:22,896 INFO [JBossCacheManager] We are using mod_jk(2) for load-balancing. Will add JvmRouteValve.
                      15:10:23,248 INFO [TomcatDeployer] deploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
                      15:10:23,355 INFO [FarmMemberService] **** pullNewDeployments ****
                      15:10:23,355 INFO [ClusterFileTransfer] Start pull of file cluster-examples-service.xml from cluster.
                      15:10:23,373 INFO [ClusterFileTransfer] Finished cluster pull of file cluster-examples-service.xml to cluster-examples-service.xml
                      15:10:23,450 INFO [Http11Protocol] Starting Coyote HTTP/1.1 on http-10.20.220.135-8080
                      15:10:23,469 INFO [AjpProtocol] Starting Coyote AJP/1.3 on ajp-10.20.220.135-8009
                      15:10:23,494 INFO [Server] JBoss (MX MicroKernel) [4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA date=200710221139)] Started in 24s:602ms
                      15:10:30,372 ERROR [STDERR] log4j:ERROR A "org.jboss.logging.appender.FileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
                      15:10:30,373 ERROR [STDERR] log4j:ERROR The class "org.apache.log4j.Appender" was loaded by
                      15:10:30,373 ERROR [STDERR] log4j:ERROR [WebappClassLoader
                      delegate: false
                      repositories:
                      /WEB-INF/classes/
                      ----------> Parent Classloader:
                      java.net.FactoryURLClassLoader@1db3aac
                      ] whereas object of type
                      15:10:30,373 ERROR [STDERR] log4j:ERROR "org.jboss.logging.appender.FileAppender" was loaded by [org.jboss.system.server.NoAnnotationURLClassLoader@1c39a2d].
                      15:10:30,373 ERROR [STDERR] log4j:ERROR Could not instantiate appender named "FILE".
                      15:10:30,517 ERROR [1]] Session attribute event listener threw exception
                      java.lang.NullPointerException
                      at nextapp.echo2.webcontainer.ContainerInstance.sessionWillPassivate(ContainerInstance.java:288)
                      at org.jboss.web.tomcat.service.session.ClusteredSession.passivate(ClusteredSession.java:901)
                      at org.jboss.web.tomcat.service.session.JBossCacheManager.storeSession(JBossCacheManager.java:646)
                      at org.jboss.web.tomcat.service.session.InstantSnapshotManager.snapshot(InstantSnapshotManager.java:49)
                      at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:98)
                      at org.jboss.web.tomcat.service.session.JvmRouteValve.invoke(JvmRouteValve.java:84)
                      at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
                      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
                      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
                      at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
                      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
                      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
                      at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
                      at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
                      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
                      at java.lang.Thread.run(Thread.java:675)
                      15:10:30,946 INFO [STDOUT] DBName---------------------casperdb
                      15:10:31,164 INFO [STDOUT] ---------------------
                      15:10:31,166 INFO [STDOUT] ---------------------com.casper.web.portal.ui.CasperSession@1609812
                      15:10:31,167 ERROR [STDERR] java.lang.IllegalArgumentException: setAttribute: Non-serializable attribute
                      15:10:31,168 ERROR [STDERR] at org.jboss.web.tomcat.service.session.ClusteredSession.setAttribute(ClusteredSession.java:535)
                      15:10:31,168 ERROR [STDERR] at org.apache.catalina.session.StandardSessionFacade.setAttribute(StandardSessionFacade.java:130)
                      15:10:31,168 ERROR [STDERR] at com.casper.web.portal.ui.CasperSession.setAttribute(CasperSession.java:132)
                      15:10:31,168 ERROR [STDERR] at com.casper.web.portal.ui.CasperSession.setCasperEnvironment(CasperSession.java:204)
                      15:10:31,168 ERROR [STDERR] at com.casper.web.portal.ui.Login.initComponents(Login.java:83)
                      15:10:31,168 ERROR [STDERR] at com.casper.web.portal.ui.Login.(Login.java:65)
                      15:10:31,168 ERROR [STDERR] at com.casper.web.portal.servlets.Application.init(Application.java:57)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.app.ApplicationInstance.doInit(ApplicationInstance.java:247)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.webcontainer.ContainerInstance.init(ContainerInstance.java:218)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.webcontainer.ContainerSynchronizeService.renderInit(ContainerSynchronizeService.java:417)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.webrender.service.SynchronizeService.service(SynchronizeService.java:269)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.webrender.WebRenderServlet.process(WebRenderServlet.java:273)
                      15:10:31,168 ERROR [STDERR] at nextapp.echo2.webrender.WebRenderServlet.doPost(WebRenderServlet.java:189)
                      15:10:31,168 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
                      15:10:31,168 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
                      15:10:31,168 ERROR [STDERR] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
                      15:10:31,168 ERROR [STDERR] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:87)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.service.session.JvmRouteValve.invoke(JvmRouteValve.java:84)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
                      15:10:31,169 ERROR [STDERR] at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
                      15:10:31,169 ERROR [STDERR] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
                      15:10:31,169 ERROR [STDERR] at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
                      15:10:31,169 ERROR [STDERR] at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
                      15:10:31,169 ERROR [STDERR] at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
                      15:10:31,170 ERROR [STDERR] at java.lang.Thread.run(Thread.java:675)



                      The mod_jk.log of apache is giving:

                      [Mon Apr 28 15:23:38 2008][29564:3086853824] [error] init_jk::mod_jk.c (2781): Initializing shm:/usr/local/apache2/run/jk.shm.29564 errno=2. Load balancing workers will not function properly.
                      [Mon Apr 28 15:23:38 2008][29564:3086853824] [warn] jk_map_validate_property::jk_map.c (410): The attribute 'worker.node1.cachesize' is deprecated - please check the documentation for the correct replacement.
                      [Mon Apr 28 15:23:38 2008][29564:3086853824] [warn] jk_map_validate_property::jk_map.c (410): The attribute 'worker.node2.cachesize' is deprecated - please check the documentation for the correct replacement.
                      [Mon Apr 28 15:23:38 2008][29564:3086853824] [info] init_jk::mod_jk.c (2825): mod_jk/1.2.26 initialized



                      The error.log of apache is
                      [Mon Apr 28 15:23:38 2008] [notice] Apache/2.0.63 (Unix) mod_jk/1.2.26 configured -- resuming normal operations

                      The access.log of apache is returned with nothing in the log file.

                      Here is my workers.properties file
                      # Define list of workers that will be used
                      # for mapping requests
                      worker.list=loadbalancer,status

                      # Define Node1
                      # modify the host as your host IP or DNS name.
                      worker.node1.port=8009
                      worker.node1.host=10.20.220.15
                      worker.node1.type=ajp13
                      worker.node1.lbfactor=1
                      worker.node1.cachesize=10

                      # Define Node2
                      # modify the host as your host IP or DNS name.
                      worker.node2.port=8009
                      worker.node2.host= 10.20.220.135
                      worker.node2.type=ajp13
                      worker.node2.lbfactor=1
                      worker.node2.cachesize=10

                      # Load-balancing behaviour
                      worker.loadbalancer.type=lb
                      worker.loadbalancer.balance_workers=node1,node2
                      worker.loadbalancer.sticky_session=1
                      #worker.list=loadbalancer

                      # Status worker for managing load balancer
                      worker.status.type=status


                      My uriworkermap.properties file
                      # Simple worker configuration file
                      #

                      # Mount the Servlet context to the ajp13 worker

                      /casper-2.1=loadbalancer

                      /casper-2.1/*=loadbalancer

                      /web-console=loadbalancer
                      /web-console/*=loadbalancer



                      You may find the server.log file in http://rifers.org/paste/show/7205
                      and boot.log in http://rifers.org/paste/show/7206

                      PS: Please let me know the modifications to be done now.

                      • 8. Re: Unable to run multiple cluster nodes.
                        Brian Stansberry Master

                        I see three things in the server.log

                        1) The logging subsystem is complaining. Have you packaged log4j.jar in your war? If yes, recommend you remove it if you can.

                        2) You nextapp.echo2.webcontainer.ContainerInstance class is throwing an NPE. Can't help you there.

                        3) Your code is trying to pass an object that doesn't implement java.io.Serializable as a value to HttpSession.setAttribute(String key, Object value). That is not allowed, as all attributes need to be serialized for session replication to work.

                        • 9. Re: Unable to run multiple cluster nodes.
                          praveena chalamcharla Newbie





                          I have deleted my log4j.jar from the "lib" of application war file. Here is the result


                          mod_jk.log file of apache

                          [root@localhost logs]# tail -f mod_jk.log
                          [Wed Apr 30 11:56:08 2008][3345:3086427840] [error] init_jk::mod_jk.c (2781): Initializing shm:/usr/local/apache2/run/jk.shm.3345 errno=2. Load balancing workers will not function properly.
                          [Wed Apr 30 11:56:08 2008][3345:3086427840] [warn] jk_map_validate_property::jk_map.c (410): The attribute 'worker.node1.cachesize' is deprecated - please check the documentation for the correct replacement.
                          [Wed Apr 30 11:56:08 2008][3345:3086427840] [warn] jk_map_validate_property::jk_map.c (410): The attribute 'worker.node2.cachesize' is deprecated - please check the documentation for the correct replacement.
                          [Wed Apr 30 11:56:08 2008][3345:3086427840] [info] init_jk::mod_jk.c (2825): mod_jk/1.2.26 initialized
                          [Wed Apr 30 11:59:14 2008]loadbalancer 10.20.220.135 0.961028
                          [Wed Apr 30 11:59:14 2008]loadbalancer 10.20.220.135 0.016325
                          [Wed Apr 30 11:59:19 2008]loadbalancer 10.20.220.135 4.573659
                          [Wed Apr 30 11:59:19 2008]loadbalancer 10.20.220.135 0.023416
                          [Wed Apr 30 11:59:19 2008]loadbalancer 10.20.220.135 0.023957
                          [Wed Apr 30 11:59:19 2008]loadbalancer 10.20.220.135 0.139374




                          error_log file of apache

                          [root@localhost logs]# tail -f error_log
                          [Wed Apr 30 11:53:16 2008] [notice] Apache/2.0.63 (Unix) mod_jk/1.2.26 configured -- resuming normal operations
                          [Wed Apr 30 11:56:07 2008] [notice] caught SIGTERM, shutting down
                          [Wed Apr 30 11:56:08 2008] [notice] Apache/2.0.63 (Unix) mod_jk/1.2.26 configured -- resuming normal operations
                          [Wed Apr 30 11:59:07 2008] [error] [client 10.20.220.15] File does not exist: /usr/local/apache2/htdocs/favicon.ico
                          [Wed Apr 30 11:59:14 2008] [error] [client 10.20.220.15] File does not exist: /usr/local/apache2/htdocs/favicon.ico



                          access_log file of apache

                          [root@localhost logs]# tail -f access_log
                          10.20.220.15 - - [30/Apr/2008:11:59:07 +0530] "GET / HTTP/1.1" 200 1456
                          10.20.220.15 - - [30/Apr/2008:11:59:07 +0530] "GET /apache_pb.gif HTTP/1.1" 200 2326
                          10.20.220.15 - - [30/Apr/2008:11:59:07 +0530] "GET /favicon.ico HTTP/1.1" 404 300
                          10.20.220.15 - - [30/Apr/2008:11:59:13 +0530] "GET /casper-2.1/app HTTP/1.1" 200 729
                          10.20.220.15 - - [30/Apr/2008:11:59:14 +0530] "GET /casper-2.1/app?serviceId=Echo.ClientEngine HTTP/1.1" 200 10767
                          10.20.220.15 - - [30/Apr/2008:11:59:14 +0530] "GET /favicon.ico HTTP/1.1" 404 300
                          10.20.220.15 - - [30/Apr/2008:11:59:14 +0530] "POST /casper-2.1/app?serviceId=Echo.Synchronize HTTP/1.1" 200 3435
                          10.20.220.15 - - [30/Apr/2008:11:59:19 +0530] "GET /casper-2.1/app?serviceId=Echo.ContentPane HTTP/1.1" 200 809
                          10.20.220.15 - - [30/Apr/2008:11:59:19 +0530] "GET /casper-2.1/app?serviceId=Echo.WebContainer HTTP/1.1" 200 609
                          10.20.220.15 - - [30/Apr/2008:11:59:19 +0530] "GET /casper-2.1/app?serviceId=22fe09_1199e06b43f_1 HTTP/1.1" 200 11333



                          server.log file of jboss server

                          [root@praveena log]# tail -f server.log
                          2008-04-30 12:15:28,674 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...
                          2008-04-30 12:15:28,674 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...
                          2008-04-30 12:15:33,467 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger] Periodic recovery - second pass <Wed, 30 Apr 2008 12:15:33>
                          2008-04-30 12:15:33,467 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger] AtomicActionRecoveryModule: Second pass
                          2008-04-30 12:15:33,467 DEBUG [com.arjuna.ats.txoj.logging.txojLoggerI18N] [com.arjuna.ats.internal.txoj.recovery.TORecoveryModule_6] - TORecoveryModule - second pass
                          2008-04-30 12:15:33,467 DEBUG [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.recovery.info.secondpass] Local XARecoveryModule - second pass
                          2008-04-30 12:15:38,675 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...
                          2008-04-30 12:15:38,675 DEBUG [org.jboss.web.tomcat.service.session.JBossCacheManager] Looking for sessions that have expired ...



                          error.log file of Jboss server


                          [root@praveena log]# tail -f boot.log
                          11:57:40,337 INFO [WebService] Using RMI server codebase: http://10.20.220.15:8083/
                          11:57:40,337 DEBUG [WebService] Created jboss:service=WebService
                          11:57:40,338 DEBUG [LocalJBossServerDomain] handleNotification: javax.management.Notification[source=jboss.system:service=ServiceController][type=org.jboss.system.ServiceMBean.create][message=]
                          11:57:40,338 DEBUG [DefaultManagedObjectFactoryMap] Failed to find factory for event: javax.management.Notification[source=jboss.system:service=ServiceController][type=org.jboss.system.ServiceMBean.create][message=]
                          11:57:40,338 DEBUG [ServiceController] Creating dependent components for: jboss:service=WebService dependents are: []
                          11:57:40,338 DEBUG [ServiceController] Creating service jboss:service=Naming
                          11:57:40,338 DEBUG [ServiceController] waiting in create of jboss:service=Naming waiting on jboss:service=NamingBeanImpl
                          11:57:40,338 DEBUG [ServiceController] Creating service jboss.system:type=Log4jService,service=Logging
                          11:57:40,338 DEBUG [Log4jService] Creating jboss.system:type=Log4jService,service=Logging
                          11:57:40,339 INFO [Log4jService$URLWatchTimerTask] Configuring from URL: resource:jboss-log4j.xml



                          Console output of the server after running it

                          [root@praveena bin]# ./run.sh -Djboss.partition.name=casper -c all -b 10.20.220.15
                          =========================================================================

                          JBoss Bootstrap Environment

                          JBOSS_HOME: /home/praveena/jboss-4.2.2.GA

                          JAVA: /home/praveena/software/jdk1.6.0_03//bin/java

                          JAVA_OPTS: -Dprogram.name=run.sh -server -Xms128m -Xmx512m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true

                          CLASSPATH: /home/praveena/jboss-4.2.2.GA/bin/run.jar:/home/praveena/software/jdk1.6.0_03//lib/tools.jar

                          =========================================================================

                          11:57:36,982 INFO [Server] Starting JBoss (MX MicroKernel)...
                          11:57:36,983 INFO [Server] Release ID: JBoss [Trinity] 4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA date=200710221139)
                          11:57:36,984 INFO [Server] Home Dir: /home/praveena/jboss-4.2.2.GA
                          11:57:36,984 INFO [Server] Home URL: file:/home/praveena/jboss-4.2.2.GA/
                          11:57:36,985 INFO [Server] Patch URL: null
                          11:57:36,985 INFO [Server] Server Name: all
                          11:57:36,985 INFO [Server] Server Home Dir: /home/praveena/jboss-4.2.2.GA/server/all
                          11:57:36,985 INFO [Server] Server Home URL: file:/home/praveena/jboss-4.2.2.GA/server/all/
                          11:57:36,985 INFO [Server] Server Log Dir: /home/praveena/jboss-4.2.2.GA/server/all/log
                          11:57:36,985 INFO [Server] Server Temp Dir: /home/praveena/jboss-4.2.2.GA/server/all/tmp
                          11:57:36,986 INFO [Server] Root Deployment Filename: jboss-service.xml
                          11:57:37,250 INFO [ServerInfo] Java version: 1.6.0_03,Sun Microsystems Inc.
                          11:57:37,250 INFO [ServerInfo] Java VM: Java HotSpot(TM) Server VM 1.6.0_03-b05,Sun Microsystems Inc.
                          11:57:37,250 INFO [ServerInfo] OS-System: Linux 2.6.23.1-42.fc8,i386
                          11:57:37,516 INFO [Server] Core system initialized
                          11:57:40,337 INFO [WebService] Using RMI server codebase: http://10.20.220.15:8083/
                          11:57:40,339 INFO [Log4jService$URLWatchTimerTask] Configuring from URL: resource:jboss-log4j.xml
                          11:57:40,689 INFO [TransactionManagerService] JBossTS Transaction Service (JTA version) - JBoss Inc.
                          11:57:40,689 INFO [TransactionManagerService] Setting up property manager MBean and JMX layer
                          11:57:40,842 INFO [TransactionManagerService] Starting recovery manager
                          11:57:40,931 INFO [TransactionManagerService] Recovery manager started
                          11:57:40,931 INFO [TransactionManagerService] Binding TransactionManager JNDI Reference
                          11:57:43,654 INFO [EJB3Deployer] Starting java:comp multiplexer
                          11:57:44,551 INFO [STDOUT]
                          -------------------------------------------------------
                          GMS: address is 10.20.220.15:32968
                          -------------------------------------------------------
                          11:57:46,576 INFO [TreeCache] viewAccepted(): [10.20.220.15:32968|0] [10.20.220.15:32968]
                          11:57:46,592 INFO [TreeCache] TreeCache local address is 10.20.220.15:32968
                          11:57:46,592 INFO [TreeCache] State could not be retrieved (we are the first member in group)
                          11:57:46,592 INFO [TreeCache] parseConfig(): PojoCacheConfig is empty
                          11:57:46,674 INFO [STDOUT] no object for null
                          11:57:46,696 INFO [STDOUT] no object for null
                          11:57:46,711 INFO [STDOUT] no object for null
                          11:57:46,726 INFO [STDOUT] no object for {urn:jboss:bean-deployer}supplyType
                          11:57:46,736 INFO [STDOUT] no object for {urn:jboss:bean-deployer}dependsType
                          11:57:48,426 INFO [NativeServerConfig] JBoss Web Services - Native
                          11:57:48,426 INFO [NativeServerConfig] jbossws-native-2.0.1.SP2 (build=200710210837)
                          11:57:49,130 INFO [TreeCache] viewAccepted(): [10.20.220.15:32968|1] [10.20.220.15:32968, 10.20.220.135:32796]
                          11:57:49,168 INFO [SnmpAgentService] SNMP agent going active
                          11:57:49,180 INFO [TreeCache] locking the subtree at / to transfer state
                          11:57:49,190 INFO [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
                          11:57:49,531 INFO [casper] Initializing
                          11:57:49,577 INFO [STDOUT]
                          -------------------------------------------------------
                          GMS: address is 10.20.220.15:32971
                          -------------------------------------------------------
                          11:57:51,585 INFO [casper] Number of cluster members: 1
                          11:57:51,585 INFO [casper] Other members: 0
                          11:57:51,585 INFO [casper] Fetching state (will wait for 30000 milliseconds):
                          11:57:51,585 INFO [casper] State could not be retrieved (we are the first member in group)
                          11:57:51,616 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/10.20.220.15
                          11:57:51,621 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /10.20.220.15:1102, group=230.0.0.4, HA-JNDI address=10.20.220.15:1100
                          11:57:52,380 INFO [TreeCache] No transaction manager lookup class has been defined. Transactions cannot be used
                          11:57:52,492 INFO [STDOUT]
                          -------------------------------------------------------
                          GMS: address is 10.20.220.15:32975
                          -------------------------------------------------------
                          11:57:54,107 INFO [casper] New cluster view for partition casper (id: 1, delta: 1) : [10.20.220.15:1099, 10.20.220.135:1099]
                          11:57:54,109 INFO [casper] I am (10.20.220.15:1099) received membershipChanged event:
                          11:57:54,109 INFO [casper] Dead members: 0 ([])
                          11:57:54,109 INFO [casper] New Members : 1 ([10.20.220.135:1099])
                          11:57:54,109 INFO [casper] All Members : 2 ([10.20.220.15:1099, 10.20.220.135:1099])
                          11:57:54,501 INFO [TreeCache] viewAccepted(): [10.20.220.15:32975|0] [10.20.220.15:32975]
                          11:57:54,505 INFO [TreeCache] TreeCache local address is 10.20.220.15:32975
                          11:57:54,668 INFO [STDOUT]
                          -------------------------------------------------------
                          GMS: address is 10.20.220.15:32977
                          -------------------------------------------------------
                          11:57:56,696 INFO [TreeCache] viewAccepted(): [10.20.220.15:32977|0] [10.20.220.15:32977]
                          11:57:56,701 INFO [TreeCache] TreeCache local address is 10.20.220.15:32977
                          11:57:56,853 INFO [TreeCache] viewAccepted(): [10.20.220.15:32975|1] [10.20.220.15:32975, 10.20.220.135:32801]
                          11:57:57,466 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000F4000102000000000D31302E32302E3232302E313500000DC8000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C00000000050100010000000105010001000101090000000105010001000000210000006C000000000000000100000000000000240000001C0000007E00000000000000010000000A3132372E302E302E31000DC900400000000000080606678102010101000000170401000806066781020101010000000764656661756C7400000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
                          11:57:57,546 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000F4000102000000000D31302E32302E3232302E313500000DC8000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C00000000050100010000000105010001000101090000000105010001000000210000006C000000000000000100000000000000240000001C0000007E00000000000000010000000A3132372E302E302E31000DC900400000000000080606678102010101000000170401000806066781020101010000000764656661756C7400000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
                          11:57:58,254 INFO [Embedded] Catalina naming disabled
                          11:57:58,430 INFO [AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/praveena/software/jdk1.6.0_03/jre/lib/i386/server:/home/praveena/software/jdk1.6.0_03/jre/lib/i386:/home/praveena/software/jdk1.6.0_03/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
                          11:57:58,495 INFO [Http11Protocol] Initializing Coyote HTTP/1.1 on http-10.20.220.15-8080
                          11:57:58,496 INFO [AjpProtocol] Initializing Coyote AJP/1.3 on ajp-10.20.220.15-8009
                          11:57:58,496 INFO [Catalina] Initialization processed in 242 ms
                          11:57:58,496 INFO [StandardService] Starting service jboss.web
                          11:57:58,498 INFO [StandardEngine] Starting Servlet Engine: JBossWeb/2.0.1.GA
                          11:57:58,537 INFO [Catalina] Server startup in 40 ms
                          11:57:58,626 INFO [TomcatDeployer] deploy, ctxPath=/, warUrl=.../deploy/jboss-web.deployer/ROOT.war/
                          11:57:59,037 INFO [TreeCache] viewAccepted(): [10.20.220.15:32977|1] [10.20.220.15:32977, 10.20.220.135:32803]
                          11:57:59,135 INFO [TomcatDeployer] deploy, ctxPath=/invoker, warUrl=.../deploy/httpha-invoker.sar/invoker.war/
                          11:57:59,258 INFO [TomcatDeployer] deploy, ctxPath=/jbossws, warUrl=.../deploy/jbossws.sar/jbossws-context.war/
                          11:57:59,321 INFO [TomcatDeployer] deploy, ctxPath=/juddi, warUrl=.../deploy/juddi-service.sar/juddi.war/
                          11:57:59,492 INFO [RegistryServlet] Loading jUDDI configuration.
                          11:57:59,493 INFO [RegistryServlet] Resources loaded from: /WEB-INF/juddi.properties
                          11:57:59,493 INFO [RegistryServlet] Initializing jUDDI components.
                          11:57:59,759 INFO [TomcatDeployer] deploy, ctxPath=/jbossmq-httpil, warUrl=.../deploy-hasingleton/jms/jbossmq-httpil.sar/jbossmq-httpil.war/
                          11:58:00,047 INFO [TomcatDeployer] deploy, ctxPath=/web-console, warUrl=.../deploy/management/console-mgr.sar/web-console.war/
                          11:58:00,758 INFO [MailService] Mail Service bound to java:/Mail
                          11:58:00,905 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-local-jdbc.rar
                          11:58:00,927 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-xa-jdbc.rar
                          11:58:00,949 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-local-jdbc.rar
                          11:58:00,977 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-xa-jdbc.rar
                          11:58:01,021 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jms/jms-ra.rar
                          11:58:01,056 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/mail-ra.rar
                          11:58:01,115 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/quartz-ra.rar
                          11:58:01,122 INFO [QuartzResourceAdapter] start quartz!!!
                          11:58:01,173 INFO [SimpleThreadPool] Job execution threads will use class loader of thread: main
                          11:58:01,195 INFO [QuartzScheduler] Quartz Scheduler v.1.5.2 created.
                          11:58:01,201 INFO [RAMJobStore] RAMJobStore initialized.
                          11:58:01,202 INFO [StdSchedulerFactory] Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
                          11:58:01,202 INFO [StdSchedulerFactory] Quartz scheduler version: 1.5.2
                          11:58:01,202 INFO [QuartzScheduler] Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
                          11:58:01,793 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=DefaultDS' to JNDI name 'java:DefaultDS'
                          11:58:02,083 INFO [A] Bound to JNDI name: queue/A
                          11:58:02,085 INFO [B] Bound to JNDI name: queue/B
                          11:58:02,087 INFO [C] Bound to JNDI name: queue/C
                          11:58:02,089 INFO [D] Bound to JNDI name: queue/D
                          11:58:02,090 INFO [ex] Bound to JNDI name: queue/ex
                          11:58:02,113 INFO [testTopic] Bound to JNDI name: topic/testTopic
                          11:58:02,115 INFO [securedTopic] Bound to JNDI name: topic/securedTopic
                          11:58:02,117 INFO [testDurableTopic] Bound to JNDI name: topic/testDurableTopic
                          11:58:02,122 INFO [testQueue] Bound to JNDI name: queue/testQueue
                          11:58:02,158 INFO [UILServerILService] JBossMQ UIL service available at : /10.20.220.15:8093
                          11:58:02,182 INFO [DLQ] Bound to JNDI name: queue/DLQ
                          11:58:02,295 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' to JNDI name 'java:JmsXA'
                          11:58:02,336 WARN [JAXWSDeployerHookPreJSE] Cannot load servlet class: com.casper.db.kb.fw.LuceneServlet
                          11:58:02,336 WARN [JAXWSDeployerHookPreJSE] Cannot load servlet class: com.casper.web.portal.servlets.CasperReportPrint
                          11:58:02,340 INFO [TomcatDeployer] deploy, ctxPath=/casper-2.1, warUrl=.../deploy/casper-2.1.war/
                          11:58:02,390 INFO [WebappClassLoader] validateJarFile(/home/praveena/jboss-4.2.2.GA/server/all/./deploy/casper-2.1.war/WEB-INF/lib/servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
                          11:58:02,836 INFO [STDOUT] DBName---------------------casperdb
                          11:58:02,843 INFO [STDOUT] Schedular Started..........................
                          11:58:02,857 INFO [STDOUT] yesterday time2008-04-30 11:58:02.856
                          11:58:02,858 INFO [STDOUT] Date of download:2008-04-30
                          11:58:02,923 INFO [JBossCacheManager] Starting JBossManager
                          11:58:02,934 INFO [JBossCacheManager] We are using mod_jk(2) for load-balancing. Will add JvmRouteValve.
                          11:58:03,059 INFO [TomcatDeployer] deploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
                          11:58:03,186 INFO [TomcatDeployer] deploy, ctxPath=/sample, warUrl=.../deploy/sample.war/
                          11:58:03,242 INFO [JBossCacheManager] Starting JBossManager
                          11:58:03,245 INFO [JBossCacheManager] We are using mod_jk(2) for load-balancing. Will add JvmRouteValve.
                          11:58:03,461 INFO [Http11Protocol] Starting Coyote HTTP/1.1 on http-10.20.220.15-8080
                          11:58:03,498 INFO [AjpProtocol] Starting Coyote AJP/1.3 on ajp-10.20.220.15-8009
                          11:58:03,509 INFO [Server] JBoss (MX MicroKernel) [4.2.2.GA (build: SVNTag=JBoss_4_2_2_GA date=200710221139)] Started in 26s:522ms
                          11:58:15,398 INFO [STDOUT] Pages Downloaded
                          11:58:15,411 INFO [STDOUT]
                          In Constructor. dbSelector = casperdb
                          11:58:16,177 INFO [STDOUT] Top Source IP's added
                          11:58:28,972 INFO [STDOUT] Port Details added
                          11:58:29,229 INFO [STDOUT] Port Trends added
                          12:00:48,833 ERROR [1]] Session attribute event listener threw exception
                          java.lang.NullPointerException
                          at nextapp.echo2.webcontainer.ContainerInstance.sessionWillPassivate(ContainerInstance.java:288)
                          at org.jboss.web.tomcat.service.session.ClusteredSession.passivate(ClusteredSession.java:901)
                          at org.jboss.web.tomcat.service.session.JBossCacheManager.storeSession(JBossCacheManager.java:646)
                          at org.jboss.web.tomcat.service.session.InstantSnapshotManager.snapshot(InstantSnapshotManager.java:49)
                          at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:98)
                          at org.jboss.web.tomcat.service.session.JvmRouteValve.invoke(JvmRouteValve.java:84)
                          at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
                          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
                          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
                          at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
                          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
                          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
                          at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
                          at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
                          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)



                          PS: I will be online for having your suggestions on tomorrow's night. Please respond me. If I impose any queries.

                          • 10. Re: Unable to run multiple cluster nodes.
                            praveena chalamcharla Newbie

                            server.log

                            2008-04-30 18:23:09,045 DEBUG [org.jboss.logging.Log4jService] Installed System.out adapter
                            2008-04-30 18:23:09,046 DEBUG [org.jboss.logging.Log4jService] Installed System.err adapter
                            2008-04-30 18:23:09,047 DEBUG [org.jboss.logging.Log4jService] Added ThrowableListener: org.jboss.logging.Log4jService$ThrowableListenerLoggingAdapter@16ea269
                            2008-04-30 18:23:09,047 DEBUG [org.jboss.logging.Log4jService] Created jboss.system:type=Log4jService,service=Logging
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.system:type=Log4jService,service=Logging dependents are: []
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Creating service jboss.rmi:type=RMIClassLoader
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.rmi:type=RMIClassLoader dependents are: []
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=WebService
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=WebService
                            2008-04-30 18:23:09,048 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=NamingBeanImpl
                            2008-04-30 18:23:09,049 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=NamingBeanImpl dependents are: [ObjectName: jboss:service=Naming
                            State: CONFIGURED
                            I Depend On:
                            jboss.system:service=ThreadPool
                            jboss:service=NamingBeanImpl
                            ]
                            2008-04-30 18:23:09,049 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=Naming
                            2008-04-30 18:23:09,049 DEBUG [org.jboss.naming.NamingService] Creating jboss:service=Naming
                            2008-04-30 18:23:09,049 DEBUG [org.jboss.naming.NamingService] Created jboss:service=Naming
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=Naming dependents are: []
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=Naming
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=Naming
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=JNDIView
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.naming.JNDIView] Creating jboss:service=JNDIView
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.naming.JNDIView] Created jboss:service=JNDIView
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=JNDIView dependents are: []
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating service jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.security.plugins.SecurityConfig] Creating jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.security.plugins.SecurityConfig] Created jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.security:service=SecurityConfig dependents are: []
                            2008-04-30 18:23:09,060 DEBUG [org.jboss.system.ServiceController] Creating service jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.security.auth.login.XMLLoginConfig] Creating jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.security.auth.login.XMLLoginConfig] Created jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.security:service=XMLLoginConfig dependents are: []
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.system.ServiceController] Creating service jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] Creating jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] Created jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.security:service=JaasSecurityManager dependents are: []
                            2008-04-30 18:23:09,061 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=XidFactory
                            2008-04-30 18:23:09,062 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=XidFactory dependents are: []
                            2008-04-30 18:23:09,062 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=TransactionManager
                            2008-04-30 18:23:09,062 DEBUG [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Creating jboss:service=TransactionManager
                            2008-04-30 18:23:09,062 DEBUG [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Created jboss:service=TransactionManager
                            2008-04-30 18:23:09,066 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=TransactionManager dependents are: [ObjectName: jboss:service=invoker,type=unified
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=TransactionManager
                            jboss.remoting:service=Connector,transport=socket
                            , ObjectName: jboss:service=invoker,type=jrmp
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=TransactionManager
                            Depends On Me:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            , ObjectName: jboss:service=invoker,type=local
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=TransactionManager
                            , ObjectName: jboss:service=invoker,type=pooled
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=TransactionManager
                            ]
                            2008-04-30 18:23:09,066 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss:service=invoker,type=unified waiting on jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Creating jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Created jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=invoker,type=jrmp dependents are: [ObjectName: jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=invoker,type=jrmp
                            Depends On Me:
                            jboss:service=ClientUserTransaction
                            , ObjectName: jboss:service=proxyFactory,target=ClientUserTransaction
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=invoker,type=jrmp
                            Depends On Me:
                            jboss:service=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Creating jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,067 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Created jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=proxyFactory,target=ClientUserTransactionFactory dependents are: [ObjectName: jboss:service=ClientUserTransaction
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss:service=ClientUserTransaction waiting on jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Creating jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Created jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=proxyFactory,target=ClientUserTransaction dependents are: [ObjectName: jboss:service=ClientUserTransaction
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.tm.usertx.server.ClientUserTransactionService] Creating jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,068 DEBUG [org.jboss.tm.usertx.server.ClientUserTransactionService] Created jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,070 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=ClientUserTransaction dependents are: []
                            2008-04-30 18:23:09,070 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=local
                            2008-04-30 18:23:09,070 DEBUG [org.jboss.invocation.local.LocalInvoker] Creating jboss:service=invoker,type=local
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.invocation.local.LocalInvoker] Created jboss:service=invoker,type=local
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=invoker,type=local dependents are: []
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.invocation.pooled.server.PooledInvoker] Creating jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.invocation.pooled.server.PooledInvoker] Created jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=invoker,type=pooled dependents are: []
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss:service=invoker,type=unified waiting on jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=local
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=invoker,type=local
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,087 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,088 DEBUG [org.jboss.system.ServiceController] Creating service jboss.remoting:service=NetworkRegistry
                            2008-04-30 18:23:09,088 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.remoting:service=NetworkRegistry dependents are: [ObjectName: jboss.remoting:service=Connector,transport=socket
                            State: CONFIGURED
                            I Depend On:
                            jboss.remoting:service=NetworkRegistry
                            Depends On Me:
                            jboss:service=invoker,type=unified
                            ]
                            2008-04-30 18:23:09,088 DEBUG [org.jboss.system.ServiceController] Creating service jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,149 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[UNINITIALIZED] did not find server socket factory configuration as mbean service or classname. Creating default server socket factory.
                            2008-04-30 18:23:09,149 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[UNINITIALIZED] created server socket factory javax.net.DefaultServerSocketFactory@5a3923
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.remoting:service=Connector,transport=socket dependents are: [ObjectName: jboss:service=invoker,type=unified
                            State: CONFIGURED
                            I Depend On:
                            jboss:service=TransactionManager
                            jboss.remoting:service=Connector,transport=socket
                            ]
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Creating service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.invocation.unified.server.UnifiedInvoker] Creating jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.invocation.unified.server.UnifiedInvoker] Created jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss:service=invoker,type=unified dependents are: []
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Creating service jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Ignoring create request for service: jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,173 DEBUG [org.jboss.system.ServiceController] Creating service jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,174 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] Creating jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,174 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] Created jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,174 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.jdbc:service=metadata dependents are: []
                            2008-04-30 18:23:09,174 DEBUG [org.jboss.system.ServiceController] Creating service jboss.deployment:type=DeploymentScanner,flavor=URL
                            2008-04-30 18:23:09,174 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Creating jboss.deployment:type=DeploymentScanner,flavor=URL
                            2008-04-30 18:23:09,210 DEBUG [org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread] Running
                            2008-04-30 18:23:09,210 DEBUG [org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread] Disabled, waiting for notification
                            2008-04-30 18:23:09,210 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Scanner thread started
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Created jboss.deployment:type=DeploymentScanner,flavor=URL
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.deployment:type=DeploymentScanner,flavor=URL dependents are: []
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.deployment.MainDeployer] Done with create step of deploying jboss-service.xml
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.deployment.MainDeployer] Begin deployment start file:/home/praveena/jboss-4.2.2.GA/server/all/conf/jboss-service.xml
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.deployment.SARDeployer] Deploying SAR, start step: url file:/home/praveena/jboss-4.2.2.GA/server/all/conf/jboss-service.xml
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.system.ServiceController] starting service jboss.management.local:j2eeType=J2EEDomain,name=Manager
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.management.local:j2eeType=J2EEDomain,name=Manager dependent components: []
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=AttributePersistenceService
                            2008-04-30 18:23:09,211 DEBUG [org.jboss.system.pm.AttributePersistenceService] Starting jboss:service=AttributePersistenceService
                            2008-04-30 18:23:09,214 DEBUG [org.jboss.system.pm.XMLAttributePersistenceManager] Constructed
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.pm.XMLAttributePersistenceManager] Using data directory: /home/praveena/jboss-4.2.2.GA/server/all/data/xmbean-attrs
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.pm.AttributePersistenceService] Started jboss:service=AttributePersistenceService
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=AttributePersistenceService dependent components: []
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.ServiceController] starting service jboss.system:service=ThreadPool
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.system:service=ThreadPool dependent components: [ObjectName: jboss:service=WebService
                            State: CREATED
                            I Depend On:
                            jboss.system:service=ThreadPool
                            , ObjectName: jboss:service=Naming
                            State: CREATED
                            I Depend On:
                            jboss.system:service=ThreadPool
                            jboss:service=NamingBeanImpl
                            ]
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=WebService
                            2008-04-30 18:23:09,215 DEBUG [org.jboss.web.WebService] Starting jboss:service=WebService
                            2008-04-30 18:23:09,218 DEBUG [org.jboss.web.WebServer] Started server: ServerSocket[addr=/10.20.220.15,port=0,localport=8083]
                            2008-04-30 18:23:09,219 DEBUG [org.jboss.web.WebService] Started WebServer with address: /10.20.220.15:8083
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.web.WebService] Started jboss:service=WebService
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=WebService dependent components: []
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=Naming
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] waiting in start jboss:service=Naming on jboss:service=NamingBeanImpl
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] starting service jboss.system:type=Log4jService,service=Logging
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.logging.Log4jService] Starting jboss.system:type=Log4jService,service=Logging
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.logging.Log4jService] Started jboss.system:type=Log4jService,service=Logging
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.system:type=Log4jService,service=Logging dependent components: []
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] starting service jboss.rmi:type=RMIClassLoader
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.rmi:type=RMIClassLoader dependent components: []
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=WebService
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=WebService
                            2008-04-30 18:23:09,220 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=NamingBeanImpl
                            2008-04-30 18:23:09,231 DEBUG [org.jnp.server.NamingBeanImpl] Using NamingServer: org.jnp.server.NamingServer@b51404
                            2008-04-30 18:23:09,231 DEBUG [org.jnp.server.NamingBeanImpl] Installed global NamingServer: org.jnp.server.NamingServer@b51404
                            2008-04-30 18:23:09,234 DEBUG [org.jnp.server.NamingBeanImpl] InitialContext Environment:
                            2008-04-30 18:23:09,234 DEBUG [org.jnp.server.NamingBeanImpl] key=java.naming.factory.initial, value(java.lang.String)=org.jnp.interfaces.NamingContextFactory
                            2008-04-30 18:23:09,234 DEBUG [org.jnp.server.NamingBeanImpl] key=java.naming.factory.url.pkgs, value(java.lang.String)=org.jboss.naming:org.jnp.interfaces
                            2008-04-30 18:23:09,237 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=NamingBeanImpl dependent components: [ObjectName: jboss:service=Naming
                            State: CREATED
                            I Depend On:
                            jboss.system:service=ThreadPool
                            jboss:service=NamingBeanImpl
                            ]
                            2008-04-30 18:23:09,237 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=Naming
                            2008-04-30 18:23:09,237 DEBUG [org.jboss.naming.NamingService] Starting jboss:service=Naming
                            2008-04-30 18:23:09,237 DEBUG [org.jboss.naming.NamingService] System.setProperty, key=java.naming.factory.initial, value=org.jnp.interfaces.NamingContextFactory
                            2008-04-30 18:23:09,238 DEBUG [org.jboss.naming.NamingService] System.setProperty, key=java.naming.factory.url.pkgs, value=org.jboss.naming:org.jnp.interfaces
                            2008-04-30 18:23:09,238 DEBUG [org.jboss.naming.Naming] Creating NamingServer stub, theServer=MBeanProxyExt[jboss:service=NamingBeanImpl],rmiPort=1098,clientSocketFactory=null,serverSocketFactory=org.jboss.net.sockets.DefaultSocketFactory@94a09a8[bindAddress=/10.20.220.15]
                            2008-04-30 18:23:09,323 DEBUG [org.jboss.naming.Naming] NamingServer stub: NamingServer_Stub[UnicastRef2 [liveRef: [endpoint:[10.20.220.15:1098,org.jboss.net.sockets.DefaultSocketFactory@94a09a8[bindAddress=/10.20.220.15]](local),objID:[704fe44c:1199f64bbf7:-7fff, -3502206036617709811]]]]
                            2008-04-30 18:23:09,331 DEBUG [org.jboss.naming.Naming] JNDI bootstrap JNP=/10.20.220.15:1099, RMI=/10.20.220.15:1098, backlog=50, no client SocketFactory, Server SocketFactory=org.jboss.net.sockets.DefaultSocketFactory@94a09a8[bindAddress=/10.20.220.15]
                            2008-04-30 18:23:09,336 DEBUG [org.jboss.naming.NamingService] Started jboss:service=Naming
                            2008-04-30 18:23:09,336 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=Naming dependent components: []
                            2008-04-30 18:23:09,336 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=Naming
                            2008-04-30 18:23:09,336 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=Naming
                            2008-04-30 18:23:09,336 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=JNDIView
                            2008-04-30 18:23:09,337 DEBUG [org.jboss.naming.JNDIView] Starting jboss:service=JNDIView
                            2008-04-30 18:23:09,337 DEBUG [org.jboss.naming.JNDIView] Started jboss:service=JNDIView
                            2008-04-30 18:23:09,337 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=JNDIView dependent components: []
                            2008-04-30 18:23:09,337 DEBUG [org.jboss.system.ServiceController] starting service jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,337 DEBUG [org.jboss.security.plugins.SecurityConfig] Starting jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,338 DEBUG [org.jboss.security.plugins.SecurityConfig] Installed JAAS Configuration service=jboss.security:service=XMLLoginConfig, config=org.jboss.security.auth.login.XMLLoginConfigImpl@1a7f9dc
                            2008-04-30 18:23:09,338 DEBUG [org.jboss.security.plugins.SecurityConfig] Started jboss.security:service=SecurityConfig
                            2008-04-30 18:23:09,338 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.security:service=SecurityConfig dependent components: []
                            2008-04-30 18:23:09,338 DEBUG [org.jboss.system.ServiceController] starting service jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,338 DEBUG [org.jboss.security.auth.login.XMLLoginConfig] Starting jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,339 DEBUG [org.jboss.security.auth.login.XMLLoginConfigImpl] Try loading config as XML, url=file:/home/praveena/jboss-4.2.2.GA/server/all/conf/login-config.xml
                            2008-04-30 18:23:09,375 DEBUG [org.jboss.security.auth.login.XMLLoginConfig] Started jboss.security:service=XMLLoginConfig
                            2008-04-30 18:23:09,375 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.security:service=XMLLoginConfig dependent components: []
                            2008-04-30 18:23:09,375 DEBUG [org.jboss.system.ServiceController] starting service jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,375 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] Starting jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,381 DEBUG [org.jboss.security.SecurityAssociation] Using ThreadLocal: false
                            2008-04-30 18:23:09,387 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] securityMgrCtxPath=java:/jaas
                            2008-04-30 18:23:09,388 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] cachePolicyCtxPath=java:/timedCacheFactory
                            2008-04-30 18:23:09,389 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] SecurityProxyFactory=org.jboss.security.SubjectSecurityProxyFactory@1071521
                            2008-04-30 18:23:09,390 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] Registered PrincipalEditor, SecurityDomainEditor
                            2008-04-30 18:23:09,390 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] ServerMode=true
                            2008-04-30 18:23:09,390 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] SecurityMgrClass=class org.jboss.security.plugins.JaasSecurityManager
                            2008-04-30 18:23:09,390 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] CallbackHandlerClass=class org.jboss.security.auth.callback.SecurityAssociationHandler
                            2008-04-30 18:23:09,391 DEBUG [org.jboss.security.plugins.JaasSecurityManagerService] Started jboss.security:service=JaasSecurityManager
                            2008-04-30 18:23:09,391 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.security:service=JaasSecurityManager dependent components: []
                            2008-04-30 18:23:09,391 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=XidFactory
                            2008-04-30 18:23:09,391 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=XidFactory dependent components: []
                            2008-04-30 18:23:09,391 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=TransactionManager
                            2008-04-30 18:23:09,391 DEBUG [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Starting jboss:service=TransactionManager
                            2008-04-30 18:23:09,391 INFO [com.arjuna.ats.jbossatx.jta.TransactionManagerService] JBossTS Transaction Service (JTA version) - JBoss Inc.
                            2008-04-30 18:23:09,391 INFO [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Setting up property manager MBean and JMX layer
                            2008-04-30 18:23:09,474 DEBUG [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.recovery.TransactionStatusManager_1] - Starting service com.arjuna.ats.arjuna.recovery.ActionStatusService on port 40528
                            2008-04-30 18:23:09,480 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger] TransactionStatusManagerItem - host: 127.0.0.1 port: 40528
                            2008-04-30 18:23:09,486 DEBUG [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.recovery.TransactionStatusManager_3] - TransactionStatusManager started on port 40528 with service com.arjuna.ats.arjuna.recovery.ActionStatusService
                            2008-04-30 18:23:09,496 DEBUG [com.arjuna.ats.tsmx.logging.tsmxLogger] Registering mbean for module 'arjuna'
                            2008-04-30 18:23:09,499 DEBUG [com.arjuna.ats.tsmx.logging.tsmxLogger] Initialising JMX agent com.arjuna.ats.internal.jbossatx.agent.LocalJBossAgentImpl
                            2008-04-30 18:23:09,503 DEBUG [com.arjuna.ats.tsmx.logging.tsmxLogger] Registering mbean for module 'jta'
                            2008-04-30 18:23:09,504 DEBUG [com.arjuna.ats.tsmx.logging.tsmxLogger] Registering mbean for module 'txoj'
                            2008-04-30 18:23:09,515 INFO [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Starting recovery manager
                            2008-04-30 18:23:09,520 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger]
                            --- Start RecoveryActivators
                            2008-04-30 18:23:09,525 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger]
                            --- ExpiredEntryMonitor ----Wed, 30 Apr 2008 18:23:09----
                            2008-04-30 18:23:09,572 DEBUG [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.recovery.noxanodes] No XA recovery nodes specified. Will only recover saved states.
                            2008-04-30 18:23:09,577 DEBUG [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.internal.arjuna.recovery.ready] RecoveryManagerImple is ready on port 59,104
                            2008-04-30 18:23:09,577 INFO [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Recovery manager started
                            2008-04-30 18:23:09,577 INFO [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Binding TransactionManager JNDI Reference
                            2008-04-30 18:23:09,593 DEBUG [com.arjuna.ats.jbossatx.jta.TransactionManagerService] Started jboss:service=TransactionManager
                            2008-04-30 18:23:09,594 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=TransactionManager dependent components: [ObjectName: jboss:service=invoker,type=unified
                            State: CREATED
                            I Depend On:
                            jboss:service=TransactionManager
                            jboss.remoting:service=Connector,transport=socket
                            , ObjectName: jboss:service=invoker,type=jrmp
                            State: CREATED
                            I Depend On:
                            jboss:service=TransactionManager
                            Depends On Me:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            , ObjectName: jboss:service=invoker,type=local
                            State: CREATED
                            I Depend On:
                            jboss:service=TransactionManager
                            , ObjectName: jboss:service=invoker,type=pooled
                            State: CREATED
                            I Depend On:
                            jboss:service=TransactionManager
                            ]
                            2008-04-30 18:23:09,594 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,594 DEBUG [org.jboss.system.ServiceController] waiting in start jboss:service=invoker,type=unified on jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,594 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,594 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Starting jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,600 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] RMI Port='4444'
                            2008-04-30 18:23:09,600 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Client SocketFactory='Default'
                            2008-04-30 18:23:09,600 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Server SocketFactory='org.jboss.net.sockets.DefaultSocketFactory@94a09a8[bindAddress=/10.20.220.15]'
                            2008-04-30 18:23:09,600 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Server SocketAddr='10.20.220.15'
                            2008-04-30 18:23:09,600 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] SecurityDomain='Default'
                            2008-04-30 18:23:09,614 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Bound JRMP invoker for JMX node
                            2008-04-30 18:23:09,614 DEBUG [org.jboss.invocation.jrmp.server.JRMPInvoker] Started jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,614 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=invoker,type=jrmp dependent components: [ObjectName: jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            State: CREATED
                            I Depend On:
                            jboss:service=invoker,type=jrmp
                            Depends On Me:
                            jboss:service=ClientUserTransaction
                            , ObjectName: jboss:service=proxyFactory,target=ClientUserTransaction
                            State: CREATED
                            I Depend On:
                            jboss:service=invoker,type=jrmp
                            Depends On Me:
                            jboss:service=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,614 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,614 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Starting jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,624 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Created JRMPPRoxy for service=jboss:service=ClientUserTransaction, nameHash=-2097396599, invoker=jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,643 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Bound proxy under jndiName=UserTransactionSessionFactory
                            2008-04-30 18:23:09,643 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Started jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,644 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=proxyFactory,target=ClientUserTransactionFactory dependent components: [ObjectName: jboss:service=ClientUserTransaction
                            State: CREATED
                            I Depend On:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,644 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,644 DEBUG [org.jboss.system.ServiceController] waiting in start jboss:service=ClientUserTransaction on jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,644 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,644 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Starting jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,647 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Created JRMPPRoxy for service=jboss:service=ClientUserTransaction, nameHash=2024786241, invoker=jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,648 DEBUG [org.jboss.invocation.jrmp.server.JRMPProxyFactory] Started jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,648 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=proxyFactory,target=ClientUserTransaction dependent components: [ObjectName: jboss:service=ClientUserTransaction
                            State: CREATED
                            I Depend On:
                            jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            jboss:service=proxyFactory,target=ClientUserTransaction
                            ]
                            2008-04-30 18:23:09,648 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,648 DEBUG [org.jboss.tm.usertx.server.ClientUserTransactionService] Starting jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,662 DEBUG [org.jboss.tm.usertx.server.ClientUserTransactionService] Started jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,662 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=ClientUserTransaction dependent components: []
                            2008-04-30 18:23:09,662 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=local
                            2008-04-30 18:23:09,662 DEBUG [org.jboss.invocation.local.LocalInvoker] Starting jboss:service=invoker,type=local
                            2008-04-30 18:23:09,664 DEBUG [org.jboss.invocation.local.LocalInvoker] Local invoker for JMX node started
                            2008-04-30 18:23:09,665 DEBUG [org.jboss.invocation.local.LocalInvoker] Started jboss:service=invoker,type=local
                            2008-04-30 18:23:09,665 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=invoker,type=local dependent components: []
                            2008-04-30 18:23:09,665 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,665 DEBUG [org.jboss.invocation.pooled.server.PooledInvoker] Starting jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.invocation.pooled.server.PooledInvoker] Bound invoker for JMX node
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.invocation.pooled.server.PooledInvoker] Started jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=invoker,type=pooled dependent components: []
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=ClientUserTransaction
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=proxyFactory,target=ClientUserTransactionFactory
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=proxyFactory,target=ClientUserTransaction
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] waiting in start jboss:service=invoker,type=unified on jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=invoker,type=jrmp
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=local
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=invoker,type=local
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss:service=invoker,type=pooled
                            2008-04-30 18:23:09,671 DEBUG [org.jboss.system.ServiceController] starting service jboss.remoting:service=NetworkRegistry
                            2008-04-30 18:23:09,672 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.remoting:service=NetworkRegistry dependent components: [ObjectName: jboss.remoting:service=Connector,transport=socket
                            State: CREATED
                            I Depend On:
                            jboss.remoting:service=NetworkRegistry
                            Depends On Me:
                            jboss:service=invoker,type=unified
                            ]
                            2008-04-30 18:23:09,672 DEBUG [org.jboss.system.ServiceController] starting service jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,675 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[UNINITIALIZED] added org.jboss.invocation.unified.server.UnifiedInvoker@adf5be for subsystem 'invoker'
                            2008-04-30 18:23:09,675 DEBUG [org.jboss.remoting.transport.socket.SocketServerInvoker] SocketServerInvoker[UNINITIALIZED] starting
                            2008-04-30 18:23:09,676 DEBUG [org.jboss.remoting.ServerInvoker] SocketServerInvoker[10.20.220.15:4446] started for locator InvokerLocator [socket://10.20.220.15:4446/?dataType=invocation&enableTcpNoDelay=true&marshaller=org.jboss.invocation.unified.marshall.InvocationMarshaller&socketTimeout=600000&unmarshaller=org.jboss.invocation.unified.marshall.InvocationUnMarshaller]
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.remoting.transport.socket.SocketServerInvoker] SocketServerInvoker[10.20.220.15:4446] started
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.remoting.transport.Connector] org.jboss.remoting.transport.Connector@1fe4169 started
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.remoting:service=Connector,transport=socket dependent components: [ObjectName: jboss:service=invoker,type=unified
                            State: CREATED
                            I Depend On:
                            jboss:service=TransactionManager
                            jboss.remoting:service=Connector,transport=socket
                            ]
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.system.ServiceController] starting service jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.invocation.unified.server.UnifiedInvoker] Starting jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,677 DEBUG [org.jboss.invocation.unified.server.UnifiedInvoker] Starting unified invoker service.
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.invocation.unified.server.UnifiedInvoker] Started jboss:service=invoker,type=unified
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss:service=invoker,type=unified dependent components: []
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.system.ServiceController] starting service jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.system.ServiceController] Ignoring start request for service: jboss.remoting:service=Connector,transport=socket
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.system.ServiceController] starting service jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,691 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] Starting jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,692 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] Loading standardjbosscmp-jdbc.xml : file:/home/praveena/jboss-4.2.2.GA/server/all/conf/standardjbosscmp-jdbc.xml
                            2008-04-30 18:23:09,727 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: FirstSQL/J
                            2008-04-30 18:23:09,729 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Ingres
                            2008-04-30 18:23:09,730 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: McKoi
                            2008-04-30 18:23:09,731 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Firebird
                            2008-04-30 18:23:09,732 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: InterBase
                            2008-04-30 18:23:09,733 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: DB2
                            2008-04-30 18:23:09,734 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Derby
                            2008-04-30 18:23:09,735 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Oracle9i
                            2008-04-30 18:23:09,737 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Oracle8
                            2008-04-30 18:23:09,738 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Oracle7
                            2008-04-30 18:23:09,739 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Sybase
                            2008-04-30 18:23:09,740 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: PostgreSQL
                            2008-04-30 18:23:09,741 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: PostgreSQL 8.0
                            2008-04-30 18:23:09,741 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: PostgreSQL 7.2
                            2008-04-30 18:23:09,742 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Hypersonic SQL
                            2008-04-30 18:23:09,743 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: PointBase
                            2008-04-30 18:23:09,743 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: SOLID
                            2008-04-30 18:23:09,744 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: mySQL
                            2008-04-30 18:23:09,745 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: MS SQLSERVER
                            2008-04-30 18:23:09,746 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: MS SQLSERVER2000
                            2008-04-30 18:23:09,747 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: DB2/400
                            2008-04-30 18:23:09,747 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: SapDB
                            2008-04-30 18:23:09,748 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: MaxDB
                            2008-04-30 18:23:09,748 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Cloudscape
                            2008-04-30 18:23:09,749 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: InformixDB
                            2008-04-30 18:23:09,750 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] added type-mapping: Mimer SQL
                            2008-04-30 18:23:09,750 DEBUG [org.jboss.ejb.plugins.cmp.jdbc.metadata.MetaDataLibrary] Started jboss.jdbc:service=metadata
                            2008-04-30 18:23:09,750 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.jdbc:service=metadata dependent components: []
                            2008-04-30 18:23:09,750 DEBUG [org.jboss.system.ServiceController] starting service jboss.deployment:type=DeploymentScanner,flavor=URL
                            2008-04-30 18:23:09,750 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Starting jboss.deployment:type=DeploymentScanner,flavor=URL
                            2008-04-30 18:23:09,754 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment of package: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,754 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment (init step) of package at: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,754 DEBUG [org.jboss.deployment.MainDeployer] using deployer org.jboss.deployment.SARDeployer@1d36dfe
                            2008-04-30 18:23:09,755 DEBUG [org.jboss.deployment.SARDeployer] Updated watch URL to: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/META-INF/jboss-service.xml
                            2008-04-30 18:23:09,757 DEBUG [org.jboss.deployment.SARDeployer] about to copy 0 local directories
                            2008-04-30 18:23:09,757 DEBUG [org.jboss.deployment.SARDeployer] looking for nested deployments in : file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.deployment.SARDeployer] nested deployment: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.deployment.SARDeployer] nested deployment: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.deployment.DeploymentInfo] createLoaderRepository from config: LoaderRepositoryConfig(repositoryName: JMImplementation:service=LoaderRepository,name=Default, repositoryClassName: null, configParserClassName: null, repositoryConfig: null)
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@126f75b, cl=org.jboss.mx.loading.UnifiedClassLoader3@57807a{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@126f75b, cl=org.jboss.mx.loading.UnifiedClassLoader3@57807a{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,758 DEBUG [org.jboss.mx.loading.UnifiedLoaderRepository3] Adding org.jboss.mx.loading.UnifiedClassLoader3@57807a{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,759 DEBUG [org.jboss.mx.loading.ClassLoaderUtils] Multiple class loaders found for pkg:
                            2008-04-30 18:23:09,759 DEBUG [org.jboss.deployment.MainDeployer] found 2 subpackages of file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,759 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment (init step) of package at: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,759 DEBUG [org.jboss.deployment.MainDeployer] Copying file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar -> /home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15040jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,761 DEBUG [org.jboss.deployment.JARDeployer] No xml files found
                            2008-04-30 18:23:09,761 DEBUG [org.jboss.deployment.MainDeployer] using deployer org.jboss.deployment.JARDeployer@178460d
                            2008-04-30 18:23:09,761 DEBUG [org.jboss.deployment.JARDeployer] looking for nested deployments in : file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,769 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] Added url: file:/home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15040jboss-annotations-ejb3.jar, to ucl: org.jboss.mx.loading.UnifiedClassLoader3@57807a{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ ,addedOrder=3}
                            2008-04-30 18:23:09,769 DEBUG [org.jboss.deployment.MainDeployer] found 0 subpackages of file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,770 DEBUG [org.jboss.deployment.MainDeployer] Watching new file: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,770 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment (init step) of package at: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,770 DEBUG [org.jboss.deployment.MainDeployer] Copying file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar -> /home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15041jboss-ejb3.jar
                            2008-04-30 18:23:09,776 DEBUG [org.jboss.deployment.JARDeployer] No xml files found
                            2008-04-30 18:23:09,776 DEBUG [org.jboss.deployment.MainDeployer] using deployer org.jboss.deployment.JARDeployer@178460d
                            2008-04-30 18:23:09,776 DEBUG [org.jboss.deployment.JARDeployer] looking for nested deployments in : file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,802 DEBUG [org.jboss.mx.loading.ClassLoaderUtils] Multiple class loaders found for pkg: org.jboss.ejb
                            2008-04-30 18:23:09,845 DEBUG [org.jboss.mx.loading.ClassLoaderUtils] Multiple class loaders found for pkg: schema
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] Added url: file:/home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15041jboss-ejb3.jar, to ucl: org.jboss.mx.loading.UnifiedClassLoader3@57807a{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ ,addedOrder=3}
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] found 0 subpackages of file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] Watching new file: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] Watching new file: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] create step for deployment file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] create step for deployment file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,853 DEBUG [org.jboss.deployment.MainDeployer] Done with create step of deploying jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,854 DEBUG [org.jboss.deployment.MainDeployer] create step for deployment file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,854 DEBUG [org.jboss.deployment.MainDeployer] Done with create step of deploying jboss-ejb3.jar
                            2008-04-30 18:23:09,854 DEBUG [org.jboss.deployment.SARDeployer] Deploying SAR, create step: url file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,854 DEBUG [org.jboss.deployment.SARDeployer] Registering service UCL=jmx.loading:UCL=57807a
                            2008-04-30 18:23:09,855 DEBUG [org.jboss.system.ServiceCreator] About to create bean: jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3 with code: org.jboss.remoting.transport.Connector
                            2008-04-30 18:23:09,857 DEBUG [org.jboss.system.ServiceCreator] Created bean: jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,857 DEBUG [org.jboss.system.ServiceController] recording that jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3 depends on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,857 DEBUG [org.jboss.system.ServiceConfigurator] considering with object name jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,858 DEBUG [org.jboss.system.ServiceConfigurator] InvokerLocator set to socket://10.20.220.15:3873 in jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,861 DEBUG [org.jboss.system.ServiceConfigurator] Configuration set to [handlers: null] in jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,861 DEBUG [org.jboss.system.ServiceCreator] About to create bean: jboss.ejb3:service=JarsIgnoredForScanning with code: org.jboss.ejb3.JarsIgnoredForScanning
                            2008-04-30 18:23:09,866 DEBUG [org.jboss.system.ServiceCreator] Created bean: jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,867 DEBUG [org.jboss.system.ServiceConfigurator] IgnoredJars set to [Ljava.lang.String;@c5122f in jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,867 DEBUG [org.jboss.system.ServiceCreator] About to create xmbean object: jboss.ejb3:service=EJB3Deployer with code: org.jboss.ejb3.EJB3Deployer with embedded descriptor
                            2008-04-30 18:23:09,883 DEBUG [org.jboss.system.ServiceCreator] Created bean: jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceConfigurator] DeployEjb3ExtensionOnly set to false in jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceController] recording that jboss.ejb3:service=EJB3Deployer depends on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceConfigurator] considering with object name jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceController] recording that jboss.ejb3:service=EJB3Deployer depends on jboss.ejb:service=EJBDeployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceConfigurator] considering with object name jboss.ejb:service=EJBDeployer
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceController] recording that jboss.ejb3:service=EJB3Deployer depends on jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,884 DEBUG [org.jboss.system.ServiceConfigurator] considering JarsIgnoredForScanning with object name jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating service jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3 waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating service jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating dependent components for: jboss.ejb3:service=JarsIgnoredForScanning dependents are: [ObjectName: jboss.ejb3:service=EJB3Deployer
                            State: CONFIGURED
                            I Depend On:
                            jboss.aop:service=AspectDeployer
                            jboss.ejb:service=EJBDeployer
                            jboss.ejb3:service=JarsIgnoredForScanning
                            ]
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.ejb3:service=EJB3Deployer waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.ejb3:service=EJB3Deployer waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] Done with create step of deploying ejb3.deployer
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] Begin deployment start file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] Begin deployment start file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] End deployment start on package: jboss-annotations-ejb3.jar
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] Begin deployment start file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/jboss-ejb3.jar
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.MainDeployer] End deployment start on package: jboss-ejb3.jar
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.deployment.SARDeployer] Deploying SAR, start step: url file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] starting service jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Start requested before create, calling create now for service: jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,886 DEBUG [org.jboss.system.ServiceController] Creating service jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3 waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in start jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3 on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] starting service jboss.ejb3:service=JarsIgnoredForScanning
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] Starting dependent components for: jboss.ejb3:service=JarsIgnoredForScanning dependent components: [ObjectName: jboss.ejb3:service=EJB3Deployer
                            State: CONFIGURED
                            I Depend On:
                            jboss.aop:service=AspectDeployer
                            jboss.ejb:service=EJBDeployer
                            jboss.ejb3:service=JarsIgnoredForScanning
                            ]
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] starting service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] Start requested before create, calling create now for service: jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] Creating service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.ejb3:service=EJB3Deployer waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in start jboss.ejb3:service=EJB3Deployer on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] starting service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] Start requested before create, calling create now for service: jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] Creating service jboss.ejb3:service=EJB3Deployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in create of jboss.ejb3:service=EJB3Deployer waiting on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,887 DEBUG [org.jboss.system.ServiceController] waiting in start jboss.ejb3:service=EJB3Deployer on jboss.aop:service=AspectDeployer
                            2008-04-30 18:23:09,922 DEBUG [org.jboss.deployment.MainDeployer] End deployment start on package: ejb3.deployer
                            2008-04-30 18:23:09,922 DEBUG [org.jboss.deployment.MainDeployer] Deployed package: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/
                            2008-04-30 18:23:09,924 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Watch URL for: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/ -> file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/ejb3.deployer/META-INF/jboss-service.xml
                            2008-04-30 18:23:09,924 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment of package: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/
                            2008-04-30 18:23:09,924 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment (init step) of package at: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/
                            2008-04-30 18:23:09,924 DEBUG [org.jboss.deployment.MainDeployer] using deployer org.jboss.deployment.SARDeployer@1d36dfe
                            2008-04-30 18:23:09,925 DEBUG [org.jboss.deployment.SARDeployer] Updated watch URL to: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/META-INF/jboss-service.xml
                            2008-04-30 18:23:09,925 DEBUG [org.jboss.deployment.SARDeployer] about to copy 0 local directories
                            2008-04-30 18:23:09,925 DEBUG [org.jboss.deployment.SARDeployer] looking for nested deployments in : file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/
                            2008-04-30 18:23:09,926 DEBUG [org.jboss.deployment.SARDeployer] nested deployment: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/trove.jar
                            2008-04-30 18:23:09,926 DEBUG [org.jboss.deployment.SARDeployer] nested deployment: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/jboss-aspect-library-jdk50.jar
                            2008-04-30 18:23:09,926 DEBUG [org.jboss.deployment.SARDeployer] nested deployment: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/jboss-aop-jdk50.jar
                            2008-04-30 18:23:09,926 DEBUG [org.jboss.deployment.DeploymentInfo] createLoaderRepository from config: LoaderRepositoryConfig(repositoryName: JMImplementation:service=LoaderRepository,name=Default, repositoryClassName: null, configParserClassName: null, repositoryConfig: null)
                            2008-04-30 18:23:09,926 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@126f75b, cl=org.jboss.mx.loading.UnifiedClassLoader3@106df95{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@126f75b, cl=org.jboss.mx.loading.UnifiedClassLoader3@106df95{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.mx.loading.UnifiedLoaderRepository3] Adding org.jboss.mx.loading.UnifiedClassLoader3@106df95{ url=file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/ ,addedOrder=0}
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.mx.loading.ClassLoaderUtils] Multiple class loaders found for pkg:
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.deployment.MainDeployer] found 3 subpackages of file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.deployment.MainDeployer] Starting deployment (init step) of package at: file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/jboss-aop-jdk50.jar
                            2008-04-30 18:23:09,927 DEBUG [org.jboss.deployment.MainDeployer] Copying file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/jboss-aop-jdk50.jar -> /home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15042jboss-aop-jdk50.jar
                            2008-04-30 18:23:09,934 DEBUG [org.jboss.deployment.JARDeployer] No xml files found
                            2008-04-30 18:23:09,934 DEBUG [org.jboss.deployment.MainDeployer] using deployer org.jboss.deployment.JARDeployer@178460d
                            2008-04-30 18:23:09,934 DEBUG [org.jboss.deployment.JARDeployer] looking for nested deployments in : file:/home/praveena/jboss-4.2.2.GA/server/all/deploy/jboss-aop-jdk50.deployer/jboss-aop-jdk50.jar
                            2008-04-30 18:23:10,009 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] Added url: file:/home/praveena/jboss-4.2.2.GA/server/all/tmp/deploy/tmp15042jboss-aop-jdk50.jar, to ucl: org.jboss.mx.loading.UnifiedCla

                            • 11. Re: Unable to run multiple cluster nodes.
                              Brian Stansberry Master

                              As I've already said, the NPE is in your code and you'll have to figure that out yourself. I've no idea what the latest server.log is supposed to be telling me.

                              If you are looking for real-time support, I suggest you contact Red Hat and engage one or another of the services described at http://www.jboss.com/services/index .

                              • 12. Re: Unable to run multiple cluster nodes.
                                praveena chalamcharla Newbie

                                HI,

                                Thank you for your suggestions.

                                We are on the way to fix all those bugs. But, a minor issue observed by me when clustering a sample application like "Hello world".

                                When the 2 cluster nodes are running and when one of them which is interacting with the browser currently(at that time) suddenly fails, until the other one is about to receive this status, the application is not being focussed to be clustered well. Of course, it is a very minute period. But the target of clustering is that, the end user must not identify the switching of the server..

                                Let me know if any alternative for this, like the server which is not currently active may take some secondary back up, so that, it may act immediately, keeping the customer transperant about this switching.

                                Thank you.

                                Praveena,
                                Securview.