4 Replies Latest reply on Aug 10, 2013 11:55 AM by Ivan Carvalho

    Where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse?

    Mason Tran Newbie

      Hello,

       

      Anyone know where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse for the profile that we setup in Fuse Server?

       

      Thanks,

      Mason Tran

        • 1. Re: Where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse?
          Mason Tran Newbie

          Hello,

           

          Bascially I wanted to setup Accumulo with JBoss Fuse.  The is what I setup for my HADOOP_HOME and ZOOKEEPER_HOME, and it seems to be working fine at the moment.

           

          HADOOP_HOME = /home/<user>/jboss-fuse-6.0.0.redhat-024/system/org/fusesource/fabric/fabric-hadoop/7.2.0.redhat-024

           

          ZOOKEEPER_HOME = /home/<user>/jboss-fuse-6.0.0.redhat-024/system/org/apache/zookeeper/zookeeper/3.3.5

           

          Thanks,

          Mason Tran

          • 2. Re: Where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse?
            Mason Tran Newbie

            Getting this error on console when I tried to setup Accumulo.  Seems like HADOOP_HOME I am using doesn't have all the hadoop jar files, so is there a way to update in JBoss Fuse so that it will download all the necessary hadoop jar file, like use FMC?  Or maybe I am using the wrong HADOOP_HOME for JBoss Fuse, if anyone know please help.

             

            Because in my HADOOP_HOME, I only have 1 jar file which is "fabric-hadoop-7.2.0.redhat-024.jar".

             

             

            2013-07-30 17:26:58,911 [util.NativeCodeLoader] WARN : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
            Thread "init" died null
            java.lang.reflect.InvocationTargetException
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:616)
            at org.apache.accumulo.start.Main$1.run(Main.java:89)
            at java.lang.Thread.run(Thread.java:679)
            Caused by: java.lang.NoClassDefFoundError: org/apache/commons/codec/binary/Base64
            at org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:79)
            at org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
            at org.apache.accumulo.server.util.Initialize.initSecurity(Initialize.java:411)
            at org.apache.accumulo.server.util.Initialize.doInit(Initialize.java:177)
            at org.apache.accumulo.server.util.Initialize.main(Initialize.java:475)
            ... 6 more
            Caused by: java.lang.ClassNotFoundException: org.apache.commons.codec.binary.Base64
            at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
            ... 11 more

            Thanks,

            Mason Tran

            • 3. Re: Where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse?
              Mason Tran Newbie

              Anyway I got it working.  Here is what I did.

               

              1.    Copy Hadoop JAR files into Accumulo - All Hadoop lib folder into Accumulo lib folder, and also Hadoop jars in hadoop-1.1.2 folder move them to Accumulo lib folder as well.

              2.    Remove duplicate different version of the JAR file from Accumulo lib

              3.     Copy Hadoop conf / xml into Accumulo bin and conf folder.  You do copy everything only stuff that you need.  Such as (hadoop, hadoop-config.sh, hadoop-env.sh, mapred-site.xml, hdfs-site.xml, core-site.xml, hadoop-policy.xml)

              4.     Follow instruction on this site to update (hadoop-env.sh, core-site.xml, hdfs-site.xml, and mapred-site.xml). http://blog.sqrrl.com/post/40578606670/quick-accumulo-install

              5.     You don't have do anything with ZooKeeper.

              6.     Update accumulo-env.sh file for JAVA_HOME, HADOOP_HOME, and ZOOKEEPER_HOME.

              7.    Bring down JBoss Fuse Server.

              8.    Go into Accumulo bin directory and run ./hadoop namenode -format

              9.     Bring JBoss Fuse Server back up.

              10.   Go into Accumulo bin directory and run ./accumulo init

               

               

              accumulo-env.sh file - replace <user> with the location you installed on.

              HADOOP_HOME=/home/<user>/accumulo-1.4.2
              ZOOKEEPER_HOME=/home/<user>/jboss-fuse-6.0.0.redhat-024/system/org/apache/zookeeper/zookeeper/3.3.5

              • 4. Re: Where is HADOOP_HOME and ZOOKEEPER_HOME in JBoss Fuse?
                Ivan Carvalho Newbie

                Hi Mason,

                 

                Just to understand, why are you using Apache Accumulo? What are your requirements for it?

                 

                cheers!

                Ivan