6 Replies Latest reply on Jan 15, 2014 2:41 PM by Ramesh Reddy

    Accumulo Connection

    adam williams Newbie

      Hello,

       

      When i try to use the Accumulo connector with 8.7 currently, I am unable to connect due to a class not found exception.  My modules look to be fine for Apache/commons/io, but is there something else I should be looking at?  I am able to establish a socket connection to the Accumulo instance, and i define my VDB and standalone-teiid as follows:\

       

      Standalone-teiid resource adapter: -- i modified the IP address here, but it is making a connection.

      <resource-adapter id="accumulo">
                  <module slot="main" id="org.jboss.teiid.resource-adapter.accumulo"/>
                  <transaction-support>NoTransaction</transaction-support>
                  <connection-definitions>
                      <connection-definition class-name="org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory"
                              jndi-name="java:/accumuloDS"
                              enabled="true"
                              use-java-context="true"
                              pool-name="teiid-accumulo-ds">
                             
                            <config-property name="ZooKeeperServerList">#.#.#.#:2181</config-property>
                            <config-property name="InstanceName">sampleInst</config-property>
                            <config-property name="Username">adam</config-property>
                            <config-property name="Password">mypass</config-property>
                            <config-property name="Roles">public</config-property>
                           
                      </connection-definition>
                  </connection-definitions>
              </resource-adapter>
      
      

       

       

       

       

      My vdb: accum-vdb.xml

      <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
        <vdb name="accum" version="1">
          <description>accum Test</description>
           <model name="accumModel">
              <source name="accumulo-connector" translator-name="accumulo" connection-jndi-name="java:/accumuloDS"/>
          </model>
      </vdb>
      
      

       

      The output:

      08:04:00,331 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2)  JBAS015876: Starting deployment of "accum-vdb.xml" (runtime-name: "accum-vdb.xml")

      08:04:00,344 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-3)  JBAS015012: Started FileSystemDeploymentService for directory /home/awilliams/jboss-eap-6.1/standalone/deployments

      08:04:00,869 INFO  [org.jboss.web] (ServerService Thread Pool -- 52)  JBAS018210: Register web context: /odata

      08:04:01,070 INFO  [org.teiid.RUNTIME] (MSC service thread 1-1)  TEIID50029 VDB accum.1 model "accumModel" metadata is currently being loaded. Start Time: 1/15/14 8:04 AM

      08:04:01,170 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1)  IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: org/apache/commons/io/output/ByteArrayOutputStream

              at org.apache.accumulo.core.client.ZooKeeperInstance.getConnector(ZooKeeperInstance.java:222)

              at org.teiid.resource.adapter.accumulo.AccumuloConnectionImpl.<init>(AccumuloConnectionImpl.java:63)

              at org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory$AccumuloConnectionFactory.getConnection(AccumuloManagedConnectionFactory.java:60)

              at org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory$AccumuloConnectionFactory.getConnection(AccumuloManagedConnectionFactory.java:49)

              at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:70) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:782) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:345) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:302) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

              at org.teiid.resource.spi.WrappedConnectionFactory.getConnection(WrappedConnectionFactory.java:58) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.translator.ExecutionFactory.getConnection(ExecutionFactory.java:170) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.translator.ExecutionFactory.getConnection(ExecutionFactory.java:191) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.query.metadata.NativeMetadataRepository.loadMetadata(NativeMetadataRepository.java:46) [teiid-engine-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.query.metadata.ChainingMetadataRepository.loadMetadata(ChainingMetadataRepository.java:55) [teiid-engine-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.jboss.VDBService$6.run(VDBService.java:403) [teiid-jboss-integration-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.teiid.jboss.VDBService$7.run(VDBService.java:450) [teiid-jboss-integration-8.7.0.Alpha1.jar:8.7.0.Alpha1]

              at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)

              at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:806)

              at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)

              at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:826)

              at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

              at org.jboss.threads.JBossThread.run(JBossThread.java:122)

      Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.output.ByteArrayOutputStream from [Module "org.apache.accumulo:main" from local module loader @bf4c346 (finder: local module finder @72402ecb (roots: /home/awilliams/jboss-eap-6.1/modules,/home/awilliams/jboss-eap-6.1/modules/system/layers/base))]

              at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190) [jboss-modules.jar:1.2.0.CR1]

              at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468) [jboss-modules.jar:1.2.0.CR1]

              at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456) [jboss-modules.jar:1.2.0.CR1]

              at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) [jboss-modules.jar:1.2.0.CR1]

              at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120) [jboss-modules.jar:1.2.0.CR1]

              ... 25 more

       

       

      08:04:01,188 WARN  [org.teiid.RUNTIME] (teiid-async-threads - 1)  TEIID50036 VDB accum.1 model "accumModel" metadata failed to load. Reason:TEIID60000 javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/accumuloDS

       

       

      Is my configuration off at all, or should i modify my Apache commons module?

       

      thanks!

        • 1. Re: Accumulo Connection
          Tom Arnold Novice

          This is just a guess, but try adding commons-io to the Accumulo module.

           

          modules/system/layers/base/org/apache/accumulo/main/module.xml

           

          Add this to the dependencies.

           

          <module name="org.apache.commons.io"/>

           

          I think you need to restart JBoss after messing with the modules.

           

          Tom

          1 of 1 people found this helpful
          • 2. Re: Accumulo Connection
            adam williams Newbie

            Good thinking!  I added it there and to the accumulo main.  I also hit a hadoop.io.text classnotfound afterwards in the accumulo translator in case that occurs for someone else.  Just added the module to the accumulo translator main.

            • 3. Re: Accumulo Connection
              Tom Arnold Novice

              I wrote an issue for this (https://issues.jboss.org/browse/TEIID-2805) so these changes get back upstream.

               

              Edited my original reply to put the path to the Accumulo module. I originally thought the problem was in Zookeeper, and forgot to update the module path when I edited the post the first time.

              • 4. Re: Accumulo Connection
                adam williams Newbie

                When i try to query accumulo right now i keep hitting this:

                 

                it says my authorizations are off.  I wrote the data to accumulo without any authorizations, so how can i do a simple select * from a table?

                org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 accumulo-connector: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

                  at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)

                  at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)

                  at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:667)

                  at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:63)

                  at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:515)

                  at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135)

                  at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40)

                  at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79)

                  at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:257)

                  at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:295)

                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                  at java.lang.reflect.Method.invoke(Method.java:606)

                  at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:102)

                  at com.sun.proxy.$Proxy1.read(Unknown Source)

                  at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:394)

                  at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:524)

                  at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)

                  at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:327)

                  at JDBCClient.execute(JDBCClient.java:107)

                  at JDBCClient.main(JDBCClient.java:67)

                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                  at java.lang.reflect.Method.invoke(Method.java:606)

                  at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

                Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 accumulo-connector: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

                  at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:381)

                  at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:154)

                  at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:369)

                  at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:278)

                  at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:136)

                  at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)

                  at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)

                  at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:159)

                  at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:141)

                  at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:435)

                  at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:320)

                  at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)

                  at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:248)

                  at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:273)

                  at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)

                  at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:214)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

                  at java.lang.Thread.run(Thread.java:662)

                Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

                  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleError(ConnectorWorkItem.java:262)

                  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:202)

                  at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:301)

                  at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:110)

                  at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:107)

                  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

                  at java.util.concurrent.FutureTask.run(FutureTask.java:138)

                  at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:58)

                  ... 6 more

                Caused by: org.teiid.core.TeiidRuntimeException: Remote org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

                  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:442)

                  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:266)

                  at org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:82)

                  at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:164)

                  at org.teiid.translator.accumulo.AccumuloQueryExecution.readNextRow(AccumuloQueryExecution.java:126)

                  at org.teiid.translator.accumulo.AccumuloQueryExecution.next(AccumuloQueryExecution.java:150)

                  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleBatch(ConnectorWorkItem.java:376)

                  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:200)

                  ... 12 more

                Caused by: org.teiid.core.TeiidRuntimeException: Remote org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException: null

                  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:4652)

                  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:4629)

                  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result.read(TabletClientService.java:4552)

                  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

                  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:209)

                  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:186)

                  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:387)

                  ... 19 more

                • 5. Re: Accumulo Connection
                  devin pinkston Newbie

                  most likely the visibility/roles are not correct so when the authorizations are created on a scanner it cannot read.

                  • 6. Re: Accumulo Connection
                    Ramesh Reddy Master

                    Adam,

                     

                    All the users must have roles in accumulo, so give the user some role like "public". This role needs to be mentioned on the resource-adapter connection properties right now.

                     

                    I have yet to clean-up the roles to propagate cleanly though teiid system, such that teiid user roles propagate to the accumulo roles or some kind of mapping etc. I will do that before before I close this issue [TEIID-2627] Accumulo as a Supported Data Source

                     

                    Adam, Tom, Devin: keep those question coming, I know I have little more work to be done on this translator. Any suggestions on improvements is most welcome. we have enough time to finalize those in 8.7 development cycle.

                     

                    Ramesh..