Accumulo Connection

[awilliams84]

Hello,

When i try to use the Accumulo connector with 8.7 currently, I am unable to connect due to a class not found exception.  My modules look to be fine for Apache/commons/io, but is there something else I should be looking at?  I am able to establish a socket connection to the Accumulo instance, and i define my VDB and standalone-teiid as follows:\

Standalone-teiid resource adapter: -- i modified the IP address here, but it is making a connection.

<resource-adapter id="accumulo">
            <module slot="main" id="org.jboss.teiid.resource-adapter.accumulo"/>
            <transaction-support>NoTransaction</transaction-support>
            <connection-definitions>
                <connection-definition class-name="org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory"
                        jndi-name="java:/accumuloDS"
                        enabled="true"
                        use-java-context="true"
                        pool-name="teiid-accumulo-ds">
                       
                      <config-property name="ZooKeeperServerList">#.#.#.#:2181</config-property>
                      <config-property name="InstanceName">sampleInst</config-property>
                      <config-property name="Username">adam</config-property>
                      <config-property name="Password">mypass</config-property>
                      <config-property name="Roles">public</config-property>
                     
                </connection-definition>
            </connection-definitions>
        </resource-adapter>

My vdb: accum-vdb.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <vdb name="accum" version="1">
    <description>accum Test</description>
     <model name="accumModel">
        <source name="accumulo-connector" translator-name="accumulo" connection-jndi-name="java:/accumuloDS"/>
    </model>
</vdb>

The output:

08:04:00,331 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2)  JBAS015876: Starting deployment of "accum-vdb.xml" (runtime-name: "accum-vdb.xml")

08:04:00,344 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-3)  JBAS015012: Started FileSystemDeploymentService for directory /home/awilliams/jboss-eap-6.1/standalone/deployments

08:04:00,869 INFO  [org.jboss.web] (ServerService Thread Pool -- 52)  JBAS018210: Register web context: /odata

08:04:01,070 INFO  [org.teiid.RUNTIME] (MSC service thread 1-1)  TEIID50029 VDB accum.1 model "accumModel" metadata is currently being loaded. Start Time: 1/15/14 8:04 AM

08:04:01,170 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1)  IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: org/apache/commons/io/output/ByteArrayOutputStream

        at org.apache.accumulo.core.client.ZooKeeperInstance.getConnector(ZooKeeperInstance.java:222)

        at org.teiid.resource.adapter.accumulo.AccumuloConnectionImpl.<init>(AccumuloConnectionImpl.java:63)

        at org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory$AccumuloConnectionFactory.getConnection(AccumuloManagedConnectionFactory.java:60)

        at org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory$AccumuloConnectionFactory.getConnection(AccumuloManagedConnectionFactory.java:49)

        at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:70) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:782) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:345) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:302) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) [ironjacamar-core-impl-1.0.15.Final.jar:1.0.15.Final]

        at org.teiid.resource.spi.WrappedConnectionFactory.getConnection(WrappedConnectionFactory.java:58) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.translator.ExecutionFactory.getConnection(ExecutionFactory.java:170) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.translator.ExecutionFactory.getConnection(ExecutionFactory.java:191) [teiid-api-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.query.metadata.NativeMetadataRepository.loadMetadata(NativeMetadataRepository.java:46) [teiid-engine-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.query.metadata.ChainingMetadataRepository.loadMetadata(ChainingMetadataRepository.java:55) [teiid-engine-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.jboss.VDBService$6.run(VDBService.java:403) [teiid-jboss-integration-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.teiid.jboss.VDBService$7.run(VDBService.java:450) [teiid-jboss-integration-8.7.0.Alpha1.jar:8.7.0.Alpha1]

        at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)

        at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:806)

        at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)

        at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:826)

        at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

        at org.jboss.threads.JBossThread.run(JBossThread.java:122)

Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.output.ByteArrayOutputStream from [Module "org.apache.accumulo:main" from local module loader @bf4c346 (finder: local module finder @72402ecb (roots: /home/awilliams/jboss-eap-6.1/modules,/home/awilliams/jboss-eap-6.1/modules/system/layers/base))]

        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190) [jboss-modules.jar:1.2.0.CR1]

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468) [jboss-modules.jar:1.2.0.CR1]

        at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456) [jboss-modules.jar:1.2.0.CR1]

        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) [jboss-modules.jar:1.2.0.CR1]

        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120) [jboss-modules.jar:1.2.0.CR1]

        ... 25 more

08:04:01,188 WARN  [org.teiid.RUNTIME] (teiid-async-threads - 1)  TEIID50036 VDB accum.1 model "accumModel" metadata failed to load. Reason:TEIID60000 javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/accumuloDS

Is my configuration off at all, or should i modify my Apache commons module?

thanks!

[tom9729]

This is just a guess, but try adding commons-io to the Accumulo module.

modules/system/layers/base/org/apache/accumulo/main/module.xml

Add this to the dependencies.

<module name="org.apache.commons.io"/>

I think you need to restart JBoss after messing with the modules.

Tom

[awilliams84]

Good thinking!  I added it there and to the accumulo main.  I also hit a hadoop.io.text classnotfound afterwards in the accumulo translator in case that occurs for someone else.  Just added the module to the accumulo translator main.

[tom9729]

I wrote an issue for this (https://issues.jboss.org/browse/TEIID-2805) so these changes get back upstream.

Edited my original reply to put the path to the Accumulo module. I originally thought the problem was in Zookeeper, and forgot to update the module path when I edited the post the first time.

[awilliams84]

When i try to query accumulo right now i keep hitting this:

it says my authorizations are off.  I wrote the data to accumulo without any authorizations, so how can i do a simple select * from a table?

org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 accumulo-connector: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

  at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)

  at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)

  at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:667)

  at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:63)

  at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:515)

  at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135)

  at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40)

  at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79)

  at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:257)

  at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:295)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:102)

  at com.sun.proxy.$Proxy1.read(Unknown Source)

  at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:394)

  at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:524)

  at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)

  at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:327)

  at JDBCClient.execute(JDBCClient.java:107)

  at JDBCClient.main(JDBCClient.java:67)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 accumulo-connector: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

  at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:381)

  at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:154)

  at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:369)

  at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:278)

  at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:136)

  at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)

  at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)

  at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:159)

  at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:141)

  at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:435)

  at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:320)

  at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)

  at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:248)

  at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:273)

  at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)

  at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:214)

  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

  at java.lang.Thread.run(Thread.java:662)

Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleError(ConnectorWorkItem.java:262)

  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:202)

  at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:301)

  at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:110)

  at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:107)

  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

  at java.util.concurrent.FutureTask.run(FutureTask.java:138)

  at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:58)

  ... 6 more

Caused by: org.teiid.core.TeiidRuntimeException: Remote org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_AUTHORIZATIONS for user root - The user does not have the specified authorizations assigned

  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:442)

  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:266)

  at org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:82)

  at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:164)

  at org.teiid.translator.accumulo.AccumuloQueryExecution.readNextRow(AccumuloQueryExecution.java:126)

  at org.teiid.translator.accumulo.AccumuloQueryExecution.next(AccumuloQueryExecution.java:150)

  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleBatch(ConnectorWorkItem.java:376)

  at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:200)

  ... 12 more

Caused by: org.teiid.core.TeiidRuntimeException: Remote org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException: null

  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:4652)

  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:4629)

  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result.read(TabletClientService.java:4552)

  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:209)

  at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:186)

  at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:387)

  ... 19 more

[pinkstondevin]

most likely the visibility/roles are not correct so when the authorizations are created on a scanner it cannot read.

[rareddy]

Adam,

All the users must have roles in accumulo, so give the user some role like "public". This role needs to be mentioned on the resource-adapter connection properties right now.

I have yet to clean-up the roles to propagate cleanly though teiid system, such that teiid user roles propagate to the accumulo roles or some kind of mapping etc. I will do that before before I close this issue [TEIID-2627] Accumulo as a Supported Data Source

Adam, Tom, Devin: keep those question coming, I know I have little more work to be done on this translator. Any suggestions on improvements is most welcome. we have enough time to finalize those in 8.7 development cycle.

Ramesh..