2 Replies Latest reply on Feb 9, 2015 11:38 AM by naskoos

    Creating a YCSB Asynchronous Client using HotRod.

    naskoos

      Hi,

       

      I'm trying to implement an asynchronous client for the YCSB utilizing the HotRod client.

      I already have an Hbase asynchronous client which uses Deferred and it could save me a lot of effort if  I could reuse that code.

       

      Searching around I found an old discussion about the differences between NotifyFuture and Deferred, however nothing came out from this conversation.

       

      Is there any implementation of Deferred inside HotRod or is there any source to help me convert a NotifyFuture implementation to Deferred?

       

      Thank you!

       

      UPDATE

       

      I have implemented the asynchronous client from scratch using NotifyingFuture, however I have a problem with the thread pool size.

      More specifically when I try to stress my DB with many read async requests I'm getting this exception:

       

      ExecutionException: Task java.util.concurrent.FutureTask@733ed393 rejected from java.util.concurrent.ThreadPoolExecutor@561f37f0[Running, pool size = 99, active threads = 99, queued tasks = 10000, completed tasks = 56783]

        at com.yahoo.ycsb.workloads.CoreWorkloadAsyncIspn.doTransaction(CoreWorkloadAsyncIspn.java:415)

        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) at com.yahoo.ycsb.async.ispn.ClientThreadAsyncIspn.run(ClientThreadAsyncIspn.java:154)

        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)

        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)

        at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)

        at org.infinispan.client.hotrod.impl.RemoteCacheImpl.getAsync(RemoteCacheImpl.java:547)

        at com.yahoo.ycsb.db.InfinispanHotRodClientAsync.read(InfinispanHotRodClientAsync.java:64)

        at com.yahoo.ycsb.async.ispn.DBAsyncWrapperIspn.read(DBAsyncWrapperIspn.java:104)

        at com.yahoo.ycsb.workloads.CoreWorkloadAsyncIspn.doTransactionRead(CoreWorkloadAsyncIspn.java:469)

        at com.yahoo.ycsb.workloads.CoreWorkloadAsyncIspn.doTransaction(CoreWorkloadAsyncIspn.java:415)

        at com.yahoo.ycsb.async.ispn.ClientThreadAsyncIspn.run(ClientThreadAsyncIspn.java:154)

       

      How can I set the thread pool size?

        • 1. Re: Creating a YCSB Asynchronous Client using HotRod and Deferred.
          nadirx

          The intention is to use Java 8's CompletableFuture in Infinispan 8 (due out in October).

          1 of 1 people found this helpful
          • 2. Re: Creating a YCSB Asynchronous Client using HotRod.
            naskoos

            Answering to myself,

             

            Trying to modify the source code to allow higher pool size I found out in the Javadoc of the RemoteCacheManager that I can pass as parameter the pool size and the queue size through the ConfigurationBuilder.

             

            org.infinispan.client.hotrod.configuration.ConfigurationBuilder cb

                   = new org.infinispan.client.hotrod.configuration.ConfigurationBuilder()

            cb.asyncExecutorFactory().addExecutorProperty("infinispan.client.hotrod.default_executor_factory.pool_size", "1000");

            cb.asyncExecutorFactory().addExecutorProperty("infinispan.client.hotrod.default_executor_factory.queue_size", "1000000");