4 Replies Latest reply on Jun 13, 2018 11:40 AM by Seto Kaiba

    Strange problem of distributed jdbc cache store for index.

    Seto Kaiba Expert

      private ConfigurationBuilder persistenceConfigLucene(ConfigurationBuilder configurationBuilder) {

        configurationBuilder.persistence()

        .addStore(JdbcStringBasedStoreConfigurationBuilder.class)

        .key2StringMapper(LuceneKey2StringMapper.class)

        .async().enable()

        .preload(preload)

        .shared(shared)

        .fetchPersistentState(fetchPersistentState)

        .ignoreModifications(ignoreModifications)

        .purgeOnStartup(purgeOnStartup)

        .table()

        .createOnStart(createOnStart)

        .dropOnExit(dropOnExit)

        .tableNamePrefix("ISPN_STRING_TABLE")

        .idColumnName("ID_COLUMN").idColumnType("VARCHAR(255)")

        .dataColumnName("DATA_COLUMN").dataColumnType("BLOB")

        .timestampColumnName("TIMESTAMP_COLUMN").timestampColumnType("BIGINT")

        .connectionPool()

        .connectionUrl(connectionUrl)

        .username(username)

        .password(password)

        .driverClass(driverClass);
        return configurationBuilder;
      }

      preload = true, shared = true, fetchPersistentState = true, ignoreModifications = false, purgeOnStartup = true

       

      It seems that I will got incorrect remaining data even purgeOnStartup is true. PurgeOnStartup for debug purpose.

       

      public void sendStoryList(DataRef<User> userDataRef, DataRef<ClientChannel> clientChannelDataRef) {

        SearchManager searchManager = distributedDataAPI.getSearchManager();
         QueryBuilder queryBuilder = searchManager.buildQueryBuilderForClass(StoryPending.class).get();
         Query query = queryBuilder.keyword().onField("user").matching(userDataRef.get().getId()).createQuery();
         System.out.println("sendList:" + searchManager.getQuery(query, StoryPending.class).getResultSize());
         CacheQuery<StoryPending> cacheQuery = searchManager.getQuery(query, StoryPending.class);
         List<StoryPending> storyPendings = cacheQuery.list();
         System.out.println("sendList:" + storyPendings.size());
         DimensionStory.StoryList.Builder builder = DimensionStory.StoryList.newBuilder();
        for (StoryPending storyPending : storyPendings) {

        System.out.println("sendList:" + storyPending.getStory());
         DimensionStory.Story.Builder builder0 = DimensionStory.Story.newBuilder().setId(storyPending.getStory());
         builder.addStories(builder0);
         }

        clientChannelDataRef.get().send(builder);
      }

      The getResultSize will get 1, but the second is 0.

      I thought that there should be not remaining incorrect data if purgeOnStartue is true.

       

      No remaining incorrect data if I deleted the tables in the mysql database before the second launch.

      Yet, my project is complicated. So I can't provide a reproducing project.

       

      But I'd like you guys to check this. Or do I misunderstand the purgeOnStartup in this case?

        • 1. Re: Strange problem of distributed jdbc cache store for index.
          Galder Zamarreño Master

          What Infinispan version? Did you try enabling TRACE logging for org.infinispan package?

          • 2. Re: Strange problem of distributed jdbc cache store for index.
            Seto Kaiba Expert

            9.2.4.Final.

            I didn't. How can I enabling TRACE only for org.infinispan?

            Here's my log4j2.xml. I don't know how to enable trace for org.infinispan only.

            If I change the root level to TRACE. A lot of other logging is printed.

            And also I'm using distributed executor service in loop. So maybe it would print unrelated info as well.

            I thought that the purge fail should be at least a warn.

            <?xml version="1.0" encoding="UTF-8"?>

            <Configuration status="INFO" packages="co.kaiba.blueeyes.impl.util.log4j">

                <Appenders>

                    <Console name="A1" target="SYSTEM_OUT">

                        <Filters>

                            <ThresholdFilter level="warn" onMatch="ACCEPT" onMismatch="NEUTRAL"/>

                            <!--<ClassNameRegexFilter regex="co\..*" onMatch="ACCEPT" onMismatch="DENY"/>-->

                        </Filters>

                        <PatternLayout pattern="%d [%t] %p %c - %m%n"/>

                    </Console>

                </Appenders>

                <Loggers>

                    <Root level="INFO">

                        <AppenderRef ref="A1"/>

                    </Root>

                </Loggers>

            </Configuration>

            • 4. Re: Strange problem of distributed jdbc cache store for index.
              Seto Kaiba Expert

              Here's the log.

              Executing Lucene query 'user:78f3ebf0-33a4-47c9-9db1-b4b27e6887db'

              This query for StoryPending should be the one with problem.

               

              This is the lucene cache containers config.

              Can you see any problem?

              @ConfigureCache("LuceneIndexesMetadata")

              @Produces
              public Configuration luceneIndexesMetadataConfig() {

                ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
                 distributeConfig(persistenceConfigLucene(configurationBuilder));
                return configurationBuilder.build();
              }

               

              @ConfigureCache("LuceneIndexesData")

              @Produces
              public Configuration luceneIndexesDataConfig() {

                ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
                 distributeConfig(persistenceConfigLucene(configurationBuilder));
                return configurationBuilder.build();
              }

               

              @ConfigureCache("LuceneIndexesLocking")

              @Produces
              public Configuration luceneIndexesLockingConfig() {

                ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
                 distributeConfig(persistenceConfigLucene(configurationBuilder));
                return configurationBuilder.build();
              }

               

              private ConfigurationBuilder distributeConfig(ConfigurationBuilder builder) {

                builder

                .clustering()

                .cacheMode(CacheMode.DIST_SYNC);
                return builder;
              }

              private ConfigurationBuilder persistenceConfigLucene(ConfigurationBuilder configurationBuilder) {

                configurationBuilder.persistence()

                .addStore(JdbcStringBasedStoreConfigurationBuilder.class)

                .key2StringMapper(LuceneKey2StringMapper.class)

                .async().enable()

                .preload(preload)

                .shared(shared)

                .fetchPersistentState(fetchPersistentState)

                .ignoreModifications(ignoreModifications)

                .purgeOnStartup(purgeOnStartup)

                .table()

                .createOnStart(createOnStart)

                .dropOnExit(dropOnExit)

                .tableNamePrefix("ISPN_STRING_TABLE")

                .idColumnName("ID_COLUMN").idColumnType("VARCHAR(255)")

                .dataColumnName("DATA_COLUMN").dataColumnType("BLOB")

                .timestampColumnName("TIMESTAMP_COLUMN").timestampColumnType("BIGINT")

                .connectionPool()

                .connectionUrl(connectionUrl)

                .username(username)

                .password(password)

                .driverClass(driverClass);
                return configurationBuilder;
              }