3 Replies Latest reply on Jun 18, 2014 7:18 AM by Radim Vansa

    Bug on language compatibility for string value

    Bingu Shim Newbie

      Hello, we are trying to use infinispan with c++ and java clients,

      but having problem with handling string value.

      (with 6.0.2.Final version of infinispan and c++ hotrod connector)

       

      The string value set by java(possibly unicode encoded Korean) is not correctly fetched with c++ client and visa versa.

       

      Here is what we have been found so far.

       

      1. C++ marshaller is not compatible with java default marshaller.

          Java marshaller is based on JBoss River Marshaller which put few bytes header for string object,

          while C++ marshaller doesn't use any header.

       

         -> We have resolved this issue by changing the c++ marshaller.

       

      2. unicode encoded Korean is not correctly marshalled on unmarshalled with c++ hotrod connector

         We are struggling with handling Korean string, so far the problem is not solved.

       

         While debugging, we found that JBoss River Marshaller is using 'org.jboss.marshalling.UTFUtil.writeUTFBytes()' method for marshalling String object,

         and the comment of the method is 'Write the modified UTF-8 form of the given string to the given output.'


         Is this because of UTFUtil.writeUTFBytes method is writing modified UTF-8 string?

         The modified UTF-8 format is standard? or just optimized format in JBoss?


      Here is the test code


      using namespace infinispan::hotrod;
      
      
      int main(int argc, char** argv){
        ConfigurationBuilder builder;
        builder.addServer().host("127.0.0.1").port(11222);
        RemoteCacheManager cacheManager(builder.build(), false);
      
        HR_SHARED_PTR<RiverMarshaller<key_t>> keyMarshaller(new RiverMarshaller<key_t>);
        HR_SHARED_PTR<RiverMarshaller<value_t>> valueMarshaller(new RiverMarshaller<value_t>);
      
        RemoteCache<key_t, value_t> cache = cacheManager.getCache<key_t, value_t>(keyMarshaller, valueMarshaller, "Janggi", true);
      
        try {
          cacheManager.start();
          std::string key("key");
      
          auto rv = cache.get(std::string(key));
          if (NULL == rv) {
            std::cout << "NOT FOUND" << std::endl;
          }
          else {
            std::cout << *rv << std::endl;
            delete rv;
          }
        } catch (HotRodClientException& e) {
          std::cout << e.what() << std::endl;
        }
        cacheManager.stop();
      }
      


      and the screen shot of result is attached



      Thanks,


      Bingu Shim

       

        • 1. Re: Bug on language compatibility for string value
          Radim Vansa Master

          HotRod operates on byte arrays, and the C++ std::string is just a container of bytes. Putting Java strings is then not equivalent to putting C++ std::strings to the cache, as the JBoss Marshaller stores the information about type, length etc. as well.

           

          You're not presenting your RiverMarshaller implementation, but it probably just does no match the Java one. I don't think you'll find here answers how to rewrite JBoss Marshallers to C++, and you probably should not do that. Instead, you can write your own simple marshaller for strings (where you can control the way how the string is marshalled into UTF)  and use that for Java clients, and compatible C++ marshaller (if the default wouldn't be sufficient).

          • 2. Re: Bug on language compatibility for string value
            Bingu Shim Newbie

            Thanks you for your answer Radim

             

            However, do you have any plan for supporting string format compatibility for Java and C++?

            Since, this would be a basic use case, when multi language is used.

             

             

            BTW, I didn't set anything about RiverMarshaller implementation, so the default RiverMarshaller (jboss-marshalling-river-1.4.4.Final.jar) was used.

            • 3. Re: Bug on language compatibility for string value
              Radim Vansa Master

              We are not going to fix String specifically (the problem is that you can put anything into the cache from Java, therefore, the cache contents have to be self-describing), but for object-level compatibility (as compared to byte array-level compatibilty) you can use the Protobuf marshallers - these are used anyway when you want to do querying over HotRod (from Java, but C++ querying support should follow). I even think there's some quickstart which uses Protobuf marshallers from C++ as well.

              1 of 1 people found this helpful