3 Replies Latest reply on Sep 25, 2012 6:48 AM by Sanne Grinovero

    Infinispan Indexing Problem - Non-transformable Key at State Transfer

    paulpa63 Newbie

      Hi,

      We are running Infinispan 5.1.5.FINAL and the following problem has been observed on RHEL5 amd Windows 7.

      In our test scenario we have a cluster of two nodes. The nodes are configured to use RAM indexing (with indexLocalOnly=false).  We start one node and load data into the node.  Then we start the second node and it joins the cluster but during state transfer(?) we see the following exception: java.lang.IllegalArgumentException: Indexing only works with entries keyed on Strings, primitives and classes that have the @Transformable annotation - you passed in a class java.util.UUID.  The second node consequently fails to acquire an index.

       

      I believe that this behaviour is different to earlier versions of Infinispan where the second node would simply synchronise its caches but silently omit to build a local index.  This meant we had been considering options for trying to implement a custom index rebuild facility / had been closely monitoring the progress of ISPN-1440.  This more recent (mis-)behaviour suggests that Infinispan will now sync a local index... we just need to ensure all keys are Transformable.  Currently I cannot see any way to register a custom key transformer for UUID such that it will be available to the internal Infinispan code which is performing the state transfer?  May this should be specifiable in the infinispan.xml configuration file?

       

      I considered changing all of our system domain objects to use String keys instead of UUID but that is quite onerous now (and affects design documents).

       

      My solution at the moment is to create a local copy of class org.infinispan.query.backend.KeyTransformationHandler and modify stringToKey and keyToString to include simple cases for java.util.UUID (one basically does "U" + key.toString() and the other UUID.fromString(s.subString(2))).  I think that use of UUID is perhaps a good / common candidate for object keys nad therefore this might be a useful patch to include into the next version of Infinispan?

       

      Paul

        • 1. Re: Infinispan Indexing Problem - Non-transformable Key at State Transfer
          Sanne Grinovero Master

          Hi Paul,

          I've sent a pull request for ISPN-1440 so I would expect it to be merged soon.

           

          Could you create a test describing your problem with UUIDs ? and yes if you have a patch to provide a default transformation for this type, I agree it would be a good idea to integrate that as well. Before that however I'd like to look at a test to better understand what problem you're having.

          1 of 1 people found this helpful
          • 2. Re: Infinispan Indexing Problem - Non-transformable Key at State Transfer
            paulpa63 Newbie

            Hi Sanne,

             

            Thanks for your response. 

             

            I am not sure that I can easily create a unit test for this problem, it was encountered within an integration scenario using two nodes in a cluster.  The main point is that the domain object being stored into the replicated cache is keyed by a non-string/primitive and the cache is of course being indexed.

             

            As I noted in my original email, I am not sure that ISPN-1440 is necessary for us any more because it is clear that recent versions of Infinispan will automatically update the local index on a node when that node joins the cluster.

             

            My patches to the KeyTransformationHaandler are very simple, see end of message.  (Apologies for formatting but changes are marked using bold.)

             

            Paul

             

            ---------------------------

            public Object stringToKey(String s, ClassLoader classLoader) {

            char type = s.charAt(0);

            switch (type) {

            case 'S':

            // this is a normal String, but NOT a SHORT. For short see case 'x'.

            return s.substring(2);

            case 'I':

            // This is an Integer

            return Integer.parseInt(s.substring(2));

            case 'Y':

            // This is a BYTE

            return Byte.parseByte(s.substring(2));

            case 'L':

            // This is a Long

            return Long.parseLong(s.substring(2));

            case 'X':

            // This is a SHORT

            return Short.parseShort(s.substring(2));

            case 'D':

            // This is a Double

            return Double.parseDouble(s.substring(2));

            case 'F':

            // This is a Float

            return Float.parseFloat(s.substring(2));

            case 'B':

            // This is a Boolean. This is NOT the case for a BYTE. For a BYTE, see case 'y'.

            return Boolean.parseBoolean(s.substring(2));

            case 'C':

            // This is a Character

            return s.charAt(2);

            case 'U':

            // This is a java.util.UUID

            return UUID.fromString(s.substring(2));

            case 'T':

            // this is a custom transformable.

            int indexOfSecondDelimiter = s.indexOf(":", 2);

            String keyClassName = s.substring(2, indexOfSecondDelimiter);

            String keyAsString = s.substring(indexOfSecondDelimiter + 1);

            Transformer t = getCustomTransformer(keyClassName, classLoader);

            if (t == null) throw new CacheException("Cannot find an appropriate Transformer for key type " + keyClassName);

            return t.fromString(keyAsString);

            }

            throw new CacheException("Unknown type metadata " + type);

            }

            public String keyToString(Object key) {

            // this string should be in the format of

            // "<TYPE>:(TRANSFORMER):<KEY>"

            // e.g.:

            // "S:my string key"

            // "I:75"

            // "D:5.34"

            // "B:f"

            // "T:com.myorg.MyTransformer:STRING_GENERATED_BY_MY_TRANSFORMER"

            char prefix = ' ';

            // First going to check if the key is a primitive or a String. Otherwise, check if it's a transformable.

            // If none of those conditions are satisfied, we'll throw an Exception.

            Transformer tf = null;

            if (isStringOrPrimitive(key)) {

            // Using 'X' for Shorts and 'Y' for Bytes because 'S' is used for Strings and 'B' is being used for Booleans.

             

            if (key instanceof String)

            prefix = 'S';

            else if (key instanceof Integer)

            prefix = 'I';

            else if (key instanceof Boolean)

            prefix = 'B';

            else if (key instanceof Long)

            prefix = 'L';

            else if (key instanceof Float)

            prefix = 'F';

            else if (key instanceof Double)

            prefix = 'D';

            else if (key instanceof Short)

            prefix = 'X';

            else if (key instanceof Byte)

            prefix = 'Y';

            else if (key instanceof Character)

            prefix = 'C';

            return prefix + ":" + key;

            } else if (key instanceof UUID) {

            return "U:" + key.toString();

            } else if ((tf = getTransformer(key.getClass())) != null) {

            // There is a bit more work to do for this case.

            return "T:" + key.getClass().getName() + ":" + tf.toString(key);

            } else

            throw new IllegalArgumentException("Indexing only works with entries keyed on Strings, primitives " +

            "and classes that have the @Transformable annotation - you passed in a " + key.getClass().toString() +

            ". Alternatively, see org.infinispan.query.SearchManager#registerKeyTransformer");

            }

            ---------------------------