5 Replies Latest reply on Aug 31, 2006 12:11 PM by Clebert Suconic

    String size limitations with TreeCacheMarshaller

    Manik Surtani Master

      See http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3968346 and JBCACHE-756.

      The story is that Strings are written using writeUTF() which has a 65535 byte limit. See:

      http://java.sun.com/j2se/1.5.0/docs/api/java/io/DataOutput.html#writeUTF(java.lang.String)

      While I initially thought that fixing this would:

      1) Involve having to write String objects directly (inefficient) as any other form of writing a byte stream will result in hitting some limit or the other.
      2) Break binary stream compat with the 1.4 series.

      I've expressed these in the JIRA task.

      Thinking about this a bit more,

      1) Perhaps we can chunk Strings into several writeUTF() calls. 2 bytes can be written first, to identify how many chunks to expect. This will give us a limit of 2^16 x 2^16 bytes, or 4GB of String data (!).

      2) We can make this backward-compatible by using the current format for Strings under 64kb, and a new magic number (MAGIC_NUMBER_LARGE_STRING ?) for a 'chunked' version, up to 4GB Strings.

      Thoughts?