AsyncCacheLoader crashes

[thibautbodart]

Hi...

We are currently facing the following problem in production.

We use a TreeCache combined with a SleepyCat cache loader. The Cache loader is writing asynchronously the data to SleepyCat.

After a while, it seems that the AsyncCacheLoader thread dies after the following, which seems to be a serialization issue:

Exception in thread "AsyncCacheLoader-1" java.lang.StackOverflowError
at java.io.ObjectStreamClass.getClassDataLayout(ObjectStreamClass.java:1052)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1321)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:291)
at java.util.ArrayList.writeObject(ArrayList.java:570)
at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:890)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1333)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:291)
at java.util.ArrayList.writeObject(ArrayList.java:570)
at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:890)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1333)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:291)
at java.util.ArrayList.writeObject(ArrayList.java:570)
....


A few minutes after the crash of the thread, the application hangs since the queue is full and all the threads are blocked on the following:
EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.put(Unknown Source) org.jboss.cache.loader.AsyncCacheLoader.enqueue(AsyncCacheLoader.java:326)
org.jboss.cache.loader.AsyncCacheLoader.put(AsyncCacheLoader.java:211)


Did anybody face a similar issue ? We can not explain this serialization issue that crashes that cache loader thread.

Thanks a lot for your support,

Thibaut

[genman]

I imagine the problem is with the serialization done for putting the data into the Berkeley cache. The thread name happens to be the async thread name. It doesn't seem to indicate a problem with the async component at all as the async component does no serialization.

Do you know what object is being put? If you can share this, it's likely there's some sort of issue with that.