cannot be cast to org.hornetq.core.server.impl.LastValueQueue$HolderReference

[cfau]

Hi,

We are using HornetQ-Server 2.4.0.FINAL, and after a restart of the application the log shows a lot of errors like the one below.

The server is ran embedded in a Web App.

Any help to understand the situation will be appreciated.

Carlos

2016-06-23 06:21:21.261 PDT [Thread-21 (HornetQ-server-HornetQServerImpl::serverUUID=9fe24002-18b9-11e6-91ce-9bd17e13b396-673384643)] ERROR core.client                     - HQ214017: Caught unexpected Throwablejava.lang.ClassCastException: org.hornetq.core.server.impl.MessageReferenceImpl cannot be cast to org.hornetq.core.server.impl.LastValueQueue$HolderReference

        at org.hornetq.core.server.impl.LastValueQueue.addHead(LastValueQueue.java:147)

        at org.hornetq.core.server.impl.QueueImpl.addHead(QueueImpl.java:432)

        at org.hornetq.core.server.impl.QueueImpl.postRollback(QueueImpl.java:2607)

        at org.hornetq.core.server.impl.QueueImpl$RefsOperation.afterRollback(QueueImpl.java:2726)

        at org.hornetq.core.transaction.impl.TransactionImpl.afterRollback(TransactionImpl.java:456)

        at org.hornetq.core.transaction.impl.TransactionImpl.access$200(TransactionImpl.java:38)

        at org.hornetq.core.transaction.impl.TransactionImpl$3.done(TransactionImpl.java:331)

        at org.hornetq.core.persistence.impl.journal.OperationContextImpl$1.run(OperationContextImpl.java:234)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:107)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

[jbertram]

Do you have a test-case or instructions to reproduce this error?

Also, I would encourage you to move to Apache ActiveMQ Artemis since HornetQ is no longer under development.  See more details about this at http://hornetq.blogspot.com/2015/06/hornetq-apache-donation-and-apache.html.

[martin.rauscher]

2017-04-07 10:55:59.869 +0000 ; Threa ; ERROR ; ; o.a.a.a.c.client ; AMQ214017: Caught unexpected Throwable: java.lang.ClassCastException: org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl cannot be cast to org.apache.activemq.artemis.core.server.impl.LastValueQueue$HolderReference at org.apache.activemq.artemis.core.server.impl.LastValueQueue.addHead(LastValueQueue.java:124) at org.apache.activemq.artemis.core.server.impl.QueueImpl.addHead(QueueImpl.java:485) at org.apache.activemq.artemis.core.server.impl.QueueImpl.postRollback(QueueImpl.java:2499) at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterRollback(RefsOperation.java:116) at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterRollback(TransactionImpl.java:472) at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$200(TransactionImpl.java:35) at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$3.done(TransactionImpl.java:351) at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl$1.run(OperationContextImpl.java:201) at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:103) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

Same here with most recent 10.1 Wildfly.

Happened under high pressure. Not really reproducible.

[jbertram]

I'm trying to piece together your use-case.  As you noted, this is happening under load and the stack-trace indicates that a transaction rollback has happened involving a message on a last-value queue.  Can you provide any more details about your use-case?

[martin.rauscher]

We use optimistic locking, modify some entities, put a message in the queue (a few kb in size) and if a parallel thread was faster, the whole transaction is rolled back. This happens rather often under certain traffic spikes. I'm not sure if we actually had an issue due to that. I simply found it in the logs and felt uncomfortable with the stack trace...

[jbertram]

I was really looking for more details specifically about the messaging aspect of your use-case.  Can you confirm you're using a last-value queue?  If so, are you using just a single "last" value property or multiple different ones?  Are you using XA transactions?

[martin.rauscher]

We're using both, last value and "normal" queues. Sometimes within the same transaction. We're using multiple values. Yes, we're using XA Txns.

[jbertram]

Do you know if the transaction where the rollback failed was using last-value and normal queues?

[martin.rauscher]

Sorry, that’s not possible to see from the log. But it is very probable.