Hi, I've encountered CASSANDRA-4206 on multiple Cassandra nodes.
The bug pertains to compaction failing. In particular it seems the metrics index won't compact:
INFO [CompactionExecutor:1720] 2014-01-20 16:28:14,860 CompactionController.java (line 156) Compacting large row rhq/metrics_index:one_hour_metrics:2014-01-19 17\:00\:00+0000 (73115218 bytes) incrementally ERROR [CompactionExecutor:1720] 2014-01-20 16:28:23,305 CassandraDaemon.java (line 192) Exception in thread Thread[CompactionExecutor:1720,1,main] java.lang.AssertionError: originally calculated column size of 25388070 but now it is 25388089 at org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:135) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:162) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:211) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)
It seems like the bug won't be fixed in Cassandra 1.2. I wonder if the index could be rebuilt? Or what can I do.