Apologies if this has been discussed already. I did some searching and couldn't find anything.
We have a need to write a lot of data to a store (Mongo) in "log file" fashion, ie. write-only. We'll be analysing the data later - the Java app doesn't have a need to retrieve the data.
I'm considering using an Infinispan cache with the following configuration:
- async mode
- a very small maxEntries size (ie. 1).
- a custom store to write the data to Mongo which always returns a miss on reads
There are a few motivations for this:
a) We're using infinispan for other caches within the app, with multiple types of configuration. We'll need to monitor how these are performing, and this write-only cache can benefit from the same monitoring
b) We can distribute writes across a number of nodes and use modificationQueueSize and threadPoolSize to tune the capacity of the cluster
c) The amount of data to be written is "peaky", so provided the load isn't too high for too long, we don't need to scale the Mongo cluster too far
An alternative is to use an async Mongo driver such as this one http://www.allanbank.com/mongodb-async-driver/index.html, but that means introducing another technology and different monitoring.
So... "mad idea" or "might just work"....?! Anyone else running a comparable setup?
Thanks!
Alfie.
