Apologies if this has been discussed already. I did some searching and couldn't find anything.
We have a need to write a lot of data to a store (Mongo) in "log file" fashion, ie. write-only. We'll be analysing the data later - the Java app doesn't have a need to retrieve the data.
I'm considering using an Infinispan cache with the following configuration:
- async mode
- a very small maxEntries size (ie. 1).
- a custom store to write the data to Mongo which always returns a miss on reads
There are a few motivations for this:
a) We're using infinispan for other caches within the app, with multiple types of configuration. We'll need to monitor how these are performing, and this write-only cache can benefit from the same monitoring
b) We can distribute writes across a number of nodes and use modificationQueueSize and threadPoolSize to tune the capacity of the cluster
c) The amount of data to be written is "peaky", so provided the load isn't too high for too long, we don't need to scale the Mongo cluster too far
An alternative is to use an async Mongo driver such as this one http://www.allanbank.com/mongodb-async-driver/index.html, but that means introducing another technology and different monitoring.
So... "mad idea" or "might just work"....?! Anyone else running a comparable setup?
I don't have a precise answer but I think it should work. However, if you have direct access to the MongoDB, you should try that. Using ISPN for that purpose, you are incurring in more overhead.
IMO, give it a try with both alternatives and see which one deals better with your application and your data load/accesses