Hi Guys,
I would like to know what is recommended/the best approach to connect nodes together into cloud for following UC:
- I need to connect up to 400 nodes together into cloud.
- Nodes are running on weak ARM processors
- Each node stores it's data in one file which consist various kinds of entities, for each entity I have cache (mostly distributed SYNC cache).
- One part of final entity can be stored on several nodes. There is relation 1 final entity consists of many partial entities stored in different nodes.
- Each node provides up to 100000 entities into several caches.
My current approach is slow but mostly working (sometimes I get StateTransferException caused by timeout):
- all nodes starts
- all nodes joins to cloud
- semaphore takes care, only once controller at the time provides it's entities into cloud. During this process, before every put, node has to check if there is not stored final entity in cache and if so, it has to attach it's part into this entity and put it again. Every read-put is encapsulated in transaction.
- when one controller provides it's data, semaphore is released and another controller starts putting data etc.
It would be nice, if there is approach when I can put data in parallel from all nodes without deadlocks or losing data.
Do you have any suggestion?
