Copying large files to members using JGroups

[forumer]

I need to copy large files (100MB to 1GB) between different machines. JGroups seems to be a good fit for doing this. Browsing the web, I find that this can be done fairly easily between two machines using state transfer which, apparently, uses TCP. However, multicasting would be a better solution because of many members and large files. I've read some conjectures about how this could be done but haven't seen a definitive solution. I understand that multicasting streams is not a viable approach for accomplishing this.

I'd appreciate any suggestions and experiences about how to go about doing this.

Thanks

P.S I hope I am posting in the right forum as I couldn't find one for JGroups.

[belaban]

There's a discussion of how this can be done at https://jira.jboss.org/jira/browse/JGRP-653.
The email lists for JGroups are at http://sourceforge.net/mail/?group_id=6081.

I would *not* (ab)use state transfer for transfer of large files. I know, some folks have done it this way, but I would not recommend it

[brian.stansberry]

The JBoss AS 4.x Farm service does file copy over JGroups; perhaps some of the code will be useful. The class that does the actual work of moving files over the wire is

https://svn.jboss.org/repos/jbossas/branches/Branch_4_2/cluster/src/main/org/jboss/ha/framework/server/ClusterFileTransfer.java

This code uses the JBoss AS HAPartition, which for this usage is just a thin layer that calls through to JGroups' RpcDispatcher. You could code directly to RpcDispatcher. You can see how the HAPartition implementation calls through to RpcDispatcher at

https://svn.jboss.org/repos/jbossas/branches/Branch_4_2/cluster/src/main/org/jboss/ha/framework/server/ClusterPartition.java