Cluster Advice Required

[pieter.martin]

Hi,

We have heavy computational needs that can be done in parallel and are thinking of using a hornetq cluster to do this for us. The plan is to have 3 fairly big machines and to have their cpus fully utilized for the computation. If each machine can have say 200 threads we are planning on having 200 nodes per machine in the cluster.

Does this sound like a good idea or is hornetq not quite the right tool for this kind of large scale parallel computation?

Thanks

Pieter

[timfox]

So you're planning on using HornetQ to deliver the computations to your clients which will do the actual computations?

Are you intending on deploying the clients on the same machines as the HornetQ servers?

[pieter.martin]

I more thought of it as having one client that delivers computational jobs (message) to the cluster. The client would be on a different physical machine to the cluster. The nodes in the cluster then does the work.

There would then be a MDB that in its execute would do the calculation. Each calculation's result would either be put into a db or perhaps onto a disk shared by the cluster. Not quite sure yet of how we are going to store each calculations result.

After further discussion it seams the physical machine is a 64 core machine which they say sort of maps to 64 java threads which I am thinking maps to 64 nodes per machine in the cluster. There might be 2 or 3 of these machines so that would give us 64*2 / 3 nodes in the cluster.

However a node in the cluster might take up more than one thread so I am not sure this logic is solid.

Thanks

Pieter

[timfox]

An MDB is a client too

So basically you want the processing to be done on the same nodes as the actual servers.

This should work, however you could also consider putting the HornetQ servers on separate machines and do the processing on other nodes.

This means the HornetQ machines won't be affected by the 100% CPU utilisation you're expecting for processing.

Also, any particular reason why you're considering MDBs?

I guess this is a matter of personal test but I'm not really a big JEE fan. I find MDBs can add complexity (and overhead) which you can avoid by just creating a simple JMS consumer.

[pieter.martin]

Ok yes I was planning on doing the calculation on the same machine. Did not even consider doing it on another machine. No reason for using a MDB. I used the wrong terminology there, we will be using a normal JMS consumer.

Ok thanks, here comes a hornetq super computer.

Pieter

[timfox]

Sounds great

Tell me how it goes.

[timfox]

I think traditionally many compute grids or parallel cluster architectures (like the old Beowulf cluster), have used many nodes for the actual compute, but only have one (or two for redundancy) specialist nodes which deliver the compute instructions to the compute nodes.

Just delivering compute packets to other nodes is not a particularly resource intensive task, especially considering many computations take a significant time to complete, traffic might not be that high.

Consequently the machine housing the messaging server can often be isolated and less powerful than the actual compute nodes.

[pieter.martin]

Hi,

After some confusion on my side regarding hornetq and pooling I have decided to use jboss appserver and a compute MDB. This seems to me a simpler solution as the app server will pool the MDB making it unnecessary for me to run and manage many nodes/consumers.

As an aside the compute MDB will not be the only app deployed on jboss in which case the app server might be an overkill.

Thanks

Pieter

[timfox]

I guess it's a matter of personal taste.

But I'd say having to manage a whole JBoss AS instance is a lot more to manage and a lot more overhead than creating a simple JMS client.

[pieter.martin]

Hi,

An update,

We are going it standalone now. Using Weld though to keep me happy with injection.

However it has been decided for better or worse that the compute grid is to compute anything now, not just the current problem.

So we entered into the world of dynamic classloading in order for the consumer to download the classes required to do whateva it wants to do.

Cheers

Pieter