JBM 2 log and ESB

[timfox]

Just a wild thought-

Have you thought about using the JBM 2.0 fast log for your ESB message store?

If JBM and ESB is running on the same machine, you could even use the same log *instance* for JBM and ESB usage - which would be better than having two log instances since the disk head wouldn't be thrashing so much.

Actually, you could even have JBoss TS, JBoss ESB and JBM all using the same log instance on the same box....

[marklittle]

If you mean use the log for the DLQ and RDQ then it's something we could definitely consider. As for the more general message store, that's probably less useful because the intention there is to allow it to be a central or federated service that is available across the network. So yes, it could still use the JBM log, but it definitely wouldn't be shared by JBossTS for a start and you wouldn't see much performance benefit since the bottleneck will be the network.

[kconner]

This is something that we are already considering, yes, in line with the architecture changes necessary for ESB 5.

[kconner]

If you mean use the log for the DLQ and RDQ then it's something we could definitely consider.

There will not be a DLQ/RDQ in the same fashion as currently exists in ESB4. The redelivery mechanism will be an integral part of the bus along with its integration to other ESB instances.

As for the more general message store, that's probably less useful because the intention there is to allow it to be a central or federated service that is available across the network. So yes, it could still use the JBM log, but it definitely wouldn't be shared by JBossTS for a start and you wouldn't see much performance benefit since the bottleneck will be the network.

My previous posting was not commenting on any integration with JBossTS as any potential integration would exist be at a lower level than we are currently dealing with. We are more concerned with an internal bus structure for the new architecture.

[timfox]

As for the more general message store, that's probably less useful because the intention there is to allow it to be a central or federated service that is available across the network. So yes, it could still use the JBM log ... you wouldn't see much performance benefit since the bottleneck will be the network.

I'd be surprised if the bottleneck is the network if the disk is just a normal disk. Standard network is 1 Gbit these days, but most disks write at around 30-80 MB/s = maybe 200 Mbits per sec.

If you're using a SAN with a load of disks in it, and parallel writing over those, then yes you could exceed write speeds > 1Gbit. (I'm looking forward to our new JMS lab to test this out).

Currently IIRC we've tested JBM writing records transactionally up to about 50MBytes/sec = 400 Mbits/sec on a cheap(ish) SCSI disk (dell workstation).

[timfox]

This is something that we are already considering, yes, in line with the architecture changes necessary for ESB 5.

Ok sounds good Kev-

If you need any help, advice etc you know where to find us :)

[marklittle]

"timfox" wrote:

I'd be surprised if the bottleneck is the network if the disk is just a normal disk. Standard network is 1 Gbit these days, but most disks write at around 30-80 MB/s = maybe 200 Mbits per sec.

If you're using a SAN with a load of disks in it, and parallel writing over those, then yes you could exceed write speeds > 1Gbit. (I'm looking forward to our new JMS lab to test this out).

Currently IIRC we've tested JBM writing records transactionally up to about 50MBytes/sec = 400 Mbits/sec on a cheap(ish) SCSI disk (dell workstation).

Yes, of course all internet communication uses 1 Gig these days ;-) And all companies can afford to move to the latest and greatest technologies ;-) We have to support legacy infrastructures: that's what SOA is all about. So it may work fine in some deployments, but it's not guaranteed to work in all.

[marklittle]

If you mean use the log for the DLQ and RDQ then it's something we could definitely consider.

There will not be a DLQ/RDQ in the same fashion as currently exists in ESB4. The redelivery mechanism will be an integral part of the bus along with its integration to other ESB instances.

Sure for 5.0, but Tim's post didn't mention versions. 4.x is still a viable alternative.