JSR-170 message store

[derek.adams]

There have been a few posts about implementing a JSR-170 (aka JCR) message store as an alternative to the default database implementation. I wrote a simple implementation using the Apache Jackrabbit libraries as a base. The new implementation has been checked into the SVN trunk and is available for testing. The required libraries and configuration files are now packaged as part of the jbossesb.sar.

To enable the JCR message store, add the following property to the "core" section of jbossesb-properties.xml in the root of the jboss-esb.sar:

<property name="org.jboss.soa.esb.persistence.base.plugin.jcr" value="org.jboss.internal.soa.esb.persistence.format.jcr.JCRMessageStorePlugin"/>

This adds the JCR plugin to the MessageStoreFactory plugins list. The JCR message store can use an existing repository via JNDI or can create a standalone instance locally on the app server. The following list of properties should be added in the "dbstore" section of jbossesb-properties.xml to configure repository access:

<property name="org.jboss.soa.esb.persistence.jcr.jndi.path" value="jcr"/>
<property name="org.jboss.soa.esb.persistence.jcr.username" value="username"/>
<property name="org.jboss.soa.esb.persistence.jcr.password" value="password"/>
<property name="org.jboss.soa.esb.persistence.jcr.root.node.path" value="JBossESB/MessageStore"/>

jcr.jndi.path - optional path in JNDI where the repository is found. If not specified, a new repository will be created based on the repository.xml located in the root of jbossesb.sar. In this case, repository data is stored in the JBossAS/server/{servername}/data/repository directory.

jcr.username - username for getting a repository session

jcr.password - password for gettging a repository session

jcr.root.node.path - the path relative to the root of the repository where messages will be stored

An easy test for whether the JCR message store is configured properly is to add the org.jboss.soa.esb.actions.persistence.StoreJCRMessage action onto an existing service. The action will attempt to store the current message to the JCR store.

Let me know if you run into any problems.

Thanks,
Derek

[kurtstam]

Very nice Derek,

I'm in the process of bringing up a default "Dead Letter Queue" service, and the default configuration may use this. I'll be checking it out :)

--Kurt

[burrsutter]

Cool.

Can you give me 2 out-of-the-box ESB actions?

MessageSave (or whatever) -returns the UUID in the message header property
MessageRetreive - has a to fetch the right one.

Therefore I could easily use them in my jboss-esb.xml without have to write Java code to the Message Store APIs (or JCR APIs).

Or just tell me if I'm crazy. :-)

Burr

[kurtstam]

I think we need one extra argument, a message store id/message classification.
I think we want to use the message store for all sorts of purposes. One example would be the DLQ I was talking about, so we need to be able to have 'substore' in this store, where I can store messages for this purpose. Is this posiible?

[burrsutter]

I thought JCR has a class/category trick but that might not be exposed through the Message Store API which should abstract it.

However, your work on the CBR wouldn't have to use the actions I was requesting. These actions are so end-users of the ESB could easily store a message and later easily retrieve one.

I can see people configuring an ESB service called something like "ToBeReviewed" as a holding tank. And have another service that A) returns all the IDs B) returns the message for a specific ID. So these messages can be reviewed and perhaps pushed back out to other services in the ESB.

[marklittle]

I think we need one extra argument, a message store id/message classification.
I think we want to use the message store for all sorts of purposes. One example would be the DLQ I was talking about, so we need to be able to have 'substore' in this store, where I can store messages for this purpose. Is this posiible?

Haven't had a chance to look at the API yet, but can we story/retrieve based on: recipient EPR or sender EPR? Would be cool to be able to send Rules to the store that are executed internally by it and used to generate the response (list of messages). Like stored procedures.

[derek.adams]

"burrsutter" wrote:

Can you give me 2 out-of-the-box ESB actions?

MessageSave (or whatever) -returns the UUID in the message header property
MessageRetreive - has a <property name="UUID" value="3214afds234"/> to fetch the right one.

I checked in two actions along with the JCR code. The first (org.jboss.soa.esb.actions.persistence.StoreMessage) stores a message to the default message store. The second (org.jboss.soa.esb.actions.persistence.StoreJCRMessage) stores a message to the JCR message store. I didn't add a message property for the resulting UID, but that would be easy to add. I can also add actions for retrieving the messages as well.

Now that I am thinking about it, maybe it would be better to pass in a message property that indicates which message store to save to/load from. If not specified, it would use the default message store.

I will take a look and get back to you soon.

Thanks,
Derek

[derek.adams]

I think we need one extra argument, a message store id/message classification.
I think we want to use the message store for all sorts of purposes. One example would be the DLQ I was talking about, so we need to be able to have 'substore' in this store, where I can store messages for this purpose. Is this posiible?

The JCR allows arbitrary metadata to be associated with each node. I was thinking we could just add a jbossesb:delivered boolean property to each message node. We could then do queries against that metadata to get the list of undelivered messages. The JCR spec allows the base repository schema to be extended with custom types, so we could have a well-defined layout for all of the info related to a message. I could see attributes like jbossesb:message-id, jbossesb:message-created-date, jbossesb:message-category, etc.. which would make reporting against messages really easy. The database message store impl would do the equivalent by adding columns in the message table.

[marklittle]

"derek.adams" wrote:

I think we need one extra argument, a message store id/message classification.
I think we want to use the message store for all sorts of purposes. One example would be the DLQ I was talking about, so we need to be able to have 'substore' in this store, where I can store messages for this purpose. Is this posiible?

The JCR allows arbitrary metadata to be associated with each node. I was thinking we could just add a jbossesb:delivered boolean property to each message node. We could then do queries against that metadata to get the list of undelivered messages. The JCR spec allows the base repository schema to be extended with custom types, so we could have a well-defined layout for all of the info related to a message. I could see attributes like jbossesb:message-id, jbossesb:message-created-date, jbossesb:message-category, etc.. which would make reporting against messages really easy. The database message store impl would do the equivalent by adding columns in the message table.

Because we can never know all possible search (and save) criteria, we want an interface that doesn't need to be updated everytime something new comes along. Do you think we've got that now, or are there more changes to be made?

[derek.adams]

I think the current implementation works well in the sense that there is not a specific interface for the properties associated with a message (i.e - no getMessageId() or getMessageCreateDate() on the Message interface itself). I added the org.jboss.soa.esb.message.properties.MessagePropertyFacade class as a wrapper for messages so that there would be consistent property naming and typing without restricting the Message interface. Does that sound like a good way to handle it?

[marklittle]

"derek.adams" wrote:
I think the current implementation works well in the sense that there is not a specific interface for the properties associated with a message (i.e - no getMessageId() or getMessageCreateDate() on the Message interface itself). I added the org.jboss.soa.esb.message.properties.MessagePropertyFacade class as a wrapper for messages so that there would be consistent property naming and typing without restricting the Message interface. Does that sound like a good way to handle it?

Yes and no ;-) Maybe I just need some clarification.

When I look at the MessageStore interface, the way of retrieving messages is either by URI:

public Message getMessage (URI uid) throws MessageStoreException;

or by the fact that the messages haven't been delivered:

public Map<URI, Message> getUndeliveredMessages() throws MessageStoreException;

One of the intentions of the message store was that it could be used for audit trail purposes ("give me all mesage exchanges between A and B during 9am and 12pm on 2nd of August") and debugging (replay the events), etc. A DLQ is obviously just one way in which we can use the messages stored. The original interface for the MessageStore fell short of our intentions, as you've seen. What we've got now is a lot more useful. However, it would be nice to have something that is more flexible. We could do this in a number of ways. For example, did you see my earlier comment about using rules?

public Map<URI, Message> getMessages(RuleBase rules) throws MessageStoreException;

Would be interested in your thoughts on generalising the interface.

Also, could you add the copyright statement to the head of the files you added? Thanks.

[jackalista]

Hi,

I'm looking at the MessageStore for use in audit / logging functionality and am interested in this thread about the JCR based MessageStore.  I tried looking through the source (trunk) for the classes mentioned in this thread however and didn't find any of them.  Is this functionality present in the JBoss ESB dist?

I should add that I have no particular attachment to a JCR impl, but *am* very much interested in the kind of querying mentioned by Mark e.g. "get all message between A & B" etc.  The JCR impl discussion seemed to imply that JCR might be a good way to do this, and also might not be the message store itself [meaning the persistrence mechanism] but might merely be a federated view that could provide searching of the kind Mark mentions.  I am unclear as to how this has progressed, if it has at all, and what the most functional interfaces are currently [at the end of 2010] for supporing audit trail-like features.  Can anyone comment on what the current state of things is?  Much thanks...