My question is how to best access the Mode Shape instance from the RCP client. At this point, I am thinking that I will develop application specific EJBs running in JBoss, that in turn invoke Mode Shape (and possibly other EE resources such as JPA EntityManager, other JCA connectors etc). The RCP client would invoke these EJBs remotely as a standard EJB client. Any feedback or suggestions on this approach or better alternatives?
You can do it with EJBs, or you can have the RCP client directly access the repository content through the RESTful service. If you're using Seam, it's probably easier for you and more conventional to write the business logic in EJBs, which have access to the full JCR API (since they're running in the same process as ModeShape). As you suggest, the RCP client would then be invoking these EJBs remotely, which is very conventional.
Also, a related question. If Mode Shape is invoked from within an EJB with CMT, possibly along with other XA resources like JPA EntityManagers or other JCA connectors, does Mode Shape participate in the JTA transaction so that if the transaction is rolled back, Mode Shape rolls back as a part of the two phase commit?
At this point, ModeShape cannot participate in JTA transactions, though we'd love to add support for this optional feature in JSR-283. While it'd be relatively straightforward when ModeShape uses a single JPA connector (or even multiple JPA connectors), it becomes much more challenging when ModeShape uses connectors to non-transactional sources like file systems.