What is VBSF? It is unclear if you want the jboss tm to control transactions in your existing code, or if you want your external application to control the jboss tm.
In the first case, you need to find some way of registering an XAResource with the jboss tm for each transaction. I recommend considering writing a jca resource adapter to do this: the ConnectionManager will take care of the registering/unregistering for you.
In the second case, you will probably need to use the Tyrex distributed transaction manager. Other than that, I have no idea how the scenario might work.
VBSF is an OO - Relational DB mapping tool
as per Toplink. On top of this tool we built
a Persistency API. This API has facilities to handle
transactions basically the Beginm Commit Rollback etc.
Basically, I wanted to link this API into the JBoss TM so that I can use CMT with its Implicit Transaction Management to control my persistency API.
It might be useful to look at how Castor is integrated, it may be doing something similar to what you want. Is VBSF a layer on top of a db driver? If so you may be able to get appropriate transaction control by wrapping the db driver in a jca wrapper and, if VBSF does caching, registering a synchronization with the tm to flush changes to the db driver before the commit starts. I'm not sure how to get the synchronizations to fire in the right order.