I found the problem -- in my EJBActivate() I made a call to a CMP managed field (to get my ID so that I could print out a debug statement). This is illegal since the bean has not yet been loaded in yet.
For some reason, the error for this only shows up as a debug level trace call, and was not printing out to my console so I didn't notice it.
In general, I guess when you get this error:
javax.transaction.TransactionRolledbackException: removing bean lock and it has tx set!
It quite likely means that an error was thrown during an earlier operation, and that is why the transaction state has not been cleared up. So if you get this error, set your trace level to a suitable level, and look for that other error.