I have finally find out what's the problem.
I'm using jboss-3.2.1_tomcat-4.1.24 and mySQL on Windows XP.
I have copied the file mysql-ds.xml from C:\jboss-3.2.1_tomcat-4.1.24\docs\examples\jca to C:\jboss-3.2.1_tomcat-4.1.24\server\default and change a few data in the file.
When I start JBoss by run.bat, the PersistenceManager cannot be started. I have to changed a line in the file C:\jboss-3.2.1_tomcat-4.1.24\server\default\deploy\jms\jbossmq-service.xml
"DELETE_MARKED_MESSAGES_WITH_TX = DELETE FROM JMS_MESSAGES WHERE TXID IN (SELECT TXID FROM JMS_TRANSACTIONS) AND TXOP=?"
"DELETE_MARKED_MESSAGES_WITH_TX = DELETE FROM JMS_MESSAGES WHERE TXID IS NOT NULL AND TXOP=?"
This is posted in http://www.thecortex.net/clover/eg/jboss/report/org/jboss/test/util/test/PropertyPatternUnitTestCase.html
Another problem is that there's an exception saying that the table jms_message doesn't exist. I suppose JBoss will have to create this table when it starts up but it seems it doesn't. So I download the source, find out the table structure and manually create the table. After that I can start JBoss without any warnings or exception and can run some sample program smoothly.