Uncommitted transactions should be automatically rolled back when you reboot.
The error message is not jms related though so I don't know how you reached the conclusion. It shows the listing of a directory by apache axis having an OOME.
It isn't even a proper OOME, it means your GC is getting stressed
This can be caused by something as simple as too much paging of memory
back and forwards to disk or that your GC isn't tuned properly?
Given that it occurred on the file access, it might also be too many calls to the
finalize() method of java.io.File due to lots of them getting constructed and then thrown away?