That's on purpose indeed. There is no installer program for ActiveMQ, so there is no need for an uninstaller then. You can simply remove the folder into which you untared/unzipped the distribution.
Thx for the confirmation!
In 5.3.x we had installer program wherein you can run the program and save a properties file for future silent installation purpose. This all is removed and moving forward we will not have installer program - it works for us. Just wanted to reconfirm the change in bundle.