oh, this sounds like RHQ 4.5. I assumed you were grabbing the latest code and distro (which is RHQ 4.6 and its build and installation is *very* different than RHQ 4.5).
The docs for RHQ 4.5 are here in the confluence RHQ45 space: https://docs.jboss.org/author/display/RHQ45
That message just means you have previously installed an RHQ on that database. You can tell the installer to overwrite the database - you'd lose all previous data and it just starts with a fresh, clean DB.
As for your other questions regarding building - you should indicate which version you are using. Because, as mentioned, RHQ 4.5 and 4.6 are very different (4.5 has JBossAS 4.2.3 under the covers and has a web installer; 4.6 has JBossAS 7.1.1.Final under the covers and has no web installer, just a command line installation script). All of this is documented on the wiki.
There are wiki pages on how to run the installer, how to install the server and agent, how to build, etc, etc.