With version of JBoss AS 5? 5.1.0 or 5.0.x?
Install the JBoss Tools 3.1.0 Eclipse plugin. The Tools team also provides a nice set of tutorials in their docs. http://docs.jboss.org/tools. The tutorials cover what types of projects to create, and how to add in extra functionality (called facets) to the project.
Yes you can keep AS running constantly. Actually, you will create a server within Eclipse and run AS using that. However, there are times when you will need to restart AS.
I am not sure if the Tools tutorials cover debugging, but there is an Eclipse tutorial on that topic (I think it uses Tomcat) but the steps are the same.