one the one hand you need to catch a HibernateException, on the other you will not get an Exception until the transaction was commited. So you perhaps have to call em.flush() to synchronize HibernateSession-state with your database. This is how i understood working with EntityManager.
Catching exception in the case of constraint violation exception you must call em.flush() in order to force the exception in you session bean. After this you are most likely going to want to create a checked exception yourself and this is where you need to mark your checked exception with the annotation @ApplicationException otherwise JBoss will raise a RuntimeException
See this for full breakdown :-