This page describes how to deploying ESB 4.x archives on JBoss AS 5.

Deploying Non-Scoped Deployments

Deploying an ESB 4.x archive should simply be a matter of adding the .esb archive to the deploy directory.

Scoped Deployments

With AS 4.x one could specify that a .esb deployment be scoped by adding the loader-repository element to deployment.xml. Starting with JBoss AS 5.1.0 a scoped deployment should instead be specified using a jboss-classloading.xml file to the META-INF directory of the .esb archive.

So, if you have a deployment that is scoped add a META-INF/jboss-classloading.xml that looks something like this:

<classloading xmlns="urn:jboss:classloading:1.0" 
    domain="simple-scoped" 
    parent-first="false" />

Some more background about why this is needed can be found in this thread :
http://www.jboss.org/index.html?module=bb&op=viewtopic&t=154586