How do I configure X for an MDB?

You will find details of configuration options in docs/dtd/jboss_3_2.dtd (or whatever version you are using)

The default configuration is in conf/standardjboss.xml. You can override the invoker-proxy-bindings and container-configurations in your own META-INF/jboss.xml

NOTE: invoker-proxy-bindings apply only to 3.2.x+. In 3.0.x this configuration was in the container-invoker-conf

Check out the examples in the testsuite