message.getProperties().setProperty("Content-Type", new ResponseHeader("Content-Type", "image/jpeg"));
See here for more info: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=130661
That fixed the Content-Type issue.
I am still unable to send back binary data from the esb. Is that possible with the jbr-provider?
Have you tried setting the payload as a byte?
The payload is returned to JBoss Remoting 'as is' and it looks like the marshaller should write the contents of the byte directly to the output stream.