MTOM enabled WS using SSB throws OOME

[kazcor]

Dear all,

I'm trying to get a webservice up an running that is capable of receiving large documents, so I decided to give MTOM/XOP a try. Although I seem to have done everything necessary, I still get OutOfMemoryExceptions on both, client and server side.
Here is what I've done so far:
The service:

@WebService
@BindingType(value = SOAPBinding.SOAP11HTTP_MTOM_BINDING)
@MTOM(enabled = true)
@Stateless
public class SampleService {

 public void sendDocument(@XmlMimeType(value = "application/octet-stream") DataHandler document) {

 if (document != null) {
 try {
 InputStream in = document.getInputStream();
 File f = File.createTempFile("mtom", ".dat", new File("/tmp"));
 if (f.exists() && f.canWrite()) {
 FileOutputStream fout = new FileOutputStream(f);
 byte[] buffer = new byte[4096];
 int rLines = -1;
 while ((rLines = in.read(buffer)) > 0) {
 fout.write(buffer, 0, rLines);
 }
 fout.close();
 in.close();
 }
 } catch (Exception e) {
 e.printStackTrace();
 }
 }
 }

}

Deploys fine. After doing wsconsume on the WSDL I end up with this client:

public class TestClient {

 public static void main(String[] args) {

 if (args.length != 1) {
 System.err.println("usage: java -jar TestMTOM-client.jar <fileToSend>");
 return;
 }

 File fin = new File(args[0]);
 if (!fin.exists()) {
 System.err.println("No such file: "+args[0]);
 return;
 }

 SampleServiceService service = new SampleServiceService();
 SampleService port = service.getSampleServicePort();
 SOAPBinding binding = (SOAPBinding)((BindingProvider)port).getBinding();
 binding.setMTOMEnabled(true);

 DataHandler handler = new DataHandler(new FileDataSource(fin));
 port.sendDocument(handler);
 }
}

Logfile and final exception with DEBUG set on org.jboss and org.jboss.management after the client was invoked with a 100MB file (and using -Xmx512m to prevent OOME on client-side):

13:05:45,032 DEBUG [DefaultSPIProvider] provide SPI 'class org.jboss.wsf.spi.management.EndpointRegistryFactory'
13:05:45,033 DEBUG [DefaultSPIProvider] class org.jboss.wsf.spi.management.EndpointRegistryFactory Implementation: org.jboss.wsf.stack.jbws.EndpointRegistryFactoryImpl@4d4f6e
13:05:45,049 DEBUG [RequestHandlerImpl] doPost: /TestMTOM/SampleService
13:05:45,050 DEBUG [RequestHandlerImpl] handleRequest: jboss.ws:context=TestMTOM,endpoint=SampleService
13:05:45,070 DEBUG [MessageContextAssociation] pushMessageContext: org.jboss.ws.core.jaxws.handler.SOAPMessageContextJAXWS@104f889 (Thread http-127.0.0.1-8080-1)
13:05:45,080 DEBUG [RequestHandlerImpl] BEGIN handleRequest: jboss.ws:context=TestMTOM,endpoint=SampleService
13:05:45,130 DEBUG [MessageFactoryImpl] createMessage: [contentType=multipart/related;

type="application/xop+xml";
boundary="uuid:16c428f3-1021-43de-af64-aecc25b7fe93";
start-info="text/xml"]
13:05:45,740 DEBUG [DefaultSPIProvider] provide SPI 'class org.jboss.wsf.spi.management.ServerConfigFactory'
13:05:45,748 DEBUG [DefaultSPIProvider] class org.jboss.wsf.spi.management.ServerConfigFactory Implementation: org.jboss.wsf.framework.management.ServerConfigFactoryImpl@8fc809
13:05:53,229 DEBUG [SwapableMemoryDataSource] Using swap file, location = file:/usr/home/kazcor/Apps/jboss-4.2.3.GA/server/default/tmp/jbossws/JBossWSattachment52750.dat size = 139810362
13:05:58,690 DEBUG [HandlerDelegateJAXWS] callRequestHandlerChain: POST
13:05:58,697 DEBUG [HandlerResolverImpl] initHandlerChain: PRE
13:05:58,777 DEBUG [HandlerResolverImpl] addHandler:
HandlerMetaDataJAXWS:
type=PRE
name=Recording Handler
class=class org.jboss.wsf.framework.invocation.RecordingServerHandler
params=[]
protocols=##SOAP11_HTTP
services=null
ports=null
13:05:58,786 DEBUG [HandlerResolverImpl] initHandlerChain: ENDPOINT
13:05:58,794 DEBUG [HandlerResolverImpl] initHandlerChain: POST
13:05:58,808 DEBUG [HandlerResolverImpl] getHandlerChain: [type=POST,info=[service={http://mtom.test.openlimit.com/}SampleServiceService,port={http://mtom.test.openlimit.com/}SampleServicePort,binding=http://schemas.xmlsoap.org/wsdl/soap/http?mtom=true]]
13:05:58,816 DEBUG [HandlerChainExecutor] Create a handler executor: []
13:05:58,826 DEBUG [SOAPMessageDispatcher] getDispatchDestination: {http://mtom.test.openlimit.com/}sendDocument
13:05:58,834 DEBUG [SOAP11BindingJAXWS] unbindRequestMessage: {http://mtom.test.openlimit.com/}sendDocument
13:05:58,846 DEBUG [EndpointInvocation] setRequestParamValue: [name={http://mtom.test.openlimit.com/}sendDocument,value=org.jboss.ws.core.soap.SOAPBodyElementDoc]
13:05:58,861 DEBUG [HandlerDelegateJAXWS] callRequestHandlerChain: ENDPOINT
13:05:58,868 DEBUG [HandlerResolverImpl] getHandlerChain: [type=ENDPOINT,info=[service={http://mtom.test.openlimit.com/}SampleServiceService,port={http://mtom.test.openlimit.com/}SampleServicePort,binding=http://schemas.xmlsoap.org/wsdl/soap/http?mtom=true]]
13:05:58,876 DEBUG [HandlerChainExecutor] Create a handler executor: []
13:05:58,883 DEBUG [HandlerDelegateJAXWS] callRequestHandlerChain: PRE
13:05:58,890 DEBUG [HandlerResolverImpl] getHandlerChain: [type=PRE,info=[service={http://mtom.test.openlimit.com/}SampleServiceService,port={http://mtom.test.openlimit.com/}SampleServicePort,binding=http://schemas.xmlsoap.org/wsdl/soap/http?mtom=true]]
13:05:58,897 DEBUG [HandlerChainExecutor] Create a handler executor: []
13:05:58,907 DEBUG [EndpointInvocation] getRequestPayload
13:05:58,914 DEBUG [EndpointInvocation] getRequestParamValue: {http://mtom.test.openlimit.com/}sendDocument
13:05:58,922 DEBUG [SOAPContentElement] -----------------------------------
13:05:58,929 DEBUG [SOAPContentElement] Transitioning from XML_VALID to OBJECT_VALID
13:05:58,936 DEBUG [XMLContent] getObjectValue [xmlType={http://mtom.test.openlimit.com/}sendDocument,javaType=class com.openlimit.test.mtom.jaxws.SendDocument]
13:05:58,949 DEBUG [JAXBDeserializer] deserialize: [xmlName={http://mtom.test.openlimit.com/}sendDocument,xmlType={http://mtom.test.openlimit.com/}sendDocument]
13:06:03,792 DEBUG [HandlerDelegateJAXWS] closeHandlerChain
13:06:03,800 DEBUG [HandlerChainExecutor] close
13:06:03,807 DEBUG [HandlerDelegateJAXWS] closeHandlerChain
13:06:03,814 DEBUG [HandlerChainExecutor] close
13:06:03,821 DEBUG [HandlerDelegateJAXWS] closeHandlerChain
13:06:03,829 DEBUG [HandlerChainExecutor] close
13:06:03,836 DEBUG [RequestHandlerImpl] END handleRequest: jboss.ws:context=TestMTOM,endpoint=SampleService
13:06:03,844 DEBUG [MessageContextAssociation] popMessageContext: org.jboss.ws.core.jaxws.handler.SOAPMessageContextJAXWS@104f889 (Thread http-127.0.0.1-8080-1)
13:06:04,179 ERROR [[SampleService]] Servlet.service() for servlet SampleService threw exception
java.lang.OutOfMemoryError: Java heap space

Seems to me like it is indeed transferred using XOP and that a swapfile is generated for the incoming request. However, afterwards on deserialization something goes wrong - I hope the former swapped attachment doesn't get deserialized in memory at this point?

Tested with:
JBoss 4.2.3GA (default settings -Xms128m -Xmx512m)
JDK 1.5.0_14-p8
JBossWS (native) 3.1.1GA

Any ideas what I've missed?

Thanks in advance,
Max

[peterj]

Sounds like you need a larger heap for JBoss AS than 512m to handle the 100MB file. Try increasing -Xmx (I would try -Xmx1024m).

[kazcor]

Sure, that's one way to deal with that issue. Non the less, it's no solution and as soon as I'm going to send a file with lets say 1GB, I'll have the same problem.

As far as I understood one benefit of using MTOM (in case of JAX-WS) is, that there is no serialization/deserialization of the attached data taking place, actually providing in-/output as streams and not allocating a new object for the whole data structure. Why else would JBoss use a swapfile for the incoming request if at the same time the whole bunch of incoming data is stored in memory as well? But maybe I'm completely wrong here - that's why I'm asking :-)
Well, in the later case - what would be a proper workaround? Provide a handler which parses the incoming SOAPMessage and extracts (or cuts out) the attached data providing a unique identifier that might be used to access this data from somewhere else, once the webservice method is invoked and all deserialization has been done?

And yes, I'm aware that a web service is not the right solution for sending such huge documents, but by thinking about a middleware connected to a CMS backend, files (e.g. PDFs) with about 20-50MB are quite normal and should not result in a DoS ;-)

[asoldano]

Please fill in a JIRA for this. Also trying with jbossws-cxf would be interesting.
Afaik the MTOM spec is about optmizing what goes to the wire, but generally speaking I agree on what you say regarding the way attachment are handled by the stack.