5 Replies Latest reply on May 16, 2008 12:59 PM by Ronald van Kuijk

    Out Of Memory problem

    Luis Miguel Rodriguez Newbie

      Hi,

      I've been using my workflow process without problems so far but did some modifications to execute a node (which calls a bash script) in a loop until some conditions are asserted. The main source code of the node's ActionHandler class is this:

       Iterator<String> it = (Iterator<String>)sitesList.iterator();
      
       LogRedirectThread lgError = null;
       LogRedirectThread lgOutput = null;
      
       while(it.hasNext()) {
       String newSite = (String)it.next();
      
       Process proc = rt.exec(pathToCrawl + " " + newSite + " " + pathToData + "/crawl-data/" + topic + " " + (String)contextInstance.getVariable("path") + " " + typeOfSources +" " + depth );
      
       lgError = new LogRedirectThread(proc.getErrorStream(),pathToData + "/logs/" + topic + "/crawler_sites_stderr_cycle" + cycleID + ".log");
       lgOutput = new LogRedirectThread(proc.getInputStream(),pathToData + "/logs/" + topic + "/crawler_sites_stdout_cycle" + cycleID + ".log");
      
       lgError.start();
       lgOutput.start();
      
       int exitVal = proc.waitFor();
      
       sitesList.remove(newSite);
      
       if (exitVal != 0) {
       throw new CrawlerException(newSite);
      
       }
      
       }
      


      that is, just call the script through the Runtime.exec() interface and creates two threads to read the stdout and stderr standard outputs.

      The problem is that I'm getting a heap OutOfMemory error when the node is executed more than 5 times (btw, my JAVA_OPTS is -Xmx1500m so that should be ok). My question are:
      How do jBPM and particularly the ActionHandler's invoker manage memory? and
      Do I need to release manually the resources created inside the ActionHandler::execute method, in my case the threads?

      Other useful information: some objects are also injected In my ActionHandler class from Spring framework. The OutOfMemory exception is thrown when, in a new cycle, the class is trying to get those beans from the spring context.

      Thanks

        • 1. Re: Out Of Memory problem
          Ronald van Kuijk Master

          in jbpm 3.x you should not make close loops without a persistent state in them. Use some other external mechanism to 'poll' for files and have that signal a persistent state.

          • 2. Re: Out Of Memory problem
            Luis Miguel Rodriguez Newbie

            Hi,

            thanks for your reply but that wasn't my question. My questions were related to jbpm memory management.
            BTW, my workflow executes more than one persistence states before going back to the same node.

            Best regards

            • 3. Re: Out Of Memory problem
              Ronald van Kuijk Master

              uhmm..... sometimes, well often, I just scan post and read them in all detail. If I see a *subject* that says 'out of memory problem' and I see things like node, loop, until in the first sentence, my answer is 'do not use loops without persistent states in between' (states where the process really 'stops' and all is saved to the db by closing the JbpmContext)

              jBPM memory management is not something it specifically does.... the jvm manages the memory. If you do not close the context, just save it, and have loops, there is no difference from not using persistent states... Lot's of references to objects will still be there.

              • 4. Re: Out Of Memory problem
                Luis Miguel Rodriguez Newbie

                Thanks for your reply.

                I'm using the jbpm-console for manage the workflow, I just wrote some action classes implementing the ActionHandler interface. So my question now is, when the JbpmContext is closed? I mean, is the JbpmContext closed when a specific action is executed and the workflow leaves the node (calling ExecutionContext.leavenode() ) or do I need to close it before leaving the node in order to release resources?

                Cheers

                • 5. Re: Out Of Memory problem
                  Ronald van Kuijk Master

                  look at the loads of testcases in the source how to use jBPM.

                  jBPM itself never closes the context, nor should you do that in nodes. It should happen in the 'controlling' logic. That is something you provide or e.g. the command executor if you use async stuff