9 Replies Latest reply on Jul 17, 2008 5:47 AM by vickyk

    4.2.2.GA IdleRemover thread inserted into my thread group

    guy_rouillier

      I have a fairly large application running under 3.2.7 that I'm trying to upgrade to 4.2.2.GA. The code upgrade was fairly trivial, and I've got the app running in test mode now. One part of this app creates a ThreadGroup to process a set of data files; the end of this processing is to insert data into an Oracle database using a no-tx-datasource. All the inserts are being completed successfully.

      But when I determine all worker threads are finished and attempt to destroy the ThreadGroup, I'm getting an IllegalThreadStateException. Upon doing a ThreadGroup.enumerate(), I'm finding a single thread with name IdleRemover.

      Why is this IdleRemover thread being inserted into my private ThreadGroup? I've spent a couple hours looking at old forum discussions, and I see that IdleRemover was present in 3.2.7. But the app running on that version didn't have this problem. So I'm guessing the implementation of IdleRemover has changed in 4.2.2.

      How do I fix this? Thanks.

        • 1. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
          vickyk

           

          "guy_rouillier" wrote:

          Why is this IdleRemover thread being inserted into my private ThreadGroup?

          I suspect that you are playing wrongly with the Threadgroups , show us the code which you are trying.

          • 2. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
            guy_rouillier

            Vicky, thanks for the reply. I finally have a reproducible test case. I just spent two hours packaging it up and providing a read.me, but now it looks like I'm not allowed to upload a file here.

            I can get this to happen using either an hsql or an Oracle datasource. I'm using no-tx-datasource with an idle timeout specified. As I say in the read.me, this problem occurs if only one no-tx-datasource is deployed. I suppose I'll include the code for the EJB below. When this is executed, the do loop terminated with "while (threadCt > 0)" never ends.

            If there is some way for me to upload a file, let me know. Thanks.

            package guyr;
            
            import java.sql.*;
            import javax.sql.*;
            import javax.naming.*;
            import javax.annotation.*;
            import javax.ejb.*;
            import org.apache.log4j.Logger;
            
            @Stateless(name = "guyr/TGroup")
            
            public class TGroupBean implements TGroup
             {
             private final Logger log = Logger.getLogger(getClass().getName());
             private final int MAX_THREADS = 1;
            
             // For HSQL
            
             // @Resource(mappedName="java:/hsqlDB")
             // private javax.sql.DataSource ds;
             //
             // String aSql = "select 'hsql' " +
             // "from information_schema.system_tables " +
             // "where table_name = 'SYSTEM_TABLES' " ;
            
             // For Oracle
            
             @Resource(mappedName="java:/rumbaDB")
             private javax.sql.DataSource ds;
            
             String aSql = "select 'oracle' " +
             "from dual " ;
            
             private class TData extends Thread
             {
             // Doesn't work if resource is inside this thread class
            
             public TData(ThreadGroup tg, String aThreadName)
             {
             super(tg, aThreadName);
             } // end constructor
            
             public void run()
             {
             log.info("Thread " + this.getName() + " started");
            
             Connection c = null;
             Statement s = null;
             ResultSet r = null;
            
            
             try
             {
             c = ds.getConnection();
             // c.setAutoCommit(false);
             // c = ((DataSource)(new InitialContext()).lookup("java:/DefaultDS")).getConnection();
             s = c.createStatement();
             r = s.executeQuery(aSql);
            
             if ((r != null) && (r.next()))
             {
             log.info("Thread " + this.getName() + " result: " + r.getString(1));
             } // end if
            
             // c.commit();
             } // end try
             catch (Exception e)
             {
             log.error("Thread " + this.getName() + " exception: ", e);
             // try
             // {
             // c.rollback();
             // } // end try
             // catch (Exception ne)
             // {
             // log.error("Thread " + this.getName() + " exception: ", ne);
             // } // end catch
             } // end catch
             finally
             {
             if (r != null)
             {
             try
             {
             r.close();
             } // end try
             catch (Exception e)
             {
             log.error("Thread " + this.getName() + " exception: ", e);
             } // end catch
             } // end if
            
             if (s != null)
             {
             try
             {
             s.close();
             } // end try
             catch (Exception e)
             {
             log.error("Thread " + this.getName() + " exception: ", e);
             } // end catch
             } // end if
            
             if (c != null)
             {
             try
             {
             // c.setAutoCommit(true);
             c.close();
             } // end try
             catch (Exception e)
             {
             log.error("Thread " + this.getName() + " exception: ", e);
             } // end catch
             } // end if
             } // end finally
             } // end run
             } // end class TData
            
             public void run()
             {
             log.info("TGroupBean.run() started");
            
             try
             {
             ThreadGroup tgTest = new ThreadGroup("TEST_THREADGROUP");
            
             for (int i = 0; i < MAX_THREADS; i++)
             {
             TData td = new TData(tgTest, "THREAD " + i);
             td.start();
             } // end for
            
             int threadCt = 0;
            
             do
             {
             Thread.sleep(2000);
            
             Thread[] activeThreads = new Thread[MAX_THREADS + 1];
             threadCt = tgTest.enumerate(activeThreads);
            
             for (int i = 0; i < threadCt; i++)
             {
             log.info("ThreadGroup " + tgTest.getName() + " thread " + i + " : " +
             activeThreads.getName());
             }
             } while (threadCt > 0); // end do
             } // end try
             catch (Exception e)
             {
             log.error("Exception caught: " + e.getMessage());
             } // end catch
            
             log.info("TGroupBean.run() ended");
             } // end run
             } // end class TGroupBean
            
            


            • 3. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
              vickyk

               

              "guy_rouillier" wrote:

              If there is some way for me to upload a file, let me know. Thanks.

              You can mail it to me at vicky.kak@jboss.com , I will have a look at it .


              • 4. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                vickyk

                Hi Guy ,

                I have been able to determine the cause of this issue .

                Here is the code which creates the IdleRemover thread

                public Object run()
                 {
                 Runnable runnable = new IdleRemoverRunnable();
                 Thread removerThread = new Thread(runnable, "IdleRemover");
                 removerThread.setDaemon(true);
                 removerThread.start();
                 return null;
                 }
                

                http://anonsvn.jboss.org/repos/jbossas/branches/Branch_4_2/connector/src/main/org/jboss/resource/connectionmanager/IdleRemover.java

                Now look at this one
                Thread removerThread = new Thread(runnable, "IdleRemover");


                This is getting invoke through the application thread context and in your case the application thread falls in the TEST_THREADGROUP threadgroup .

                If you look at the java docs you will read these things

                Allocates a new Thread object. This constructor has the same effect as Thread(null, target, name).


                If group is null and there is a security manager, the group is determined by the security manager's getThreadGroup method. If group is null and there is not a security manager, or the security manager's getThreadGroup method returns null, the group is set to be the same ThreadGroup as the thread that is creating the new thread.


                http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Thread.html#Thread(java.lang.Runnable,%20java.lang.String)

                This would explain why the IdleThread goes in the TEST_THREADGROUP .

                The same can be viewed from listThreadDump() operations of the jboss.system:type=ServerInfo MBean .

                Ideally the jboss generated Threads should go in the jboss ThreadGroup , I am going to make changes for that .

                I am interested to know how does this behavior creates an issues in your application .

                btw , here is the jsp version of the problem .

                <%@ page import="javax.xml.*,javax.sql.*,javax.naming.*,java.sql.*" %>
                <%!
                 ThreadGroup tgroup = null ;
                 int counter = 0;
                 String aSql = "select 'hsql' " +
                
                 "from information_schema.system_tables " +
                
                 "where table_name = 'SYSTEM_TABLES' " ;
                
                
                 private class TData extends Thread
                 {
                
                 public TData(ThreadGroup tg, String aThreadName)
                
                 {
                
                 super(tg, aThreadName);
                
                 } // e
                
                 public TData(String aThreadName)
                
                 {
                
                 super(aThreadName);
                
                 } // e
                 public void run(){
                 try{
                 InitialContext context = new InitialContext();
                 DataSource ds = (DataSource)context.lookup("java:/hsqlDB");
                 Connection con = ds.getConnection();
                 Statement stmt = con.createStatement();
                
                 ResultSet rs = stmt.executeQuery(aSql);
                 while(rs.next())
                 System.out.println("--->"+rs.getString(1));
                 Thread.sleep(1000);
                 System.out.println("DOne");
                 }
                 catch(Exception e)
                 {
                 }
                 }
                 }
                %>
                
                <%
                
                 ThreadGroup tgTest = new ThreadGroup("TEST_THREADGROUP");
                 out.println("coutner is "+counter+"<br>");
                 if(counter == 0){
                
                 tgroup = tgTest;
                 }
                
                 //v.add(tgTest);
                 for (int i = 0; i < 2; i++)
                
                 {
                
                 TData td = new TData(tgTest, "THREAD " + i);
                 //TData td = new TData("THREAD " + i);
                
                 td.start();
                
                 }
                
                 Thread[] activeThreads = new Thread[10];
                 int threadCt = tgTest.enumerate(activeThreads);
                
                 out.println(threadCt+"<br>");
                 out.println(tgTest.getParent()+"<br>");
                 for (int i = 0; i < threadCt; i++)
                
                 {
                
                 out.println("ThreadGroup " + tgTest.getName() + " thread " + i + " : " +
                
                 activeThreads.getName()+"<br>");
                
                 }
                
                 out.println("First group count ::: "+tgroup.activeCount()+"<br>");
                 Thread[] activeList = new Thread[tgroup.activeCount()+1];
                 int threadCt1 = tgroup.enumerate(activeList);
                 for (int i = 0; i < threadCt1; i++)
                
                 {
                 out.println(activeList.getName()+"<br>");
                 }
                 counter = counter + 1;
                
                %>



                • 5. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                  vickyk

                   

                  "vickyk" wrote:

                  I am interested to know how does this behavior creates an issues in your application

                  It has been explained in initial post .

                  Let me see how things were done in the Jboss3.2.* series .



                  • 6. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                    vickyk

                    I get this error when I run jsp on Jboss3.2.* series so I wonder if you have tried the same code over there or not

                    14:33:53,058 ERROR [STDERR] java.sql.SQLException: Table not found: INFORMATION_SCHEMA.SYSTEM_TABLES in statement [select 'hsql' from information_schema.system_tables where table_name = 'SYSTEM_TABLES'




                    • 7. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                      guy_rouillier

                      Vicky, thanks very much for your detective work, and for confirming the issue. Regarding your last post, I can only guess that JBoss 3.2.x shipped with an earlier version of hsql, and that version apparently does not have a table named INFORMATION_SCHEMA.SYSTEM_TABLES. You should be able to substitute any other available table for which you can isolate a single row; as you can see from the query, the results are immaterial. I'll look for a substitute table later today or tomorrow.

                      Unfortunately, the workaround I proposed in the read.me of introducing a dummy hsql no-tx-datasource works for me in my Windows development environment but not in our Linux production environment. Don't know why, but for the time being I'm setting our Oracle datasource to use idle-timeout-minutes = 0. This prevents the IdleRemover thread from being created. Not an optimal solution, but will have to do until your fix makes its way into the code base.

                      • 8. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                        guy_rouillier

                         

                        "vickyk" wrote:
                        I get this error when I run jsp on Jboss3.2.* series so I wonder if you have tried the same code over there or not

                        14:33:53,058 ERROR [STDERR] java.sql.SQLException: Table not found: INFORMATION_SCHEMA.SYSTEM_TABLES in statement [select 'hsql' from information_schema.system_tables where table_name = 'SYSTEM_TABLES'




                        I got your JSP page running under JBoss 3.2.7. Just get rid of the schema qualifier "information_schema." from in from of "system_tables".

                        • 9. Re: 4.2.2.GA IdleRemover thread inserted into my thread grou
                          vickyk