12 Replies Latest reply on Feb 7, 2010 10:28 AM by brian.stansberry

    RMI Classloading Broken

    brian.stansberry

      Discussion related to https://jira.jboss.org/jira/browse/JBAS-7274.

       

      The cause of the test failures in the tests-client-unit target is the move of jboss:service=WebService and jboss.rmi:type=RMIClassLoader out of conf/jboss-service.xml and into deploy/legacy-conf-service.xml.

       

      This breaks things because RMI classloading is controlled via system properties that are only read once -- on load of the relevant JDK classes:

       

      Property java.rmi.server.RMIClassLoaderSpi is read via a static initialization block in class java.rmi.server.RMIClassLoader

       

      Property java.rmi.server.codebase is read via a static initialization block in class sun.rmi.server.LoaderHandler.

       

      So, for RMI classloading to work we need to have these properties set up before any use of the RMI subsystem. The current earliest use of that is during deployment of deployers/ when org.jnp.server.Main creates a java.rmi.MarshalledObject. But of course that could easily shift around.

       

      Simply moving the two mbeans back into conf/jboss-service.xml doesn't work; it seems the create/start lifecycle methods don't get invoked until later in the startup, after org.jnp.server.Main. I suspect that to get this working I'd have to move the MainDeployer mbean back into conf/jboss-service.xml as well.

       

      This issue along with the presence of deployers/core-naming-jboss-beans.xml make me question whether getting rid of the "phase" formerly represented by conf/jboss-service.xml makes sense. Wouldn't it make more sense to acknowledge that certain stuff isn't part of the "bootstrap" but has to be done before we move on to creating deployers? Perhaps a conf/jboss-beans.xml? Going forward the ProfileService can do more sophisticated ordering of things based on capabilities/requirements, but for AS 6 (or at least M2) should we follow the KISS principle?

        • 1. Re: RMI Classloading Broken
          brian.stansberry
          Potentially we could separate the setting of these system properties from the actual opening of the port 8083 socket, doing the latter via a service in deploy/. Downside to that is if the listener service isn't deployed, we'll be setting an incorrect java.rmi.server.codebase value.
          • 2. Re: RMI Classloading Broken
            brian.stansberry

            Alternative:

             

            1) Figure out if/why anything in deployers/ needs access to a remotable naming service.

            2) Fix whatever we find in 1)

            3) Move RemoteNamingBean from deployers/ to deploy/. Or refactor such that the remoting aspects can move to deploy/.

            4) Find anything in deploy/ that exports an RMI stub or creates a java.rmi.server.MarshalledObject. Add a dependency on jboss:service=WebServer (or some refactored version thereof).

            5) Move ServiceBindingManager out of conf and into deploy/ since nothing outside of deploy/ depends on it (which is as it should be).

            • 3. Re: RMI Classloading Broken
              wolfc
              I would say that deployers startup should not depend on any facility. I even want to go even farther and say that deployers startup must never block (I/O etc).
              • 4. Re: RMI Classloading Broken
                brian.stansberry

                For 1, 2, 3 above: https://jira.jboss.org/jira/browse/JBAS-7580

                 

                For 4 above, the original testsuite failure JIRA https://jira.jboss.org/jira/browse/JBAS-7274 is now for that

                 

                for 5 above: https://jira.jboss.org/jira/browse/JBAS-7583

                • 5. Re: RMI Classloading Broken
                  brian.stansberry

                  Agreed in theory, but in practice it may be a bit tricky; would be good to solve.

                   

                  The tricky thing is if the deployer needs the service to do it's deploy() work (e.g. org.jboss.beanvalidation.deployers.ValidatorFactoryJndiBinderDeployer binds stuff in JNDI, so it needs JNDI) then the service has to be there before deploy() is called.  Really, it's whatever it's deploying that depends on JNDI, not the deployer itself. But that's not so simple to express, since when the deployer does it's work, the MC might not even be aware of the deployment and thus can't enforce such a dependency. This ValidatorFactoryJndiBinderDeployer is PRE_REAL.

                   

                  One approach to this kind of thing would be for the deployer to not bind anything in JNDI; rather it could create and ask the MC to install a BeanMetaData for an on-demand bean that does the binding. Then add a dependency on this bean to the DeploymentUnit.

                   

                  Note that I don't see ValidatorFactoryJndiBinderDeployer itself being deployed anywhere, so this specific example may not be a real use case.

                  • 6. Re: RMI Classloading Broken
                    dallen6

                    I think this is the same problem with the Profile Service no longer working remotely.  I tried to update our Weld TCK Test Harness for 6.0.0-SNAPSHOT, but the client impl classes for the Profile Service are no longer available.  They were removed from the JBossAS client libs sometime after M1 and are now included in the server lib directory.  Debugging only showed me that all three classloading options in ClassLoaderUtil on the client side were exhausted with no luck.

                     

                    I held off on filing a Profile Service bug till I could see if something more pervasive, like this problem, existed in AS 6.0.0 trunk.  On the other hand, I could file the bug and reference the JIRA reports mentioned above?

                    • 7. Re: RMI Classloading Broken
                      brian.stansberry
                      David, as we discussed privately while the forums weren't working, Emanuel's looking into why the ProfileService classes aren't showing up in the proper locations.
                      • 8. Re: RMI Classloading Broken
                        brian.stansberry

                        The ProfileService changes potentially in the works for M3 may make all existing issues related to deployment phases (conf/bindings.beans, conf/jboss-service.xml, deployers. deploy) somewhat obsolete. Or at least very different in nature.

                         

                        So, I don't want to risk stability in trunk by moving everything to deploy/ for M2. For M2 I'm just going to add a hack to put an mbean in conf/jboss-service.xml that sets java.rmi.server.codebase. Jaikiran has already dealt with java.rmi.server.RMIClassLoaderSpi in his fix for https://jira.jboss.org/jira/browse/JBAS-7588

                        • 9. Re: RMI Classloading Broken
                          jaikiran

                          bstansberry@jboss.com wrote:

                           

                           

                           

                          The tricky thing is if the deployer needs the service to do it's deploy() work (e.g. org.jboss.beanvalidation.deployers.ValidatorFactoryJndiBinderDeployer binds stuff in JNDI, so it needs JNDI) then the service has to be there before deploy() is called.  Really, it's whatever it's deploying that depends on JNDI, not the deployer itself.


                          bstansberry@jboss.com wrote in https://jira.jboss.org/jira/browse/JBAS-7588:

                           

                          A month ago I worked through and tested how to move everything into deploy, it's mostly a matter of checking whether M2 actually *has* deployers that directly bind stuff in JNDI and then making a go/no-go decision. There were some in the plans a month ago, but last I checked they weren't actually implemented.


                          For the record - EJB3.1 nointerface view deployer processes a deployment unit and installs a MC bean (BeanMetaData) into the kernel. The MC bean on its start binds proxy to the JNDI. However, the BeanMetaData does not (currently) have any explicit dependencies on this RMI classloading service. IMO, i think the RMI classloading is an internal impl dependency of the naming service, so adding it as a dependency in the BeanMetaData perhaps isn't right from a deployer point of view.

                          • 10. Re: RMI Classloading Broken
                            brian.stansberry

                            Absolutely. Your BeanMetaData should have a dependency on LocalNamingBean, or some alias thereof, or if it binds in some child context on whatever bean creates that child context. It's then up to the configuration of LocalNamingBean to ensure this classloading stuff gets done first. Right now that could be done by adding a depends to LocalNamingBean. The ProfileService stuff Emanuel's planning will allow something a bit cleaner, an "optional dependency" notion that imposes ordering if the dependency exists but doesn't fail the dependent item if the dependency doesn't exist.

                             

                            BTW, for M2 I already put in the conf/jboss-service.xml hack I described above.

                            • 11. Re: RMI Classloading Broken
                              jaikiran

                              bstansberry@jboss.com wrote:

                               

                              Absolutely. Your BeanMetaData should have a dependency on LocalNamingBean, or some alias thereof, or if it binds in some child context on whatever bean creates that child context. It's then up to the configuration of LocalNamingBean to ensure this classloading stuff gets done first.


                              Ah right! Makes sense to add a dependency on the NamingBean. I earlier, incorrectly, thought that we were discussing about adding a dependency on the classloading stuff.

                              • 12. Re: RMI Classloading Broken
                                brian.stansberry

                                Yeah, sorry, my stuff above wasn't very clear.

                                 

                                There are a number of beans that just assume local JNDI is available without adding any dependency; those should be cleaned up:

                                 

                                https://jira.jboss.org/jira/browse/JBAS-7709