1 2 Previous Next 15 Replies Latest reply on Feb 13, 2009 4:58 PM by Richard Opalka

    Dynamic Endpoint Deployment

    Richard Opalka Master

      Kabir Khan wrote:

      If you've seen the dev-list discussions yesterday regarding JBAS-6489, I can work around this in AOP, but I have found the cause to be something to do with org.jboss.test.webservice.endpoint.EndpointTestCase. It deploys jaxws-endpoint-servlet.war.

      Hot-deploying the war seems to go through the deployer chain twice. Once, when it goes through usual hot-deployment (See Stack1 below), then it does something in the tomcat/service-layer, I guess to deploy the WS endpoint (See Stack2 below).

      Now the problem is that the first time it deploys this, it correctly includes a Module attachment with the deploymentunit which links the classloader to a scoped classloading domain called "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war" (See variables1 below). When it deploys the second time, it incorrectly includes a Module attachment with the deploymentunit linking the classloader to the main/non-scoped classloading domain called DefaultDomain. Somehow the new deployment created internally needs to either include the original Module, or the correct classloading metadata to reuse the name "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war". I think this happens around this point of Stack2:


      DynamicEndpointDeploymentAspect.create(Deployment) line: 88
      DeploymentAspectManagerImpl.deploy(Deployment) line: 115
      DeploymentAspectHttpServer.publish(HttpContext, Endpoint) line: 91

      Since deployment 2 also passes through the AOPClassLoader deployer and it is getting the wrong information, AOP gets confused about the AOP domains used.

      Stack1:
      Daemon System Thread [RMI TCP Connection(10)-127.0.0.1] (Suspended (breakpoint at line 58 in VFSClassLoaderDomainRegistry))
      VFSClassLoaderDomainRegistry.initMapsForLoader(ClassLoader, Module, ScopedVFSClassLoaderDomain, ClassLoader) line: 58
      AOPClassLoaderInitializer.registerLoaders(AOPClassLoaderScopingPolicyWithRegistry, DeploymentUnit) line: 78
      AOPClassLoaderInitializer.initializeForUnit(DeploymentUnit) line: 41
      AOPClassLoaderDeployer.internalDeploy(DeploymentUnit) line: 65
      AOPClassLoaderDeployer(AbstractRealDeployer).deploy(DeploymentUnit) line: 50
      DeployerWrapper.deploy(DeploymentUnit) line: 171
      DeployersImpl.doDeploy(Deployer, DeploymentUnit) line: 1439
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1157
      DeployersImpl.install(ControllerContext, ControllerState, ControllerState) line: 1098
      DeploymentControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1598
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1062
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      DeployersImpl.process(List, List) line: 781
      MainDeployerImpl.process() line: 572
      MainDeployer.deploy(URL) line: 812
      MainDeployer.redeploy(URL) line: 587

      Stack2:
      Daemon System Thread [RMI TCP Connection(10)-127.0.0.1] (Suspended (breakpoint at line 58 in VFSClassLoaderDomainRegistry))
      VFSClassLoaderDomainRegistry.initMapsForLoader(ClassLoader, Module, ScopedVFSClassLoaderDomain, ClassLoader) line: 58
      AOPClassLoaderInitializer.registerLoaders(AOPClassLoaderScopingPolicyWithRegistry, DeploymentUnit) line: 78
      AOPClassLoaderInitializer.initializeForUnit(DeploymentUnit) line: 41

      AOPClassLoaderDeployer.internalDeploy(DeploymentUnit) line: 65
      AOPClassLoaderDeployer(AbstractRealDeployer).deploy(DeploymentUnit) line: 50
      DeployerWrapper.deploy(DeploymentUnit) line: 171
      DeployersImpl.doDeploy(Deployer, DeploymentUnit) line: 1439
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1157
      DeployersImpl.install(ControllerContext, ControllerState, ControllerState) line: 1098
      DeploymentControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1598
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1062
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      DeployersImpl.process(List, List) line: 781
      MainDeployerImpl.deploy(Deployment...) line: 445
      DynamicEndpointDeploymentAspect.create(Deployment) line: 88
      DeploymentAspectManagerImpl.deploy(Deployment) line: 115
      DeploymentAspectHttpServer.publish(HttpContext, Endpoint) line: 91
      EndpointImpl.publish(Object) line: 172
      EndpointServlet.init(ServletConfig) line: 70
      StandardWrapper.loadServlet() line: 1048
      StandardWrapper.load() line: 950
      StandardContext.loadOnStartup(Container[]) line: 4122
      StandardContext.start() line: 4421
      TomcatDeployment.performDeployInternal(WebApplication, String, String) line: 315
      TomcatDeployment.performDeploy(WebApplication, String) line: 147
      TomcatDeployment(AbstractWarDeployment).start(DeploymentUnit, JBossWebMetaData) line: 461
      WebModule.startModule() line: 118
      WebModule.start() line: 97
      NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
      NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
      DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
      Method.invoke(Object, Object...) line: 597
      ReflectedDispatcher.invoke(Invocation) line: 157
      Invocation.dispatch() line: 96
      Invocation.invoke() line: 88
      XMBean(AbstractMBeanInvoker).invoke(String, Object[], String[]) line: 264
      MBeanServerImpl.invoke(ObjectName, String, Object[], String[]) line: 668
      ServiceProxy.invoke(Object, Method, Object[]) line: 206
      $Proxy36.start() line: not available
      StartStopLifecycleAction.installAction(ServiceControllerContext) line: 42
      StartStopLifecycleAction.installAction(ControllerContext) line: 37
      StartStopLifecycleAction(SimpleControllerContextAction).simpleInstallAction(T) line: 62
      StartStopLifecycleAction(AccessControllerContextAction<S,T>).install(ControllerContext) line: 71
      ServiceControllerContextActions(AbstractControllerContextActions).install(ControllerContext, ControllerState, ControllerState) line: 51
      ServiceControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      ServiceControllerContext.install(ControllerState, ControllerState) line: 286
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1598
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1062
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      ServiceController.doChange(KernelController, ServiceControllerContext, ControllerState, String) line: 688
      ServiceController.start(ObjectName) line: 460
      ServiceDeployer.start(ServiceContext) line: 163
      ServiceDeployer.deploy(DeploymentUnit, ServiceMetaData) line: 99
      ServiceDeployer.deploy(DeploymentUnit, Object) line: 46
      ServiceDeployer(AbstractSimpleRealDeployer).internalDeploy(DeploymentUnit) line: 62
      ServiceDeployer(AbstractRealDeployer).deploy(DeploymentUnit) line: 50
      DeployerWrapper.deploy(DeploymentUnit) line: 171
      DeployersImpl.doDeploy(Deployer, DeploymentUnit) line: 1439
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1157
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1178
      DeployersImpl.install(ControllerContext, ControllerState, ControllerState) line: 1098
      DeploymentControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1598
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1062
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      DeployersImpl.process(List, List) line: 781
      MainDeployerImpl.process() line: 572
      MainDeployer.deploy(URL) line: 812
      MainDeployer.redeploy(URL) line: 587

      Variables1:
      loader BaseClassLoader (id=641)
      module VFSDeploymentClassLoaderPolicyModule (id=676)
      capabilities CopyOnWriteArrayList (id=677)
      classLoader BaseClassLoader (id=641)
      classLoadingMetaData ClassLoadingMetaData (id=678)
      context DeploymentControllerContext (id=637)
      contextName "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war" (id=666)
      domain Domain (id=361)
      classLoading ClassLoading (id=701)
      modules CopyOnWriteArrayList (id=703)
      modulesByName ConcurrentHashMap<K,V> (id=704)
      name "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war" (id=261)
      parentDomainName "DefaultDomain" (id=585)
      parentFirst false
      excludedRoots VirtualFile[0] (id=679)
      name "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war" (id=666)
      policy VFSClassLoaderPolicy (id=680)
      requirementDependencies null
      requirements null
      space null
      system DefaultClassLoaderSystem (id=389)
      unit AbstractVFSDeploymentUnit (id=638)
      version Version (id=681)
      vfsRoots VirtualFile[2] (id=683)

        • 1. Re: Dynamic Endpoint Deployment
          Richard Opalka Master

          In this thread we're talking about the following part of the stack trace:

          MainDeployerImpl.deploy(Deployment...) line: 445
          DynamicEndpointDeploymentAspect.create(Deployment) line: 88
          DeploymentAspectManagerImpl.deploy(Deployment) line: 115
          DeploymentAspectHttpServer.publish(HttpContext, Endpoint) line: 91
          EndpointImpl.publish(Object) line: 172
          EndpointServlet.init(ServletConfig) line: 70
          StandardWrapper.loadServlet() line: 1048
          StandardWrapper.load() line: 950
          StandardContext.loadOnStartup(Container[]) line: 4122
          StandardContext.start() line: 4421
          TomcatDeployment.performDeployInternal(WebApplication, String, String) line: 315
          TomcatDeployment.performDeploy(WebApplication, String) line: 147
          TomcatDeployment(AbstractWarDeployment).start(DeploymentUnit, JBossWebMetaData) line: 461
          WebModule.startModule() line: 118
          WebModule.start() line: 97



          Hot-deploying the war seems to go through the deployer chain twice. Once, when it goes through usual hot-deployment (See Stack1 below), then it does something in the tomcat/service-layer, I guess to deploy the WS endpoint (See Stack2 below).

          Correct, it goes through deployers chain twice. This is evident from the stack trace above ;)

          Now the problem is that the first time it deploys this, it correctly includes a Module attachment with the deploymentunit which links the classloader to a scoped classloading domain called "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war" (See variables1 below).

          Correct because first walk through deployers chain is pure web app deployment (Servlet in war)

          When it deploys the second time, it incorrectly includes a Module attachment with the deploymentunit linking the classloader to the main/non-scoped classloading domain called DefaultDomain.

          We do construct "minimal Deployment" needed to pass the deployers chain programatically (because we're calling MainDeployer.deploy() from servlet code).
          Regarding the classloading issue you're facing it's because
          our DynamicEndpointDeploymentAspect puts
          Servlet context classloader to the constructed "minimal Deployment".

          Somehow the new deployment created internally needs to either include the original Module,

          I don't know how to achieve that because web service endpoint is published from servlet and we do not have access to DeploymentUnit from it :(
          or the correct classloading metadata to reuse the name "vfszip:/Users/kabir/sourcecontrol/jbossas/Branch_5_0/testsuite/output/lib/jaxws-endpoint-servlet.war".

          How to achieve that if we don't have access to DeploymentUnit? Is it possible to construct it dynamically?


          • 2. Re: Dynamic Endpoint Deployment
            Kabir Khan Master

             

            "richard.opalka@jboss.com" wrote:


            Somehow the new deployment created internally needs to either include the original Module,

            I don't know how to achieve that because web service endpoint is published from servlet and we do not have access to DeploymentUnit from it :(

            If you have the deployment's ClassLoader, you can obtain the Module by calling
            org.jboss.classloading.spi.dependency.ClassLoading.getModuleForClassLoader()
            This method exists in the version of jboss-cl that exists in AS Branch_5_0.

            I am not sure if adding the Module as an attachment to the created deployment will have any effect since it looks like a Module is always created by AbstractClassLoaderDescribeDeployer/VFSClassLoaderDescribeDeployer, but you can try.

            If that doesn't work I think you need to create a ClassLoadingMetaData from the Module, and put that into the deployment as an attachment.

            • 3. Re: Dynamic Endpoint Deployment
              Dimitris Andreadis Master

              Guys, I need to understand if any of this work should hold the AS 5.0.1 release and what exactly would have to be changed. Any pending component updates? Thanks.

              • 4. Re: Dynamic Endpoint Deployment
                Kabir Khan Master

                This is what causes the error in https://jira.jboss.org/jira/browse/JBAS-6489. I am testing a fix in AOP to make it more resistant to things like this, but I think this root cause needs addressing. I still need to do an AOP 2.0.1.GA which I am aiming to do today.

                As far as I can tell the work needed for the issue on this thread is in the AS source in org.jboss.wsf.container.jboss50.transport.DynamicEndpointDeploymentAspect.

                • 5. Re: Dynamic Endpoint Deployment
                  Richard Opalka Master

                   

                  "kabir.khan@jboss.com" wrote:

                  If that doesn't work I think you need to create a ClassLoadingMetaData from the Module, and put that into the deployment as an attachment.

                  We don't have the Module reference available :( As I said above, we're doing all the magic from servlet code (we're not in deployers processing chain when creating that "minimalistic Deployment") :(

                  • 6. Re: Dynamic Endpoint Deployment
                    Kabir Khan Master

                    To be clear I can work around this with my AOP release. So this is not blocking me

                    • 7. Re: Dynamic Endpoint Deployment
                      Richard Opalka Master

                       

                      "kabir.khan@jboss.com" wrote:
                      To be clear I can work around this with my AOP release. So this is not blocking me

                      All the dynamic webservice endpoint publish hara-kiri is one big ugly hack.
                      This JAX-WS spec. requirement is real challenge to be implemented properly. If you can workaround it, then do it please.

                      • 8. Re: Dynamic Endpoint Deployment
                        Richard Opalka Master

                        However hints/suggestions from AlesJ, AdrianB and others are welcomed to implement it properly.

                        • 9. Re: Dynamic Endpoint Deployment
                          Kabir Khan Master

                           

                          "richard.opalka@jboss.com" wrote:
                          "kabir.khan@jboss.com" wrote:

                          If that doesn't work I think you need to create a ClassLoadingMetaData from the Module, and put that into the deployment as an attachment.

                          We don't have the Module reference available :( As I said above, we're doing all the magic from servlet code (we're not in deployers processing chain when creating that "minimalistic Deployment") :(


                          You obviously have access to the ClassLoader, so:

                          "kabir.khan@jboss.com" wrote:

                          If you have the deployment's ClassLoader, you can obtain the Module by calling
                          org.jboss.classloading.spi.dependency.ClassLoading.getModuleForClassLoader()
                          This method exists in the version of jboss-cl that exists in AS Branch_5_0.



                          • 10. Re: Dynamic Endpoint Deployment
                            Dimitris Andreadis Master

                            Can you guys create a JIRA so this DeynamicEndpointDeploymentAspect is fixed properly in a following release?

                            • 11. Re: Dynamic Endpoint Deployment
                              Richard Opalka Master

                              Follow up:

                              JBAS-6496 for AS 5.0.1.GA
                              JBAS-6497 for AS 5.1.0.Beta1 and AS 6.0.0.Alpha1

                              • 13. Re: Dynamic Endpoint Deployment
                                Jason Greene Master

                                The Endpoint API is really intended for out-of-container usage. The TCK for EE5 doesn't verify it any long since the test used to assume sun implementation details.

                                That said, to "fix" in container usage, I would recommend creating an AS endpoint deployment/management service that manages all of the AS details. Then you can just delegate calls from the Endpoint API to the endpoint service instead of calling MainDeployer from the web container.

                                -Jason

                                • 14. Re: Dynamic Endpoint Deployment
                                  Jason Greene Master

                                  Spoke to soon. I glanced at the code, and you kind of have this already, so I don't see the problem in passing the correct information. You just need to enhance the service to track the deployment information associated with a context.

                                  Btw DeploymentAspectHttpService isn't thread safe.

                                  1 2 Previous Next