Ideas for a DeploymentService

[dimitris]

I'm prototyping a DeploymentService that can generate simple deployment descriptors (e.g. to create a jms queue) or more complex unpacked archives (e.g. the ./deploy/jms or /jmx-console) that are created by copying a directory structure and overlaying/rendering any deployment descriptor using apache-velocity templates.

This allows the programmatic creation of "durable" services that survive server restarts, and the development of GUI apps that allow the administrator to instantiate services on the server (e.g. JMS) without having to write the descriptors.

In the prototype I can output the modules directly to ./deploy for the scanner to pick up, but I thought it would be better to use an intermediate directory (e.g. undeploy) where modules could be generated OR uploaded and reside there in an "undeployed" state, until they are explicitly moved to ./deploy (or back to undeploy).

So the API would look something like
Upload()
Create() (all 3 operating in the undeploy dir)
Remove()

Then I need a deploy() and undeploy(). In those 2 operations I would like to synchronise with the MainDeployer, somehow, so that I don't wait for the deployment scanner (faster) and get more immediate input on the outcome. So If I just move the module to deploy then immediately call MainDeployer.deploy() are there any issues conflicting with the Scanner? What if it happens to get there first? And if so, how will I learn about the fate and possible problem of the deployment (or undeployment)?

Also, it would be as well easy to move modules to ./farm or ha-singleton dirs, however, whom do I call for farmed or ha-singleton deployments? How the MainDeployer behaves in this case?

Any ideas and suggestion are welcome.

[dimitris]

Yes, exactly the Scanner works on a background thread so even if you do addURL() you don't get immediate feedback.

Instead, I think I'll get the subdeployer notification that a module was either started or destoyed, then I'll guery the main deployer for the DeploymentInfo object that contains the reason, e.g.

org.jboss.deployment.DeploymentInfo@bf6032d7 { url=file:/X:/cvs/jboss-public/jboss-4.0/build/output/jboss-4.0.1RC1/server/default/deploy/lolo-service.xml }
 deployer: org.jboss.deployment.SARDeployer@5e55ab
 status: Deployment FAILED reason: Trying to install an already registered mbean: jboss.mq.destination:name=Z,service=Queue
 state: FAILED
 watch: file:/X:/cvs/jboss-public/jboss-4.0/build/output/jboss-4.0.1RC1/server/default/deploy/lolo-service.xml
 altDD: null
 lastDeployed: 1098198222960
 lastModified: 1098198222950
 mbeans:

I might event call checkIncompleteDeployments() and see if I can figure-out something from the exception.

[dimitris]

Sorry it doe's call dstroy but when I'm creatin Thread in the init() method
and then try to destroy() my Thread in destroy I get the following exception :(my servlet name is StartErverServlet)

11:37:18,828 ERROR [Engine] StandardWrapper[/server:StartServerServlet]: Servlet StartServerServlet threw unload() exceptio
javax.servlet.ServletException: Servlet.destroy() for servlet StartServerServlet threw exception
at org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1135)
at org.apache.catalina.core.StandardWrapper.stop(StandardWrapper.java:1356)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:1036)
at org.jboss.web.tomcat.tc4.SingleSignOnContextConfig.stop(SingleSignOnContextConfig.java:814)
at org.jboss.web.tomcat.tc4.SingleSignOnContextConfig.lifecycleEvent(SingleSignOnContextConfig.java:257)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:166)
at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3690)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:1036)
at org.apache.catalina.core.StandardHostDeployer.remove(StandardHostDeployer.java:470)
at org.apache.catalina.core.StandardHost.remove(StandardHost.java:868)
at org.jboss.web.tomcat.tc4.EmbeddedTomcatService.performUndeploy(EmbeddedTomcatService.java:348)
at org.jboss.web.AbstractWebContainer.stop(AbstractWebContainer.java:476)
at org.jboss.deployment.MainDeployer.stop(MainDeployer.java:489)
at org.jboss.deployment.MainDeployer.stop(MainDeployer.java:504)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:472)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:467)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:440)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy6.undeploy(Unknown Source)
at org.jboss.deployment.scanner.URLDeploymentScanner.undeploy(URLDeploymentScanner.java:332)
at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:452)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:201)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:212)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:191)
11:37:18,875 ERROR [Engine] ----- Root Cause -----
java.lang.NoSuchMethodError
at java.lang.Thread.destroy(Thread.java:779)
at com.idi.framework.environment.server.servlet.StartServerServlet.destroy(StartServerServlet.java:311)
at org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1124)
at org.apache.catalina.core.StandardWrapper.stop(StandardWrapper.java:1356)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:1036)
at org.jboss.web.tomcat.tc4.SingleSignOnContextConfig.stop(SingleSignOnContextConfig.java:814)
at org.jboss.web.tomcat.tc4.SingleSignOnContextConfig.lifecycleEvent(SingleSignOnContextConfig.java:257)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:166)
at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3690)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:1036)
at org.apache.catalina.core.StandardHostDeployer.remove(StandardHostDeployer.java:470)
at org.apache.catalina.core.StandardHost.remove(StandardHost.java:868)
at org.jboss.web.tomcat.tc4.EmbeddedTomcatService.performUndeploy(EmbeddedTomcatService.java:348)
at org.jboss.web.AbstractWebContainer.stop(AbstractWebContainer.java:476)
at org.jboss.deployment.MainDeployer.stop(MainDeployer.java:489)
at org.jboss.deployment.MainDeployer.stop(MainDeployer.java:504)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:472)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:467)
at org.jboss.deployment.MainDeployer.undeploy(MainDeployer.java:440)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy6.undeploy(Unknown Source)
at org.jboss.deployment.scanner.URLDeploymentScanner.undeploy(URLDeploymentScanner.java:332)
at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:452)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:201)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:212)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:191)

[adrian.brock]

Obviously you want this to be interactive so you can display errors back to the user.

But I'm not convinced the deployers are thread safe enough to make this possible.
e.g. after MainDeployer.deploy() it will check for incomplete deployments
which will give spurious errors if there are concurrent requests.

[adrian.brock]

My preferred solution would be to do this via the Scanner.
The farm is a subclass of DeploymentScanner.

i.e. Telll the scanner you are adding a new url and get it to suspend its work
while you deploy. This will also allow it to update its deployment/timestamp
correctly so it doesn't try to redeploy.

The only downside to this, is that the Scanner can monitor any URLs.
You would have to matchup the file URL with the Scanner that is monitoring it.
What do you do when there are no file URLs being monitored, e.g. netboot?

[adrian.brock]

There is a TODO to make deploy-hasingleton an extension to Farm/Scanner.
Currently it does not hot deploy new urls.
The idea is that it would work like the Farm in distributing deployments, but they
would only hotdeploy on the singleton server.

[dimitris]

I'm not sure if I understand.

The Scanner points to ./deploy and this is where I plan to "move" the module once it is ready (move's supposed to be atomic operation in the filesystem). I might even use a subdir, like ./deploy/manager to separate the "managed" deployments.

Why should I add a new URL or suspend the Scanner? It'll already be looking there for new deployments. I would have to do this if I was using a different directory outside ./deploy e.g. ./deploy2, or if I wanted to add each module's URL individually (e.g. ./deploy2/module1.sar).

Could I just move the module in deploy and wait for a JMX Notification from the MainDeployer, they go and find out the status?

[adrian.brock]

I'm not sure if I understand.

The Scanner points to ./deploy and this is where I plan to "move" the module once it is ready (move's supposed to be atomic operation in the filesystem). I might even use a subdir, like ./deploy/manager to separate the "managed" deployments.

Why should I add a new URL or suspend the Scanner? It'll already be looking there for new deployments. I would have to do this if I was using a different directory outside ./deploy e.g. ./deploy2, or if I wanted to add each module's URL individually (e.g. ./deploy2/module1.sar).

Could I just move the module in deploy and wait for a JMX Notification from the MainDeployer, they go and find out the status?

How would you collect the logging so you can display what went wrong
on the gui/client?
The scanner does its work on a background thread.
This also needs to be integrated/usable with the jsr88 code
that Thomas wrote.

[dimitris]

Yes, exactly the Scanner works on a background thread so even if you do addURL() you don't get immediate feedback.

Instead, I think I'll get the subdeployer notification that a module was either started or destoyed, then I'll guery the main deployer for the DeploymentInfo object that contains the reason, e.g.

org.jboss.deployment.DeploymentInfo@bf6032d7 { url=file:/X:/cvs/jboss-public/jboss-4.0/build/output/jboss-4.0.1RC1/server/default/deploy/lolo-service.xml }
 deployer: org.jboss.deployment.SARDeployer@5e55ab
 status: Deployment FAILED reason: Trying to install an already registered mbean: jboss.mq.destination:name=Z,service=Queue
 state: FAILED
 watch: file:/X:/cvs/jboss-public/jboss-4.0/build/output/jboss-4.0.1RC1/server/default/deploy/lolo-service.xml
 altDD: null
 lastDeployed: 1098198222960
 lastModified: 1098198222950
 mbeans:

I might event call checkIncompleteDeployments() and see if I can figure-out something from the exception.

[adrian.brock]

With the way it works at the moment you might not get a reason.
e.g. It says the EAR is deployed but an ejb it contains has failed.

Some of the messages won't be very revealing, e.g.
"Verification of ejb-jar.xml failed - see above messages for more information"

You only need to look in the forums to see that people often post
the last exception (deployment failed or something not found)
when the real error is further back in the log.

[starksm64]

The template mechanism seems too disconnected from the current jmx notions. Effectively this is a custom ant build process embedded into jboss and has poor integration with future notions of heirarchical metadata, is coupled tightly to jboss dist layouts and service interface versions, and is too tightly coupled to the existing file system based configuration. I thought we had discussed an abstraction like a deployment service and configuration service that would allow for a more natural view of the server configuration, peristence and versioning as another aspect of the server.

[dimitris]

I agree it is most of those "bad" things. But I don't see it as a long term approach (i.e. for JBoss 5.x). I see it as an easy way to layer factory/deployment functionality on top of the the 3.2 and possibly 4.0 codebase, until we have better support in the new kernel.

If we break the problem into pieces, we have the issue of creating a new "service" that will survive a server restart (service being possibly a collection of mbeans) What options do exist now that wouldn't distrupt the jboss deployment model, apart from creating one or more deployment descriptor and either move them to deploy or submit them directly to the main deployer?

I think I should just commit the prototype that does the basic template/based service creation into ./undeploy dir, moving back and forth to ./deploy and removing from undeploy (without any feedback on the fate of the deployment), so you could all have a look and we can talk more specifically about it.

[adrian.brock]

I agree with Dimitris to an extent. The problem he is trying to solve is
deploy(URL) that survives a restart. This is relevent to 3.2.x regardless of
the enhancements coming with the new micro-kernel.

I don't think we need to consider the other problems for this thread, since
this is just a "better"
gui/ant > mydeployment
cp mydeployment server/[config]/deploy

The other problems still exist regardless of this service. Though this service can
take advantage of the solutions to those problems.

[starksm64]

Ok, as a bootstrap for getting something working on 3.2 this is fine.