1 2 Previous Next 17 Replies Latest reply on Jun 21, 2010 1:38 PM by alesj

    AutoInstall semantics in supported target containers

    thomas.diesler

      The  OSGiAutoInstallPlugin is currently only used by the standalone Runtime and not by the AS.

       

      In the Runtime there is no native automatic deployment of bundles. The Framework starts up and that's it. If the AutoInstallPlugin is configured, which it is by default, it will first install the union of bundles configured in the autoInstall and autoStart properties. Additionally, the bundles in autoStart get started.

       

      Part of that procedure the jboss-osgi-hotdeploy bundle gets installed and started. The HotDeploymentService continously monitors changes in the 'deploy' folder and installs/starts/uninstalls bundles appropriately.

       

      A side effect of this procedure is, that those bundles start in a defined order.

       

      ----------------------------

       

      In JBossAS, there already is a MainDeployer service that takes care of deployments in the deploy folder. So there cant be a competing service that tries the same. Also, the AutoInstallPlugin cannot install bundles that are located in the deploy folder, because those would then get installed twice.

       

      A possible solution would be to auto install bundles from a different location (i.e. osgi.deployers/conf/some-bundle.jar) However, care must be taken that the content of those jars does not end up in the CL domain of the OSGi service itself.

       

      ---------------------------- 

       

      I mention this, because it might be usefull to get the AutoInstallPlugin running in AS for initial bundle provisioning first before looking at the added functionality of start level configuration.

        • 1. Re: AutoInstall semantics in supported target containers
          bosschaert

          Rather than taking a parallel approach in AS with the AutoInstallPlugin performing somewhat the same thing that the MainDeployer is doing would it not make sense to try and extend the MainDeployer so that it can also deploy OSGi bundles? Or does it support those already? Ultimately we'd might want to get to a stage where each AS deployment is internally represented as an OSGi bundle so at that point in time the MainDeployer must already either directly or indirectly be deploying OSGi bundles.

           

          I guess the question then becomes: how do we give the MainDeployer optional additional metadata for certain bundles. Currently the additional OSGi metadata would be whether or not a bundle is started or only installed and what its start level is, if the default isn't used.

           

          One possibility is to introduce companion files to go along with the deployables that describe this metadata. You might be deploying simple-bundle.jar. If you want to provide additional metadata with that file you could make sure that there is an additional simple-bundle.jar.md file in the deploy directory. That file then contains configuration such as the start level.

           

          There are two constraints with this approach. First you need to make sure that the .md file is there before the actual deployment. Second all of the configuration in the .md file needs to be optional. It should not be required that this file is present for every deployed artefact.

           

          One thought that might come up is whether this start level information shouldn't be in bundle manifest. My experience is that it can't be there. Start levels are not very typical in OSGi bundle use. However they can be very helpful when deploying libraries that weren't written with OSGi in mind. In the CXF-DOSGi project we used many libraries like that and some of these required a certain ordering to work. The start level service proved very helpful here.

          In OSGi-native deployments dynamic dependency management should really be done through the OSGi Service Registry, but for libraries that are not aware of OSGi this is not an option...

          Bottom line is that the start level information changes from deployment to deployment - you don't want to have to crack open the manifest very every use of a certain bundle.

          • 2. Re: AutoInstall semantics in supported target containers
            alesj

            With default MC/AS deployments they are deployed/installed as far as they can go.

            e.g. no dependency is holding them back or/and their required stage is INSTALLED

             

            What you call start level is in VDF called required stage.

            One way of explicitly setting this up is through existing DeploymentMetaData / jboss-deployment.xml.

             

            Or am I missing something here?

            • 3. Re: AutoInstall semantics in supported target containers
              bosschaert

              So there are two things that you want to be able to control without having to modify the deployable (bundle) itself.

              * Whether it is started or just installed

              * If it is started, what start level it will have (if different from the default)

              Reason that you don't want to put this in the bundle is that depending on your system this information may vary, especially the start level. You don't want to have to recreate a bundle depending on its target context.

               

              Is it possible to provide the jboss-deployment.xml alongside the deployables (i.e. not inside)? This what I was getting at with the idea of companion files...

              • 4. Re: AutoInstall semantics in supported target containers
                alesj
                Is it possible to provide the jboss-deployment.xml alongside the deployables (i.e. not inside)?

                You could simply have a deployer which would based on some criteria add DeploymentMD into bundle's attachments in very early stage.

                • 5. Re: AutoInstall semantics in supported target containers
                  thomas.diesler

                  In AS a bundle (like any other deployment) gets installed by virtue of being in the deploy folder. Bundle.start() is currently handled by a "hack" that wraps the Deployers. Have a look at https://jira.jboss.org/browse/JBOSGI-340 for details on that.

                   

                  The StartLevel service should get the bundle's start level from the OSGiMetaData, which is the aggregated view of all metadata that pertains to a bundle. This is the consuming part of the start level metadata and should be discussed seperately from the providing part of that metadata.

                   

                  The providing part (i.e. where does the start level definition for a particular bundle come from) can be addressed in a variety of ways. A central configuration bean is possible, so is a jboss specific attribute in the manifest, so is a @StartLevel annotation, a seperate XML descriptor, etc ... The difference between these different sources of metadata is marginally and may be applicable in certain use cases and not in others. All of them feed that piece of start level metadata into OSGiMetaData. The important bit is that the StartLevel service does not care where that metadata comes from.

                   

                  A flexible solution could be to use a deployer in the DESCRIBE phase that consumes OSGiMetaData (i.e. addInput(OSGiMetadata.class)), reads the start level config for a bundle from somewhere and sets the property on OSGiMetadata accordingly. If multiple sources of that metadata are supported we could have multiple deployers that all extend OSGiStartLevelMetaDataDeployer each of them dealing with a different source.

                  • 6. Re: AutoInstall semantics in supported target containers
                    bosschaert

                    So in the deploy directory I would like to have something like mybundle.jar  and mybundle.md where the latter is a file containing the startlevel metadata for  that bundle. So I guess I need to write a deployer for those *.md files? Or can I pick them up in some other way in the deployer that already handles the jar files and process them both in one go... The .md file would always be beside the .jar file...

                     

                    If I need to create two separate deployers I guess I need to create some shared location where each can leave their information, or does the deployment framework already provide a means to share information across deployers?

                    Alternatively the .md deployer would need to get hold of the bundle deployer and provide it with the metadata found. Is there a way to do this?

                    • 7. Re: AutoInstall semantics in supported target containers
                      alesj

                      Each file would be / is a separate deployment, hence it would be processed separately.

                      But since we do deployment per stage per deployment, if both were present at some "atomic" time,

                      you could hack things so one would see other's deployment unit.

                      See MainDeployerStructure for more useful methods. ;-)

                       

                      I can hack you some pseudo code or you can do it yourself to learn a few tricks. :-)

                      Let me know which one you prefer.

                      • 8. Re: AutoInstall semantics in supported target containers
                        alesj
                        hack things so one

                        things == one == new deployer ... the important detail :-)

                         

                        Yes, you would need a new .md file deployer,

                        which would try to find jar's deployment unit, and attach it the required metadata / info.

                        • 9. Re: AutoInstall semantics in supported target containers
                          bosschaert

                          Ales Justin wrote:

                           

                          I can hack you some pseudo code or you can do it yourself to learn a few tricks. :-)

                          Let me know which one you prefer.

                          I had a few ideas last night which I like to try so yes - I'll give it a go! Thanks for the clarifications!

                          • 10. Re: AutoInstall semantics in supported target containers
                            bosschaert

                            Ok - here's the beginnings of what I started putting together.

                             

                            I have a new deployer class that can read a metadata file and puts it into an OSGiDeploymentMetaData object:

                            {code}public class OSGiCompanionFileMetaDataDeployer extends AbstractVFSParsingDeployer<OSGiDeploymentMetaData>

                            { ... }{code}

                             

                            For every bundle that gets installed in the system, OSGiBundleManager.installBundle(Deployment dep) gets called, so that seemed like a good place to try and deploy the additional metadata before the bundle itself is deployed:

                             

                            {code}

                            public AbstractBundleState installBundle(Deployment dep) throws BundleException
                               {
                                  // ... snip ...
                                  try
                                  {
                                     // Deploy bundle.jar.md file if it exists in the same location
                                     VirtualFile root = dep.getRoot();
                                     VirtualFile parent = root.getParent();
                                     VirtualFile mdFile = parent.getChild(root.getName() + ".md");
                                     OSGiDeploymentMetaData deploymentMD = null;
                                     if (mdFile != null)
                                     {
                                        org.jboss.deployers.client.spi.Deployment mdDeployment = AbstractDeployment.createDeployment(mdFile);
                                        deployerClient.deploy(mdDeployment);
                                        DeploymentUnit mdUnit = ((MainDeployerStructure)deployerClient).getDeploymentUnit(mdDeployment.getName());
                                        deploymentMD = mdUnit.getAttachment(OSGiDeploymentMetaData.class);
                                     }{code}

                             

                            Then, before I deploy the actual bundle, I attach the deploymentMD to the bundle deployment:

                             

                            {code}

                                     org.jboss.deployers.client.spi.Deployment deployment = AbstractDeployment.createDeployment(dep.getRoot());

                                     MutableAttachments att = (MutableAttachments)deployment.getPredeterminedManagedObjects();

                                     att.addAttachment(Deployment.class, dep);

                                     if (deploymentMD != null)

                                        att.addAttachment(OSGiDeploymentMetaData.class, deploymentMD);

                             

                                     //...

                                     deployerClient.deploy(deployment);{code}

                            When the OSGiBundleState gets constructed I can check whether the OSGiDeploymentMetaData is in the attachments and apply it, e.g. to set the start level:

                             

                            {code}public OSGiBundleState(OSGiBundleManager bundleManager, DeploymentUnit unit)

                               {

                                  super(bundleManager, unit);

                             

                                  StartLevelPlugin slp = bundleManager.getOptionalPlugin(StartLevelPlugin.class);

                                  if (slp != null) {

                                     OSGiDeploymentMetaData dmd = unit.getAttachment(OSGiDeploymentMetaData.class);

                                     if (dmd != null && dmd.hasStartLevel())

                                        startLevel = dmd.getStartLevel();

                                     else

                                        startLevel = slp.getInitialBundleStartLevel();

                                  }

                               }{code}

                             

                            Seems to work

                            Would this be considered a sensible approach? I'm still learning lots of stuff so if there are better ways I'd love to know...

                            • 11. Re: AutoInstall semantics in supported target containers
                              alesj

                              Yuck, what a hack. :-)

                               

                              If the .md file was next to .jar file, it should / will be picked up by main deployer at the same time.

                              So, all you need is two deployers.

                               

                              1st deployer -- simple .md file parsing deployer --> SomeMDMetaData attachment is created

                              2nd deployer -- at POST_PARSE state

                              * picks up this SomeMDMetaData

                              * tries to locate its companion .jar file's DeploymentUnit

                              * if there is such DU and its state is not yet installed, attach new DeploymentMetaData created from SomeMDMD

                               

                              The OSGiBundleManager already has too much stuff in it, not need for such details / hacks.

                              That's why we have deployers. ;-)

                               

                              And what's with all this OSGiX renaming / extending of things.

                              e.g. OSGiDeploymentMetaData, OSGiClassLoaderSystem, OSGiClassLoaderDomain, OSGiClassLoaderFactory ...

                              No wonder nothing generic works, as everything is made custom, w/o any "grand aka service-mix" logic.

                              OK, but this rant is from another story ... just spiked me when seeing the OSGiDMD ...

                              • 12. Re: AutoInstall semantics in supported target containers
                                alesj
                                1st deployer -- simple .md file parsing deployer --> SomeMDMetaData attachment is created

                                2nd deployer -- at POST_PARSE state

                                * picks up this SomeMDMetaData

                                * tries to locate its companion .jar file's DeploymentUnit

                                * if there is such DU and its state is not yet installed, attach new DeploymentMetaData created from SomeMDMD

                                This is what I had in mind

                                 

                                public class MDCompanionDeployer extends AbstractSimpleRealDeployer<MDCompanion>
                                {
                                   public MDCompanionDeployer()
                                   {
                                      super(MDCompanion.class);
                                      setStage(DeploymentStages.POST_PARSE);
                                   }
                                
                                   @Override
                                   public void deploy(DeploymentUnit unit, MDCompanion deployment) throws DeploymentException
                                   {
                                      String jarDeploymentName = deployment.getName();
                                      if (jarDeploymentName != null)
                                      {
                                         MainDeployer md = unit.getMainDeployer();
                                         if (md instanceof MainDeployerStructure)
                                         {
                                            MainDeployerStructure mds = (MainDeployerStructure) md;
                                            DeploymentUnit jarDU = mds.getDeploymentUnit(jarDeploymentName, false);
                                            if (jarDU != null)
                                            {
                                               // TODO -- perhaps check jarDU's state?
                                               DeploymentMetaData dmd = new DeploymentMetaData();
                                               // TODO add MCCompanion stuff
                                               jarDU.addAttachment(DeploymentMetaData.class, dmd);
                                            }
                                         }
                                      }
                                   }
                                }

                                 

                                where metadata class looks like

                                 

                                public class MDCompanion
                                {
                                   private VirtualFile mdFile;
                                
                                   public String getName()
                                   {
                                      VirtualFile parent = mdFile.getParent();
                                      String mdName = mdFile.getName();
                                      String jarName = mdName.substring(0, mdName.lastIndexOf(".")) + ".jar";
                                      VirtualFile jarFile = parent.getChild(jarName);
                                      if (jarFile.exists() == false)
                                         return null;
                                
                                      Deployment deployment = VFSDeploymentFactory.getInstance().createVFSDeployment(jarFile);
                                      return deployment.getName();
                                   }
                                }
                                
                                • 13. Re: AutoInstall semantics in supported target containers
                                  thomas.diesler

                                  We said that OSGiMetaData is the unified view of standard *and* custom (i.e. jboss specific) meta data related to OSGi deployments. Fragmenting the the metadata across multiple objects is not a good idea because if forces clients (i.e. who want to know the SL of a bundle) to be aware of that fragmentation.

                                   

                                  With this "two files" approach - how is the atomicity problem addressed? AFAICS there is no guaranty that the MainDeployer will present both files to the deployers in one cycle. A user might drop both files in the deploy directory, The deployment scanner however kicks in in between.

                                   

                                  Although it might be useful to Support specifying  start levels for bundles deployed in the deploy director in such (or a similar) a way it is not a required feature for the upcommig Beta8 release.

                                   

                                  If we want to claim StartLevel support in AS, these three however should be resolved.

                                   

                                   

                                  In practise, a user could configure the SL in the plugin and have a unified view of all SLs for all bundles. Bundles can be deployed via the console if a SL needs to be specified. Test cases use the eOSGi JMX interface already, which can be used to write SL test cases.

                                   

                                  Later we can add support for SL for hot-deployed bundles when we have a good understanding of how to do that.

                                  • 14. Re: AutoInstall semantics in supported target containers
                                    bosschaert

                                    Thomas Diesler wrote:

                                     

                                     

                                    That bug is really just an alternative way of specifying start levels to what was being discussed in this thread (JBOSGI-345). Which one you like best is just a matter of taste IMHO. I just happen to prefer keeping things localized in one place. If you deploy a bundle (which typically goes through a deploy directory) I would prefer not to edit a global file to specify its start level. I'd rather specify that alongside the deployed bundle - but again there may be use cases for the centralized approach.

                                     

                                    While I agree that finishing the bug you mention might be quicker way to achieve a way to specify this for the user, looking at JBOSGI-345 gives me a chance to look into the deployers which is useful for me in my learning process...

                                    1 2 Previous Next