1 2 Previous Next 23 Replies Latest reply on Jan 21, 2009 10:14 AM by alesj

    JBAS-6061 or weird nested jar

    alesj

      Some explanation about https://jira.jboss.org/jira/browse/JBAS-6061.

      If you follow Luc's instructions and download the jar with the driver,
      you'll see a weird structure inside.

      sqljdbc.jar
      - com
      - META-INF
      - sqljdbc.jar (<-- 0 size jar; the actual cause)
      - sqljdbc.jar.old

      What happens is that when VFS tries to read top sqljdbc.jar it finds inner sqljdbc.jar.
      Currently the policy in JBossAS is to unpack nested jar into temp and again handle them as top level jars.
      But in this case - probably due to 0 size - JDK's Zip handling fails while handling this newly unpacked inner sqljdbc.jar,
      propagating the error all the way to the top - classloading part.

      I don't see how we could exclude such failures,
      unless we suppress all errors during inner jars unpack phase,
      which is not the way to go imo.

        • 1. Re: JBAS-6061 or weird nested jar

           

          "alesj" wrote:

          What happens is that when VFS tries to read top sqljdbc.jar it finds inner sqljdbc.jar.
          Currently the policy in JBossAS is to unpack nested jar into temp and again handle them as top level jars.
          But in this case - probably due to 0 size - JDK's Zip handling fails while handling this newly unpacked inner sqljdbc.jar,
          propagating the error all the way to the top - classloading part.


          It's only referenced by the VFS because the JarStructure is adding it as a
          subdeployment. You could add an option to JarStructure that says something
          like:
          <property name="ignoreBrokenJars">true</property>
          


          Then in jar structure
           else if (JarUtils.isArchive(file.getName()))
           {
          + if (ignoreBrokenJars && validateZipFile(file) == false)
          + {
          + log.warn("Ignoring broken archive: " + file);
          + return false;
          + }
          
           if (trace)
           log.trace("... ok - its an archive or at least pretending to be.");
           }
          


          where validateZipFile() checks to see whether the x.jar or y.zip is really
          an archive.

          It might be an idea to do the validateZipFile() anyway since we'll then have
          "fail early" behaviour when it looks at the structure of the deployment
          if one of the jars is broken.

          • 2. Re: JBAS-6061 or weird nested jar
            alesj

             

            "adrian@jboss.org" wrote:

            It's only referenced by the VFS because the JarStructure is adding it as a
            subdeployment.

            No, that's not the case here. ;-)

            If you look at the actual JIRA issue,
            you'll see that Luc has added it to server/some-as-config/lib
            - as that's how/why conf/jboss-service.xml is able to see it in its classpath -
            meaning it's referenced via this piece of xml (from jboss-service.xml):
            <server>
            
             <!-- Load all jars from the JBOSS_DIST/server/<config>/lib directory. This
             can be restricted to specific jars by specifying them in the archives
             attribute.
             -->
             <classpath codebase="${jboss.server.lib.url:lib}" archives="*"/>
            

            Being added to conf/jboss-service.xml as part of its classpath in SARDeployer::processXMLClasspath.

            Hence the actual jar is checked/read in some CL impl details,
            no way to add some flag and ignore it if it's broken.

            • 3. Re: JBAS-6061 or weird nested jar

               

              "alesj" wrote:
              "adrian@jboss.org" wrote:

              It's only referenced by the VFS because the JarStructure is adding it as a
              subdeployment.

              No, that's not the case here. ;-)

              If you look at the actual JIRA issue,
              you'll see that Luc has added it to server/some-as-config/lib
              - as that's how/why conf/jboss-service.xml is able to see it in its classpath -
              meaning it's referenced via this piece of xml (from jboss-service.xml):
              <server>
              
               <!-- Load all jars from the JBOSS_DIST/server/<config>/lib directory. This
               can be restricted to specific jars by specifying them in the archives
               attribute.
               -->
               <classpath codebase="${jboss.server.lib.url:lib}" archives="*"/>
              

              Being added to conf/jboss-service.xml as part of its classpath in SARDeployer::processXMLClasspath.

              Hence the actual jar is checked/read in some CL impl details,
              no way to add some flag and ignore it if it's broken.


              Well in that case, a more generic solution would be to write a deployer
              that runs in PostParse.

              It would then check that all the VFS files in getClassPath()
              are valid and remove those that aren't. This would of course be an
              optional deployer (not configured by default) for backwards compatiblity purposes.

              Looking at the old MainDeployer, it ignored everything:

               private void makeLocalCopy(DeploymentInfo sdi)
               {
               try
               {
               if (sdi.url.getProtocol().equals("file") && (!copyFiles || sdi.isDirectory))
               {
               // If local copies have been disabled, do nothing
               sdi.localUrl = sdi.url;
               return;
               }
               // Are we already in the localCopyDir?
               else if (inLocalCopyDir(sdi.url))
               {
               sdi.localUrl = sdi.url;
               return;
               }
               else
               {
               String shortName = sdi.shortName;
               File localFile = File.createTempFile("tmp", shortName, tempDir);
               sdi.localUrl = localFile.toURL();
               copy(sdi.url, localFile);
               }
               }
               catch (Exception e)
               {
              
              // HERE ignore problem and log an error
              
               log.error("Could not make local copy for " + sdi.url, e);
               }
               }
              


              • 4. Re: JBAS-6061 or weird nested jar

                 

                "adrian@jboss.org" wrote:

                Well in that case, a more generic solution would be to write a deployer
                that runs in PostParse.


                Or maybe Describe if we aren't going to trust other deployers that add things
                to the classpath? :-)

                • 5. Re: JBAS-6061 or weird nested jar
                  alesj

                   

                  "adrian@jboss.org" wrote:

                  Well in that case, a more generic solution would be to write a deployer
                  that runs in PostParse.

                  It would then check that all the VFS files in getClassPath()
                  are valid and remove those that aren't. This would of course be an
                  optional deployer (not configured by default) for backwards compatiblity purposes.

                  Makes sense.

                  But there's a catch.
                  This behavior is only triggered when somebody tries to read something from that jar - in this case load some class.
                  Which starts zip entries read, stumbling upon that broken jar.

                  How would you know what read/lookup to trigger in this deployer?
                  Or how would you actually check if all entries - e.g. some similar weird double nested crap - is all legit?

                  Perhaps we should add another (hard keeping track of all) flag to VFS,
                  that would also ignore everything - as did the old code.

                  • 6. Re: JBAS-6061 or weird nested jar

                     

                    "alesj" wrote:

                    Perhaps we should add another (hard keeping track of all) flag to VFS,
                    that would also ignore everything - as did the old code.


                    A single flag in VFS would make it hard to control. i.e. it might be ok
                    for the deployers to ignore the error in the classpath but for other uses it
                    would not be ok.

                    Also the VFS would have to cache/remember the brokeness since there is no
                    way to "weed out" the problem during startup. Otherwise it would continually
                    try to copy the jar and get the same error.

                    I personally don't think it is even ok for the deployers/classloader to ignore the error
                    and the example Luc provides is correctly rejected.
                    i.e. the real solution is to fix the deployment

                    The reason I think it needs addressing is because we never really used to
                    look at nested deployments in server/default/lib (i.e. those specifed on
                    a -service.xml classpath).

                    The reason we are doing so now is because the PackageVisitor is triggering
                    the VFS to unpack the archive when it is scanning for the packages in the jars.

                    Maybe a simpler (but less inclusive fix) would be to have the PackageVisitor
                    ignore archives in its VFS visit()?

                    e.g.
                     public boolean accepts(VirtualFile file)
                     {
                     // This is our current root
                     if (file.equals(root))
                     return true;
                    
                    + // Don't go into nested archives
                    + if (file.isArchive())
                    + return false;
                    
                     // Some other root, it will be handled later
                     for (VirtualFile other : roots)
                     {
                     if (file.equals(other))
                     return false;
                     }
                     // Is this an excluded roots?
                     if (excludedRoots != null)
                     {
                     for (VirtualFile other : excludedRoots)
                     {
                     if (file.equals(other))
                     return false;
                     }
                     }
                    
                     // Ok
                     return true;
                     }
                    


                    But I'm not sure if that is too late in terms of the archive getting unpacked?

                    It would certainly be an optimization in terms of the scanning anyway. ;-)

                    • 7. Re: JBAS-6061 or weird nested jar
                      alesj

                       

                      "adrian@jboss.org" wrote:

                      Maybe a simpler (but less inclusive fix) would be to have the PackageVisitor ignore archives in its VFS visit()?

                      But I'm not sure if that is too late in terms of the archive getting unpacked?

                      Yeah, I thought of that too.
                      It would solve some of the problems - depending on how lazy is the VirtualFileHandler initialization.
                      But not in this case - at least not in the way how it's currently implemented.
                      Although it doesn't read/init the broken zip, it will break, since it already fails to create zip file from where it would eventually lazily read entries.
                       private ZipFile ensureZipFile() throws IOException
                       {
                       if (zipFile == null)
                       {
                       zipFile = new ZipFile(file);
                      


                      • 8. Re: JBAS-6061 or weird nested jar
                        maxandersen

                        Just my 2 cents for this.

                        I find it scary that AS 5 will be so fragile to real life jars, but I do understand that in some cases it might be caused by actual user packaging errors.

                        Why not why not have an an "ignoreBrokenArchives" option for all VFS that is disabled by default and then you fail with an exception saying..."oops - your xyz.jar is broken because of Blah. Set 'ignoreBrokenJar=true' in bla blah to ignore these"

                        Then when users set that you just print warning for it instead of failing.

                        This would allow for a hard fail, but at least users would be able to use the probably second most used database in the java world.

                        • 9. Re: JBAS-6061 or weird nested jar
                          rzumwalt

                          I was seeing this same problem as Luc in CR2, but the original fix for this issue only partially resolved my problem. Our archive file actually includes sqljdbc.jar in it's WEB-INF/lib directory. We do this because we ship an EAR file out to customers who then install it on one of the app servers we support (JBoss being one of them). If I remove the JAR from our EAR and place it in the server's lib directory, then everything works just fine. I'd prefer to be able to deploy our EAR file with the jar inside of it since it reduces the number of steps to deploy our app.

                          the stack trace I see is below:

                          07:17:40,080 ERROR [AbstractKernelController] Error installing to ClassLoader: name=vfszip:/E:/jboss-5.0.0.GA/server/default/deploy/pr-war.war state=Describe mode=Manual requiredState=ClassLoader
                          org.jboss.deployers.spi.DeploymentException: Error creating classloader for vfszip:/E:/jboss-5.0.0.GA/server/default/deploy/pr-war.war
                           at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(DeploymentException.java:49)
                           at org.jboss.deployers.structure.spi.helpers.AbstractDeploymentContext.createClassLoader(AbstractDeploymentContext.java:576)
                           at org.jboss.deployers.structure.spi.helpers.AbstractDeploymentUnit.createClassLoader(AbstractDeploymentUnit.java:159)
                           at org.jboss.deployers.spi.deployer.helpers.AbstractClassLoaderDeployer.deploy(AbstractClassLoaderDeployer.java:53)
                           at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
                           at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
                           at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
                           at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
                           at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                           at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1598)
                           at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                           at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1062)
                           at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                           at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                           at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                           at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
                           at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:545)
                           at org.jboss.system.server.profileservice.ProfileServiceBootstrap.loadProfile(ProfileServiceBootstrap.java:304)
                           at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:205)
                           at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:405)
                           at org.jboss.Main.boot(Main.java:209)
                           at org.jboss.Main$1.run(Main.java:547)
                           at java.lang.Thread.run(Thread.java:595)
                          Caused by: java.lang.Error: Error visiting DelegatingHandler@27305459[path=pr-war.war/WEB-INF/lib/sqljdbc-1.0.jar context=file:/E:/jboss-5.0.0.GA/server/default/deploy/ real=file:/E:/jboss-5.0.0.GA/server/default/deploy/pr-war.war/WEB-INF/lib/sqljdbc-1.0.jar]
                           at org.jboss.classloading.plugins.vfs.PackageVisitor.determineAllPackages(PackageVisitor.java:98)
                           at org.jboss.deployers.vfs.plugins.classloader.VFSDeploymentClassLoaderPolicyModule.determineCapabilities(VFSDeploymentClassLoaderPolicyModule.java:108)
                           at org.jboss.classloading.spi.dependency.Module.getCapabilities(Module.java:485)
                           at org.jboss.classloading.spi.dependency.Module.determinePackageNames(Module.java:544)
                           at org.jboss.classloading.spi.dependency.Module.getPackageNames(Module.java:529)
                           at org.jboss.deployers.vfs.plugins.classloader.VFSDeploymentClassLoaderPolicyModule.determinePolicy(VFSDeploymentClassLoaderPolicyModule.java:129)
                           at org.jboss.deployers.vfs.plugins.classloader.VFSDeploymentClassLoaderPolicyModule.determinePolicy(VFSDeploymentClassLoaderPolicyModule.java:48)
                           at org.jboss.classloading.spi.dependency.policy.ClassLoaderPolicyModule.getPolicy(ClassLoaderPolicyModule.java:149)
                           at org.jboss.deployers.vfs.plugins.classloader.VFSDeploymentClassLoaderPolicyModule.getPolicy(VFSDeploymentClassLoaderPolicyModule.java:122)
                           at org.jboss.deployers.vfs.plugins.classloader.VFSDeploymentClassLoaderPolicyModule.getPolicy(VFSDeploymentClassLoaderPolicyModule.java:48)
                           at org.jboss.classloading.spi.dependency.policy.ClassLoaderPolicyModule.registerClassLoaderPolicy(ClassLoaderPolicyModule.java:88)
                           at org.jboss.deployers.plugins.classloading.AbstractLevelClassLoaderSystemDeployer.createClassLoader(AbstractLevelClassLoaderSystemDeployer.java:120)
                           at org.jboss.deployers.structure.spi.helpers.AbstractDeploymentContext.createClassLoader(AbstractDeploymentContext.java:562)
                           ... 21 more
                          Caused by: java.lang.RuntimeException: Failed to read zip file: org.jboss.virtual.plugins.context.zip.ZipFileWrapper@25211b - E:\jboss-5.0.0.GA\server\default\tmp\vfs-nested.tmp\e7d09837_sqljdbc.jar
                           at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:563)
                           at org.jboss.virtual.plugins.context.zip.ZipEntryContext.checkIfModified(ZipEntryContext.java:693)
                           at org.jboss.virtual.plugins.context.zip.ZipEntryContext.getChildren(ZipEntryContext.java:765)
                           at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.getChildren(ZipEntryHandler.java:149)
                           at org.jboss.virtual.plugins.context.DelegatingHandler.getChildren(DelegatingHandler.java:120)
                           at org.jboss.virtual.plugins.context.AbstractVFSContext.getChildren(AbstractVFSContext.java:171)
                           at org.jboss.virtual.plugins.context.AbstractVFSContext.visit(AbstractVFSContext.java:288)
                           at org.jboss.virtual.plugins.context.AbstractVFSContext.visit(AbstractVFSContext.java:258)
                           at org.jboss.virtual.VFS.visit(VFS.java:404)
                           at org.jboss.virtual.VirtualFile.visit(VirtualFile.java:407)
                           at org.jboss.virtual.VirtualFile.getChildren(VirtualFile.java:356)
                           at org.jboss.virtual.VirtualFile.getChildren(VirtualFile.java:337)
                           at org.jboss.classloading.plugins.vfs.PackageVisitor.visit(PackageVisitor.java:200)
                           at org.jboss.virtual.plugins.vfs.helpers.WrappingVirtualFileHandlerVisitor.visit(WrappingVirtualFileHandlerVisitor.java:62)
                           at org.jboss.virtual.plugins.context.AbstractVFSContext.visit(AbstractVFSContext.java:313)
                           at org.jboss.virtual.plugins.context.AbstractVFSContext.visit(AbstractVFSContext.java:258)
                           at org.jboss.virtual.VFS.visit(VFS.java:404)
                           at org.jboss.virtual.VirtualFile.visit(VirtualFile.java:407)
                           at org.jboss.classloading.plugins.vfs.PackageVisitor.determineAllPackages(PackageVisitor.java:94)
                           ... 33 more
                          Caused by: java.util.zip.ZipException: error in opening zip file
                           at java.util.zip.ZipFile.open(Native Method)
                           at java.util.zip.ZipFile.<init>(ZipFile.java:203)
                           at java.util.zip.ZipFile.<init>(ZipFile.java:234)
                           at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.ensureZipFile(ZipFileWrapper.java:175)
                           at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.acquire(ZipFileWrapper.java:245)
                           at org.jboss.virtual.plugins.context.zip.ZipEntryContext.initEntries(ZipEntryContext.java:461)
                           at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:554)
                           ... 51 more
                          


                          I assumed from the comments for the fix that the intent was to be able to specify additional exception handlers for problematic jars in conf/bootstrap/vfs.xml when needed. I tried adding an additional exception handler (that used the server's tmp/vfs-nested.tmp/ directory as the key since that is where the JAR is read from when deploying) to the vfs.xml file but that did not work. Perhaps I did something wrong, here was the entry I added to vfs.xml

                          <entry>
                           <key>${jboss.server.home.url}tmp/vfs-nested.tmp/</key>
                           <value><inject bean="VfsNamesExceptionHandler"/></value>
                          </entry>
                          


                          • 10. Re: JBAS-6061 or weird nested jar
                            alesj

                             

                            "rzumwalt" wrote:
                            Perhaps I did something wrong, here was the entry I added to vfs.xml

                            <entry>
                             <key>${jboss.server.home.url}tmp/vfs-nested.tmp/</key>
                             <value><inject bean="VfsNamesExceptionHandler"/></value>
                            </entry>
                            

                            You should actually pre-cache (and watch for exceptions) ${jboss.server.home.url}deploy directory.
                            The tmp directory is impl detail - only kicking in if we do a copy vfs handling.


                            • 11. Re: JBAS-6061 or weird nested jar
                              rzumwalt

                              That did the trick, Ales, thanks a lot. I had specified the tmp directory because that was where the exception originated from. The deploy directory makes much more sense and seems so obvious now... thanks again!

                              • 12. Re: JBAS-6061 or weird nested jar
                                alesj

                                NP. ;-)

                                I'll probably add this as a default for next release.

                                • 13. Re: JBAS-6061 or weird nested jar
                                  jaikiran

                                  One of the users is seeing this error on JBoss-5.0 GA http://www.jboss.com/index.html?module=bb&op=viewtopic&t=148740

                                  • 14. Re: JBAS-6061 or weird nested jar
                                    alesj

                                     

                                    "jaikiran" wrote:
                                    One of the users is seeing this error on JBoss-5.0 GA http://www.jboss.com/index.html?module=bb&op=viewtopic&t=148740

                                    I don't see what's so different from having sqljdbc.jar in common/lib vs. server/(some-config)/lib.
                                    Both roots are 'covered' with the same exception handler.
                                    Perhaps it's not getting passed from root to more exact handler ...

                                    1 2 Previous Next