1 2 3 Previous Next 35 Replies Latest reply on Jan 16, 2008 8:44 AM by Ales Justin

    Embbedded - broken - some fixes - still not working

    Adrian Brock Master

      I tried to get the ide test working for embedded inside Eclipse
      to understand how Bill's assembly vfs stuff works.
      (Aside - the public interface for these classes shouldn't be in org.jboss.vfs.plugins :-)

      Problems found so far:
      1) Resources from the ejb3 testsuite were leaking into the embeded project.
      In particular jndi.properties
      2) The ejb3-interceptors-aop.xml is out-of-date
      3) The default ejb3 persistence properties was missing
      4) jboss-local-jdbc.rar was missing
      5) The jca stylesheet was in the wrong place
      6) There were many missing jars from the classpath

      I've managed to fix all these (at least as they relate to the VFS test in the IDE)
      although this shows just how stupid this solution is.

      Why do we need to maintain multiple copies of configurations files and implementation
      details all over the place. It's an impossible task to keep them in synchronization.

      Finally, it's still broken, because Hibernate can't find the entities.

      WARN 21-11 15:44:14,717 (?:?:?) -Unable to find file (ignored): vfs://11956562541581/
      java.lang.RuntimeException: Cannot open stream
       at org.jboss.virtual.plugins.context.vfs.AssembledDirectoryHandler.openStream(AssembledDirectoryHandler.java:117)
       at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:216)
       at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:117)
       at java.net.URL.openStream(URL.java:1007)
       at org.hibernate.ejb.packaging.InputStreamZippedJarVisitor.doProcessElements(InputStreamZippedJarVisitor.java:33)
       at org.hibernate.ejb.packaging.JarVisitor.getMatchingEntries(JarVisitor.java:240)
       at org.hibernate.ejb.Ejb3Configuration.addScannedEntries(Ejb3Configuration.java:275)
      


      It doesn't undertand the virtual file system and assumes its a zipped jar
      which it fails to open - obviously.
      This behaviour is not pluggable in hibernate. And the choice of how to scan
      the jars/filesystem is even in a private method.

      A policy decision in a private non-overridable method? :-)

      Finally the test itself uses a jboss image in the main src checkout.
      Which SVN tells me should be checked in! ;-)
      [ejort@warjort embedded]$ svn status | grep \?
      ? src/main/resources/javase/tmp
      ? src/main/resources/javase/data
      


        • 1. Re: Embbedded - broken - some fixes - still not working
          Steve Ebersole Apprentice

          1) What exactly does a vfs:// url represent? Any type of file resource?

          2) Are we (Hibernate) being handed an actual java.net.URL here? If so, is this a vfs-specific URL implementation? In other words could we rely on using the URLConnection/ContentHandler mechanism and have that be handled correctly by the underlying vfs?

          • 2. Re: Embbedded - broken - some fixes - still not working
            Max Rydahl Andersen Master

            1) its a jboss virtual file system. Afaik only accessible through the jboss vfs api i think (would be cool though if it could be mapped behind a file system in java's api ;)

            2) that is what i thought so too and if you look in embedded source code there are some attempts on doing it via HandlerHack's etc. But from what I understood from the code hibernate assumes its either a file or a jar connection and the VFS is neither of these two.....but if jboss can traverse it we should also be able to map it to something that the current traversal code can handle....

            • 3. Re: Embbedded - broken - some fixes - still not working
              Adrian Brock Master

               

              "steve.ebersole@jboss.com" wrote:
              1) What exactly does a vfs:// url represent? Any type of file resource?


              Its an abstract directory structure. The underlying resources are scattered
              around the file system. i.e. no need to package a jar (not even as an
              expanded filesystem that looks like a jar).
              See the VFSTestCase in embedded.


              2) Are we (Hibernate) being handed an actual java.net.URL here? If so, is this a vfs-specific URL implementation? In other words could we rely on using the URLConnection/ContentHandler mechanism and have that be handled correctly by the underlying vfs?


              Yes, but you don't use it. Your logic is something like:
              if (fileURL()) then use java.io.File
              else assume a jar file.

              This is neither of those. You need to define your own simple VFS api
              that we can plug into. In fact, you should really be defining something
              more high level, since all you really want is a list of relevant annotated class URLs
              (the real ones) which we can give you.

              • 4. Re: Embbedded - broken - some fixes - still not working
                Steve Ebersole Apprentice

                 

                "adrian@jboss.org" wrote:

                Yes, but you don't use it. Your logic is something like:
                if (fileURL()) then use java.io.File
                else assume a jar file.

                Yes, I understand that much; I am a dolt, not a *complete* dolt ;)

                The question is whether if we did use that approach the vfs URL would actually do TheRightThing...

                "adrian@jboss.org" wrote:

                This is neither of those. You need to define your own simple VFS api
                that we can plug into. In fact, you should really be defining something
                more high level, since all you really want is a list of relevant annotated class URLs
                (the real ones) which we can give you.

                You can, sure. But what about other app severs/environments... If the URLConnection/ContentHandler approach is more "correct" and the vfs stuff will really handle it correctly, then whats the issue with using that? Seems like a more universally applicable solution. But of course, I'm a dolt (^^) so please enlighten me ;)

                • 5. Re: Embbedded - broken - some fixes - still not working
                  Adrian Brock Master

                   

                  "steve.ebersole@jboss.com" wrote:

                  You can, sure. But what about other app severs/environments... If the URLConnection/ContentHandler approach is more "correct" and the vfs stuff will really handle it correctly, then whats the issue with using that? Seems like a more universally applicable solution. But of course, I'm a dolt (^^) so please enlighten me ;)


                  Where in the URL api does it have a listChildren(), etc.? :-)

                  You have to use a specific api, e.g. java.io.File or java.util.jar.JarFile
                  before you can look at structure.

                  This adhoc processing (usually done wrong in multiple places :-) is one of
                  the reasons we introduced the VFS, so we can do it correctly in one place
                  and extend it.

                  All I'm asking is for a way to plugin where currently hibernate takes a
                  non-overriable decision. If other appservers want to plugin as well
                  then ok. Even something as simple as:

                  package org.hiberante.vfs
                  
                  public interface URLLister
                  {
                   URL[] getChildren(URL parent);
                  }
                  


                  would mean we could do:
                  -Dorg.hibnerate.vfs.URLLister.vfs=org.jboss.hibernate.vfs.VFSURLLister
                  i.e. a system/hibernate property org.hiberate.vfs.URLLister.{protocol}

                  I'm sure you can come up with something better :-)

                  I'd prefer something at a higher level. Mainly because
                  we are already scanning classes for annotations for other reasons so we
                  can optimize your search for those that are marked @Entity or whatever
                  filtering you need.

                  • 6. Re: Embbedded - broken - some fixes - still not working
                    Steve Ebersole Apprentice

                    I guess it comes down to the definition of "correctness" I asked about before. I should have been more specific. Concretely I was thinking of URLConnection.getContentType(), and wondering what would get returned here for the vfs URL impl. But based on your answers I have to assume it is something non-useful like "vfs' or some-such.

                    I have not seen Emmanuel's code for this discovery, so I cannot give the best answer. But I would assume something like this registerable with the Hibernate Configuration. Probably even the discovery itself could get encapsulated into a registerable contract which you could then provide. A contact like:

                    public interface EntityScanner {
                     /**
                     * Note sure of the most appropriate return type here, but
                     * it seems best to stay away from URL on vfs ;)
                     */
                     File[] getAnnotatedEntities(URL root);
                    }
                    


                    • 7. Re: Embbedded - broken - some fixes - still not working
                      Adrian Brock Master

                       

                      "steve.ebersole@jboss.com" wrote:
                      I guess it comes down to the definition of "correctness" I asked about before. I should have been more specific. Concretely I was thinking of URLConnection.getContentType(), and wondering what would get returned here for the vfs URL impl. But based on your answers I have to assume it is something non-useful like "vfs' or some-such.


                      Well it should return the underlying real answer. But that
                      depends upon how good the protocol handler is.
                      In this case, Bill wrote it, so its a NYI :-)


                      I have not seen Emmanuel's code for this discovery, so I cannot give the best answer. But I would assume something like this registerable with the Hibernate Configuration. Probably even the discovery itself could get encapsulated into a registerable contract which you could then provide. A contact like:
                      public interface EntityScanner {
                       /**
                       * Note sure of the most appropriate return type here, but
                       * it seems best to stay away from URL on vfs ;)
                       */
                       File[] getAnnotatedEntities(URL root);
                      }
                      


                      URL[] is better than file. e.g. it could be a jar entry.

                      But like I said before, this doesn't really optimize it. e.g. at least in JBoss
                      we'll already have a cache of the annotations so there would be no
                      need for you to re-read them. e.g. you currently take the stream
                      and parse the byte code (we've already done that looking for other annotations)

                      Anyway, let's keep the two issues seperate. Just being able
                      to tell you what the real class and META-INF/persistence.xml URLs are would be a start. :-)

                      • 8. Re: Embbedded - broken - some fixes - still not working
                        Adrian Brock Master

                         

                        "adrian@jboss.org" wrote:

                        URL[] is better than file. e.g. it could be a jar entry.


                        We'll actually
                        Iterator<URL>
                        will scale better when there are lots of classes.


                        • 9. Re: Embbedded - broken - some fixes - still not working
                          Steve Ebersole Apprentice

                           

                          But like I said before, this doesn't really optimize it. e.g. at least in JBoss
                          we'll already have a cache of the annotations so there would be no
                          need for you to re-read them. e.g. you currently take the stream
                          and parse the byte code (we've already done that looking for other annotations)

                          The idea is that whatever implementation of the EntityScanner contract was registered would be the one "finding" annotated classes. In your case, as you said, that work has already been done, so your impl could be quite optimized ;)


                          Iterator<URL>

                          Personally, I like this:
                          interface EntityProcessor {
                           void processAnnotatedClass(Class clazz);
                           ...
                          }
                          interface EntityScanner {
                           void findAnnotatedEntities(EntityProcessor processor);
                          }
                          

                          But like I said, I'll leave this up to Emmanuel...

                          • 10. Re: Embbedded - broken - some fixes - still not working
                            Emmanuel Bernard Master

                            Hey guys,
                            There are some reasons why I assume either a JAR structure or a directory from these URLs.
                            The contract between the app server and the persistence provider is defined by javax.persistence.spi.PersistenceUnitInfo

                            When the container creates a PU, it calls

                            /**
                             * Called by the container when an EntityManagerFactory
                             * is to be created.
                             *
                             * @param info Metadata for use by the persistence provider
                             * @param map A Map of integration-level properties for use
                             * by the persistence provider (may be null if no properties
                             * are specified).
                             * @return EntityManagerFactory for the persistence unit
                             * specified by the metadata
                             */
                             public EntityManagerFactory createContainerEntityManagerFactory(PersistenceUnitInfo info, Map map);


                            Specifically the interesting bits of the contract are


                            /**
                             * Interface implemented by the container and used by the
                             * persistence provider when creating an EntityManagerFactory.
                             */
                            public interface PersistenceUnitInfo {
                             [...]
                            
                             /**
                             * @return The list of JAR file URLs that the persistence
                             * provider must examine for managed classes of the persistence
                             * unit. Each jar file URL corresponds to a named <jar-file>
                             * element in the persistence.xml file.
                             */
                             public List<URL> getJarFileUrls();
                            
                             /**
                             * @return The URL for the jar file or directory that is the
                             * root of the persistence unit. (If the persistence unit is
                             * rooted in the WEB-INF/classes directory, this will be the
                             * URL of that directory.)
                             */
                             public URL getPersistenceUnitRootUrl();
                            
                             [...]
                            }


                            The reading of the JavaDoc (and the intend of the spec) is that the persistence provider expect either a directory or a JAR file.

                            I scan the root URL for 3 reasons:
                            - get the annotated classes
                            - get hbm.xml files
                            - get ORM.xml files

                            2 is proprietary to Hibernate of course.

                            Even if we define a proprietary abstraction implemented by JBoss AS and passed as an argument to createContainerEntityManagerFactory (in the Map), it does not solves the whole problem.
                            JBoss AS has to conform to the PersistenceUnitInfo contract so that other providers can bootstrap :)

                            • 11. Re: Embbedded - broken - some fixes - still not working
                              Max Rydahl Andersen Master

                              emmanuel - Isn't this way of starting (with PersistenceUnitInfo) different from what happens when hibernate scans for entities "on its own" in j2se scenario ?

                              • 12. Re: Embbedded - broken - some fixes - still not working
                                Adrian Brock Master

                                 

                                "epbernard" wrote:

                                Even if we define a proprietary abstraction implemented by JBoss AS and passed as an argument to createContainerEntityManagerFactory (in the Map), it does not solves the whole problem.
                                JBoss AS has to conform to the PersistenceUnitInfo contract so that other providers can bootstrap :)


                                I'm not saying don't support the spec. I'm saying we can optimize it.

                                There are really two seperate issues:

                                1) The embedded stuff where we don't require a packaging step to run tests,
                                we create a VFS context that collects the disparate resources from across
                                the IDE project. A sort of in memory symbolic link.
                                See the VFSTestCase in the embedded project.

                                2) Optimizing the scanning/parsing of the class files for annotations.
                                Scanning we can trim the list the list to speed things up
                                since we already did the scan and know which are @Entity.
                                And we've already parse those classes that do match.

                                The issue that started this thread was (1).

                                (2) is a nice to have. At the moment we have EJB3, Hibernate, Tomcat, AOP
                                and in the future the MC all needing to scan packages for annotated classes.
                                Doing it N times for N different modules is just a waste.

                                • 13. Re: Embbedded - broken - some fixes - still not working
                                  Max Rydahl Andersen Master

                                  ...but why is this concern of where to get resources from leaked out via the urls passed down to hibernate via the urls it gets ? Can't we avoid that somehow?

                                  • 14. Re: Embbedded - broken - some fixes - still not working
                                    Adrian Brock Master

                                     

                                    "max.andersen@jboss.com" wrote:
                                    ...but why is this concern of where to get resources from leaked out via the urls passed down to hibernate via the urls it gets ? Can't we avoid that somehow?


                                    I don't understand your question? You need some identifier.
                                    JavaEE generally uses URLs (but they are not navigable
                                    there is no generic getChildURLs()).

                                    Currently hibernate gets the top level url and barfs because it only knows
                                    how to navigate file:, jar:
                                    There is no way to make it understand anything else.

                                    We're discussing how to go beyond these for reasons of optimization
                                    and ease of use/development.

                                    If you want to avoid the problem then we say you always have to package
                                    as a jar or exploded archive and we will always scan everything (multiple times :-)


                                    1 2 3 Previous Next