7 Replies Latest reply on Jan 21, 2009 7:07 AM by vyemialyanchyk

    Incorporate Hibernate Search into Hibernate Tools idea

    vyemialyanchyk

      Hi. I have an idea to incorporate Hibernate Search into Hibernate Tools.
      Of cause I'm intresting to get some feedback about it... What do you think?
      There are several propositions for startup:
      1) Hibernate Cosole Cofiguration:
      a) possible to configure (view/edit) "hibernate.search.default.IndexBaase"
      b) possible to view list of entities for the index (@Indexed), possible to view hierarchy of index entities relation (@ContainedIn and @IndexedEmbedded)
      c) add possibility to display objects for some query (seems be usefull for testing)
      2) automatic annotation creation for some entity/entities - at least in the simplest case - @Indexed & @Field, in more complex - @ContainedIn and @IndexedEmbedded, may be some other?
      3) may be there is a sense to create for Eclipse plugins which will incorporate some Luke functionality? - may be not all of it, but functions to index manage, simplest search and some statistics;
      4) as one more idea - it would useful if all of these has some common switch in interface which is not very bind to Lucene;
      5) may be some functions which will help adjust index for better search result and performance - it is similar for clause 3 but not exactly the same;
      may be something other? your opinion?

        • 1. Re: Incorporate Hibernate Search into Hibernate Tools idea
          sannegrinovero

          cool! I would like to use a tool like that.

          2) automatic annotation creation for some entity/entities - at least in the simplest case - @Indexed & @Field

          What do you mean by "automatic"? I don't think it's possible to find a way to guess the correct positions and options for a Field annotation; maybe from the queries but it seems to me quite hard.

          4) as one more idea - it would useful if all of these has some common switch in interface which is not very bind to Lucene;

          you refer to an interface update to the luke tools? what's wrong with being tightly coupled to Lucene for a tool like this?

          b) possible to view list of entities for the index (@Indexed), possible to view hierarchy of index entities relation (@ContainedIn and @IndexedEmbedded)

          I would highly appreciate a view like the Hibernate Tools mapping diagram, showing the resulting index fields and how they are mapped to the entities.

          5) may be some functions which will help adjust index for better search result and performance - it is similar for clause 3 but not exactly the same;

          A practical use case?

          • 2. Re: Incorporate Hibernate Search into Hibernate Tools idea
            vyemialyanchyk

             

            What do you mean by "automatic"?

            here I mean something like "Generate Hibernate/JPA annotations" in Hibernate Tools. Of cause it is rather difficult to define correct places for the Field annotation - here I propose generate it only for Strings fields as a first step.
            you refer to an interface update to the luke tools? what's wrong with being tightly coupled to Lucene for a tool like this?

            that is right. Luke is a good tool exactly for Lucene, Lucene is a good index search tool, but there are several other and if they will get a common switch in common interface, the developers could select the best index search tool for their needs. As I think this will not add too much complexity, but will give a real flexibility like Hibernate has with plenty DBMS.
            A practical use case?

            As I think it be useful for development if there are several indexes for different purposes - it could be a big index for production with milions of documents/items and several small one for different test corpora, it is possible to have situation, when there is a sense to have several indexes for a one project, etc. And it seems useful if be possible to view information about index, optimize index, enter search expresions with content assistance proposals. And use Eclipse/JBoss tools for this as a one environment for development. Is it answer the question or?

            • 3. Re: Incorporate Hibernate Search into Hibernate Tools idea
              maxandersen

               

              "vyemialyanchyk" wrote:
              What do you mean by "automatic"?

              here I mean something like "Generate Hibernate/JPA annotations" in Hibernate Tools. Of cause it is rather difficult to define correct places for the Field annotation - here I propose generate it only for Strings fields as a first step.


              I very deliberately haven't added any automatic code generation that is non-deterministic/dangerous to hibernate tools so let's not start doing that ;)

              You could do something like use meta attributes that hibernate tools support to annotate which things you want to be searched and users can then either use reveng.xml to indicate which fields needs indexing or use a programmatic strategy to compute it.

              So if the generation is pretty simplistic (i.e. nothing fancy) then it could be integrated into reverse engineering/code generation but not sure if its really useful/will be used.


              you refer to an interface update to the luke tools? what's wrong with being tightly coupled to Lucene for a tool like this?

              that is right. Luke is a good tool exactly for Lucene, Lucene is a good index search tool, but there are several other and if they will get a common switch in common interface, the developers could select the best index search tool for their needs. As I think this will not add too much complexity, but will give a real flexibility like Hibernate has with plenty DBMS.


              What other good index search tools are there and why should we generalize them ? Hibernate only support many db's because the db's has a common language (SQL, tables, rows, columns,..). Index search tools doesn't afaik...and again - why not just focus on Lucene ?



              • 4. Re: Incorporate Hibernate Search into Hibernate Tools idea
                vyemialyanchyk

                 

                You could do something like use meta attributes that hibernate tools support to annotate which things you want to be searched and users can then either use reveng.xml to indicate which fields needs indexing or use a programmatic strategy to compute it.

                This is seems be better. I have a smattering knowledge of Hibernate Search.

                So if the generation is pretty simplistic (i.e. nothing fancy) then it could be integrated into reverse engineering/code generation but not sure if its really useful/will be used.

                I think it could be usefull.

                What other good index search tools are there and why should we generalize them ? Hibernate only support many db's because the db's has a common language (SQL, tables, rows, columns,..). Index search tools doesn't afaik...and again - why not just focus on Lucene ?

                This is could be a topic for a long discussion... In a general case - index search tool consist of index (analog of DB) for some documents/items, and query in a common case is just a string (Lucene generalize this, but in common case this is just a string) - no SQL just a string in a common case (of cause it is possible to use some preconditioning). What does mean a good index tool? There are many parameters here as portability, index speed, search speed, maximum number of documents in index, relevancy of results, simplicity to configure, etc. I have to test a lot of index tools cause this is intresting question for me. Lucene is a most famous, it has a god infrastructure and many projects use it, but it is not the best for all parameters which I describe above. I will not admit some other index search tool - cause this is not a main topic here, but I admit that there are others, and possibility to select is a greate posibility for the user, for developer. "focus on Lucene?" why not :)

                • 5. Re: Incorporate Hibernate Search into Hibernate Tools idea
                  sannegrinovero

                   

                  This is could be a topic for a long discussion...

                  Not so long maybe: there are no plans to support anything different than Lucene, for several good reasons, so at the moment the discussion doesn't even need to start in the tooling context.
                  If you would like to discuss future Search development we should switch to the H.Search forum, I'm curious to hear about other tools: all decent solutions I've found are either DB coupled or based on Lucene.
                  Also it should be noted that Lucene is not only very good but also very active and constantly improving: I doubt some other project could catch up to it in the short time, let alone doing better.

                  In a general case - index search tool consist of index (analog of DB) for some documents/items, and query in a common case is just a string (Lucene generalize this, but in common case this is just a string)

                  Lucene doesn't generalize at all, it's Search which makes conversions to and from the String world from the objects world.

                  As I think it be useful for development if there are several indexes for different purposes - it could be a big index for production with milions of documents/items and several small one for different test corpora, it is possible to have situation, when there is a sense to have several indexes for a one project, etc. And it seems useful if be possible to view information about index, optimize index, enter search expresions with content assistance proposals. And use Eclipse/JBoss tools for this as a one environment for development. Is it answer the question or?

                  yes that makes sense, but as "practical example" I was curious about how such a tool could be useful. Optimization can be done automatically in Search, just enable it in configuration or use it programmatically; the big feature would be to be able to look into the index, testing queries, testing analyzers, explaining queries.
                  Would be nice to have content assistance in queries, but that is IMHO not a priority.

                  • 6. Re: Incorporate Hibernate Search into Hibernate Tools idea
                    epbernard

                    I would really truly love a revamping of Luke into a more user friendly interface and integrated in the IDE.
                    Even better, I would like to read in-memory Lucene Directory state when I put a debug point in my program. That would be ideal for unit tests analysis.

                    • 7. Re: Incorporate Hibernate Search into Hibernate Tools idea
                      vyemialyanchyk

                       

                      revamping of Luke into a more user friendly interface and integrated in the IDE

                      For me it is seems rather rough and in the case of input search expression it would be nice to have content assistance.

                      I would like to read in-memory Lucene Directory state when I put a debug point in my program.

                      seems as the intresting feature to implement. You mean a RAM-based directory, isn't it?

                      I'm curious to hear about other tools: all decent solutions I've found are either DB coupled or based on Lucene

                      I told about solutions which is not obligatory open source. There plenty of these. It is possible that some of them will want to become open source. That is my point.