-
1. Re: How to remove all the duplicate results in Hibernate Search with Infinispan?
rvansa Jun 16, 2014 6:33 AM (in response to abey4315266)1 of 1 people found this helpfulI can't answer your question positively (it really seems it's not available), but are you sure you want to get all those object in one huge list? It's not scaling, and you're asking for OOME. Have you considered using iterator with lazy fetch options instead?
But anyway, if you think you'll always be able to hold all results in-memory, you can just use HashSet and ignore the duplicities, can't you?
-
2. Re: How to remove all the duplicate results in Hibernate Search with Infinispan?
sannegrinovero Jun 16, 2014 10:03 AM (in response to abey4315266)Hi Stephan,
no there is no function like "DISTINCT" as it is not a relational database (The Hibernate Criteria API need a database, in this case neither Infinispan nor a Lucene index includes de-duplication functionality).
In your case I would have the query sorted on your "id" field, so you can iterate on the list to skip results in a more efficient way: you'd only need to verify if the next result is the same as the previous one. At least that's more efficient than using the contains method.
In a future version of Infinispan we hope to provide an integration of Map/Reduce on query results, so you wouldn't have to fetch the results first but will be able to run a "de-duplication job" on the results before fetching them.
-
3. Re: How to remove all the duplicate results in Hibernate Search with Infinispan?
abey4315266 Jun 17, 2014 3:07 AM (in response to abey4315266)Thanks for the reponses. They are really helpful.