-
1. Re: Loading large amount of entries in cache
nikolay1981 Oct 31, 2011 9:44 AM (in response to pcraveiro)Use a cache loader to restore cache state from a disk between tests.
-
2. Re: Loading large amount of entries in cache
pcraveiro Oct 31, 2011 12:26 PM (in response to nikolay1981)Thanks for your reply Nikolay ....
I'm already using a cache loader (FS) to store and restore the state. The problem I'm facing is because of the store and the index creation/update (i think)
Just for information, from friday until now only 2.500.000 entries were loaded on the cache. I'm trying different JVM configurations to see if something gets better. I really would like to know if there is some ISPN case with the load described before. Must of the examples and test cases works only with 10K or 100K ...
Thanks in advance.
-
3. Re: Loading large amount of entries in cache
sannegrinovero Nov 1, 2011 6:36 AM (in response to pcraveiro)Hi Pedro,
insertion/update performance when indexing is enabled is very sensitive to proper tuning. There are many advanced propierties you can experiment with to improve performance, and all are mentioned in the reference documentation of Hibernate Search.
I'd suggest to start with exclusive_index_use, this should be set to true (it is by default in Infinispan 5.1 but I don't know which vesion you're running).
Also make sure to dedicate some memory for the index writing queue, set something around 128MB
ram_buffer_size
as a good starting point for further tuning. -
4. Re: Loading large amount of entries in cache
pcraveiro Nov 2, 2011 6:22 PM (in response to sannegrinovero)Sanne,
Thanks for your reply. I'm using ISPN 5.0.0.FINAL.
I tried your suggestions but seems that ISPN is ignoring them, maybe something I'm doing wrong. Here is a snippet from my configuration file:
<indexing enabled="true" indexLocalOnly="false">
<properties>
<property name="hibernate.search.default.exclusive_index_use" value="true" />
<property name="hibernate.search.default.indexwriter.batch.ram_buffer_size" value="128MB" />
<property name="hibernate.search.default.indexBase" value="/var/index/${ispn.cluster.nodeName}/ISPNLuceneLocal" />
<property name="hibernate.search.default.directory_provider" value="filesystem" />
</properties>
</indexing>
As you can see I also configured the hibernate.search.default.indexBase, but ISPN is creating the index in a different location ...
There are some other way to configure the Hibernate Search ?
Thanks.
-
5. Re: Loading large amount of entries in cache
sannegrinovero Nov 2, 2011 7:32 AM (in response to pcraveiro)Hi Pedro,
you're being affected by ISPN-1320 ; I'm sorry for that issue to be still unsolved, there is a patch coming which rewrites all che configuration parsing so I'm not supposed to change that code; should be fixed by next week.
Could you run your experiments by configuring the default cache instead of using a named cache?
-
6. Re: Loading large amount of entries in cache
pcraveiro Nov 3, 2011 4:17 PM (in response to sannegrinovero)Thanks again Sanne !
I tried your sugestion and ISPN now recognize the properties defined in the configuration file.
Now I'm trying to find the optimal configuration using the Hibernate Search properties.
Cheers,
Pedro Igor
-
7. Re: Loading large amount of entries in cache
sannegrinovero Nov 3, 2011 4:25 PM (in response to pcraveiro)nice. please post back your findings (when you have) as they might be useful to others.
-
8. Re: Loading large amount of entries in cache
pcraveiro Nov 3, 2011 5:02 PM (in response to sannegrinovero)Ok. I will post the results as soon as I finish.