-
1. Re: 3 nodes problem
galder.zamarreno Sep 11, 2009 10:35 AM (in response to mcmati)How are these machines communicating? Via udp or tcp?
If using udp, can you switch to tcp and see if you have the same issues?
The easiest way to switch to tcp is to use a transport config like this:<transport clusterName="infinispan-cluster"> <properties> <property name="configurationFile" value="tcp.xml"/> </properties> </transport>
And then pass the initial hosts to each node via system property, exampple:-Djgroups.tcpping.initial_hosts=host1[7800],host2[7800],host3[7800]
Are the 2 nodes that work fine started in the same machine or different machines?
And the 3rd node? Is it a different machine? -
2. Re: 3 nodes problem
mcmati Sep 14, 2009 4:29 AM (in response to mcmati)Those machines communicate via default protocol (I believe it is udp).
When I changed that to tcp there was no improvement.
It doesn't matter if those two nodes that work fine are on the same machine or two different machines - it works ok. Problems start when I add more nodes, and here again it doesn't matter if those new nodes are created on one of those 2 machines that are already running or if I run it on 3rd machine. Anyway, after I add 3rd node, my cluster works a lot worse.
It is weird, because when I have 3 nodes or more, some of them still work fine, but some don't, and I can't find any dependencies which work better and which worse. It just seems to be quite random. -
3. Re: 3 nodes problem
galder.zamarreno Sep 15, 2009 2:36 AM (in response to mcmati)Could you post your full configuration file?
What's the Infinispan version you're using? Beta1?
I'd suggest enabling TRACE logging (see http://www.jboss.org/community/wiki/infinispantechnicalfaqs#How_can_I_enable_logging) in each of the 3 nodes and try to replicate the issue where you get those timeouts. -
4. Re: 3 nodes problem
mcmati Sep 15, 2009 2:48 AM (in response to mcmati)well I've been trying lots of configuration files, but as for now I'm using this one:
<?xml version="1.0" encoding="UTF-8"?> <infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:infinispan:config:4.0"> <global> <transport clusterName="infinispan-cluster"> <properties> <property name="configurationFile" value="udp.xml"/> </properties> </transport> </global> <default> <locking isolationLevel="REPEATABLE_READ" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500"/> <!-- Used to register a transaction manager and participate in ongoing transactions. --> <transaction transactionManagerLookupClass="org.infinispan.transaction.lookup.GenericTransactionManagerLookup" syncRollbackPhase="false" syncCommitPhase="false" useEagerLocking="true"/> <clustering mode="distribution"> <l1 enabled="true" lifespan="10000"/> <hash numOwners="2" rehashWait="12000" rehashRpcTimeout="60000"/> <sync replTimeout="200000"/> </clustering> </default> </infinispan>
I know that here I use UDP, but I also tried TCP. And there were versions without transaction and locking part.
So that's it. Now I'm going to enable trace logging and we'll see ;)
Best regards,
Martin -
5. Re: 3 nodes problem
mcmati Sep 15, 2009 2:51 AM (in response to mcmati)I forgot. I'm using Beta1. Sorry for posting two replyes in a row, but I couldn't find "edit" button.
-
6. Re: 3 nodes problem
galder.zamarreno Sep 15, 2009 12:47 PM (in response to mcmati)Hi, I don't think you should be using useEagerLocking unless you have specific case for it, i.e. cache a shared counter that can be modified by different nodes at the same time.
Please make sure that apart from noting anything specific of the log files, you actually send me the full TRACE log files to my email address which is my username. -
7. Re: 3 nodes problem
galder.zamarreno Sep 24, 2009 12:19 PM (in response to mcmati)mcmati, solution to the issue you're encountering with adding big maps to the cache is being discussed in http://lists.jboss.org/pipermail/infinispan-dev/2009-September/001457.html