-
1. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Jul 21, 2011 12:48 PM (in response to mookythemook)Oops, I meant to add: Im using 2.2.5 Final
-
2. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Jul 21, 2011 3:21 PM (in response to mookythemook)You probably have Paging mode as blocking. That means you won't be able to produce any more messages until you consume them.
-
3. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Jul 21, 2011 6:05 PM (in response to clebert.suconic)All the messages in the destination had been consumed - it was empty. (or have I misunderstood what you meant?)
-
4. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Jul 21, 2011 6:43 PM (in response to mookythemook)Are you sending transactionally? Maybe your transaction is bigger than the max sizes?
-
5. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Jul 21, 2011 7:49 PM (in response to clebert.suconic)Nope. No transactions.
-
6. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Jul 21, 2011 10:35 PM (in response to mookythemook)Hard to tell what you're doing.
Common mistakes:
- People using core-api, and not setting consumer-batch-size properly
- Big transactions
- Lost subscriptions
-
7. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Jul 22, 2011 11:41 AM (in response to clebert.suconic)Im sending one message at a time, untransacted session. Its to a queue - so no awol durable subscriptions. No messages left on queue anyway.
Is there a chance Im doing nothing wrong - and theres some concurrency bug? Ive never seen this happen before or since...
-
8. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Jul 22, 2011 11:57 AM (in response to mookythemook)I suspect you are doing something wrong.
How you are consuming...
You should post some code. it's hard to guess.
-
9. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Jul 28, 2011 4:16 AM (in response to clebert.suconic)Message consumption is via a MessageListener. Untransacted session, default prefetch, default (ie none) consumer rate throttling.
But ultimately, the queue is empty - so consumer has consumed everything in it.
I'll try post some code when Im back in the office - but its pretty standard JMS code - albeit with a bit of our abstraction which will confuse matters.
-
10. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Jul 28, 2011 12:03 PM (in response to mookythemook)albeit with a bit of our abstraction which will confuse matters.
We need something simple.
Have you checked into Page Mode blocking versus Paging?
default prefetch,
I'm not sure what this means ^^^
-
11. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Aug 8, 2011 9:29 AM (in response to clebert.suconic)Thanks Clebert
>> >> default prefetch
>> I'm not sure what this means ^^^
HornetQConnectionFactory.setConsumerWindowSize(HornetQClient.DEFAULT_CONSUMER_WINDOW_SIZE);
>> We need something simple.
I can try to reduce the code down to just the pure JMS code - theres nothing exciting there - its pretty vanilla jms usage. Will do as soon as I can.
That said, I dont think it will be repeatable - Ive only seen this happen once in much testing & live usage.
>> Have you checked into Page Mode blocking versus Paging?
Destination Queue was empty. Zero messages.
Also, no limits set anywhere (so any limits are default limits - broker and connection factory are all configured in code).
Not sure if its pertinent, but there is a possibility that the free memory on the machine the producers & consumers & broker was running on (all running in the same process in the IDE) might have been low... but GC activity came to a standstill (so it wasnt choking on full GC's)
-
12. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Aug 8, 2011 11:55 AM (in response to mookythemook)I can try to reduce the code down to just the pure JMS code - theres nothing exciting there - its pretty vanilla jms usage. Will do as soon as I can.
You are assuming there's a bug here, when I don't think that's the case.
The idea I asked you some code was to see what you are doing (and try helping you identify your issue).
You are probabably sending it transactioned, not commiting messages and hence just using memory at the server what will block page-blocking.
On my first email I asked you about paging mode and you never answered the question.
-
13. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
mookythemook Aug 8, 2011 1:11 PM (in response to clebert.suconic)Thanks Clebert,
You are assuming there's a bug here, when I don't think that's the case.
I hope Im not making any assumption there is a bug. I always tend to start with the assumption that Im doing something wrong - but I havent been able to work out what Im doing wrong yet...
Given the behaviour has only been observed once, its got to be timing-related (and not something fundamentally dumb that it doesnt work at all)
You are probabably sending it transactioned, not commiting messages and hence just using memory at the server what will block page-blocking.
Definitely no transacted session. Definitely no transactions.
Here is the ConnectionFactory code
Map<String, Object> connectionParams = new HashMap<String, Object>();
connectionParams.put(TransportConstants.PORT_PROP_NAME, 61600);
connectionParams.put(TransportConstants.HOST_PROP_NAME, "localhost");
connectionParams.put(TransportConstants.USE_NIO_PROP_NAME, false);
TransportConfiguration transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class.getName(), connectionParams);
// Step 3 Directly instantiate the JMS ConnectionFactory object using that TransportConfiguration
HornetQConnectionFactory factory = HornetQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.CF, transportConfiguration);
factory.setConsumerWindowSize(HornetQClient.DEFAULT_CONSUMER_WINDOW_SIZE);
factory.setClientFailureCheckPeriod(DEFAULT_CLIENT_FAILURE_CHECK_PERIOD);
factory.setRetryInterval(DEFAULT_RETRY_INTERVAL);
factory.setRetryIntervalMultiplier(1.5);
factory.setMaxRetryInterval(10000);
factory.setReconnectAttempts(-1);
// NAM: A bit of a hack to disable large messages - virus scanner playing havoc with performance
factory.setMinLargeMessageSize(Integer.MAX_VALUE);
connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
I thought I had answered your question about Paging - but I guess I didnt answer it completely.
Im running the embedded JMS server, and therefore not using the xml config. No specific code for address settings, so its using all defaults. The default, from what I can tell is AddressFullMessagePolicy.PAGE.
Here is the code for setting up and starting the broker:
// Step 2. Create the Configuration, and set the properties accordingly
Configuration configuration = new ConfigurationImpl();
configuration.setPersistenceEnabled(false);
configuration.setSecurityEnabled(false);
configuration.setThreadPoolMaxSize(ConfigurationImpl.DEFAULT_THREAD_POOL_MAX_SIZE * 2); // not using NIO
configuration.setLogDelegateFactoryClassName(Log4jLogDelegateFactory.class.getName());
configuration.setBindingsDirectory(new File("messaging", "bindings").getCanonicalPath());
configuration.setJournalDirectory(new File("messaging", "journal").getCanonicalPath());
configuration.setLargeMessagesDirectory(new File("messaging", "largemessages").getCanonicalPath());
configuration.setPagingDirectory(new File("messaging", "paging").getCanonicalPath());
Map<String, Object> connectionParams = new HashMap<String, Object>();
connectionParams.put(TransportConstants.PORT_PROP_NAME, 61600);
connectionParams.put(TransportConstants.HOST_PROP_NAME, "localhost");
connectionParams.put(TransportConstants.USE_NIO_PROP_NAME, false);
TransportConfiguration transpConf = new TransportConfiguration(NettyAcceptorFactory.class.getName(), connectionParams);
HashSet<TransportConfiguration> setTransp = new HashSet<TransportConfiguration>();
setTransp.add(transpConf);
configuration.setAcceptorConfigurations(setTransp);
JMSConfiguration jmsConfig = new JMSConfigurationImpl();
server = new EmbeddedJMS();
server.setConfiguration(configuration);
server.setJmsConfiguration(jmsConfig);
server.start();
Here is the code for creating the message producer:
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
Here is the code for sending the message:
producer.send(message);
I hope the code is clear to read....
Thanks again.
Cheers.
-
14. Re: Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...
clebert.suconic Aug 8, 2011 3:06 PM (in response to mookythemook)// NAM: A bit of a hack to disable large messages - virus scanner playing havoc with performance
factory.setMinLargeMessageSize(Integer.MAX_VALUE);
^^ if you send a large message, you will be immediately out of credits, what *will* be causing your issue.
If you want to do this, disable producer flow control.