are the consumers distributed over a number of connections? In other words, when you see the subscriptions in the jmx console, do you see corresponding connections. It would be best to use multiple connections for such a large number of consumers.
Also, it may make sense to reduce the prefetch value from the default, because for each consumer, the broker will keep prefecth quantity of messages in memory if the producers are sufficiently fast. This could cause the producer side to block.