HornetQ Connection Leak in 2.2.10
mrobson May 25, 2012 12:54 PMHey,
System:
RHEL 6.2 + EAP 5.1.2 w/ HormetQ 2.2.10
I have a Remote HornetQ configuration using MDBs.
There is a front end web service which accepts and writes messages to a remote HornetQ server and a MDB server which reads messages off the queue and processes them.
Everything is done via Multicast (broadcast / discovery groups).
The issue I am seeing, is a consistent connection leak on the front end and MDB servers in terms of the number of established TCP connections to HornetQ on port 5445.
I have a total of 5 Queue's, each one opens 15 connections, meaning there should only be 75 open connections.
I am also seeing a consistent leak on the actual HornetQ server, in which it seems to be opening connections to itsself via localhost on port 5445.
When I start the HornetQ Server, I has 2 connections on localhost. 25 minutes later, 26 connections...
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49465 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49472 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49474 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49469 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49468 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49469 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49470 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49465 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49464 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49467 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49468 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49470 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49466 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49476 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49471 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49473 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49476 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49466 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49471 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49464 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49467 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49475 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49475 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:5445 | 127.0.0.1:49472 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49474 | 127.0.0.1:5445 | ESTABLISHED 15211/java | |
tcp | 0 | 0 127.0.0.1:49473 | 127.0.0.1:5445 | ESTABLISHED 15211/java |
There is no traffic or anything running on any of the servers, the connections just grow and grow until they crash due to too many processes.
I have another service, which is currently still running on AS 6.1 w/ HornetQ 2.2.5 using the exact same configuration for HornetQ and it does not have any of the above issues with any leaked connections.
Matt