Problems with Messages after shutdown on Jboss 3.2.2RC3

[nollario]

Hi!

I am currently testing JMS on JBoss and I am facing the following scenario:

When I send messages to a queue and try to receive them - there is no problem.

If I shutdown the server and start it again, there are already some error messages during the boot process (attached).

Furthermore, the MQ services don't start (Queue not bound...).

There were 5000 messages in the queue and I am using the standard Hypersonic database.

Why does the server not restart with the messages kept in the queue?

Thanks

Christian


2003-09-22 10:52:39,602 ERROR [org.jboss.mq.pm.jdbc2.PersistenceManager] Starting failed
org.jboss.mq.SpyJMSException: Could not resolve uncommited transactions. Message recovery may not be accurate; - nested throwable: (org.jboss.util.NestedSQLException: Could not create connection; - nested throwable: (java.sql.SQLException: Connection is broken: Connection refused); - nested throwable: (org.jboss.resource.JBossResourceException: Could not create connection; - nested throwable: (java.sql.SQLException: Connection is broken: Connection refused)))
at org.jboss.mq.pm.jdbc2.PersistenceManager.resolveAllUncommitedTXs(PersistenceManager.java:295)
at org.jboss.mq.pm.jdbc2.PersistenceManager.startService(PersistenceManager.java:1315)
at org.jboss.system.ServiceMBeanSupport.start(ServiceMBeanSupport.java:192)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:976)
at $Proxy12.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:394)
at org.jboss.system.ServiceController.start(ServiceController.java:411)
at org.jboss.system.ServiceController.start(ServiceController.java:411)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy4.start(Unknown Source)
at org.jboss.deployment.SARDeployer.start(SARDeployer.java:226)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:824)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:632)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:605)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)

....

10:52:45,909 ERROR [URLDeploymentScanner] MBeanException: Exception in MBean operation 'checkIncompleteDeployments()'
Cause: Incomplete Deployment listing:
Packages waiting for a deployer:

Incompletely deployed packages:

MBeans waiting for classes:

MBeans waiting for other MBeans:
[ObjectName: jboss.mq:service=InvocationLayer,type=HTTP
state: CREATED
I Depend On: jboss.mq:service=Invoker
jboss.web:service=WebServer

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=JVM
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=UIL2
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: , ObjectName: jboss.mq:service=DestinationManager
state: CREATED
I Depend On: jboss.mq:service=MessageCache
jboss.mq:service=PersistenceManager
jboss.mq:service=StateManager

Depends On Me: jboss.mq.destination:service=Topic,name=testTopic
jboss.mq.destination:service=Topic,name=securedTopic
jboss.mq.destination:service=Topic,name=testDurableTopic
jboss.mq.destination:service=Queue,name=testQueue
jboss.mq.destination:service=Queue,name=A
jboss.mq.destination:service=Queue,name=B
jboss.mq.destination:service=Queue,name=C
jboss.mq.destination:service=Queue,name=D
jboss.mq.destination:service=Queue,name=ex
jboss.mq:service=SecurityManager
jboss.mq.destination:service=Queue,name=DLQ
, ObjectName: jboss.mq:service=PersistenceManager
state: FAILED
I Depend On: jboss.jca:service=LocalTxCM,name=DefaultDS

Depends On Me: jboss.mq:service=DestinationManager
org.jboss.mq.SpyJMSException: Could not resolve uncommited transactions. Message recovery may not be accurate; - nested throwable: (org.jboss.util.NestedSQLException: Could not create connection; - nested throwable: (java.sql.SQLException: Connection is broken: Connection refused); - nested throwable: (org.jboss.resource.JBossResourceException: Could not create connection; - nested throwable: (java.sql.SQLException: Connection is broken: Connection refused))), ObjectName: jboss.mq.destination:service=Topic,name=testTopic
state: CREATED
I Depend On: jboss.mq:service=DestinationManager
jboss.mq:service=SecurityManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Topic,name=securedTopic
state: CREATED
I Depend On: jboss.mq:service=DestinationManager
jboss.mq:service=SecurityManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Topic,name=testDurableTopic
state: CREATED
I Depend On: jboss.mq:service=DestinationManager
jboss.mq:service=SecurityManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=testQueue
state: CREATED
I Depend On: jboss.mq:service=DestinationManager
jboss.mq:service=SecurityManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=A
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=B
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=C
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=D
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: , ObjectName: jboss.mq.destination:service=Queue,name=ex
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: , ObjectName: jboss.mq:service=Invoker
state: CREATED
I Depend On: jboss.mq:service=TracingInterceptor

Depends On Me: jboss.mq:service=InvocationLayer,type=HTTP
jboss.mq:service=InvocationLayer,type=JVM
jboss.mq:service=InvocationLayer,type=UIL2
jboss.mq:service=InvocationLayer,type=OIL2
jboss.mq:service=InvocationLayer,type=OIL
jboss.mq:service=InvocationLayer,type=RMI
jboss.mq:service=InvocationLayer,type=UIL
, ObjectName: jboss.mq:service=TracingInterceptor
state: CREATED
I Depend On: jboss.mq:service=SecurityManager

Depends On Me: jboss.mq:service=Invoker
, ObjectName: jboss.mq:service=SecurityManager
state: CREATED
I Depend On: jboss.mq:service=DestinationManager

Depends On Me: jboss.mq.destination:service=Topic,name=testTopic
jboss.mq.destination:service=Topic,name=securedTopic
jboss.mq.destination:service=Topic,name=testDurableTopic
jboss.mq.destination:service=Queue,name=testQueue
jboss.mq:service=TracingInterceptor
jboss.mq.destination:service=Queue,name=DLQ
, ObjectName: jboss.mq.destination:service=Queue,name=DLQ
state: CREATED
I Depend On: jboss.mq:service=DestinationManager
jboss.mq:service=SecurityManager

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=OIL2
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=OIL
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=RMI
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: , ObjectName: jboss.mq:service=InvocationLayer,type=UIL
state: CREATED
I Depend On: jboss.mq:service=Invoker

Depends On Me: ]
10:52:45,913 INFO [URLDeploymentScanner] Started
10:52:45,985 INFO [MainDeployer] Deployed package: file:/home/jboss/jboss-3.2.2RC3/server/default/conf/jboss-service.xml
10:52:45,988 INFO [Server] JBoss (MX MicroKernel) [3.2.2RC3 (build: CVSTag=JBoss_3_2_2_RC3 date=200308241317)] Started in 34s:205ms

[stephanenicoll]

Your are probably using a transactional JMS session without calling commit on the session.

Regards,

Stephane

[nollario]

Hi!

No, the send of the messages is commited... I found another exception that occurs during the startup of the server.

Probably, I missed to explain the sequence of the events:

1.) I sent 5000 messages to the JMS queue
2.) Shutdown the server
3.) Restart...

During the restart, the problems occur.

Is my problem probably connected to this topic created my Rod Macpherson?

http://www.mail-archive.com/[e-mail]/msg32107.html//www.mail-archive.com/[e-mail]/msg32107.html

Thanks

Christian


org.jboss.resource.JBossResourceException: Could not create connection; - nested throwable: (java.sql.SQLException: Connection is broken: Connection refused)
at org.jboss.resource.adapter.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:153)
at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnection(InternalManagedConnectionPool.java:372)
at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:165)
at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:445)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:426)
at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:331)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:509)
at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:839)
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:102)
at org.jboss.mq.pm.jdbc2.PersistenceManager.getConnection(PersistenceManager.java:1225)
at org.jboss.mq.pm.jdbc2.PersistenceManager.resolveAllUncommitedTXs(PersistenceManager.java:214)
at org.jboss.mq.pm.jdbc2.PersistenceManager.startService(PersistenceManager.java:1315)
at org.jboss.system.ServiceMBeanSupport.start(ServiceMBeanSupport.java:192)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:976)
at $Proxy12.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:394)
at org.jboss.system.ServiceController.start(ServiceController.java:411)
at org.jboss.system.ServiceController.start(ServiceController.java:411)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy4.start(Unknown Source)
at org.jboss.deployment.SARDeployer.start(SARDeployer.java:226)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:824)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:632)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:605)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy6.deploy(Unknown Source)
at org.jboss.deployment.scanner.URLDeploymentScanner.deploy(URLDeploymentScanner.java:302)
at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:476)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:201)
at org.jboss.deployment.scanner.AbstractDeploymentScanner.startService(AbstractDeploymentScanner.java:274)
at org.jboss.system.ServiceMBeanSupport.start(ServiceMBeanSupport.java:192)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:976)
at $Proxy0.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:394)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy4.start(Unknown Source)
at org.jboss.deployment.SARDeployer.start(SARDeployer.java:226)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:824)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:632)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:605)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:589)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
at $Proxy5.deploy(Unknown Source)
at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:384)
at org.jboss.system.server.ServerImpl.start(ServerImpl.java:291)
at org.jboss.Main.boot(Main.java:150)
at org.jboss.Main$1.run(Main.java:388)
at java.lang.Thread.run(Thread.java:534)
Caused by: java.sql.SQLException: Connection is broken: Connection refused
at org.hsqldb.Trace.getError(Unknown Source)
at org.hsqldb.Trace.error(Unknown Source)
at org.hsqldb.jdbcConnection.reconnectHSQL(Unknown Source)
at org.hsqldb.jdbcConnection.openHSQL(Unknown Source)
at org.hsqldb.jdbcConnection.(Unknown Source)
at org.hsqldb.jdbcDriver.connect(Unknown Source)
at org.jboss.resource.adapter.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:143)

[stephanenicoll]

The 'Queue not bound' stuff is because JMS failed to start. Try creatuing a non transactionnal JMS session and remove the calls to commit / rollback to see if you still have the problem

(just to check whether it's coming from your transactionnal session or not). What connection factory are you using?

Regards,

Stephane

[nollario]

Hi!

Thanks for the help, Lord! ;-)

No, even without a transaction - MQ is not started, the same exceptions occur. :(

The different is, that the default.script file (Hypersonic) grows up to 27 MB....

Any ideas?

Christian

[stephanenicoll]

The 27Mo is your messages ;-) and HSQL seems to be broken :/

If you are testing stuff remove the data directory (rm -rf data) and restart JBoss (which will restart ok this time) Please note that removing the data directory will imply the lost of your messages!

Then relauch your tests whith

createSession(false, Session.AUTO_ACKNOWLEDGE)

Regards,

Stephane

[nollario]

thank you for your help, stephane!

yes! that is obvious...

the problem is we really need a performance of 5000 messages sent in a single transaction.

i thought hypersonic would be able to support this...
:(

and when the server is started, and there are messages left in the queue... they should not be lost!

thanks

christian

[stephanenicoll]

It seems that your messages are not committed properly. I had the problem a few months ago (message lost on restart). If you are inside a J2EE component, you don't need to specify a JMS transactionnel session, the global transaction context will carry that for you.

Regards,

Stephane

[stephanenicoll]

Regarding my last post, I thing this it. Have a look to the stacktrace:

at org.jboss.mq.pm.jdbc2.PersistenceManager.resolveAllUncommitedTXs(PersistenceManager.java:295)

The method's name is quite explicit, don't you think?

Regards,

Stephane

[adrian.brock]

jboss-3.2.2RC4 includes a change to the configuration
for hsqldb to use "create cached table"
This reverses the assumption (everything is flushed
to disk with a memory cache rather than memory first with
period savepoints).

You should seriously consider your use of hsqldb if
you can't afford to lose a message/downtime.
hsqldb is in the same rank as MS access. Ok as a desktop
database and testing but not an enterprise db.

Regards,
Adrian