-
15. Re: Re: Re: Teiid performance Linux v.s. Windows
kchen007 Apr 10, 2014 9:44 AM (in response to rareddy)Ramesh:
I tried the servlet, it is about the same time on both windows and linux, around 10s for 100 top level entities, inlinie with above windows timing when using socket and much better than linux's data(>60s).
-
16. Re: Re: Re: Teiid performance Linux v.s. Windows
rareddy Apr 10, 2014 1:09 PM (in response to kchen007)Kevin,
This proves that it definitely something do with the network configuration in your Linux environment. Now switching gears, if your client accesses the DBMS directly do you still see the lag or the lag is only with Teiid in the mix?
Ramesh..
-
17. Re: Re: Re: Teiid performance Linux v.s. Windows
kchen007 Apr 10, 2014 6:08 PM (in response to rareddy) -
18. Re: Teiid performance Linux v.s. Windows
kchen007 Apr 11, 2014 3:19 PM (in response to kchen007)We got the bottom of it, if we disable the Nagle's alorithm, then the problem went away.
BTW, what is the best way to disable the Nagle's alorithm without changing the Teiid client code or change the machine wide setting.
thanks.
-
19. Re: Teiid performance Linux v.s. Windows
shawkins Apr 11, 2014 4:34 PM (in response to kchen007)> We got the bottom of it, if we disable the Nagle's alorithm, then the problem went away.
We'll look into that as well and see if local/remote socket connections in our scenarios have a performance degradation.
> BTW, what is the best way to disable the Nagle's alorithm without changing the Teiid client code or change the machine wide setting.
There is the system/Teiid client property org.teiid.sockets.conserveBandwidth, but it already defaults to false (which maps to Socket.setTCPNodelay(true)) which should disable Nagle's algorithm.
-
20. Re: Teiid performance Linux v.s. Windows
kchen007 Apr 11, 2014 4:59 PM (in response to shawkins)That is strange, If I started my client program with -Dorg.teiid.sockets.conserveBandwidth=true, instead of false, it will make the program performance better. maybe the flag got flipped again somewhere?
-
21. Re: Teiid performance Linux v.s. Windows
rareddy Apr 11, 2014 6:31 PM (in response to kchen007)I believe the issue is with how it was confusingly described in the javadoc, which led to a bug in Teiid interpretation
{code}
* Enable/disable TCP_NODELAY (disable/enable Nagle's algorithm).
*
* @param on <code>true</code> to enable TCP_NODELAY,
* <code>false</code> to disable.
{code}
true = enable tcp_delay = disable Nagle
Either we can fix doc, or can log a JIRA. I do remember that it was conscious decision not to disable Nagle's algorithm while back, but do remember the reasons behind it now.
Ramesh..
-
22. Re: Teiid performance Linux v.s. Windows
shawkins Apr 11, 2014 7:45 PM (in response to rareddy)> I believe the issue is with how it was confusingly described in the javadoc, which led to a bug in Teiid interpretation
I'm not sure I follow you. Our property, org.teiid.sockets.conserveBandwidth is doc'ed as true=Nagle's algorithm. Then when we setTCPNoDelay we flip the value, so conserveBandwidth=true which means TCP_NODELAY=false which means enable Nagle's algorithm. Do you see something else?
-
23. Re: Teiid performance Linux v.s. Windows
rareddy Apr 13, 2014 8:43 PM (in response to shawkins)You are correct, I read wrong.
org.teiid.sockets.conserveBandwidth = false is default setting, so Teiid is setting TCP_DELAY = true, that disable Nagle's algorithm by default. Flipping it to true, should enable Nagle's algorithm.
Kevin: how did you disable Nagle's algorithm first time when you found out about it? I guess my confusion started there thinking Teiid by default enabled Nagle's algorithm.
Ramesh..
-
24. Re: Teiid performance Linux v.s. Windows
kchen007 Apr 14, 2014 9:40 AM (in response to rareddy)I recompile Teiid client code after changing the boolean to true. then it started to perform well.
-
25. Re: Teiid performance Linux v.s. Windows
kchen007 Apr 14, 2014 10:44 AM (in response to kchen007)When I suggested one of our user to set the flag, he told me from time to time, the client cannot connect to server with the following error msgs.
thanks.
on Server side:
09:22:24,257 ERROR [org.teiid.TRANSPORT] (New I/O worker #5) TEIID40017 Unexpected exception for session null: org.teiid.core.crypto.CryptoException:
TEIID10006 Decryption failed: javax.crypto.BadPaddingException Given final block not properly padded
at org.teiid.core.crypto.BasicCryptor.decrypt(BasicCryptor.java:94) [teiid-common-core-8.4.0.Final.jar:8.4.0.Final]
at org.teiid.core.crypto.BasicCryptor.unsealObject(BasicCryptor.java:176) [teiid-common-core-8.4.0.Final.jar:8.4.0.Final]
at org.teiid.transport.ServerWorkItem.run(ServerWorkItem.java:77) [teiid-runtime-8.4.1.jar:8.4.1]
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:269) [teiid-engine-8.4.1.jar:8.4.1]
at org.teiid.transport.SocketClientInstance.processMessagePacket(SocketClientInstance.java:192) [teiid-runtime-8.4.1.jar:8.4.1]
at org.teiid.transport.SocketClientInstance.receivedMessage(SocketClientInstance.java:178) [teiid-runtime-8.4.1.jar:8.4.1]
at org.teiid.transport.SSLAwareChannelHandler.messageReceived(SSLAwareChannelHandler.java:201) [teiid-runtime-8.4.1.jar:8.4.1]
at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:787) [netty-3.6.2.Final.jar:]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:787) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.2.Final.jar:]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.2.Final.jar:]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.2.Final.jar:]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.2.Final.jar:]
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.2.Final.jar:]
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.2.Final.jar:]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
and here is from the client side:
SEVERE: Could not create connection
org.teiid.jdbc.TeiidSQLException: TEIID20018 Unable to find a component used authenticate on to Teiid
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
at org.teiid.jdbc.SocketProfile.connect(SocketProfile.java:56)
at org.teiid.jdbc.TeiidDriver.connect(TeiidDriver.java:107)
at org.teiid.jdbc.TeiidDriver.connect(TeiidDriver.java:55)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at com.lgc.dsds.performance.TestPrepStatementClose.runTest(TestPrepStatementClose.java:29)
at com.lgc.dsds.performance.TestPrepStatementClose.main(TestPrepStatementClose.java:208)
Caused by: org.teiid.net.CommunicationException: TEIID20018 Unable to find a component used authenticate on to Teiid
at org.teiid.net.socket.SocketServerConnection.selectServerInstance(SocketServerConnection.java:144)
at org.teiid.net.socket.SocketServerConnection.<init>(SocketServerConnection.java:92)
at org.teiid.net.socket.SocketServerConnectionFactory.getConnection(SocketServerConnectionFactory.java:312)
at org.teiid.jdbc.SocketProfile.connect(SocketProfile.java:54)
... 6 more
Caused by: org.teiid.core.TeiidComponentException: TEIID10006 Decryption failed: javax.crypto.BadPaddingException Given final block not properly padded
at org.teiid.client.util.ExceptionUtil.convertException(ExceptionUtil.java:59)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:388)
at $Proxy0.logon(Unknown Source)
at org.teiid.net.socket.SocketServerConnection.logon(SocketServerConnection.java:173)
at org.teiid.net.socket.SocketServerConnection.selectServerInstance(SocketServerConnection.java:125)
... 9 more
Caused by: org.teiid.core.crypto.CryptoException: TEIID10006 Decryption failed: javax.crypto.BadPaddingException Given final block not properly padded
at org.teiid.core.crypto.BasicCryptor.decrypt(BasicCryptor.java:94)
at org.teiid.core.crypto.BasicCryptor.unsealObject(BasicCryptor.java:176)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.convertResult(SocketServerInstanceImpl.java:340)
at org.teiid.client.util.ResultsFuture.get(ResultsFuture.java:116)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:371)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:380)
... 12 more
Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID20018 Unable to find a component used authenticate on to Teiid
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
at org.teiid.jdbc.SocketProfile.connect(SocketProfile.java:56)
at org.teiid.jdbc.TeiidDriver.connect(TeiidDriver.java:107)
at org.teiid.jdbc.TeiidDriver.connect(TeiidDriver.java:55)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at com.lgc.dsds.performance.TestPrepStatementClose.runTest(TestPrepStatementClose.java:29)
at com.lgc.dsds.performance.TestPrepStatementClose.main(TestPrepStatementClose.java:208)
Caused by: org.teiid.net.CommunicationException: TEIID20018 Unable to find a component used authenticate on to Teiid
at org.teiid.net.socket.SocketServerConnection.selectServerInstance(SocketServerConnection.java:144)
at org.teiid.net.socket.SocketServerConnection.<init>(SocketServerConnection.java:92)
at org.teiid.net.socket.SocketServerConnectionFactory.getConnection(SocketServerConnectionFactory.java:312)
at org.teiid.jdbc.SocketProfile.connect(SocketProfile.java:54)
... 6 more
Caused by: org.teiid.core.TeiidComponentException: TEIID10006 Decryption failed: javax.crypto.BadPaddingException Given final block not properly padded
at org.teiid.client.util.ExceptionUtil.convertException(ExceptionUtil.java:59)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:388)
at $Proxy0.logon(Unknown Source)
at org.teiid.net.socket.SocketServerConnection.logon(SocketServerConnection.java:173)
at org.teiid.net.socket.SocketServerConnection.selectServerInstance(SocketServerConnection.java:125)
... 9 more
Caused by: org.teiid.core.crypto.CryptoException: TEIID10006 Decryption failed: javax.crypto.BadPaddingException Given final block not properly padded
at org.teiid.core.crypto.BasicCryptor.decrypt(BasicCryptor.java:94)
at org.teiid.core.crypto.BasicCryptor.unsealObject(BasicCryptor.java:176)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.convertResult(SocketServerInstanceImpl.java:340)
at org.teiid.client.util.ResultsFuture.get(ResultsFuture.java:116)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:371)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:380)
... 12 more
-
26. Re: Teiid performance Linux v.s. Windows
shawkins Apr 14, 2014 2:20 PM (in response to kchen007)I'm not entirely sure how the two would be related. You wouldn't expect to get these bad padding exceptions except if the wrong key is being used to decrypt, which shouldn't be possible. How reproducible is this, and what if any other changes are in effect?
-
27. Re: Teiid performance Linux v.s. Windows
kchen007 Apr 14, 2014 3:24 PM (in response to shawkins)I cannot reproduce it myself. After examine the call stack again, It looks like there is mismatch between runtime jar and common-core jar. We built the runtime jar to pickup some fix for 8.4.0 and it requuires corresponding common-core.jar. I have given him the new common-core module and hopefully that will fix the problem.
It only happened twice for him too.
thanks
Kevin