连接上的mysql-connector阻塞不再存在。 Tomcat冻结了

We use JDBCRealm with mysql for form authentication in Tomcat. This works fine almost always.
When we stop/restart our mysql database (in test), we notice that one of the Tomcat HTTP threads will wait endlessly for some authentication related data, instead of failing (e.g. IOException or time out or something). This causes all other HTTP threads to block as well, as soon as they require a shared synchronized method. The result is that more and more HTTP threads are created, all blocking, until the maximum is reached.
The frozen thread never awakes or times out, even though I used netstat to confirm that the connection on which it is blocking, is effectively gone.
There is a firewall between both sides, although I don't think this has anything to do with the problem here. The connections are set up and taken down "normally".
Quick solution is ofcourse to stop/start tomcat with the database, but it seems odd that the call would not simply fail. Also I'm afraid this would happen in production in similar situations.
Is there something we can do other than restart tomcat or manually kill the blocking thread?
Thanks in advance for your thoughts.

The setup:

我们在Tomcat中使用JDBCRealm和mysql进行表单身份验证。这几乎总是很好。当我们停止/重新启动我们的mysql数据库(在测试中)时,我们注意到其中一个Tomcat HTTP线程会无休止地等待一些与身份验证相关的数据,而不是失败(例如IOException或超时等)。这会导致所有其他HTTP线程一旦需要共享同步方法就阻塞。结果是创建了越来越多的HTTP线程,所有HTTP线程都被阻塞,直到达到最大值。冻结的线程永远不会唤醒或超时,即使我使用netstat确认它阻塞的连接实际上已经消失了。双方都有防火墙,虽然我不认为这与这里的问题有任何关系。建立连接并“正常”取下。快速解决方案是使用数据库停止/启动tomcat,但看起来奇怪的是调用不会简单地失败。另外我担心这会在类似情况下的制作中发生。除了重启tomcat还是手动杀死阻塞线程,我们能做些什么吗?提前感谢您的想法。设置:

tomcat6
JDBCRealm for form authentication

JDBCRealm用于表单身份验证

mysql 5.5 database

mysql 5.5数据库

mysql-connector-java-5.1.7-bin.jar

relevant part of server.xml:

server.xml的相关部分:

  <Realm className="org.apache.catalina.realm.LockOutRealm" lockOutTime="300">
       <Realm className="org.apache.catalina.realm.JDBCRealm"
            connectionName="xxx"
            connectionPassword="xxx"
            connectionURL="jdbc:mysql://xxx:6446/xxx?autoReconnect=true"
            digest="xxx"
            driverName="com.mysql.jdbc.Driver"
            roleNameCol="xxx"
            userCredCol="xxx"
            userNameCol="xxx"
            userRoleTable="xxx"
            userTable="xxx"
            maxActive="30"
            maxIdle="30"
            maxWait="10000"
            factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"/>
  </Realm>

Example of a blocked thread (blocking on entry of a "synchronized" method):

被阻止线程的示例(阻止输入“synchronized”方法):

Name: http-8080-60
State: BLOCKED on org.apache.catalina.realm.JDBCRealm@edeb18b owned by: http-8080-61
Total blocked: 27  Total waited: 63

Stack trace: 
 org.apache.catalina.realm.JDBCRealm.authenticate(JDBCRealm.java:353)
org.apache.catalina.realm.CombinedRealm.authenticate(CombinedRealm.java:178)
org.apache.catalina.realm.LockOutRealm.authenticate(LockOutRealm.java:196)
org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:260)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:454)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:394)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Unknown Source)

The single thread that is causing the blocks in all other threads (notice that the "available()" call is waiting/blocking on a connection that isn't there anymore):

导致所有其他线程中的块的单个线程(注意“available()”调用正在等待/阻塞不再存在的连接):

Name: http-8080-61
State: RUNNABLE
Total blocked: 132  Total waited: 1,198

Stack trace: 
 java.net.PlainSocketImpl.socketAvailable(Native Method)
java.net.PlainSocketImpl.available(Unknown Source)
   - locked java.net.SocksSocketImpl@c679aa4
java.net.SocketInputStream.available(Unknown Source)
com.mysql.jdbc.util.ReadAheadInputStream.available(ReadAheadInputStream.java:231)
com.mysql.jdbc.MysqlIO.clearInputStream(MysqlIO.java:941)
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1887)
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2101)
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2548)
   - locked java.lang.Object@65d98b58
com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1560)
   - locked java.lang.Object@65d98b58
org.apache.catalina.realm.JDBCRealm.getPassword(JDBCRealm.java:583)
   - locked org.apache.catalina.realm.JDBCRealm@edeb18b
org.apache.catalina.realm.JDBCRealm.authenticate(JDBCRealm.java:414)
   - locked org.apache.catalina.realm.JDBCRealm@edeb18b
org.apache.catalina.realm.JDBCRealm.authenticate(JDBCRealm.java:361)
   - locked org.apache.catalina.realm.JDBCRealm@edeb18b
org.apache.catalina.realm.CombinedRealm.authenticate(CombinedRealm.java:178)
org.apache.catalina.realm.LockOutRealm.authenticate(LockOutRealm.java:196)
org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:260)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:454)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:394)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Unknown Source)

2 个解决方案

#1

Try seting a read timeout on JDBC connection with socketTimeout.

尝试使用socketTimeout在JDBC连接上设置读取超时。

jdbc:mysql://xxx:6446/xxx?autoReconnect=true&connectTimeout=60000&socketTimeout=60000

#2

Do you use Database only for auth° ? Imho you should better use a datasource coupled with a DataSourceRealm and your LockOutRealm. Maybe this last one is not well integrated with the JDBCRealm.

您是否仅将数据库用于auth°? Imho你应该更好地使用数据源加上DataSourceRealm和你的LockOutRealm。也许这最后一个与JDBCRealm没有很好地集成。

Here is a JDBCRealm javadoc extract :

这是一个JDBCRealm javadoc提取:

For a Realm implementation that supports connection pooling and doesn't require synchronisation of authenticate(), getPassword(), roles() and getPrincipal() or the ugly connection logic use the DataSourceRealm.

对于支持连接池,并且不需要authenticate()的,getPassword来(),角色()和getPrincipal()或难看连接逻辑使用DataSourceRealm的同步的领域实施。

#1