Oracle ORA-03137: TTC protocol internal error : [12333] 故障分析

程序通过JDBC 连接数据库异常，报 ORA-03137[12333]的错误。

当前程序的JDBC 驱动版本：ojdbc16-11.2.0.1.0.jar

数据库版本： 11.2.0.3

一. Log 信息

1.1 alert log

TNS-12637: Packet receive failed

ns secondary err code: 12532

nt main err code: 0

nt secondary err code: 0

nt OS err code: 0

opiodr aborting process unknown ospid(28518) as a result of ORA-609

Fri Aug 09 15:41:20 2013

Errors in file /u01/app/oracle/diag/rdbms/tserpdb/tserpdb/trace/tserpdb_ora_28329.trc (incident=60746):

ORA-03137: TTC protocolinternal error : [12333] [19] [3] [14] [] [] [] []

Incident details in:/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/incident/incdir_60746/tserpdb_ora_28329_i60746.trc

Fri Aug 09 15:41:21 2013

Sweep [inc][60746]: completed

Sweep [inc2][60746]: completed

Fri Aug 09 15:41:21 2013

Dumping diagnostic data indirectory=[cdmp_20130809154121], requested by (instance=1, osid=28329),summary=[incident=60746].

Fri Aug 09 15:59:42 2013

1.2 trace 文件

Dump file/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/incident/incdir_60746/tserpdb_ora_28329_i60746.trc

Oracle Database 11g Enterprise EditionRelease 11.2.0.3.0 - 64bit Production

With the Partitioning, Automatic StorageManagement, OLAP, Data Mining

and Real Application Testing options

ORACLE_HOME =/u01/app/oracle/product/11.2.0/dbhome_1

System name: Linux

Node name: sh-tsiagent-003180

Release: 2.6.18-308.el5

Version: #1SMP Tue Feb 21 20:06:06 EST 2012

Machine: x86_64

Instance name: tserpdb

Redo thread mounted by this instance: 1

Oracle process number: 92

Unix process pid: 28329, image:oracle@sh-tsiagent-003180

*** 2013-08-09 15:41:20.138

*** SESSION ID:(1911.53519) 2013-08-0915:41:20.138

*** CLIENT ID:() 2013-08-09 15:41:20.138

*** SERVICE NAME:(tsdb3180.develop)2013-08-09 15:41:20.138

*** MODULE NAME:(JDBC Thin Client)2013-08-09 15:41:20.138

*** ACTION NAME:() 2013-08-09 15:41:20.138

Dump continued from file:/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/trace/tserpdb_ora_28329.trc

ORA-03137: TTC protocol internal error :[12333] [19] [3] [14] [] [] [] []

========= Dump for incident 60746 (ORA 3137[12333]) ========

*** 2013-08-09 15:41:20.139

dbkedDefDump(): Starting incident defaultdumps (flags=0x2, level=3, mask=0x0)

----- Current SQL Statement for thissession (sql_id=35tr4jfq0x3jw) -----

select mpzone0_.ZONEID as ZONEID83_0_,mpzone0_.AREACODE as AREACODE83_0_, mpzone0_.CARDTYPE as CARDTYPE83_0_,mpzone0_.city as city83_0_, mpzone0_.province as province83_0_ fromIAGENT.MPZONE mpzone0_ where mpzone0_.ZONEID=:1

----- Call Stack Trace -----

calling call entry argument values in hex

location type point (? means dubious value)

-------------------- ---------------------------- ----------------------------

skdstdst()+36 call kgdsdst() 000000000 ?000000000 ?

7FFF91403248 ? 000000001 ?

000000001 ?000000002 ?

…

__libc_start_main() call main() 000000002 ?7FFF91414458 ?

+244 000000001 ? 000000000 ?

009B910C0 ? 000000000 ?

_start()+36 call __libc_start_main() 000A0AF38 ? 000000002 ?

7FFF91414448 ? 000000000 ?

009B910C0 ? 000000002 ?

…

--------------------- Binary Stack Dump---------------------

========== FRAME [1] (skdstdst()+36 ->kgdsdst()) ==========

defined by frame pointers0x7fff91407b00 and 0x7fff91407af0

CALL TYPE: call ERROR SIGNALED: no COMPONENT: (null)

RDI 0000000000000000 RSI 0000000000000000RDX 00007FFF91403248

RCX 0000000000000001 R8 0000000000000001 R90000000000000002

RAX 0000000000000000 RBX 0000000000000003RBP 00007FFF91407B00

R10 00007FFF913FFA80 R11 0000000000000000R12 0000000000000003

R13 0000000000000002 R14 0000000000000000R15 0000000000000001

RSP 00007FFF91407B00 RIP 0000000006772B38

Dump of memory from 0x7fff91407af0 to0x7fff91407b00

7FFF91407AF0 91407B00 00007FFF 06772B3D00000000 [.{@.....=+w.....]

========== FRAME [2] (ksedst1()+98 ->skdstdst()) ==========

defined by frame pointers0x7fff91407bb0 and 0x7fff91407b00

CALL TYPE: call ERROR SIGNALED: no COMPONENT: KSE

1.3 Java 错误

org.springframework.transaction.TransactionSystemException:Could not commit Hibernate transaction; nested exception isorg.hibernate.TransactionException: JDBC commit failed

org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:660)

org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)

org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)

org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394)

java.sql.SQLRecoverableException:No more data to read from socket

oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1200)

oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1155)

oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:279)

oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)

oracle.jdbc.driver.T4C7Ocommoncall.doOCOMMIT(T4C7Ocommoncall.java:75)

oracle.jdbc.driver.T4CConnection.doCommit(T4CConnection.java:558)

oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3674)

oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3680)

org.apache.tomcat.dbcp.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)

org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:211)

org.hibernate.transaction.JDBCTransaction.commitAndResetAutoCommit(JDBCTransaction.java:166)

二.解决方法

在MOS上搜了一下，该问题与Bug9445675 非常像。

Bug 9445675 NO MOREDATA TO READ FROM SOCKET WHEN USING END-TO-END METRICS

This bug does affect the JDBC driver. This bug may be the cause when all of the following conditions are met:

1) You are using the 10.1.x.x orthe 11.2.0.1 JDBC driver; the bug does not affect 10.2.x.x, or 11.1.x.xversions of the driver, nor versions 11.2.0.2 or above

2) You are using end-to-endmetrics in your Java code

3) The server side ORA-3137[12333] error is accompanied by the client side Java exception "No moredata to read from socket"

This bug is fixed in the11.2.0.2 version of the JDBC driver and above.

而当前版本的JDBC Driver 也是：11.2.0.1.0。所以将JDBC DRIVER升级到11.2.0.2 之后，故障解决。

参考MOS：

Understanding and Diagnosing ORA-00600[12333] ORA-3137 [12333] Errors (文档 ID 389713.1)

ORA-03137 TTC Protocol Internal Error [12333] Using JDBC Driver (文档 ID 752297.1)

Troubleshooting ORA-3137 [12333] ErrorsEncountered When Using Oracle JDBC Driver (文档 ID1361107.1)

--------------------------------------------------------------------------------------------

QQ:492913789

Email:ahdba@qq.com

Blog: http://www.cndba.cn/dave