环境描述:
操作系统版本:Red Hat Enterprise Linux Server release 6.5 (Santiago)
数据库版本:Oracle 11.2.0.4 RAC
场景描述:
Oracle RAC环境搭建完成之后,数据库可以启动到mounted状态,无法启动到open状态;其他的CRS集群服务、ASM服务状态正常!!!
处理过程:
[oracle@oracle01 ~]$ sqlplus / as sysdba; SQL*Plus: Release 11.2.0.4.0 Production on Thu Jun 15 12:48:00 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options SQL>
SQL> select open_mode from v$database; OPEN_MODE
--------------------
MOUNTED SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01154: database busy. Open, close, mount, and dismount not allowed now
++++++++++++++++++++++++++++++++++++++++++
再次尝试:
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01154: database busy. Open, close, mount, and dismount not allowed now 启动数据库一直停留在Database mounted.
最后报ORA-03113: end-of-file on communication channel
++++++++++++++++++++++++++++++++++++++++
查看告警日志,输出如下错误信息:
<txt>ORACLE Instance EBJDB1 - Archival Error
</txt>
</msg>
<msg time='2017-06-15T14:38:26.640+08:00' org_id='oracle' comp_id='rdbms'
client_id='' type='UNKNOWN' level='16'
host_id='oracle01' host_addr='127.0.0.1' module=''
pid='7977'>
<txt>ORA-16014: log 1 sequence# 5 not archived, no available destinations
ORA-00312: online log 1 thread 1: '+DATA/ebjdb/onlinelog/group_1.261.945309179'
ORA-00312: online log 1 thread 1: '+FRA/ebjdb/onlinelog/group_1.257.945309181'
</txt>
</msg>
<msg time='2017-06-15T14:38:26.857+08:00' org_id='oracle' comp_id='rdbms'
client_id='' type='UNKNOWN' level='16'
host_id='oracle01' host_addr='127.0.0.1' module=''
pid='7977'>
<txt>Archiver process freed from errors. No longer stopped
</txt>
</msg>
有戏!!!
初步分析是:日志不能正常归档,导致了数据库只能在mounted状态,在往open状态切换的时候,找不到相关的日志归档路径,导致数据库不能启动到open状态
++++++++++++++++++++++++++++++++++++++++
开始处理:
首先查看数据库当前启动状态:
SQL> select open_mode from v$database; OPEN_MODE
--------------------
MOUNTED
查看当前数据库的归档路径设置:
注:数据库在open状态下,可以通过如下命令查看归档信息,
SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination +FRA/archivelog/
Oldest online log sequence 8
Next log sequence to archive 9
Current log sequence 9
数据库在mounted状态下的时候,需要通过查看参数文件来确认:
SQL> set linesize 200
SQL> set pagesize 100
SQL> show parameter log_ NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
audit_syslog_level string
commit_logging string
db_create_online_log_dest_1 string
db_create_online_log_dest_2 string
db_create_online_log_dest_3 string
db_create_online_log_dest_4 string
db_create_online_log_dest_5 string
enable_ddl_logging boolean FALSE
log_archive_config string
log_archive_dest string
.. .. .. .. .. .. .. .......
log_archive_dest_1 string LOCATION=+FRA/archivelog/
log_archive_dest_state_4 string enable
log_archive_dest_state_5 string enable
log_archive_dest_state_9 string enable
log_archive_duplex_dest string
log_archive_format string %t_%s_%r.dbf
log_archive_local_first boolean TRUE
log_archive_max_processes integer 4
log_archive_min_succeed_dest integer 1
log_archive_start boolean FALSE
log_archive_trace integer 0
log_buffer integer 2379776
log_checkpoint_interval integer 0
log_checkpoint_timeout integer 1800
log_checkpoints_to_alert boolean FALSE
log_file_name_convert string
remote_login_passwordfile string EXCLUSIVE
sec_case_sensitive_logon boolean TRUE
sec_max_failed_login_attempts integer 10
可以看出归档日志的路径是:LOCATION=+FRA/archivelog/
+++++++++++++++++++++++++++++++++++
切换到oracle用户或者grid用户下,执行asmcmd查看磁盘组信息:
[grid@oracle01 ~]$ asmcmd
ASMCMD> ls
BAK/
CRS/
DATA/
FRA/
ASMCMD>
ASMCMD> cd FRA
ASMCMD> ls
EBJDB/
ASMCMD> 发现没有+FRA/archivelog目录,我们在+FRA目录下,创建archivelog目录:
ASMCMD> pwd
+FRA
ASMCMD> mkdir archivelog
archivelog/
ASMCMD> cd archivelog
ASMCMD> pwd
+FRA/archivelog
ASMCMD>
OK!!!!!!归档日志目录设置完毕!
++++++++++++++++++++++++++++++++++++++
回到sqlplus界面,
SQL> alter database open; SQL> select instance_name,status from v$instance; INSTANCE_NAME STATUS
---------------- ------------
EBJDB1 OPEN
查看集群服务状态:
[grid@oracle01 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.BAK.dg
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.CRS.dg
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.DATA.dg
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.FRA.dg
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.LISTENER.lsnr
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.asm
ONLINE ONLINE oracle01 Started
ONLINE ONLINE oracle02 Started
ora.gsd
ONLINE OFFLINE oracle01
ONLINE OFFLINE oracle02
ora.net1.network
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.ons
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
ora.registry.acfs
ONLINE ONLINE oracle01
ONLINE ONLINE oracle02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE oracle01
ora.cvu
1 ONLINE ONLINE oracle01
ora.db.db
1 ONLINE ONLINE oracle01 Open
2 ONLINE ONLINE oracle02 Open
ora.oracle01.vip
1 ONLINE ONLINE oracle01
ora.oracle02.vip
1 ONLINE ONLINE oracle02
ora.oc4j
1 ONLINE ONLINE oracle01
ora.scan1.vip
1 ONLINE ONLINE oracle01
OK!!! 焦急的心情稍许平静!!!!有木有!!!
网上查找各种资料的时候,心情特别难受,都开始怀疑人生了,对面的纯开发同事,人家一天啥事没有。。。。。这次问题虽然解决了,以后估计还会有相同的想法, 哈哈!!!
其他参考:
http://blog.csdn.net/tianlesoftware/article/details/6051672
http://blog.csdn.net/jingleli21/article/details/45888295
http://www.itpub.net/thread-1510308-1-1.html
https://wenku.baidu.com/view/755ddc25bcd126fff7050bdc.html
http://blog.chinaunix.net/uid-28813259-id-3987260.html