环境描述：
| 节点一 | 节点二 | |
| 主机名 |
rac1 |
rac2 |
| IP | 192.168.10.11 | 192.168.10.12 |
| IP-VIP | 192.168.10.111 | 192.168.10.112 |
| IP-priv | 10.10.10.11 | 10.10.10.12 |
| rac-scan | 192.168.10.200 | |
1.安装操作系统
1.1安装操作系统
---同样操作创建rac2
---设置网卡，挂载光盘,打开虚拟机安装系统：
---密码默认oracle
---同样操作安装rac2的操作系统
1.2关闭无用的服务：（所有节点）
|
service NetworkManager stop service acpid stop service anacron stop service apmd stop service atd stop service auditd stop service autofs stop service avahi-daemon stop service avahi-dnsconfd stop service bluetooth stop service conman stop service cpuspeed stop service cups stop service dnsmasq stop service dund stop service firstboot stop service gpm stop service haldaemon stop service hidd stop service ip6tables stop service ipmi stop service ipmievd stop service iptables stop service irda stop service irqbalance stop service kdump stop service kudzu stop service mcstrans stop service mdmpd stop service microcode_ctl stop service netconsole stop service netfs stop service netplugd stop service nfs stop service nfslock stop service nscd stop service ntpd stop service pand stop service pcscd stop service portmap stop service psacct stop service rawdevices stop service rdisc stop service readahead_early stop service readahead_later stop service restorecond stop service rhnsd stop service rpcgssd stop service rpcidmapd stop service rpcsvcgssd stop service saslauthd stop service sendmail stop service setroubleshoot stop service smartd stop service vncserver stop service wdaemon stop service winbind stop service wpa_supplicant stop service xfs stop service xinetd stop service ypbind stop service yum-updatesd stop |
chkconfig NetworkManager off chkconfig acpid off chkconfig anacron off chkconfig apmd off chkconfig atd off chkconfig auditd off chkconfig autofs off chkconfig avahi-daemon off chkconfig avahi-dnsconfd off chkconfig bluetooth off chkconfig conman off chkconfig cpuspeed off chkconfig cups off chkconfig dnsmasq off chkconfig dund off chkconfig firstboot off chkconfig gpm off chkconfig haldaemon off chkconfig hidd off chkconfig ip6tables off chkconfig ipmi off chkconfig ipmievd off chkconfig iptables off chkconfig irda off chkconfig isdn off chkconfig irqbalance off chkconfig kdump off chkconfig kudzu off chkconfig mcstrans off chkconfig mdmpd off chkconfig microcode_ctl off chkconfig netconsole off chkconfig netfs off chkconfig netplugd off chkconfig nfs off chkconfig nfslock off chkconfig nscd off chkconfig ntpd off chkconfig pand off chkconfig pcscd off chkconfig portmap off chkconfig psacct off chkconfig rawdevices off chkconfig rdisc off chkconfig readahead_early off chkconfig readahead_later off chkconfig restorecond off chkconfig rhnsd off chkconfig rpcgssd off chkconfig rpcidmapd off chkconfig rpcsvcgssd off chkconfig saslauthd off chkconfig sendmail off chkconfig setroubleshoot off chkconfig smartd off chkconfig vncserver off chkconfig wdaemon off chkconfig winbind off chkconfig wpa_supplicant off chkconfig xfs off chkconfig xinetd off chkconfig ypbind off chkconfig yum-updatesd off |
1.3关闭系统ntp服务，采用oracle自带的时间同步服务：
[root@rac1 ~]# chkconfig ntpd off
[root@rac1 ~]# mv /etc/ntp.conf /etc/ntp.conf.bak
2.环境配置（所有节点）：
2.1hosts文件设置：
[root@rac1 ~]# vi /etc/hosts # Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost 192.168.10.11 rac1
192.168.10.12 rac2 192.168.10.111 rac1-vip
192.168.10.112 rac2-vip 10.10.10.11 rac1-priv
10.10.10.12 rac2-priv 192.168.10.200 rac-scan
2.2创建用户和组：
---创建用户和组：
groupadd -g 200 oinstall
groupadd -g 201 dba
groupadd -g 202 oper
groupadd -g 203 asmadmin
groupadd -g 204 asmoper
groupadd -g 205 asmdba
useradd -u 200 -g oinstall -G dba,asmdba,oper oracle
useradd -u 201 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
---创建相关的目录：
mkdir -p /u01/app/oraInventory
chown -R grid:oinstall /u01/app/oraInventory/
chmod -R 775 /u01/app/oraInventory/
mkdir -p /u01/11.2.0/grid
chown -R grid:oinstall /u01/11.2.0/grid/
chmod -R 775 /u01/11.2.0/grid/
mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/cfgtoollogs
mkdir -p /u01/app/oracle/product/11.2.0/db_1
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
---密码设置：
[root@rac1 ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@rac1 ~]# passwd grid
Changing password for user grid.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@rac1 ~]#
2.3修改内核参数：
--修改的时候，直接写入文件最下方即可。
--也可以不提前修改，在安装的时候会有修改提示。
[root@rac1 ~]# vi /etc/sysctl.conf
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586 <tc/sysctl.conf" 50L, 1390C written
[root@rac1 ~]# ——参数生效 [root@rac1 ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
[root@rac1 ~]#
---安全限制修改：
[root@rac1 ~]# vi /etc/security/limits.conf oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240 ~
<ty/limits.conf" 62L, 2210C written
[root@rac1 ~]#
---修改其他参数：
[root@rac1 ~]# vi /etc/pam.d/login
session required /lib/security/pam_limits.so [root@rac1 ~]# vi /etc/profile :
if [ $USER = "oracle" ]||[ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
2.4修改oracle用户和grid用户的环境变量：
——rac1 ORACLE_SID=PROD1 ORACLE_SID=+ASM1
——rac2 ORACLE_SID=PROD2 ORACLE_SID=+ASM2
——环境变量必须大写，dbca建库，实例名默认大写
——Oracle用户
——Oracle用户 [root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ vi .bash_profile
# .bash_profile # Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi # User specific environment and startup programs PATH=$PATH:$HOME/bin export EDITOR=vi
export ORACLE_SID=PROD1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022 <9L, 426C written
[oracle@rac1 ~]$ ——grid用户 [oracle@rac1 ~]$ su - grid
Password:
[grid@rac1 ~]$ vi .bash_profile # .bash_profile # Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi # User specific environment and startup programs PATH=$PATH:$HOME/bin export EDITOR=vi
export ORACLE_SID=+ASM1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/11.2.0/grid
export GRID_HOME=/u01/11.2.0/grid
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export THREADS_FLAG=native
export PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022 ~
~
~
~
~
~
<3L, 473C written
[grid@rac1 ~]$
3.rpm包安装：
——需要先加载光盘
[grid@rac1 ~]$ su - root
Password:
[root@rac1 ~]# mount /dev/cdrom /mnt
mount: block device /dev/cdrom is write-protected, mounting read-only
[root@rac1 ~]# vi /etc/yum.repos.d/public-yum-el5.repo [oel5]
name = Enterprise Linux 5.6 DVD
baseurl=file:///mnt/Server/
gpgcheck=0
enabled=1 <um-el5.repo" [New] 7L, 90C written
[root@rac1 ~]# [root@rac1 ~]# yum install oracle-validated
4.时间同步
——查看时间是否同步，时间差不要超过20秒 [root@rac1 ~]# date
Sat Oct 29 10:05:19 CST 2016
[root@rac1 ~]# [root@rac2 ~]# date
Sat Oct 29 10:05:19 CST 2016
[root@rac2 ~]#
5.存储
5.1配置共享存储
——关闭虚拟机，注意取消挂载的光盘，避免进入重新安装系统界面
——新创建的4块虚拟磁盘都要设置
—然后在节点2（rac2）添加虚拟磁盘
——注意，添加的顺序要和节点一一致
——打开虚拟机，查看磁盘信息
——节点一
[root@rac1 ~]# fdisk -l Disk /dev/sda: 51.5 GB, 51539607552 bytes
255 heads, 63 sectors/track, 6266 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 535 4192965 82 Linux swap / Solaris
/dev/sda3 536 6266 46034257+ 83 Linux Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdd doesn't contain a valid partition table Disk /dev/sde: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sde doesn't contain a valid partition table
[root@rac1 ~]# ——节点二
[root@rac2 ~]# fdisk -l Disk /dev/sda: 51.5 GB, 51539607552 bytes
255 heads, 63 sectors/track, 6266 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 535 4192965 82 Linux swap / Solaris
/dev/sda3 536 6266 46034257+ 83 Linux Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdd doesn't contain a valid partition table Disk /dev/sde: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sde doesn't contain a valid partition table
[root@rac2 ~]#
5.2配置裸设备：
——在节点一上临时设置，主要为了查看major
[root@rac1 ~]# raw /dev/raw/raw1 /dev/sdb
/dev/raw/raw1: bound to major 8, minor 16
[root@rac1 ~]# raw /dev/raw/raw2 /dev/sdc
/dev/raw/raw2: bound to major 8, minor 32
[root@rac1 ~]# raw /dev/raw/raw3 /dev/sdd
/dev/raw/raw3: bound to major 8, minor 48
[root@rac1 ~]# raw /dev/raw/raw4 /dev/sde
/dev/raw/raw4: bound to major 8, minor 64
[root@rac1 ~]# ——设置参数文件（所有节点）
[root@rac1 ~]# vi /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1. ACTION=="add", KERNEL=="/dev/sdb",RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="16",RUN+="/bin/raw /dev/raw/raw1 %M %m"
ACTION=="add", KERNEL=="/dev/sdc",RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="32",RUN+="/bin/raw /dev/raw/raw2 %M %m"
ACTION=="add", KERNEL=="/dev/sdd",RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="48",RUN+="/bin/raw /dev/raw/raw3 %M %m"
ACTION=="add", KERNEL=="/dev/sde",RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="64",RUN+="/bin/raw /dev/raw/raw4 %M %m"
KERNEL=="raw[1-4]", OWNER="grid", GROUP="asmadmin", MODE="660" ——启动服务或者重启节点（所有节点）
[root@rac1 ~]# start_udev
Starting udev: [ OK ]
[root@rac1 ~]# ——查看
——节点一
[root@rac1 ~]# ls -lrt /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 3 Oct 29 11:15 /dev/raw/raw3
crw-rw---- 1 grid asmadmin 162, 2 Oct 29 11:15 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 4 Oct 29 11:15 /dev/raw/raw4
crw-rw---- 1 grid asmadmin 162, 1 Oct 29 11:15 /dev/raw/raw1
[root@rac1 ~]# ——节点二
[root@rac2 ~]# ls -lrt /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 4 Oct 29 11:17 /dev/raw/raw4
crw-rw---- 1 grid asmadmin 162, 3 Oct 29 11:17 /dev/raw/raw3
crw-rw---- 1 grid asmadmin 162, 2 Oct 29 11:17 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 1 Oct 29 11:17 /dev/raw/raw1
[root@rac2 ~]#
6.安装GI（单一节点操作）
6.1上传安装介质
——在grid用户下创建soft文件夹，安装介质上传到soft文件夹后解压缩
6.2安装前检测：
[grid@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
6.3安装GI
——添加节点一和节点二的grid用户互信
——查看IP地址段是否正确
——DNS错误，忽略就可以了
——优先在节点一执行脚本，执行完一个脚本后，在执行一个
——可以忽略报错
6.4验证安装是否成功：
[grid@rac1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....ER.lsnr ora....er.type ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type ONLINE ONLINE rac1
ora.OCR.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.cvu ora.cvu.type ONLINE ONLINE rac1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type ONLINE ONLINE rac1
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora.scan1.vip ora....ip.type ONLINE ONLINE rac1
[grid@rac1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@rac1 ~]$
7.安装oracle软件（单一节点操作）：
7.1上传安装介质：
——在oracle用户中创建soft文件夹，上传介质到soft文件夹中，解压缩
7.2安装oracle：
——创建两个节点的oracle用户互信
——在所有节点的root用户下依次执行脚本
8.创建asm磁盘组：
——在节点一的grid用户下，使用asmca打开图形化进行创建
——需要注意是否所有节点都mount上了
9.DBCA建库
——在oracle用户下，使用dbca图形化建库
——通过rman恢复的方式建库，快一些
---至此，rac以全部搭建完成，过程很多很杂。