Oracle 11g RAC on Oracle Virtual Machine
Oracle 11g RAC on Oracle Virtual Machine
Oracle 11g RAC on Oracle Virtual Machine
Milap Joshi
Oracle 11gR2 RAC on Virtual M
9) Choose adapter 2 for private network ad below. (We’ll setup static IP addresses at later point for RAC
cluster’s Interconnectivity).
10) Choose Image or disk option where Linux Installation software or image is placed.
Partition Size
/boot 200MB
Swap 4GB
/u01 20GB
/ 10GB
compat-libstdc++-33-3.2.3-61.x86_64.rpm
elfutils-libelf-devel-0.137-3.el5.x86_64.rpm
elfutils-libelf-devel-static-0.137-3.el5.x86_64.rpm
kernel-headers-2.6.18-164.el5.x86_64.rpm
glibc-headers-2.5-42.x86_64.rpm
glibc-devel-2.5-42.x86_64.rpm
libgomp-4.4.0-6.el5.x86_64.rpm
gcc-4.1.2-46.el5.x86_64.rpm
libstdc++-devel-4.1.2-46.el5.x86_64.rpm
gcc-c++-4.1.2-46.el5.x86_64.rpm
libaio-devel-0.3.106-3.2.x86_64.rpm
libstdc++-devel-4.1.2-46.el5.x86_64.rpm
sysstat-7.0.2-3.el5.x86_64.rpm
unixODBC-2.2.11-7.1.x86_64.rpm
unixODBC-devel-2.2.11-7.1.x86_64.rpm
15) Perform Below pre checks and make required changes in memory or space parameters if
necessary.
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmmax = 2147483648
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=4194304
net.core.rmem_max=4194304
net.core.wmem_default=1048576
net.core.wmem_max=1048576
20) Check /etc/selinux/config file for SELINUX option and make it disabled.
SELINUX=disabled
22) Make required directories on the OS for GRID and Database Installation
24) Now switch to oracle user and make required changes in bash profile as shown below.
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
27) Configure eth1 with static IP with different series than other IPs as eth1 will be used for interconnection
between 2 nodes using these private IPs.
oracleasm-support-2.1.8-1.el5.x86_64.rpm
oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm
kernel-debug-2.6.18-164.el5.x86_64.rpm
oracleasm-2.6.18-164.el5debug-2.0.5-1.el5.x86_64.rpm
kernel-xen-2.6.18-164.el5.x86_64.rpm
oracleasm-2.6.18-164.el5xen-2.0.5-1.el5.x86_64.rpm
oracleasm-2.6.18-164.el5-debuginfo-2.0.5-1.el5.x86_64.rpm
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
31) Shut down RAC1 Machine and Perform Full clone of the Machine with a name as RAC2
32) Create 3 sharable disks for OCR and 1 Datadisk for ASM installation on RAC1 node.
34) Choose existing shared disks created from rac1 on rac2 node.
Choose the option to add hard disk and choose option to use existing drive as below.
35) Change MAC Address’ last 2 digit to something else on RAC2 as it is duplicated from RAC 1. Change MAC
address for both adapter 1 and adapter 2 on RAC2 as below.
/etc/sysconfig/network
/etc/hosts files
37) Check network ip address and remove .bak connections and set private ip for eth1. Activate both eth0
and eth1 IPs
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
40) Scan ASM Disks on both the nodes and list it.
# Private
192.168.10.1 rac1-priv.localdomain rac1-priv
192.168.10.2 rac2-priv.localdomain rac2-priv
# Public
192.168.42.71 rac1.localdomain rac1
192.168.42.39 rac2.localdomain rac2
# Virtual
192.168.42.181 rac1-vip.localdomain rac1-vip
192.168.42.182 rac2-vip.localdomain rac2-vip
# SCAN
192.168.42.191 rac-scan.localdomain rac-scan
192.168.42.192 rac-scan.localdomain rac-scan
192.168.52.193 rac-scan.localdomain rac-scan
43) Copy database and grid software on /u01 using any ftp tool.
46) Choose install and configure grid infrastructure for a cluster option as below.
50) Create SSH connectivity between 2 nodes. Provide oracle password and click setup to create
passwordless connectivity.
55) Oracle cluster verification utility and Configure oracle Grid Infrastructure for a cluster will fail as there is
no DNS configured and scan entries are only modified in /etc/hosts file. Grid installed successfully.
58) Choose the option Real Application cluster database installation option and select both the nodes.
61) Now Configure ASM diskgroup for data storage for newly created database. Set ASM environment and
execute asmca.
62) Choose option create new diskgroup, provide new name to diskgroup & choose the disk on which
diskgroup needs to be created.
63) Set Database Home and start DBCA to create and configure database.
64) Choose Oracle Real Application cluster database option and create database.
65) Provide Global Database name and SID. Also select both the nodes to configure database on both the
nodes.Ex. If SID is PROD on both node it’ll create instance as PROD1 & PROD2 on respective nodes.
66) Choose storage type as ASM and diskgroup on which database needs to be configured.
1)
[oracle@node1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle
[oracle@node1 ~]$ echo $ORACLE_HOME
/u01/app/11.2.0/grid
[oracle@node1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.eons ora.eons.type ONLINE ONLINE node1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsd application OFFLINE OFFLINE
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsd application OFFLINE OFFLINE
ora.node2.ons application ONLINE ONLINE node2
ora.node2.vip ora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.type OFFLINE OFFLINE
ora.ons ora.ons.type ONLINE ONLINE node1
ora.prod.db ora....se.type ONLINE ONLINE node1
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node1
2)
[oracle@node1 ~]$ crsctl status resource
NAME=ora.DATA.dg
TYPE=ora.diskgroup.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.LISTENER.lsnr
TYPE=ora.listener.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.LISTENER_SCAN1.lsnr
TYPE=ora.scan_listener.type
TARGET=ONLINE
STATE=ONLINE on node1
NAME=ora.asm
TYPE=ora.asm.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.eons
TYPE=ora.eons.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.gsd
TYPE=ora.gsd.type
TARGET=OFFLINE, OFFLINE
STATE=OFFLINE, OFFLINE
NAME=ora.net1.network
TYPE=ora.network.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.node1.vip
TYPE=ora.cluster_vip_net1.type
TARGET=ONLINE
STATE=ONLINE on node1
NAME=ora.node2.vip
TYPE=ora.cluster_vip_net1.type
TARGET=ONLINE
STATE=ONLINE on node2
NAME=ora.oc4j
TYPE=ora.oc4j.type
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.ons
TYPE=ora.ons.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.prod.db
TYPE=ora.database.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type
TARGET=ONLINE , ONLINE
STATE=ONLINE on node1, ONLINE on node2
NAME=ora.scan1.vip
TYPE=ora.scan_vip.type
TARGET=ONLINE
STATE=ONLINE on node1
3)
[oracle@node1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
4)
[oracle@node1 ~]$ ps -ef | grep d.bin
root 2684 1 0 20:06 ? 00:00:08 /u01/app/11.2.0/grid/bin/ohasd.bin reboot
oracle 2809 1 0 20:06 ? 00:00:03 /u01/app/11.2.0/grid/bin/oraagent.bin
oracle 2822 1 0 20:06 ? 00:00:00 /u01/app/11.2.0/grid/bin/gipcd.bin
oracle 2827 1 0 20:06 ? 00:00:00 /u01/app/11.2.0/grid/bin/mdnsd.bin
oracle 2841 1 0 20:06 ? 00:00:01 /u01/app/11.2.0/grid/bin/gpnpd.bin
root 2852 1 0 20:06 ? 00:00:01 /u01/app/11.2.0/grid/bin/cssdmonitor
root 2872 1 0 20:06 ? 00:00:01 /u01/app/11.2.0/grid/bin/cssdagent
root 2874 1 0 20:06 ? 00:00:00 /u01/app/11.2.0/grid/bin/orarootagent.bin
oracle 2902 1 0 20:06 ? 00:00:07 /u01/app/11.2.0/grid/bin/ocssd.bin
oracle 2906 1 0 20:06 ? 00:00:01 /u01/app/11.2.0/grid/bin/diskmon.bin -d -f
root 2955 1 0 20:07 ? 00:00:00 /u01/app/11.2.0/grid/bin/octssd.bin reboot
oracle 2973 1 0 20:07 ? 00:00:04 /u01/app/11.2.0/grid/bin/evmd.bin
oracle 3212 1 0 20:07 ? 00:00:01 /u01/app/11.2.0/grid/bin/oclskd.bin
root 3229 1 0 20:07 ? 00:00:08 /u01/app/11.2.0/grid/bin/crsd.bin reboot
root 3269 1 0 20:07 ? 00:00:01 /u01/app/11.2.0/grid/bin/oclskd.bin
oracle 3317 2973 0 20:07 ? 00:00:00 /u01/app/11.2.0/grid/bin/evmlogger.bin -o
/u01/app/11.2.0/grid/evm/log/evmlogger.info -l /u01/app/11.2.0/grid/evm/log/evmlogger.log
oracle 3359 1 0 20:07 ? 00:00:03 /u01/app/11.2.0/grid/bin/oraagent.bin
root 3363 1 0 20:07 ? 00:00:06 /u01/app/11.2.0/grid/bin/orarootagent.bin
oracle 3490 1 0 20:08 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
oracle 3574 1 0 20:08 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -
inherit
oracle 4977 4671 0 20:28 pts/2 00:00:00 grep d.bin
5)
[oracle@node1 ~]$ ps -ef | grep tns
oracle 3490 1 0 20:08 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
oracle 3574 1 0 20:08 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -
inherit
oracle 4987 4671 0 20:28 pts/2 00:00:00 grep tns
6)
[oracle@node2 ~]$ cat /etc/hosts
# Private
192.168.10.1 node1-priv.localdomain node1-priv
192.168.10.2 node2-priv.localdomain node2-priv
# Public
192.168.42.186 node1.localdomain node1
192.168.42.23 node2.localdomain node2
# Virtual
192.168.42.181 node1-vip.localdomain node1-vip
192.168.42.182 node2-vip.localdomain node2-vip
# SCAN
192.168.42.191 node-scan.localdomain node-scan
192.168.42.192 node-scan.localdomain node-scan
192.168.52.193 node-scan.localdomain node-scan
6)
[root@node1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:70:49:71
inet addr:192.168.42.71 Bcast:192.168.42.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe70:4971/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1854 errors:0 dropped:0 overruns:0 frame:0
TX packets:2087 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:280022 (273.4 KiB) TX bytes:310865 (303.5 KiB)
Base address:0xd010 Memory:f0000000-f0020000
collisions:0 txqueuelen:1000
RX bytes:31427482 (29.9 MiB) TX bytes:19663826 (18.7 MiB)
Base address:0xd240 Memory:f0820000-f0840000
7)
[oracle@rac1 ~]$ db_env
8) ASM Commands