ORACLE RAC CONFIGURATION

ENVIRONMENT:

Node/Server Private IP Public IP Virtual IP

DB1 192.168.1.101 192.168.0.101 192.168.0.111
DB2 192.168.1.102 192.168.0.102 192.168.1.112

Scan IP:

192.168.0.121

192.168.0.122

192.168.0.123

HOST FILE:
Enter these entries in both nodes host file:

vi /etc/hosts

# Public

192.168.0.101 db1.db.com db1

192.168.0.102 db2.db.com db2

# Private

192.168.1.101 db1-priv.db.com db1-priv

192.168.1.102 db2-priv.db.com db2-priv

# Virtual

192.168.0.111 db1-vip.db.com db1-vip

192.168.0.112 db2-vip.db.com db2-vip

# SCAN

192.168.0.121 db-scan.db.com db-scan

192.168.0.122 db-scan.db.com db-scan

192.168.0.123 db-scan.db.com db-scan

DISABLE IPV6:
Add the following lines to /etc/sysctl.conf:

ifconfig | grep inet

vi /etc/sysctl.conf

# Disable IPv6

net.ipv6.conf.all.disable_ipv6 = 1

net.ipv6.conf.default.disable_ipv6 = 1

net.ipv6.conf.lo.disable_ipv6 = 1

Now execute this for apply:

/sbin/sysctl –p

For checking:

ifconfig | grep inet

DISABLE FIREWALL:
systemctl disable firewalld

systemctl stop filewalld

CREATE USERS AND GROUPS:

groupadd dba -g 1600

useradd -g dba -G dba,vboxsf -s /bin/bash oracle -u 1601

useradd -g dba -G dba,vboxsf -s /bin/bash grid -u 1602

echo "password" | passwd --stdin oracle

echo "password" | passwd --stdin grid

echo "%dba ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers

INSTALL NECESSARY PACKAGES:

Preinstall packages:

yum install -y compat-openssl10

yum install -y ksh

yum install -y libnsl

yum install -y sysstat

yum install -y xterm

yum install libaio-devel*

Oracle preinstall:

yum install -y oracle-database-preinstall-19c

If above command not working so download like that:

curl -o oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm
https://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/getPackage/oracle-database-preinstall-19c-1.0-
1.el7.x86_64.rpm

rpm -i oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm

Install this on browser and SCP to Node:

rpm -i compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm

Checking:

cd /etc/security/limits.d/

ls -l

cat oracle-database-preinstall-19c.conf

https://yum.oracle.com/oracle-linux-7.html

https://yum.oracle.com/repo/OracleLinux/OL7/8/base/x86_64/index.html

ADD BELOW LIMITS FOR ORACLE AND GRID USER:

vi /etc/security/limits.conf

grid soft stack 10240

oracle soft stack 10240

grid soft nofile 4096

grid hard nofile 63536

oracle soft nofile 4096

oracle hard nofile 63536

INSTALL ORACLE ASM:

yum install -y oracleasm*
yum install kmod-oracleasm -y

If above command not install ASM so SCP oracle ASM from this website:

https://www.oracle.com/linux/downloads/linux-asmlib-v7-downloads.html

rpm -i oracleasmlib-2.0.15-1.el7.x86_64.rpm

CONFIGURE ORACLE ASM:

To configure it run below command:

oracleasm configure -i

EXAMPLE:

Look the example and set this data:

[root@db1 u01]# oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting <ENTER> without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid

Default group to own the driver interface []: dba

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

Start the service:

systemctl list-unit-files | grep asm

systemctl enable oracleasm.service

systemctl start oracleasm.service

systemctl list-unit-files | grep asm

CREATE DIRECTORIES:
Create directories and give permissions according to you we have two home and base one for grid and second for oracle:

mkdir /u01

mkdir /u01/oracle/

mkdir /u01/oracle/V19BaseDatabase

mkdir /u01/oracle/V19Database

mkdir /u01/oracle/V19BaseGrid

mkdir /u01/oracle/V19Grid

chmod 777 /u01

chmod 777 /u01/oracle/

chown oracle:dba /u01/oracle/V19Database

chown oracle:dba /u01/oracle/V19BaseDatabase

chown grid:dba /u01/oracle/V19BaseGrid

chown grid:dba /u01/oracle/V19Grid

cd /u01/oracle; ls -l

SETUP NTP SERVICES (*3RD ADAPTER NEEDED): ---- OPTIONAL

Note: If you need to open the internet on your environment so need to add this if not so skip this part:

systemctl enable chronyd

systemctl restart chronyd

chronyc -a 'burst 4/4'

chronyc -a makestep

OPTIONAL – ADD BELOW LINE IN .BASHRC TO SET ENVIROMENT:

For Oracle user:

Set according to your directories and names:

vi /home/oracle/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=ora19c1 ; export ORACLE_SID

ORACLE_UNQNAME=ora19c ; export ORACLE_UNQNAME

ORACLE_GLOBAL_NAME=ora19c1.db.com ; export ORACLE_GLOBAL_NAME

ORACLE_BASE=/u01/oracle/V19BaseDatabase ; export ORACLE_BASE

ORACLE_HOME=/u01/oracle/V19Database ; export ORACLE_HOME

ORACLE_GRID=/u01/oracle/V19Grid ; export ORACLE_GRID

ORACLE_HOSTNAME=db1.db.com ; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib ; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

For Grid User:

Set according to your directories and names:

vi /home/grid/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=+ASM1 ; export ORACLE_SID

ORACLE_BASE=/u01/oracle/V19BaseGrid ; export ORACLE_BASE

ORACLE_HOME=/u01/oracle/V19Grid ; export ORACLE_HOME

ORACLE_GRID=/u01/oracle/V19Grid ; export ORACLE_GRID

ORACLE_HOSTNAME=db1.db.com ; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib ; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

IMPORTANT NOTE:
Every things will be same on bash for node 2 accept ASM SID and HOSTNAME will be change on node 2 like that:

ORACLE_SID=+ASM2 ; export ORACLE_SID

ORACLE_HOSTNAME=db2.db.com ; export ORACLE_HOSTNAME

ISCASI RPM DOWNLOAD: ------ (OPTIONAL ONLY NEED IN OPENFILER)

yum install iscsi-initiator-utils* -y

yum install iscsi-initiator-utils-devel* -y

CLONE NODES FROM BASE IMAGE:

Note: If you using virtual machine so after cloning do some changes on clone node if using physical machine so need to
follow these steps:

Change the MAC address in the Virtual Box Adapter both nodes.

Change the hostname that will give above:

vi /etc/hosts

Change the IP address of 2nd Node and set according that give above:

hostnamectl set-hostname db2.db.com

Change bash file.

Note: No changes are required to first node.

INSTALL AND CONFIGURE OPEN FILER VM AND SHARED DISK:

Note: Open file is use to shred disk in both nodes if you not using open filer so skip this:

REFRENCE LINK TO INSTALL AND CONFIGURE OPEN FILER:

http://abilim.blogspot.com/2014/01/step-by-step-configuring-automatic.html

Download Open filer ISO:

http://www.openfiler.com/community/download

Installation of Open Filer:

1- Add openfiler iso on vmware (with assign storage 15 gb and ram 4 gb) must you have hdd or scsi drive

2- Begin instllation follow some easy steps

3- You must have remember assign static ip address and local hostname in the time of installation

4- After the installation you have to power off the virtual machine and add hardive capacity for being purpose of sharing

5- Afterward power on the vm machine and trigger url address on screen just copy and paste on the web browser

6- You must have ssl ver 1 min and max 3 then you could access the machine

ERROR:

Solve open filer interface is not opening for that we use Firefox there we configure:

Enter this url:

about:config

Now enter this on bar and set 1:

security.tls.version.min

Configuration steps:

1- After you open the page of open filer via browser

Id : openfiler

Password : password

2- Then go to system option and check IP address and local host name

3- After check the configuration your start three services initiator.i target,scst target

4- After you select volume tab and select option create new volume group and enter volume group name.

5- Then go to again volume tab and scroll down and fill the some option available and select disk full size. (File system select
block)

6- Then go to ISCI target and select add option after that generate a address.

7- Then go to network ACL and enter network id.

8- Then go to LUN mapping option and click on map after that generate and address.
CONFIGURE OPEN-FILER RAC NODES:
Check ISCSI RPM:

rpm –qa iscsi*

Now enable shared storage on node:

Note: the LUN map and IP will be different according to your environment:

iscsiadm -m discovery -t sendtargets -p 192.168.0.109

iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.3d59eeab4b63 -p 192.168.0.109 -l

iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.3d59eeab4b63 -p 192.168.0.109 --op update -n node.startup -v

automatic

SETUP SHARED STORAGE:

Now create partition with fdisk:

RAC 1:

fdisk /dev/sdi

N>>>>W

Now setup oracle ASM to shared disk:

oracleasm createdisk ASMDATA /dev/sde1

oracleasm createdisk ASMOCR /dev/sdd1

RAC 2:

oracleasm scandisk

INSTALL GRID ON RAC1:

As a grid user:

Download the grid zip from oracle website and SCP then unzip it:

unzip -qq LINUX.X64_193000_grid_home.zip -d /u01/oracle/V19Grid/

As grid user run the gridSetup on RAC 1:

/u01/oracle/V19Grid/gridSetup.sh
1)

2)
3) add your scan

4) add second node for that click > add

5) Add the host name of RAC2

6) setup RAC 2 SSH

7) correct the private to asm private

8)
9)

10) it don’t show oracle asm disk by default so add that

11) add the path of oracle asm: /dev/oracleasm/disks/*

12) now select OSR and rename it and select external

13)

14)
15)

16)
17)

18)
19)

20) ignore that it will show because of DNS

21)

22) ignore this it will show because of DNS you can verify from details
23)

ERROR:

IF YOU SEE IN INSTALLATIO THAT NODE 2 FILE COPYING ERROR SO CHECK THE PERMISSION OF RAC1 AND RAC2 ALSO:
chmod 777 /u01 -R on both node

Add entry in /etc/oratab on both Node:

+ASM1:/u01/oracle/V19Grid:Y

RAC2:

+ASM2:/u01/oracle/V19Grid:Y

INSTALL RPM:
# as root

/u01/oracle/V19Grid/cv/rpm

CVUQDISK_GRP=dba; export CVUQDISK_GRP

rpm -i cvuqdisk-1.0.10-1.rpm

ON NODE 2:
scp cvuqdisk-1.0.10-1.rpm root@db2:/tmp/

ssh db2

/u01/oracle/V19Grid/cv/rpm

CVUQDISK_GRP=dba; export CVUQDISK_GRP

AFTER GRID INSTALLATION CHECK THE GRID IS INTALLED ON BHOT NODE:

ps -ef | grep pmon

crsctl stat res -t

INSTALL ORACLE HOME AS ORACLE USER:

As oracle user:

cd /u01

unzip -qq Oracle_DB_19cR300_Enterprise_LIN64_Home.zip -d /u01/oracle/V19Database/

As oracle user run the run Installer on First Node:

/u01/oracle/V19Database/runInstaller

1)
2)

3) Do ssh connectivity for node 2

4)

5)
6)

7)
8)

9) This prerequisite failed for DNS SCAN so ignore that

CREATE DATABASE ON RAC 1:
As oracle user:

cd $ORACLE_HOME/bin

./dbca

1)
2)

3)
4)

5)
6)

7) --- Solution --- SELECT name, compatibility FROM v$asm_diskgroup; (for-check) ALTER DISKGROUP ASMDATA1 SET
ATTRIBUTE 'compatible.asm' = '19.0'; (change)
8) add storage asmdata2 like asmdata1 add (optional enable archive)

9)
10)

11)
12)

13)
14)

15) SAME DNS ERROR

16)

17)
Add entry in /etc/oratab

+ora19c1:/u01/oracle/V19Database:Y

CREATE ASM DATA DISK FOR THE DATABASE RAC 1:

As grid user:

. oraenv

+ASM1

sqlplus / as sysasm

CREATE DISKGROUP ASMDATA1 EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/ASMDATA1';

SELECT STATE, NAME FROM V$ASM_DISKGROUP;

ALTER DISKGROUP ASMDATA MOUNT;

COL name FOR A10;

COL compatibility FOR A15;

SELECT name, compatibility FROM v$asm_diskgroup;

ALTER DISKGROUP ASMDATA SET ATTRIBUTE 'compatible.asm' = '19.0';

CREATE DISKGROUP ASMDATA2 EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/ASMDATA2';

SELECT STATE, NAME FROM V$ASM_DISKGROUP;

ALTER DISKGROUP ASMDATA2 MOUNT;

SELECT name, compatibility FROM v$asm_diskgroup;

ALTER DISKGROUP ASMDATA2 SET ATTRIBUTE 'compatible.asm' = '19.0';

TEST THE RAC:

Create table on RAC 1 and select from RAC 2:

-- Create the suleman table:

CREATE TABLE suleman5 (

sulemanloyee_id NUMBER PRIMARY KEY,

sulemanloyee_name VARCHAR2(50),

hire_date DATE
);

-- Insert dummy data:

INSERT INTO suleman5 (sulemanloyee_id, sulemanloyee_name, hire_date) VALUES (1, 'Suleman1', TO_DATE('2023-01-01',
'YYYY-MM-DD'));

INSERT INTO suleman5 (sulemanloyee_id, sulemanloyee_name, hire_date) VALUES (2, 'Suleman2', TO_DATE('2023-02-01',
'YYYY-MM-DD'));

INSERT INTO suleman5 (sulemanloyee_id, sulemanloyee_name, hire_date) VALUES (3, 'Suleman3', TO_DATE('2023-03-01',
'YYYY-MM-DD'));

-- Commit the changes

COMMIT;

RAC 2:

select * from suleman;

STARTUP AND SHUTDOWN ORACLE RAC BOTH NODES:

RAC START:
Startup RAC step by step like that:

1 CRS

2 NODE

3 ASM

4 DB INSTANCE

5 DB

1. CRS:

crsctl check cluster -all

crsctl start crs

crsctl start crs

2. NODE:

GRID:

srvctl status nodeapps -n db1

srvctl status nodeapps -n rac2

3. ASM:

GRID:

srvctl status asm -n db1

srvctl status asm -n db2

srvctl start asm -n rac1

srvctl start asm -n rac2

srvctl stop asm -n rac2

srvctl stop asm -n rac2

4. DB INSTANCE:

STOP AND START DB + INSTANCE

srvctl status database -d ora19c

srvctl stop database -d ora19c

srvctl start database -d ora19c

5. DB:

ONLY INSTANCE + PARTICULAR ONE INSTANCE

srvctl start instance -d RAC -i instance

srvctl start instance -d RAC -i instance

RAC SHUTDOWN:
Shutdown RAC step by step like that:

1 DB

2 DB INSTANCE

3 ASM

4 NODE

5 CRS

Run above commands with shutdown clause.

DOCUMENT MADE BY: SULEMAN SIDDIQUI

IF YOU FACE ANY PROBLEM CONTACT ME:

sulemansidd992@gmail.com