Thinking Out Loud

May 20, 2013

RAC KISS on VirtualBox

Filed under: 11g,oracle,RAC — mdinh @ 4:52 am

Practice makes perfect, but it does not have to be difficult. Please don’t short yourself by not trying to understand what is going on.

This is my third RAC install and I got tired of doing it the hard manual way; hence, decided to script what I can.

ASM will still need to be configured after rackiss:

oracleasm configure -i
oracleasm init
oracleasm createdisk asm01 /dev/sdb1 ...
oracleasm scandisks
oracleasm listdisks

Tested Configuration: Linux rac01.localdomain 2.6.39-200.24.1.el6uek.x86_64 using V33411-01.iso on Virtual Box 4.2.12r84980

Modify script based on network configuration for the following:

/etc/resolv.conf
/etc/hosts
/etc/dnsentries.txt

CUSTOMIZE MINIMAL INSTALL

Base System
> Base
> Client Management Tools
Desktops
> Desktop
> Desktop Platform
> Graphical Administration Tools
> X Window System
Applications
> Internet Browser
740 packages
NTP not configured
KDUMP not enabled

Using GUI from VirtualBox Guest OS

Disaable Software Updates:
System > Preferences > System > Software Updates

Install Guest Additions
Devices > Install Guest Additions (ISO)

SCRIPT:  rackiss.sh

#/bin/sh
{
echo -e "\n******** Disable packagekit refresh ********"
sed -i 's/enabled=1/enabled=0/g' /etc/yum/pluginconf.d/refresh-packagekit.conf
grep enabled /etc/yum/pluginconf.d/refresh-packagekit.conf

echo -e "\n******** Disable avahi-daemon 1501093.1 ********"
/etc/init.d/avahi-daemon stop
chkconfig avahi-daemon off
chkconfig avahi-daemon --list

echo -e "\n******** Silent Install oracle-rdbms-server-11gR2-preinstall ********"
yum install oracle-rdbms-server-11gR2-preinstall -q -y

echo -e "\n********* Silent Install dnsmaq, oracleasm, kernel *********"
yum install dnsmasq oracleasm-support oracleasmlib kernel-uek-devel-2.6.39-200.24.1.el6uek.x86_64 -q -y

echo -e "\n******** Download oracleasmlib ********"
wget http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el6.x86_64.rpm

echo -e "\n******** Install oracleasmlib ********"
yum localinstall oracleasmlib-2.0.4-1.el6.x86_64.rpm -y

echo -e "\n******** Verify yum install ********"
yum install oracle-rdbms-server-11gR2-preinstall -q -y
yum install dnsmasq oracleasm-support oracleasmlib kernel-uek-devel-2.6.39-200.24.1.el6uek.x86_64 -q -y
yum list install oracleasmlib-2.0.4-1.el6.x86_64

echo -e "\n******** Update rp_filter Note 1501093.1 ********"
sed -i 's/rp_filter = 1/rp_filter = 0/g' /etc/sysctl.conf
grep rp_filter  /etc/sysctl.conf

echo -e "\n******** Disable Firewall ********"
service iptables save
service iptables stop
chkconfig iptables off
chkconfig iptables --list

echo -e "\n******** AutoStart dnsmasq ********"
chkconfig dnsmasq on
chkconfig dnsmasq --list

echo -e "\n******** Update /etc/sysconfig/network ********"
echo "NOZEROCONF=yes" >> /etc/sysconfig/network
cat /etc/sysconfig/network

echo -e "\n******** Update /etc/grub.conf ********"
sed -i 's/quiet/quiet divider=10/g' /etc/grub.conf
grep divider /etc/grub.conf

echo -e "\n******** Update /etc/pam.d/login ********"
echo "session    required     pam_limits.so" >> /etc/pam.d/login
grep pam_limits.so /etc/pam.d/login

echo -e "\n******** Disable SELINUX ********"
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
grep SELINUX=disabled /etc/selinux/config

echo -e "\n******** Rename /etc/ntp.conf"
mv /etc/ntp.conf /etc/ntp.conf.org
ls -l /etc/ntp.conf.org
chmod 0444 /etc/ntp.conf.org

echo -e "\n******** Rename /etc/dnsmasq.conf ********"
mv /etc/dnsmasq.conf /etc/dnsmasq.conf.org
chmod 0444 /etc/dnsmasq.conf.org
ls -l /etc/dnsmasq.conf.org

echo -e "\n******** Configure /etc/hosts ********"
echo "192.168.56.11  rac01 rac01.localdomain" >> /etc/hosts
echo "192.168.56.12  rac02 rac02.localdomain" >> /etc/hosts
echo "192.168.56.13  rac03 rac03.localdomain" >> /etc/hosts
echo "192.168.56.21  rac01-vip rac01-vip.localdomain" >> /etc/hosts
echo "192.168.56.22  rac02-vip rac02-vip.localdomain" >> /etc/hosts
echo "192.168.56.23  rac03-vip rac03-vip.localdomain" >> /etc/hosts
echo "10.10.10.11    rac01-priv rac02-priv.localdomain" >> /etc/hosts
echo "10.10.10.12    rac02-priv rac02-priv.localdomain" >> /etc/hosts
echo "10.10.10.13    rac03-priv rac03-priv.localdomain" >> /etc/hosts
chmod 0444 /etc/hosts
cat /etc/hosts

echo -e "\n******** Configure /etc/dnsentries.txt for dnsmasq ********"
echo "192.168.56.11  rac01.localdomain"      > /etc/dnsentries.txt
echo "192.168.56.12  rac02.localdomain"     >> /etc/dnsentries.txt
echo "192.168.56.13  rac03.localdomain"     >> /etc/dnsentries.txt
echo "192.168.56.31  dinh-scan.localdomain" >> /etc/dnsentries.txt
echo "192.168.56.32  dinh-scan.localdomain" >> /etc/dnsentries.txt
echo "192.168.56.33  dinh-scan.localdomain" >> /etc/dnsentries.txt
chmod 0444 /etc/dnsentries.txt
cat /etc/dnsentries.txt

echo -e "\n******** Configure /etc/dnsmasq.conf for dnsmasq ********"
echo "listen-address=127.0.0.1"        > /etc/dnsmasq.conf
echo "strict-order"                   >> /etc/dnsmasq.conf
echo "domain-needed"                  >> /etc/dnsmasq.conf
echo "bogus-priv"                     >> /etc/dnsmasq.conf
echo "no-poll"                        >> /etc/dnsmasq.conf
echo "no-hosts"                       >> /etc/dnsmasq.conf
echo "local=/localdomain/"            >> /etc/dnsmasq.conf
echo "addn-hosts=/etc/dnsentries.txt" >> /etc/dnsmasq.conf
echo "cache-size=150"                 >> /etc/dnsmasq.conf
chmod 0444 /etc/dnsmasq.conf
cat /etc/dnsmasq.conf

echo -e "\n******** Configure /etc/resolv.conf for dnsmasq ********"
echo "nameserver 127.0.0.1"   > /etc/resolv.conf
echo "nameserver 172.16.0.1" >> /etc/resolv.conf
echo "nameserver 8.8.8.8"    >> /etc/resolv.conf
echo "search localdomain"    >> /etc/resolv.conf
chattr +i /etc/resolv.conf
cat /etc/resolv.conf

echo -e "\n******** Start dnsmasq ********"
service dnsmasq start

echo -e "\n******** Test nslookup ********"
nslookup rac01
nslookup rac02
nslookup rac03
nslookup dinh-scan
nslookup dinh-scan
nslookup dinh-scan
nslookup www.google.com

echo -e "\n******** Install Guest Additions ********"
/media/VBOXADDITIONS_4.2.12_84980/VBoxLinuxAdditions.run

echo -e "\n******** Add asmadmin asmdba ********"
groupadd -g 54325 asmadmin
groupadd -g 54326 asmdba

echo -e "\n******** Update Oracle's group ********"
usermod -g dba -G dba,asmadmin,asmdba,vboxsf oracle
id oracle

echo -e "\n******** Create directories ********"
mkdir -p /u01/app/oraInventory
mkdir -p /u01/app/11.2.0.3/grid
mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/cfgtoollogs
mkdir -p /u01/app/oracle/product/11.2.0.3/dbhome_1
chown -R oracle:dba /u01
chmod -R 775 /u01
find /u01 -type d -ls
} > /tmp/rackiss.log 2>&1
echo -e "\n******** Check /tmp/rackiss.log for errors ********"
exit

Verify mount for VBOXADDITIONS exist

[root@rac01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg01-lv_root
                       28G  1.8G   25G   7% /
tmpfs                1004M  100K 1004M   1% /dev/shm
/dev/sda1             485M   55M  405M  12% /boot
/dev/sr0               57M   57M     0 100% /media/VBOXADDITIONS_4.2.12_84980

Create and Run rackiss.sh

[root@rac01 ~]# vi rackiss.sh
[root@rac01 ~]# chmod 755 rackiss.sh
[root@rac01 ~]# time ./rackiss.sh

******** Check /tmp/rackiss.log for errors ********

real    33m43.730s
user    0m49.952s
sys     0m33.525s

Review Log

[root@rac01 ~]# cat /tmp/rackiss.log

******** Disable packagekit refresh ********
enabled=0

******** Disable avahi-daemon 1501093.1 ********
Shutting down Avahi daemon:                                [  OK  ]
avahi-daemon    0:off   1:off   2:off   3:off   4:off   5:off   6:off

******** Silent Install oracle-rdbms-server-11gR2-preinstall ********
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Importing GPG key 0xEC551F03:
 Userid: "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>"
 From  : http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6

********* Silent Install dnsmaq, oracleasm, kernel *********

******** Download oracleasmlib ********
--2013-05-19 20:35:49--  http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el6.x86_64.rpm
Resolving download.oracle.com... 63.141.192.73, 63.141.192.10
Connecting to download.oracle.com|63.141.192.73|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13300 (13K) [application/x-redhat-package-manager]
Saving to: âoracleasmlib-2.0.4-1.el6.x86_64.rpmâ

2013-05-19 20:35:50 (234 KB/s) - âoracleasmlib-2.0.4-1.el6.x86_64.rpmâ

******** Install oracleasmlib ********
Loaded plugins: security
Setting up Local Package Process
Examining oracleasmlib-2.0.4-1.el6.x86_64.rpm: oracleasmlib-2.0.4-1.el6.x86_64
Marking oracleasmlib-2.0.4-1.el6.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package oracleasmlib.x86_64 0:2.0.4-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package        Arch     Version       Repository                          Size
================================================================================
Installing:
 oracleasmlib   x86_64   2.0.4-1.el6   /oracleasmlib-2.0.4-1.el6.x86_64    27 k

Transaction Summary
================================================================================
Install       1 Package(s)

Total size: 27 k
Installed size: 27 k
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : oracleasmlib-2.0.4-1.el6.x86_64                              1/1
  Verifying  : oracleasmlib-2.0.4-1.el6.x86_64                              1/1

Installed:
  oracleasmlib.x86_64 0:2.0.4-1.el6

Complete!

******** Verify yum install ********
Package oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64 already installed and latest version
Package dnsmasq-2.48-13.el6.x86_64 already installed and latest version
Package oracleasm-support-2.1.8-1.el6.x86_64 already installed and latest version
Package kernel-uek-devel-2.6.39-200.24.1.el6uek.x86_64 already installed and latest version
Loaded plugins: security
Installed Packages
oracleasmlib.x86_64        2.0.4-1.el6         @/oracleasmlib-2.0.4-1.el6.x86_64

******** Update rp_filter Note 1501093.1 ********
net.ipv4.conf.default.rp_filter = 0

******** Disable Firewall ********
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:                               [  OK  ]
iptables        0:off   1:off   2:off   3:off   4:off   5:off   6:off

******** AutoStart dnsmasq ********
dnsmasq         0:off   1:off   2:on    3:on    4:on    5:on    6:off

******** Update /etc/sysconfig/network ********
NETWORKING=yes
HOSTNAME=rac01.localdomain
NOZEROCONF=yes

******** Update /etc/grub.conf ********
        kernel /vmlinuz-2.6.39-200.24.1.el6uek.x86_64 ro root=/dev/mapper/vg01-lv_root rd_LVM_LV=vg01/lv_swap rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 rd_LVM_LV=vg01/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet divider=10 numa=off
        kernel /vmlinuz-2.6.32-279.el6.x86_64 ro root=/dev/mapper/vg01-lv_root rd_LVM_LV=vg01/lv_swap rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg01/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet divider=10 numa=off

******** Update /etc/pam.d/login ********
session    required     pam_limits.so

******** Disable SELINUX ********
SELINUX=disabled

******** Rename /etc/ntp.conf
-rw-r--r--. 1 root root 1917 Jul  6  2010 /etc/ntp.conf.org

******** Rename /etc/dnsmasq.conf ********
-r--r--r--. 1 root root 21214 May 19 20:35 /etc/dnsmasq.conf.org

******** Configure /etc/hosts ********
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.56.11  rac01 rac01.localdomain
192.168.56.12  rac02 rac02.localdomain
192.168.56.13  rac03 rac03.localdomain
192.168.56.21  rac01-vip rac01-vip.localdomain
192.168.56.22  rac02-vip rac02-vip.localdomain
192.168.56.23  rac03-vip rac03-vip.localdomain
10.10.10.11    rac01-priv rac02-priv.localdomain
10.10.10.12    rac02-priv rac02-priv.localdomain
10.10.10.13    rac03-priv rac03-priv.localdomain

******** Configure /etc/dnsentries.txt for dnsmasq ********
192.168.56.11  rac01.localdomain
192.168.56.12  rac02.localdomain
192.168.56.13  rac03.localdomain
192.168.56.31  dinh-scan.localdomain
192.168.56.32  dinh-scan.localdomain
192.168.56.33  dinh-scan.localdomain

******** Configure /etc/dnsmasq.conf for dnsmasq ********
listen-address=127.0.0.1
strict-order
domain-needed
bogus-priv
no-poll
no-hosts
local=/localdomain/
addn-hosts=/etc/dnsentries.txt
cache-size=150

******** Configure /etc/resolv.conf for dnsmasq ********
nameserver 127.0.0.1
nameserver 172.16.0.1
nameserver 8.8.8.8
search localdomain

******** Start dnsmasq ********
Starting dnsmasq:                                          [  OK  ]

******** Test nslookup ********
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   rac01.localdomain
Address: 192.168.56.11

Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   rac02.localdomain
Address: 192.168.56.12

Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   rac03.localdomain
Address: 192.168.56.13

Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   dinh-scan.localdomain
Address: 192.168.56.33
Name:   dinh-scan.localdomain
Address: 192.168.56.31
Name:   dinh-scan.localdomain
Address: 192.168.56.32

Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   dinh-scan.localdomain
Address: 192.168.56.31
Name:   dinh-scan.localdomain
Address: 192.168.56.32
Name:   dinh-scan.localdomain
Address: 192.168.56.33

Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   dinh-scan.localdomain
Address: 192.168.56.32
Name:   dinh-scan.localdomain
Address: 192.168.56.33
Name:   dinh-scan.localdomain
Address: 192.168.56.31

Server:         127.0.0.1
Address:        127.0.0.1#53

Non-authoritative answer:
Name:   www.google.com
Address: 74.125.224.177
Name:   www.google.com
Address: 74.125.224.178
Name:   www.google.com
Address: 74.125.224.179
Name:   www.google.com
Address: 74.125.224.180
Name:   www.google.com
Address: 74.125.224.176

******** Install Guest Additions ********
Verifying archive integrity... All good.
Uncompressing VirtualBox 4.2.12 Guest Additions for Linux............
VirtualBox Guest Additions installer
Copying additional installer modules ...
Installing additional modules ...
Removing existing VirtualBox non-DKMS kernel modules       [  OK  ]
Building the VirtualBox Guest Additions kernel modules
The headers for the current running kernel were not found. If the following
module compilation fails then this could be the reason.
The missing package can be probably installed with
yum install kernel-uek-devel-2.6.39-200.24.1.el6uek.x86_64

Building the main Guest Additions module                   [  OK  ]
Building the shared folder support module                  [  OK  ]
Building the OpenGL support module                         [  OK  ]
Doing non-kernel setup of the Guest Additions              [  OK  ]
Starting the VirtualBox Guest Additions                    [  OK  ]
Installing the Window System drivers
Installing X.Org Server 1.10 modules                       [  OK  ]
Setting up the Window System to use the Guest Additions    [  OK  ]
You may need to restart the hal service and the Window System (or just restart
the guest system) to enable the Guest Additions.

Installing graphics libraries and desktop services componen[  OK  ]

******** Add asmadmin asmdba ********

******** Update Oracle's group ********
uid=54321(oracle) gid=54322(dba) groups=54322(dba),54323(vboxsf),54325(asmadmin),54326(asmdba)

******** Create directories ********
654346    4 drwxrwxr-x   3 oracle   dba          4096 May 19 21:06 /u01
654347    4 drwxrwxr-x   5 oracle   dba          4096 May 19 21:06 /u01/app
654349    4 drwxrwxr-x   3 oracle   dba          4096 May 19 21:06 /u01/app/11.2.0.3
654350    4 drwxrwxr-x   2 oracle   dba          4096 May 19 21:06 /u01/app/11.2.0.3/grid
654351    4 drwxrwxr-x   4 oracle   dba          4096 May 19 21:06 /u01/app/oracle
654352    4 drwxrwxr-x   2 oracle   dba          4096 May 19 21:06 /u01/app/oracle/cfgtoollogs
654353    4 drwxrwxr-x   3 oracle   dba          4096 May 19 21:06 /u01/app/oracle/product
654354    4 drwxrwxr-x   3 oracle   dba          4096 May 19 21:06 /u01/app/oracle/product/11.2.0.3
654355    4 drwxrwxr-x   2 oracle   dba          4096 May 19 21:06 /u01/app/oracle/product/11.2.0.3/dbhome_1
654348    4 drwxrwxr-x   2 oracle   dba          4096 May 19 21:06 /u01/app/oraInventory
[root@rac01 ~]#
About these ads

1 Comment »

  1. IMPORTANT: since there is no error checking for changes already made, the script can only be run once !!!. Suggest snapshot created before running script to be able to rollback.

    Comment by mdinh — May 20, 2013 @ 4:55 am | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 318 other followers

%d bloggers like this: