No, this is not another post on how to install RAC on Virtual Box. Instead, it is a post on my mistakes and notes so that you don’t make the same.
Adapter 1 – eth0 (DHCP) / Adapter 2 – eth1 (STATIC IP) / Adapter 3 (STATIC IP) (Private)
Hosts
[root@rac02 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.56.11 rac01 rac01.localdomain 192.168.56.12 rac02 rac02.localdomain 192.168.56.13 rac03 rac03.localdomain 192.168.56.21 rac01-vip rac01-vip.localdomain 192.168.56.22 rac02-vip rac02-vip.localdomain 192.168.56.23 rac03-vip rac03-vip.localdomain 10.10.10.11 rac01-priv rac02-priv.localdomain 10.10.10.12 rac02-priv rac02-priv.localdomain 10.10.10.13 rac03-priv rac03-priv.localdomain
Setup same eth interface across all nodes
[root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 NAME="System eth0" TYPE="Ethernet" DEVICE="eth0" BOOTPROTO=dhcp NM_CONTROLLED="yes" ONBOOT=yes DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no HWADDR=08:00:27:DE:4D:20 PEERDNS=yes PEERROUTES=yes [root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE="eth1" BOOTPROTO=none NM_CONTROLLED="yes" ONBOOT=yes TYPE="Ethernet" IPADDR=192.168.56.12 PREFIX=24 GATEWAY=192.168.56.1 DNS1=8.8.8.8 DOMAIN=localdomain DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth1" HWADDR=08:00:27:EE:48:CF [root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth2 DEVICE="eth2" BOOTPROTO=none NM_CONTROLLED="yes" ONBOOT=yes TYPE="Ethernet" IPADDR=10.10.10.12 PREFIX=8 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth2" HWADDR=08:00:27:FC:83:39
Use generic name for volume group. Otherwise, rac02 clone will have vg-rac01.
[root@rac02 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root
12G 2.0G 8.9G 18% /
tmpfs 1004M 88K 1004M 1% /dev/shm
/dev/sda1 485M 55M 405M 12% /boot
Linux 466G 45G 422G 10% /media/sf_Linux
DNS configuration – dynamic and no change needed when cloning
[root@rac02 ~]# cat /etc/dnsmasq.conf listen-address=127.0.0.1 interface=eth1 strict-order domain-needed bogus-priv no-poll no-hosts local=/localdomain/ addn-hosts=/etc/racdns cache-size=150 [root@rac02 ~]# cat /etc/racdns 192.168.56.11 rac01.localdomain 192.168.56.12 rac02.localdomain 192.168.56.13 rac03.localdomain 192.168.56.31 dinh-scan.localdomain 192.168.56.32 dinh-scan.localdomain 192.168.56.33 dinh-scan.localdomain
Prevent DHCP from modifying /etc/resolv.conf
[root@rac02 ~]# chattr +i /etc/resolv.conf
Multiple DNS
[root@rac02 ~]# cat /etc/resolv.conf nameserver 127.0.0.1 nameserver 8.8.8.8 nameserver 172.16.0.1 search gateway.2wire.net localdomain
Script – I get tired of having to type the same thing over and over again
[root@rac02 ~]# cat t.sh ping -c 3 rac01 ping -c 3 rac02 ping -c 3 rac03 ping -c 3 www.google.com nslookup rac01 nslookup rac02 nslookup rac03 nslookup www.google.com nslookup dinh-scan nslookup dinh-scan nslookup dinh-scan
Verify Network – ping and nslookup
+ ping -c 3 rac01 [root@rac02 ~]# sh -x ./t.sh PING rac01 (192.168.56.11) 56(84) bytes of data. 64 bytes from rac01 (192.168.56.11): icmp_seq=1 ttl=64 time=0.508 ms 64 bytes from rac01 (192.168.56.11): icmp_seq=2 ttl=64 time=0.471 ms 64 bytes from rac01 (192.168.56.11): icmp_seq=3 ttl=64 time=0.565 ms --- rac01 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.471/0.514/0.565/0.046 ms + ping -c 3 rac02 PING rac02 (192.168.56.12) 56(84) bytes of data. 64 bytes from rac02 (192.168.56.12): icmp_seq=1 ttl=64 time=0.038 ms 64 bytes from rac02 (192.168.56.12): icmp_seq=2 ttl=64 time=0.021 ms 64 bytes from rac02 (192.168.56.12): icmp_seq=3 ttl=64 time=0.042 ms --- rac02 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.021/0.033/0.042/0.011 ms + ping -c 3 rac03 PING rac03 (192.168.56.13) 56(84) bytes of data. 64 bytes from rac03 (192.168.56.13): icmp_seq=1 ttl=64 time=0.536 ms 64 bytes from rac03 (192.168.56.13): icmp_seq=2 ttl=64 time=0.438 ms 64 bytes from rac03 (192.168.56.13): icmp_seq=3 ttl=64 time=0.471 ms --- rac03 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.438/0.481/0.536/0.047 ms + ping -c 3 www.google.com PING www.google.com (74.125.224.178) 56(84) bytes of data. 64 bytes from lax02s01-in-f18.1e100.net (74.125.224.178): icmp_seq=1 ttl=51 time=29.5 ms 64 bytes from lax02s01-in-f18.1e100.net (74.125.224.178): icmp_seq=2 ttl=49 time=28.6 ms 64 bytes from lax02s01-in-f18.1e100.net (74.125.224.178): icmp_seq=3 ttl=51 time=27.5 ms --- www.google.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2030ms rtt min/avg/max/mdev = 27.559/28.571/29.517/0.824 ms + nslookup rac01 Server: 127.0.0.1 Address: 127.0.0.1#53 Name: rac01.localdomain Address: 192.168.56.11 + nslookup rac02 Server: 127.0.0.1 Address: 127.0.0.1#53 Name: rac02.localdomain Address: 192.168.56.12 + nslookup rac03 Server: 127.0.0.1 Address: 127.0.0.1#53 Name: rac03.localdomain Address: 192.168.56.13 + nslookup www.google.com Server: 127.0.0.1 Address: 127.0.0.1#53 Non-authoritative answer: Name: www.google.com Address: 74.125.224.179 Name: www.google.com Address: 74.125.224.177 Name: www.google.com Address: 74.125.224.180 Name: www.google.com Address: 74.125.224.176 Name: www.google.com Address: 74.125.224.178 + nslookup dinh-scan Server: 127.0.0.1 Address: 127.0.0.1#53 Name: dinh-scan.localdomain Address: 192.168.56.33 Name: dinh-scan.localdomain Address: 192.168.56.31 Name: dinh-scan.localdomain Address: 192.168.56.32 + nslookup dinh-scan Server: 127.0.0.1 Address: 127.0.0.1#53 Name: dinh-scan.localdomain Address: 192.168.56.31 Name: dinh-scan.localdomain Address: 192.168.56.32 Name: dinh-scan.localdomain Address: 192.168.56.33 + nslookup dinh-scan Server: 127.0.0.1 Address: 127.0.0.1#53 Name: dinh-scan.localdomain Address: 192.168.56.32 Name: dinh-scan.localdomain Address: 192.168.56.33 Name: dinh-scan.localdomain Address: 192.168.56.31 [root@rac02 ~]#
!!! BACKUP!!!
!!! BACKUP!!! I FUBAR the install where ASM disks were listed as MEMBER versus CANDIDATE. !!! BACKUP!!!
!!! BACKUP!!!
Now’s a good time since everything looks good. Next step is to create ASM disks.
Best way to backup, shutdown VM copy ALL folders to new directory. To restore, copy everything back. Does not get easier.
Some times I feel like GUI, some times, I don’t.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\falcon>d:
D:\>cd "VirtualBox VMs"
D:\VirtualBox VMs>dir
Volume in drive D is SOFTWARE
Volume Serial Number is 8A61-FD0C
Directory of D:\VirtualBox VMs
04/24/2013 06:27 PM .
04/24/2013 06:27 PM ..
04/24/2013 07:51 AM asm
04/24/2013 07:51 AM GOLD
04/24/2013 07:37 PM rac01
04/24/2013 07:37 PM rac02
04/24/2013 07:37 PM rac03
0 File(s) 0 bytes
7 Dir(s) 658,327,556,096 bytes free
D:\VirtualBox VMs>cd asm
D:\VirtualBox VMs\asm>dir
Volume in drive D is SOFTWARE
Volume Serial Number is 8A61-FD0C
Directory of D:\VirtualBox VMs\asm
04/24/2013 07:51 AM .
04/24/2013 07:51 AM ..
0 File(s) 0 bytes
2 Dir(s) 658,327,556,096 bytes free
D:\VirtualBox VMs\asm>VBoxManage createhd --filename asm1.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Disk image created. UUID: 65b57fff-26d4-4691-9d29-54924b7a870f
D:\VirtualBox VMs\asm>VBoxManage createhd --filename asm2.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Disk image created. UUID: 345917d0-d033-4ef3-af85-f4a4cd82669f
D:\VirtualBox VMs\asm>VBoxManage createhd --filename asm3.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Disk image created. UUID: 40064c7e-5795-4544-908d-bb19d9545b02
D:\VirtualBox VMs\asm>VBoxManage createhd --filename asm4.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Disk image created. UUID: dc1320c0-34b7-46b7-afca-1ec2b5319aeb
D:\VirtualBox VMs\asm>VBoxManage createhd --filename asm5.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Disk image created. UUID: 6fb56b8d-76e5-43a9-865c-48b5aed5cdac
D:\VirtualBox VMs\asm>VBoxManage storageattach rac01 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
D:\VirtualBox VMs\asm>VBoxManage storageattach rac01 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
D:\VirtualBox VMs\asm>VBoxManage storageattach rac01 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
D:\VirtualBox VMs\asm>VBoxManage storageattach rac01 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable
D:\VirtualBox VMs\asm>VBoxManage storageattach rac01 --storagectl "SATA" --port 5 --device 0 --type hdd --medium asm5.vdi --mtype shareable
D:\VirtualBox VMs\asm>VBoxManage modifyhd asm1.vdi --type shareable
D:\VirtualBox VMs\asm>VBoxManage modifyhd asm2.vdi --type shareable
D:\VirtualBox VMs\asm>VBoxManage modifyhd asm3.vdi --type shareable
D:\VirtualBox VMs\asm>VBoxManage modifyhd asm4.vdi --type shareable
D:\VirtualBox VMs\asm>VBoxManage modifyhd asm5.vdi --type shareable
D:\VirtualBox VMs\asm>
Verify partitions
[root@rac01 ~]# /sbin/partprobe /dev/sdb [root@rac01 ~]# /sbin/partprobe /dev/sdc [root@rac01 ~]# /sbin/partprobe /dev/sdd [root@rac01 ~]# /sbin/partprobe /dev/sde [root@rac01 ~]# /sbin/partprobe /dev/sdf [root@rac01 ~]# ll /dev/sd* brw-rw---- 1 root disk 8, 0 Apr 24 19:44 /dev/sda brw-rw---- 1 root disk 8, 1 Apr 24 19:44 /dev/sda1 brw-rw---- 1 root disk 8, 2 Apr 24 19:44 /dev/sda2 brw-rw---- 1 root disk 8, 16 Apr 24 19:49 /dev/sdb brw-rw---- 1 root disk 8, 17 Apr 24 19:49 /dev/sdb1 brw-rw---- 1 root disk 8, 32 Apr 24 19:49 /dev/sdc brw-rw---- 1 root disk 8, 33 Apr 24 19:49 /dev/sdc1 brw-rw---- 1 root disk 8, 48 Apr 24 19:49 /dev/sdd brw-rw---- 1 root disk 8, 49 Apr 24 19:49 /dev/sdd1 brw-rw---- 1 root disk 8, 64 Apr 24 19:49 /dev/sde brw-rw---- 1 root disk 8, 65 Apr 24 19:49 /dev/sde1 brw-rw---- 1 root disk 8, 80 Apr 24 19:49 /dev/sdf brw-rw---- 1 root disk 8, 81 Apr 24 19:49 /dev/sdf1
[root@rac01 ~]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules [root@rac01 ~]# /sbin/udevadm control --reload-rules [root@rac01 ~]# /sbin/start_udev Starting udev: [ OK ] [root@rac01 ~]# ls -al /dev/asm* brw-rw---- 1 oracle dba 8, 17 Apr 24 19:54 /dev/asm-disk1 brw-rw---- 1 oracle dba 8, 33 Apr 24 19:54 /dev/asm-disk2 brw-rw---- 1 oracle dba 8, 49 Apr 24 19:54 /dev/asm-disk3 brw-rw---- 1 oracle dba 8, 65 Apr 24 19:54 /dev/asm-disk4 brw-rw---- 1 oracle dba 8, 81 Apr 24 19:54 /dev/asm-disk5
OOPS! Need to shutdown rac01
D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable VBoxManage.exe: error: Medium 'D:\VirtualBox VMs\asm\asm1.vdi' is locked for reading by another task VBoxManage.exe: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Medium, interface IMedium, callee IUnknown VBoxManage.exe: error: Context: "COMSETTER(Type)(mediumType)" at line 678 of file VBoxManageStorageController.cpp VBoxManage.exe: error: Failed to set the medium type
Attach shared disks to VM
D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac02 --storagectl "SATA" --port 5 --device 0 --type hdd --medium asm5.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac03 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac03 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac03 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac03 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable D:\VirtualBox VMs\asm>VBoxManage storageattach rac03 --storagectl "SATA" --port 5 --device 0 --type hdd --medium asm5.vdi --mtype shareable
VERIFY ASM DISKS FOR ALL NODES – TRUST ME! YOU DON’T WANT TO FIND OUT AFTER THE FACT.
[oracle@rac01:]/home/oracle $ ls -al /dev/asm* brw-rw---- 1 oracle dba 8, 17 Apr 25 20:34 /dev/asm-disk1 brw-rw---- 1 oracle dba 8, 33 Apr 25 20:34 /dev/asm-disk2 brw-rw---- 1 oracle dba 8, 49 Apr 25 20:34 /dev/asm-disk3 brw-rw---- 1 oracle dba 8, 65 Apr 25 20:34 /dev/asm-disk4 brw-rw---- 1 oracle dba 8, 81 Apr 25 20:34 /dev/asm-disk5 [oracle@rac02:]/home/oracle $ ls -al /dev/asm* brw-rw---- 1 oracle dba 8, 17 Apr 25 20:34 /dev/asm-disk1 brw-rw---- 1 oracle dba 8, 33 Apr 25 20:34 /dev/asm-disk2 brw-rw---- 1 oracle dba 8, 49 Apr 25 20:34 /dev/asm-disk3 brw-rw---- 1 oracle dba 8, 65 Apr 25 20:34 /dev/asm-disk4 brw-rw---- 1 oracle dba 8, 81 Apr 25 20:34 /dev/asm-disk5 [oracle@rac03:]/home/oracle $ ls -al /dev/asm* brw-rw---- 1 oracle dba 8, 17 Apr 25 20:34 /dev/asm-disk1 brw-rw---- 1 oracle dba 8, 33 Apr 25 20:34 /dev/asm-disk2 brw-rw---- 1 oracle dba 8, 49 Apr 25 20:34 /dev/asm-disk3 brw-rw---- 1 oracle dba 8, 65 Apr 25 20:34 /dev/asm-disk4 brw-rw---- 1 oracle dba 8, 81 Apr 25 20:34 /dev/asm-disk5
NOTE: I tried to install everything under oracle and was struggling.
If I am going to create a environment, then why not create it production like!
Using grid and NOT oracle for grid infrastructure instal.
[grid@rac01:]/home/grid $ ll /dev/asm*;id brw-rw---- 1 grid asmadmin 8, 17 Apr 28 14:49 /dev/asm-disk1 brw-rw---- 1 grid asmadmin 8, 33 Apr 28 14:49 /dev/asm-disk2 brw-rw---- 1 grid asmadmin 8, 49 Apr 28 14:49 /dev/asm-disk3 brw-rw---- 1 grid asmadmin 8, 65 Apr 28 14:44 /dev/asm-disk4 brw-rw---- 1 grid asmadmin 8, 81 Apr 28 14:44 /dev/asm-disk5 uid=54322(grid) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54324(asmoper),54325(asmadmin),54326(asmdba) [grid@rac02:]/home/grid $ ll /dev/asm*;id brw-rw---- 1 grid asmadmin 8, 17 Apr 28 14:49 /dev/asm-disk1 brw-rw---- 1 grid asmadmin 8, 33 Apr 28 14:49 /dev/asm-disk2 brw-rw---- 1 grid asmadmin 8, 49 Apr 28 14:49 /dev/asm-disk3 brw-rw---- 1 grid asmadmin 8, 65 Apr 28 14:44 /dev/asm-disk4 brw-rw---- 1 grid asmadmin 8, 81 Apr 28 14:44 /dev/asm-disk5 uid=54322(grid) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54324(asmoper),54325(asmadmin),54326(asmdba) [grid@rac03:]/home/grid $ ll /dev/asm*;id brw-rw---- 1 grid asmadmin 8, 17 Apr 28 14:49 /dev/asm-disk1 brw-rw---- 1 grid asmadmin 8, 33 Apr 28 14:49 /dev/asm-disk2 brw-rw---- 1 grid asmadmin 8, 49 Apr 28 14:49 /dev/asm-disk3 brw-rw---- 1 grid asmadmin 8, 65 Apr 28 14:44 /dev/asm-disk4 brw-rw---- 1 grid asmadmin 8, 81 Apr 28 14:44 /dev/asm-disk5 uid=54322(grid) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54324(asmoper),54325(asmadmin),54326(asmdba)
Just don’t cut and paste and follow. Do some digging and googling.
Pretty cool features, eh.
[oracle@rac01:]/media/sf_Linux/grid/sshsetup $ ./sshUserSetup.sh Please specify a valid and existing cluster configuration file. Either user name or host information is missing Usage ./sshUserSetup.sh -user [ -hosts "" | -hostfile ] [ -advanced ] [ -verify] [ -exverify ] [ -logfile ] [-confirm] [-shared] [-help] [-usePassphrase] [-noPromptPassphrase] [oracle@rac01:]/media/sf_Linux/grid/sshsetup $ /media/sf_Linux/grid/sshsetup/sshUserSetup.sh -user oracle -hosts "rac01 rac02 rac03" -noPromptPassphrase -confirm -advanced -exverify The output of this script is also logged into /tmp/sshUserSetup_2013-04-25-20-47-55.log Hosts are rac01 rac02 rac03 user is oracle Platform:- Linux
/media/sf_Linux/grid/runcluvfy.sh stage -pre crsinst -n rac01,rac02,rac03 = That’s how I found out about
Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... NTP Configuration file check passed No NTP Daemons or Services were found to be running PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s): rac03,rac02,rac01 Clock synchronization check using Network Time Protocol(NTP) failed
mv /etc/ntp.conf /etc/ntp.conf.org
Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed
Face it. Not every system is going to be perfect.
cluvfy is your friend http://docs.oracle.com/cd/E11882_01/rac.112/e16794/cvu.htm
Script it
[root@rac03 ~]# cat crdir.sh mkdir -p /u01/app/oraInventory chown -R grid:oinstall /u01/app/oraInventory mkdir -p /u01/app/grid chown -R grid:oinstall /u01/app/grid mkdir -p /u01/app/11.2.0.3/grid chown -R grid:oinstall /u01/app/11.2.0.3 mkdir -p /u01/app/oracle mkdir -p /u01/app/oracle/cfgtoollogs mkdir -p /u01/app/oracle/product/11.2.0.3/dbhome_1 chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01
!!! Primary group for grid must be dba, not oinstall for my 11.2.0.3 !!!
!!! Must let root.sh complete on the first node, then parallel !!!
SAVE the file
$ You can find the log of this install session at: /u01/app/oraInventory/logs/installActions2013-04-26_08-50-45PM.log
That’s messed up! I build cluster as rac03,02,01
But ran root.sh in order rac03,01,02 – evident by time for /usr/local/bin
[root@rac03 ~]# cat /etc/oratab #Backup file is /u01/app/11.2.0.3/grid/srvm/admin/oratab.bak.rac03 line added by Agent +ASM1:/u01/app/11.2.0.3/grid:N # line added by Agent $ cat /etc/oratab #Backup file is /u01/app/11.2.0.3/grid/srvm/admin/oratab.bak.rac01 line added by Agent +ASM2:/u01/app/11.2.0.3/grid:N # line added by Agent [grid@rac01:]/home/grid $ ll /usr/local/bin/ total 20 -rwxr-xr-x 1 grid root 5778 Apr 26 21:13 coraenv -rwxr-xr-x 1 grid root 2415 Apr 26 21:13 dbhome -rwxr-xr-x 1 grid root 6183 Apr 26 21:13 oraenv [root@rac02 ~]# cat /etc/oratab #Backup file is /u01/app/11.2.0.3/grid/srvm/admin/oratab.bak.rac02 line added by Agent +ASM3:/u01/app/11.2.0.3/grid:N # line added by Agent [root@rac02 ~]# ll /usr/local/bin/ total 20 -rwxr-xr-x 1 grid root 5778 Apr 26 21:14 coraenv -rwxr-xr-x 1 grid root 2415 Apr 26 21:14 dbhome -rwxr-xr-x 1 grid root 6183 Apr 26 21:14 oraenv
What’s my cluster and scan listener?
$ olsnodes -help
Usage: olsnodes [ [-n] [-i] [-s] [-t] [ | -l [-p]] | [-c] ] [-g] [-v]
where
-n print node number with the node name
-p print private interconnect address for the local node
-i print virtual IP address with the node name
print information for the specified node
-l print information for the local node
-s print node status - active or inactive
-t print node type - pinned or unpinned
-g turn on logging
-v Run in debug mode; use at direction of Oracle Support only.
-c print clusterware name
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ olsnodes
rac03
rac01
rac02
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ olsnodes -c
dinh-cluster
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ olsnodes -n -i -s -t
rac03 1 rac03-vip Active Unpinned
rac01 2 rac01-vip Active Unpinned
rac02 3 rac02-vip Active Unpinned
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ olsnodes -n -i -s -t -l -p
rac01 2 10.10.10.11 rac01-vip Active Unpinned
$ srvctl config -help
The SRVCTL config command displays the configuration for the object stored in the OCR.
Usage: srvctl config database [-d [-a] ] [-v]
Usage: srvctl config service -d [-s ] [-v]
Usage: srvctl config nodeapps [-a] [-g] [-s]
Usage: srvctl config vip { -n | -i }
Usage: srvctl config network [-k ]
Usage: srvctl config asm [-a]
Usage: srvctl config listener [-l ] [-a]
Usage: srvctl config scan [-i ]
Usage: srvctl config scan_listener [-i ]
Usage: srvctl config srvpool [-g ]
Usage: srvctl config oc4j
Usage: srvctl config filesystem -d
Usage: srvctl config gns [-a] [-d] [-k] [-m] [-n ] [-p] [-s] [-V] [-q ] [-l] [-v]
Usage: srvctl config cvu
For detailed help on each command and object and its options use:
srvctl -h
Usage (for backward compatibility): srvctl config
Usage (for backward compatibility): srvctl config -p -n
Usage (for backward compatibility): srvctl config -V
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config nodeapps
Network exists: 1/192.168.56.0/255.255.255.0/eth1, type static
VIP exists: /rac01-vip/192.168.56.21/192.168.56.0/255.255.255.0/eth1, hosting node rac01
VIP exists: /rac02-vip/192.168.56.22/192.168.56.0/255.255.255.0/eth1, hosting node rac02
VIP exists: /rac03-vip/192.168.56.23/192.168.56.0/255.255.255.0/eth1, hosting node rac03
GSD exists
ONS exists: Local port 6100, remote port 6200, EM port 2016
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config network
Network exists: 1/192.168.56.0/255.255.255.0/eth1, type static
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config asm
ASM home: /u01/app/11.2.0.3/grid
ASM listener: LISTENER
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config listener
Name: LISTENER
Network: 1, Owner: grid
Home:
End points: TCP:1521
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config scan
SCAN name: dinh-scan, Network: 1/192.168.56.0/255.255.255.0/eth1
SCAN VIP name: scan1, IP: /dinh-scan/192.168.56.33
SCAN VIP name: scan2, IP: /dinh-scan/192.168.56.31
SCAN VIP name: scan3, IP: /dinh-scan/192.168.56.32
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1551
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1551
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1551
$ srvctl config srvpool
Server pool name: Free
Importance: 0, Min: 0, Max: -1
Candidate server names:
Server pool name: Generic
Importance: 0, Min: 0, Max: -1
Candidate server names:
[grid@rac01:+ASM2]/u01/app/11.2.0.3/grid/bin
$ srvctl config cvu
CVU is configured to run once every 360 minutes
Where’s my telnet? – yum install telnet -y
Notice that scan tried all 3 addresses
[root@rac02 ~]# telnet dinh-scan Trying 192.168.56.31... telnet: connect to address 192.168.56.31: Connection refused Trying 192.168.56.33... telnet: connect to address 192.168.56.33: Connection refused Trying 192.168.56.32... telnet: connect to address 192.168.56.32: Connection refused [root@rac02 ~]# telnet dinh-scan 1551 Trying 192.168.56.31... Connected to dinh-scan. Escape character is '^]'. ^] telnet> quit Connection closed.
Shutting down cluster – Good Night
[root@rac03 ~]# /u01/app/11.2.0.3/grid/bin/crsctl stop cluster -all CRS-2673: Attempting to stop 'ora.crsd' on 'rac03' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac03' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac03' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac03' CRS-2673: Attempting to stop 'ora.oc4j' on 'rac03' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac03' CRS-2673: Attempting to stop 'ora.cvu' on 'rac03' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.rac03.vip' on 'rac03' CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac03' CRS-2677: Stop of 'ora.scan3.vip' on 'rac03' succeeded CRS-2677: Stop of 'ora.rac03.vip' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.crsd' on 'rac02' CRS-2673: Attempting to stop 'ora.crsd' on 'rac01' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac02' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac02' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac01' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac01' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac01' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac01' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac02' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac02' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac01' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.rac02.vip' on 'rac02' CRS-2677: Stop of 'ora.scan1.vip' on 'rac01' succeeded CRS-2677: Stop of 'ora.rac02.vip' on 'rac02' succeeded CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.rac01.vip' on 'rac01' CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac02' CRS-2677: Stop of 'ora.rac01.vip' on 'rac01' succeeded CRS-2677: Stop of 'ora.scan2.vip' on 'rac02' succeeded CRS-2677: Stop of 'ora.oc4j' on 'rac03' succeeded CRS-2677: Stop of 'ora.cvu' on 'rac03' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac03' CRS-2677: Stop of 'ora.asm' on 'rac03' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac01' CRS-2677: Stop of 'ora.DATA.dg' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac02' CRS-2677: Stop of 'ora.asm' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac01' CRS-2677: Stop of 'ora.asm' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac02' CRS-2677: Stop of 'ora.ons' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac01' CRS-2677: Stop of 'ora.net1.network' on 'rac01' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac01' has completed CRS-2677: Stop of 'ora.crsd' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac01' CRS-2673: Attempting to stop 'ora.evmd' on 'rac01' CRS-2673: Attempting to stop 'ora.asm' on 'rac01' CRS-2677: Stop of 'ora.ons' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac02' CRS-2677: Stop of 'ora.net1.network' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac03' CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac02' has completed CRS-2677: Stop of 'ora.ons' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac03' CRS-2677: Stop of 'ora.net1.network' on 'rac03' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac03' has completed CRS-2677: Stop of 'ora.crsd' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac02' CRS-2673: Attempting to stop 'ora.evmd' on 'rac02' CRS-2673: Attempting to stop 'ora.asm' on 'rac02' CRS-2677: Stop of 'ora.crsd' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac03' CRS-2673: Attempting to stop 'ora.evmd' on 'rac03' CRS-2673: Attempting to stop 'ora.asm' on 'rac03' CRS-2677: Stop of 'ora.evmd' on 'rac01' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac02' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac03' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac01' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac02' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac03' succeeded CRS-2677: Stop of 'ora.asm' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac01' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac01' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac01' CRS-2677: Stop of 'ora.asm' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac02' CRS-2677: Stop of 'ora.asm' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac03' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac02' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac02' CRS-2677: Stop of 'ora.cssd' on 'rac01' succeeded CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac03' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac03' CRS-2677: Stop of 'ora.cssd' on 'rac02' succeeded CRS-2677: Stop of 'ora.cssd' on 'rac03' succeeded [root@rac03 ~]#
CRAP! CRAP! CRAP!
I proceed to install database software and received insufficient space error.
There was only 4GB left and that not enough since software requires 4.7GB if memory serves me correctly.
After some digging, yum was taking up some space.
Next time, allocate 16GB and not 12GB
yum clean all
[root@rac01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root
12G 6.9G 4.0G 64% /
tmpfs 1004M 212M 792M 22% /dev/shm
/dev/sda1 485M 55M 405M 12% /boot
Linux 466G 45G 422G 10% /media/sf_Linux
[root@rac01 ~]# cd /var/cache
[root@rac01 cache]# du -sh *
4.0K abrt-di
12K cups
72K fontconfig
4.0K foomatic
16K gdm
328K hald
40K ldconfig
544K man
96K PackageKit
12K rpcbind
1.4G yum
[root@rac01 cache]# yum clean all
Loaded plugins: refresh-packagekit, security
Cleaning repos: ol6_UEK_latest ol6_latest
Cleaning up Everything
[root@rac01 cache]# du -sh *
4.0K abrt-di
12K cups
72K fontconfig
4.0K foomatic
16K gdm
328K hald
40K ldconfig
544K man
96K PackageKit
12K rpcbind
36K yum
[root@rac01 cache]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root
12G 5.6G 5.3G 52% /
tmpfs 1004M 212M 792M 22% /dev/shm
/dev/sda1 485M 55M 405M 12% /boot
Linux 466G 45G 422G 10% /media/sf_Linux
[root@rac01 cache]#
From Metalink Step By Step guide for installing Oracle RAC 11gR2 (11.2.0.2) on Linux
#/usr/sbin/useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper grid
#/usr/sbin/useradd -u 502 -g oinstall -G dba,asmdba oracle
Even Document 1442768.1 – RACGuide: 11.2.0.3 RAC Installation on Linux [Video] add oinstall as primary.
PRIMARY GROUP NEEDS TO BE DBA for my 11.2.0.3 – VERY STRANGE.
[oracle@rac01:]/home/oracle $ id uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [oracle@rac01:]/home/oracle $ logout [root@rac01 cache]# /tmp/CVU_11.2.0.3.0_oracle/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.response Enable file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.enable Log file location: /tmp/CVU_11.2.0.3.0_oracle/orarun.log uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [root@rac01 cache]# su - oracle [oracle@rac01:]/home/oracle $ id uid=54321(oracle) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54326(asmdba) [oracle@rac02:]/home/oracle $ id uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [oracle@rac02:]/home/oracle $ logout [root@rac02 ~]# /tmp/CVU_11.2.0.3.0_oracle/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.response Enable file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.enable Log file location: /tmp/CVU_11.2.0.3.0_oracle/orarun.log uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [root@rac02 ~]# su - oracle [oracle@rac02:]/home/oracle $ id uid=54321(oracle) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54326(asmdba) [oracle@rac03:]/home/oracle $ id uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [oracle@rac03:]/home/oracle $ logout [root@rac03 ~]# /tmp/CVU_11.2.0.3.0_oracle/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.response Enable file being used is :/tmp/CVU_11.2.0.3.0_oracle/fixup.enable Log file location: /tmp/CVU_11.2.0.3.0_oracle/orarun.log uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf),54326(asmdba) [root@rac03 ~]# su - oracle [oracle@rac03:]/home/oracle $ id uid=54321(oracle) gid=54322(dba) groups=54322(dba),54321(oinstall),54323(vboxsf),54326(asmdba)
LOGOUT and LOGIN again; otherwise, error as shown below.
When cloning VM, Don’t forget to change hostname and add to /etc/sysconfig/network
NOZEROCONF=yes
Run raccheck and modify accordingly.
WARNING => avahi-daemon process is running
CSSD Fails to Join the Cluster After Private Network Recovered if avahi Daemon is up and Running [ID 1501093.1] Modified:Dec 5, 2012
# /etc/init.d/avahi-daemon stop
# /sbin/chkconfig avahi-daemon off
Don’t forget to change hostname and add to /etc/sysconfig/network
NOZEROCONF=yes
WARNING => kernel parameter rp_filter is set to 1
rp_filter for multiple private interconnects and Linux Kernel 2.6.32+ [ID 1286796.1] Modified:Dec 3, 2012
# grep rp_filter /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 1
Changed to net.ipv4.conf.default.rp_filter = 0
WARNING => kernel.shmmax parameter is NOT configured according to recommendation
# grep kernel.shmmax /etc/sysctl.conf
# oracle-rdbms-server-11gR2-preinstall setting for kernel.shmmax is 4398046511104 on x86_64
# oracle-rdbms-server-11gR2-preinstall setting for kernel.shmmax is 4294967295 on i386
kernel.shmmax = 4398046511104
To be continued …
Thinking Out Loud 
Hi,
I have followed this wonderful post to get a 2-node RAC working but after a week, I am still stuck at being able to ping the other node.
Firewall is disabled on both nodes but I am still getting:
[root@dap-rac2 etc]# ping -c 3 dap-rac1
PING dap-rac1.localdomain (169.254.36.101) 56(84) bytes of data.
From dap-rac2.localdomain (169.254.36.102) icmp_seq=1 Destination Host Unreachable
From dap-rac2.localdomain (169.254.36.102) icmp_seq=2 Destination Host Unreachable
From dap-rac2.localdomain (169.254.36.102) icmp_seq=3 Destination Host Unreachable
— dap-rac1.localdomain ping statistics —
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3000ms
pipe 3
Please help!
Comment by dapogbite — September 18, 2013 @ 2:02 pm |
Please check your gateway. Are you able to ping dap-rac2 from dap-rac1?
Comment by mdinh — September 23, 2013 @ 1:33 am |
Could you please seme me the details of the ifconfig -a from your nodes. I reckon it is an issue with nework config but I can’t figure out what the problems is.
Thanks and best regards
Comment by dapo gbite — October 3, 2013 @ 12:45 pm |
I hope this helps you.
Adapter 1 (NAT) – eth0 (DHCP)
Adapter 2 (Host Only) – eth1 (STATIC IP)
Adapter 3 (Internal) – eth2 (STATIC IP) (Private)
[root@rac01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=”eth0″
NAME=”System eth0″
BOOTPROTO=”dhcp”
TYPE=”Ethernet”
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
HWADDR=08:00:27:14:4F:98
[root@rac01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=”eth1″
NAME=”System eth1″
TYPE=”Ethernet”
BOOTPROTO=none
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PREFIX=24
GATEWAY=192.168.56.1
HWADDR=08:00:27:F9:E0:54
IPADDR=192.168.56.11
[root@rac01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=”eth2″
NAME=”System eth2″
TYPE=”Ethernet”
BOOTPROTO=none
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PREFIX=8
HWADDR=08:00:27:1E:77:37
IPADDR=10.10.10.11
[root@rac01 ~]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 08:00:27:14:4F:98
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe14:4f98/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2360 (2.3 KiB) TX bytes:2100 (2.0 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:F9:E0:54
inet addr:192.168.56.11 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fef9:e054/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:369 errors:0 dropped:0 overruns:0 frame:0
TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:36591 (35.7 KiB) TX bytes:28065 (27.4 KiB)
eth2 Link encap:Ethernet HWaddr 08:00:27:1E:77:37
inet addr:10.10.10.11 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::a00:27ff:fe1e:7737/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:240 (240.0 b) TX bytes:830 (830.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:382 (382.0 b) TX bytes:382 (382.0 b)
[root@rac01 ~]#
[root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=”eth0″
NAME=”System eth0″
BOOTPROTO=”dhcp”
TYPE=”Ethernet”
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
HWADDR=08:00:27:36:3A:C9
[root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=”eth1″
NAME=”System eth1″
TYPE=”Ethernet”
BOOTPROTO=none
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PREFIX=24
GATEWAY=192.168.56.1
HWADDR=08:00:27:BA:72:9B
IPADDR=192.168.56.12
[root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=”eth2″
NAME=”System eth2″
TYPE=”Ethernet”
BOOTPROTO=none
NM_CONTROLLED=”yes”
ONBOOT=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
PREFIX=8
HWADDR=08:00:27:39:87:19
IPADDR=10.10.10.12
[root@rac02 ~]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 08:00:27:36:3A:C9
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe36:3ac9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2360 (2.3 KiB) TX bytes:2100 (2.0 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:BA:72:9B
inet addr:192.168.56.12 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:feba:729b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:277 errors:0 dropped:0 overruns:0 frame:0
TX packets:167 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:27515 (26.8 KiB) TX bytes:22591 (22.0 KiB)
eth2 Link encap:Ethernet HWaddr 08:00:27:39:87:19
inet addr:10.10.10.12 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::a00:27ff:fe39:8719/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:360 (360.0 b) TX bytes:746 (746.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:382 (382.0 b) TX bytes:382 (382.0 b)
[root@rac02 ~]#
[root@rac01 ~]# ping -c 3 rac01
PING rac01 (192.168.56.11) 56(84) bytes of data.
64 bytes from rac01 (192.168.56.11): icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from rac01 (192.168.56.11): icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from rac01 (192.168.56.11): icmp_seq=3 ttl=64 time=0.039 ms
— rac01 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2011ms
rtt min/avg/max/mdev = 0.028/0.036/0.041/0.005 ms
[root@rac01 ~]# ping -c 3 rac02
PING rac02 (192.168.56.12) 56(84) bytes of data.
64 bytes from rac02 (192.168.56.12): icmp_seq=1 ttl=64 time=0.360 ms
64 bytes from rac02 (192.168.56.12): icmp_seq=2 ttl=64 time=0.577 ms
64 bytes from rac02 (192.168.56.12): icmp_seq=3 ttl=64 time=0.510 ms
— rac02 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2008ms
rtt min/avg/max/mdev = 0.360/0.482/0.577/0.092 ms
[root@rac01 ~]#
[root@rac02 ~]# ping -c 3 rac01
PING rac01 (192.168.56.11) 56(84) bytes of data.
64 bytes from rac01 (192.168.56.11): icmp_seq=1 ttl=64 time=0.525 ms
64 bytes from rac01 (192.168.56.11): icmp_seq=2 ttl=64 time=0.451 ms
64 bytes from rac01 (192.168.56.11): icmp_seq=3 ttl=64 time=0.447 ms
— rac01 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2053ms
rtt min/avg/max/mdev = 0.447/0.474/0.525/0.040 ms
[root@rac02 ~]# ping -c 3 rac02
PING rac02 (192.168.56.12) 56(84) bytes of data.
64 bytes from rac02 (192.168.56.12): icmp_seq=1 ttl=64 time=0.112 ms
64 bytes from rac02 (192.168.56.12): icmp_seq=2 ttl=64 time=0.039 ms
64 bytes from rac02 (192.168.56.12): icmp_seq=3 ttl=64 time=0.038 ms
— rac02 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.038/0.063/0.112/0.034 ms
[root@rac02 ~]#
Comment by mdinh — October 4, 2013 @ 10:56 pm |
Thanks and I am really grateful for your time. I have compared and with the little change I made, I can now ping the node’s public IPs but I cannot ping the private IPs
I have noticed this difference on my rac2 but don’t know where to change or where it went wrong:
eth0 Link encap:Ethernet HWaddr 08:00:27:BD:5C:66
inet6 addr: fe80::a00:27ff:febd:5c66/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:720 (720.0 b) TX bytes:5508 (5.3 KiB)
This line is missing:
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
I reckon that may be causing the problem
Comment by dapo gbite — October 8, 2013 @ 1:12 pm
I don’t think private IPs are ping-able. Don’t have access to my system at the moment.
[root@rac02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth# to review eth configuration.
Comment by mdinh — October 13, 2013 @ 12:53 pm |
Thanks so much.
I made a 3rd clone and that works. I could have messed up the 2nd node somehow
With your help and others mainly oraclebase.com, I have now installed my 2 node rac which I can now explore and hopefully, it will boost my chance for getting another job.
I followed different notes which didn’t work for me for reasons which are network related but yours….having NAT,Host-only and internal adapter with the IPs appears to be the one that worked for me.
Regards
Comment by dapo gbite — October 16, 2013 @ 3:10 pm |
Thanks for the feedback and I am glad that it helped.
Comment by mdinh — October 21, 2013 @ 1:43 pm |