Thinking Out Loud

April 28, 2019

Simplest Automation: Source RAC GI/DB Environment

Filed under: automation,RAC — mdinh @ 1:18 pm

The conundrum I am facing is whether or not to prepend or append db_name for it’s specific environment.

For example, if db_name is hawk, then hawkprod, hawkqa, hawkdev, etc…

If the hosts have the specific environment, e.g. prodhost, qahost, devhost, then isn’t hawk database on qahost a qa database?

Understandably, every organizations have different requirements and conventions as there’s no one size fits all.

One thing to consider is consistency for easy automation.

So there I was, preparing to patch 8-nodes RAC cluster and thinking how can I make this easier and automated.

First thought was to have a consistent method to source RAC GI/DB.

Once that is done, create shell script to apply the patch.

Patching VM can take hours and too lazy to copy and paste all the commands.

Just run the script and be done with it. I could have scripted patching to loop through all instances vs running them individually per host.

[root@racnode-dc1-1 ~]# /media/patch/jan2019_patch_28833531.sh
[root@racnode-dc1-2 ~]# /media/patch/jan2019_patch_28833531.sh

Have I deployed this in real life? Not to the extreme where everything is scripted because there are too many inconsistencies across environments.

Framework and demo.

[oracle@racnode-dc2-1 ~]$ df -h /media/patch/
Filesystem      Size  Used Avail Use% Mounted on
media_patch     3.7T  463G  3.2T  13% /media/patch

[oracle@racnode-dc2-1 ~]$ ll /media/patch/*.env
-rwxrwxrwx 1 vagrant vagrant  180 Mar 26 04:49 /media/patch/asm.env
-rwxrwxrwx 1 vagrant vagrant 1295 Sep 28  2018 /media/patch/bp.env
-rwxrwxrwx 1 vagrant vagrant 1113 Feb  2 16:00 /media/patch/ch_gi_prereq.env
-rwxrwxrwx 1 vagrant vagrant  511 Mar 27 15:22 /media/patch/crap.env
-rwxrwxrwx 1 vagrant vagrant  130 Jan 31 23:16 /media/patch/db.env
-rwxrwxrwx 1 vagrant vagrant  105 Feb 20 19:04 /media/patch/ggs.env
-rwxrwxrwx 1 vagrant vagrant  444 Apr 14 15:45 /media/patch/gi.env
-rwxrwxrwx 1 vagrant vagrant  518 Apr 14 00:49 /media/patch/hawk.env
-rwxrwxrwx 1 vagrant vagrant  944 Apr 22  2018 /media/patch/psu.env

### 12.2 GI

[oracle@racnode-dc2-1 ~]$ . /media/patch/gi.env
The Oracle base has been set to /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/12.2.0.1/grid
ORACLE_HOME=/u01/app/12.2.0.1/grid
Oracle Instance alive for sid "+ASM1"

### 12.2 DB

[oracle@racnode-dc2-1 ~]$ . /media/patch/hawk.env
The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/12.2.0.1/db1
Oracle Instance alive for sid "hawk1"

[oracle@racnode-dc2-1 ~]$ ps -ef|grep pmon
oracle    8756     1  0 14:27 ?        00:00:00 asm_pmon_+ASM1
oracle    9663     1  0 14:28 ?        00:00:00 ora_pmon_hawk1
oracle   14319 12020  0 14:30 pts/0    00:00:00 grep --color=auto pmon

[oracle@racnode-dc2-1 ~]$ ssh racnode-dc2-1
Last login: Sun Apr 28 14:28:59 2019

----------------------------------------
Welcome to racnode-dc2-1
OracleLinux 7.5 x86_64

FQDN: racnode-dc2-1.internal.lab
IP's:
enp0s3: 10.0.2.15
enp0s8: 192.168.7.100
enp0s9: 172.16.7.10

Processor: Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz
#CPU's:    2
Memory:    5708 MB
Kernel:    4.1.12-112.16.4.el7uek.x86_64

----------------------------------------

[oracle@racnode-dc2-1 ~]$ df -h /media/patch/
Filesystem      Size  Used Avail Use% Mounted on
media_patch     3.7T  463G  3.2T  13% /media/patch

[oracle@racnode-dc2-1 ~]$ ll /media/patch/*.env
-rwxrwxrwx 1 vagrant vagrant  180 Mar 26 04:49 /media/patch/asm.env
-rwxrwxrwx 1 vagrant vagrant 1295 Sep 28  2018 /media/patch/bp.env
-rwxrwxrwx 1 vagrant vagrant 1113 Feb  2 16:00 /media/patch/ch_gi_prereq.env
-rwxrwxrwx 1 vagrant vagrant  511 Mar 27 15:22 /media/patch/crap.env
-rwxrwxrwx 1 vagrant vagrant  130 Jan 31 23:16 /media/patch/db.env
-rwxrwxrwx 1 vagrant vagrant  105 Feb 20 19:04 /media/patch/ggs.env
-rwxrwxrwx 1 vagrant vagrant  444 Apr 14 15:45 /media/patch/gi.env
-rwxrwxrwx 1 vagrant vagrant  518 Apr 14 00:49 /media/patch/hawk.env
-rwxrwxrwx 1 vagrant vagrant  944 Apr 22  2018 /media/patch/psu.env

[oracle@racnode-dc2-1 ~]$ . /media/patch/gi.env
The Oracle base has been set to /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/12.2.0.1/grid
ORACLE_HOME=/u01/app/12.2.0.1/grid
Oracle Instance alive for sid "+ASM1"
[oracle@racnode-dc2-1 ~]$ . /media/patch/hawk.env
The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/12.2.0.1/db1
Oracle Instance alive for sid "hawk1"
[oracle@racnode-dc2-1 ~]$

====================================================================================================

[oracle@racnode-dc1-1 ~]$ df -h /media/patch/
Filesystem      Size  Used Avail Use% Mounted on
media_patch     3.7T  463G  3.2T  13% /media/patch

[oracle@racnode-dc1-1 ~]$ ll /media/patch/*.env
-rwxrwxrwx 1 vagrant vagrant  180 Mar 26 04:49 /media/patch/asm.env
-rwxrwxrwx 1 vagrant vagrant 1295 Sep 28  2018 /media/patch/bp.env
-rwxrwxrwx 1 vagrant vagrant 1113 Feb  2 16:00 /media/patch/ch_gi_prereq.env
-rwxrwxrwx 1 vagrant vagrant  511 Mar 27 15:22 /media/patch/crap.env
-rwxrwxrwx 1 vagrant vagrant  130 Jan 31 23:16 /media/patch/db.env
-rwxrwxrwx 1 vagrant vagrant  105 Feb 20 19:04 /media/patch/ggs.env
-rwxrwxrwx 1 vagrant vagrant  444 Apr 14 15:45 /media/patch/gi.env
-rwxrwxrwx 1 vagrant vagrant  518 Apr 14 00:49 /media/patch/hawk.env
-rwxrwxrwx 1 vagrant vagrant  944 Apr 22  2018 /media/patch/psu.env

### 18.3 GI

[oracle@racnode-dc1-1 ~]$ . /media/patch/gi.env
The Oracle base has been set to /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/18.3.0.0/grid
ORACLE_HOME=/u01/18.3.0.0/grid
Oracle Instance alive for sid "+ASM1"

### 12.1 DB

[oracle@racnode-dc1-1 ~]$ . /media/patch/hawk.env
The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/12.1.0.1/db1
Oracle Instance alive for sid "hawk1"

[oracle@racnode-dc1-1 ~]$ ssh racnode-dc1-2
Last login: Wed Apr 24 04:23:30 2019

----------------------------------------
Welcome to racnode-dc1-2
OracleLinux 7.3 x86_64

FQDN: racnode-dc1-2.internal.lab
IP:   10.0.2.15

Processor: Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz
#CPU's:    2
Memory:    5709 MB
Kernel:    4.1.12-61.1.18.el7uek.x86_64

----------------------------------------

[oracle@racnode-dc1-2 ~]$ df -h /media/patch/
Filesystem      Size  Used Avail Use% Mounted on
media_patch     3.7T  463G  3.2T  13% /media/patch

[oracle@racnode-dc1-2 ~]$ ll /media/patch/*.env
-rwxrwxrwx 1 vagrant vagrant  180 Mar 26 04:49 /media/patch/asm.env
-rwxrwxrwx 1 vagrant vagrant 1295 Sep 28  2018 /media/patch/bp.env
-rwxrwxrwx 1 vagrant vagrant 1113 Feb  2 16:00 /media/patch/ch_gi_prereq.env
-rwxrwxrwx 1 vagrant vagrant  511 Mar 27 15:22 /media/patch/crap.env
-rwxrwxrwx 1 vagrant vagrant  130 Jan 31 23:16 /media/patch/db.env
-rwxrwxrwx 1 vagrant vagrant  105 Feb 20 19:04 /media/patch/ggs.env
-rwxrwxrwx 1 vagrant vagrant  444 Apr 14 15:45 /media/patch/gi.env
-rwxrwxrwx 1 vagrant vagrant  518 Apr 14 00:49 /media/patch/hawk.env
-rwxrwxrwx 1 vagrant vagrant  944 Apr 22  2018 /media/patch/psu.env

[oracle@racnode-dc1-2 ~]$ . /media/patch/gi.env
The Oracle base has been set to /u01/app/oracle
ORACLE_SID=+ASM2
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/18.3.0.0/grid
ORACLE_HOME=/u01/18.3.0.0/grid
Oracle Instance alive for sid "+ASM2"

[oracle@racnode-dc1-2 ~]$ . /media/patch/hawk.env
The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk2
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/12.1.0.1/db1
Oracle Instance alive for sid "hawk2"
[oracle@racnode-dc1-2 ~]$ 

====================================================================================================

[oracle@racnode-dc1-1 ~]$ /media/patch/crs_Query.sh
+ . /media/patch/gi.env
++ set +x
The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/18.3.0.0/grid
ORACLE_HOME=/u01/18.3.0.0/grid
Oracle Instance alive for sid "+ASM1"
+ crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [18.0.0.0.0]
+ crsctl query crs softwareversion
Oracle Clusterware version on node [racnode-dc1-1] is [18.0.0.0.0]
+ crsctl query crs softwarepatch
Oracle Clusterware patch level on node racnode-dc1-1 is [2532936542].
+ crsctl query crs releasepatch
Oracle Clusterware release patch level is [2532936542] and the complete list of patches [27908644 27923415 28090523 28090553 28090557 28256701 28435192 28547619 28822489 28864593 28864607 ] have been applied on the local node. The release patch string is [18.5.0.0.0].
+ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [18.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [2532936542].
+ exit
[oracle@racnode-dc1-1 ~]$

====================================================================================================

[oracle@racnode-dc1-2 ~]$ cat /media/patch/gi.env
### Michael Dinh : Mar 26, 2019
### Source RAC GI environment
### Prerequisites for hostname: last char from hostname must be digit
### Allow: prodhost01, racnode-dc1-1
### DisAllow: prod01host
set +x
unset ORACLE_UNQNAME
ORAENV_ASK=NO
h=$(hostname -s)
### Extract last character from hostname to create ORACLE_SID
export ORACLE_SID=+ASM${h:${#h} - 1}
. oraenv <<< $ORACLE_SID
export GRID_HOME=$ORACLE_HOME
env|egrep 'ORA|GRID'
sysresv|tail -1
[oracle@racnode-dc1-2 ~]$

====================================================================================================

[oracle@racnode-dc1-2 ~]$ cat /media/patch/hawk.env
### Michael Dinh : Mar 26, 2019
### Source RAC DB environment
### Prerequisites for hostname: last char from hostname must be digit
### Allow: prodhost01, racnode-dc1-1
### DisAllow: prod01host
set +x
unset GRID_HOME
h=$(hostname -s)
### Extract filename without extension (.env)
ORAENV_ASK=NO
export ORACLE_UNQNAME=$(basename $BASH_SOURCE .env)
### Extract last character from hostname to create ORACLE_SID
export ORACLE_SID=$ORACLE_UNQNAME${h:${#h} - 1}
. oraenv <<< $ORACLE_SID
env|egrep 'ORA|GRID'
sysresv|tail -1
[oracle@racnode-dc1-2 ~]$

In conclusion, convention and consistency are important to avoid complications with automation.

Advertisements

February 21, 2019

Simplest Automation: Use Environment Variables

Filed under: automation,GoldenGate — mdinh @ 1:25 pm

Copy the last five Goldengate trail files from source to destination.

Here are high level steps:

Copy trail with prefix (aa*) to new destination:
1. export OLD_DIRDAT=/media/patch/dirdat
2. export NEW_DIRDAT=/media/swrepo/dirdat
3. export TRAIL_PREFIX=rt*
5. ls -l $NEW_DIRDAT
6. ls $OLD_DIRDAT/$TRAIL_PREFIX | head -5
7. ls $OLD_DIRDAT/$TRAIL_PREFIX | tail -5
8. cp -fv $(ls $OLD_DIRDAT/$TRAIL_PREFIX | tail -5) $NEW_DIRDAT
9. ls -l $NEW_DIRDAT/*

Copy trail with prefix (ab*) to new destination:
export TRAIL_PREFIX=ab*
Repeat steps 5-9

Blog at WordPress.com.