Thinking Out Loud

March 10, 2019

Oracle Resources: VirtualBox, Vagrant, Linux, Docker, Database, Clusterware, GoldenGate, and More

Filed under: oracle — mdinh @ 2:08 am

Oracle Clusterware

Oracle VM VirtualBox

Oracle Linux Vagrant boxes (Might consider building a customized one.)

Oracle Vagrant configuration

Oracle Linux Download

Official Docker configurations

Oracle Database on Docker

Oracle GoldenGate on Docker

Oracle Linux Images for Hands-On Labs

Pre-Built Developer VMs for Oracle VM VirtualBox

VM Virtual Box for Oracle Enterprise Manager Cloud Control 13c Release 1 (13.1.0.0)

VM Virtual Box for Oracle Enterprise Manager Cloud Control 13c Release 2 (13.2.0.0)

Oracle Enterprise Manager Downloads

 

February 28, 2019

ORA-17503: ksfdopn:10 Failed to open spfile

Filed under: 12c,oracle — mdinh @ 7:21 pm
[oracle@racnode-dc1-2 dbs]$ srvctl start database -d hawk
PRCR-1079 : Failed to start resource ora.hawk.db
CRS-5017: The resource action "ora.hawk.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/HAWK/spfilehawk.ora'
ORA-17503: ksfdopn:10 Failed to open file +DATA/HAWK/spfilehawk.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 54321 (oinstall), current egid = 54322 (dba)
. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/racnode-dc1-2/crs/trace/crsd_oraagent_oracle.trc".

CRS-2674: Start of 'ora.hawk.db' on 'racnode-dc1-2' failed
CRS-5017: The resource action "ora.hawk.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/HAWK/spfilehawk.ora'
ORA-17503: ksfdopn:10 Failed to open file +DATA/HAWK/spfilehawk.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 54321 (oinstall), current egid = 54322 (dba)
. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/racnode-dc1-1/crs/trace/crsd_oraagent_oracle.trc".

CRS-2674: Start of 'ora.hawk.db' on 'racnode-dc1-1' failed
CRS-2632: There are no more servers to try to place resource 'ora.hawk.db' on that would satisfy its placement policy

Starting from sqlplus did not help either and why does it even matter?

[oracle@racnode-dc1-2 dbs]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Thu Feb 28 18:11:58 2019

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to an idle instance.

18:11:59 SYS @ hawk2:>startup;
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/HAWK/spfilehawk.ora'
ORA-17503: ksfdopn:10 Failed to open file +DATA/HAWK/spfilehawk.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 54321 (oinstall), current egid = 54322 (dba)
18:12:01 SYS @ hawk2:>exit
Disconnected
[oracle@racnode-dc1-2 dbs]$

SRVCTL Fails to Start Instance with ORA-17503 ORA-27303 But sqlplus Startup is Fine [1322959.1]

Even though the situation did not match support note, the solution provided did work

[oracle@racnode-dc1-2 dbs]$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54318(asmdba),54322(dba),54323(backupdba),54324(oper),54325(dgdba),54326(kmdba)

[oracle@racnode-dc1-2 dbs]$ ls -l $GRID_HOME/bin/oracle
-rwxrwxr-x 1 oracle oinstall 292020952 Feb 27 22:57 /u01/app/12.1.0.1/grid/bin/oracle

[oracle@racnode-dc1-2 dbs]$ chmod 6751 $GRID_HOME/bin/oracle
[oracle@racnode-dc1-2 dbs]$ ls -l $GRID_HOME/bin/oracle
-rwsr-s--x 1 oracle oinstall 292020952 Feb 27 22:57 /u01/app/12.1.0.1/grid/bin/oracle

[oracle@racnode-dc1-2 dbs]$ ls -l $ORACLE_HOME/bin/oracle
-rwxrwsr-x 1 oracle dba 324409192 Feb 27 22:51 /u01/app/oracle/12.1.0.1/db1/bin/oracle
[oracle@racnode-dc1-2 dbs]$

==========================================================================================

[oracle@racnode-dc1-1 dbs]$ ls -l $GRID_HOME/bin/oracle
-rwxrwxr-x 1 oracle oinstall 292020952 Feb 27 21:41 /u01/app/12.1.0.1/grid/bin/oracle

[oracle@racnode-dc1-1 dbs]$ chmod 6751 $GRID_HOME/bin/oracle
[oracle@racnode-dc1-1 dbs]$ ls -l $GRID_HOME/bin/oracle
-rwsr-s--x 1 oracle oinstall 292020952 Feb 27 21:41 /u01/app/12.1.0.1/grid/bin/oracle

[oracle@racnode-dc1-1 dbs]$ ls -l $ORACLE_HOME/bin/oracle
-rwxrwsr-x 1 oracle dba 324409192 Feb 27 21:35 /u01/app/oracle/12.1.0.1/db1/bin/oracle
[oracle@racnode-dc1-1 dbs]$

[oracle@racnode-dc1-1 dbs]$ srvctl start database -d hawk
[oracle@racnode-dc1-1 dbs]$ srvctl status database -d hawk -v
Instance hawk1 is running on node racnode-dc1-1. Instance status: Open.
Instance hawk2 is running on node racnode-dc1-2. Instance status: Open.
[oracle@racnode-dc1-1 dbs]$

[oracle@racnode-dc1-1 dbs]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.DATA.dg
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.FRA.dg
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.asm
               ONLINE  ONLINE       racnode-dc1-1            Started,STABLE
               ONLINE  ONLINE       racnode-dc1-2            Started,STABLE
ora.net1.network
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.ons
               ONLINE  ONLINE       racnode-dc1-1            STABLE
               ONLINE  ONLINE       racnode-dc1-2            STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       racnode-dc1-1            169.254.203.248 172.
                                                             16.9.10,STABLE
ora.cvu
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.hawk.db
      1        ONLINE  ONLINE       racnode-dc1-1            Open,STABLE
      2        ONLINE  ONLINE       racnode-dc1-2            Open,STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       racnode-dc1-1            Open,STABLE
ora.oc4j
      1        OFFLINE OFFLINE                               STABLE
ora.racnode-dc1-1.vip
      1        ONLINE  ONLINE       racnode-dc1-1            STABLE
ora.racnode-dc1-2.vip
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       racnode-dc1-2            STABLE
--------------------------------------------------------------------------------
[oracle@racnode-dc1-1 dbs]$

February 13, 2019

Source Oracle Environment Easily

Filed under: oracle,shell scripting — mdinh @ 3:50 am

I have been patching a lot lately and wanted a fast and easy method to source Oracle environment.

The objective is to copy, paste from action plan vs having to selectively copy, edit, paste.

Example: . /media/patch/gi.env vs . oraenv — +ASM[n]

Started by creating gi.env which will be used to source GI for all RAC hosts.

You are probably thinking, isn’t it a PITA to have to edit and maintain all the gi.env per host, e.g. 6 nodes RAC cluster?

Rightfully so and it’s a PITA unless it’s dynamic.

There is one requirement: host# = instance#

Hence, +ASM1 is running on host05 will not work.

Next step would probably be to script the tasks.

DEMO1:

[oracle@racnode-dc1-1 ~]$ ps -ef|grep [p]mon
oracle 10818 1 0 03:58 ? 00:00:00 asm_pmon_+ASM1
oracle 11456 1 0 03:58 ? 00:00:00 ora_pmon_hawk1
oracle 11763 1 0 03:58 ? 00:00:00 mdb_pmon_-MGMTDB

[oracle@racnode-dc1-1 ~]$ . /media/patch/gi.env
ORACLE_SID = [+ASM1] ? The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/12.2.0.1/grid
ORACLE_HOME=/u01/app/12.2.0.1/grid
Oracle Instance alive for sid “+ASM1”

[oracle@racnode-dc1-1 ~]$ . /media/patch/hawk.env
ORACLE_SID = [+ASM1] ? The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/11.2.0.4/db1
Oracle Instance alive for sid “hawk1”

[oracle@racnode-dc1-1 ~]$ srvctl status database -d $ORACLE_UNQNAME
Instance hawk1 is running on node racnode-dc1-1
Instance hawk2 is running on node racnode-dc1-2

[oracle@racnode-dc1-1 ~]$ df -h /media/patch/
Filesystem Size Used Avail Use% Mounted on
media_patch 3.7T 413G 3.3T 12% /media/patch

[oracle@racnode-dc1-1 ~]$

[oracle@racnode-dc1-2 ~]$ ps -ef|grep [p]mon
oracle 7339 1 0 03:56 ? 00:00:00 asm_pmon_+ASM2
oracle 8904 1 0 03:57 ? 00:00:00 ora_pmon_hawk2

[oracle@racnode-dc1-2 ~]$ . /media/patch/gi.env
ORACLE_SID = [oracle] ? The Oracle base has been set to /u01/app/oracle
ORACLE_SID=+ASM2
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/12.2.0.1/grid
ORACLE_HOME=/u01/app/12.2.0.1/grid
Oracle Instance alive for sid “+ASM2”

[oracle@racnode-dc1-2 ~]$ . /media/patch/hawk.env
ORACLE_SID = [+ASM2] ? The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk2
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/11.2.0.4/db1
Oracle Instance alive for sid “hawk2”

[oracle@racnode-dc1-2 ~]$ srvctl status database -d $ORACLE_UNQNAME
Instance hawk1 is running on node racnode-dc1-1
Instance hawk2 is running on node racnode-dc1-2

[oracle@racnode-dc1-2 ~]$ cat /media/patch/gi.env
set +x
unset ORACLE_UNQNAME
h=$(hostname -s)
n=1
. oraenv <<< +ASM${h:${#h} – $n}
export GRID_HOME=$ORACLE_HOME
env|egrep ‘ORACLE|GRID’
sysresv|tail -1

[oracle@racnode-dc1-2 ~]$

[oracle@racnode-dc1-2 ~]$ cat /media/patch/hawk.env
set +x
h=$(hostname -s)
n=1
export ORACLE_UNQNAME=hawk
. oraenv <<< $ORACLE_UNQNAME${h:${#h} – $n}
env|grep ORACLE
sysresv|tail -1
[oracle@racnode-dc1-2 ~]$

DEMO2:

[oracle@racnode-dc1-1 ~]$ export PATCH_TOP_DIR=/u01/stage/patch/Jan2019
[oracle@racnode-dc1-1 ~]$
[oracle@racnode-dc1-1 ~]$ . /media/patch/gi.env
ORACLE_SID = [+ASM1] ? The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/12.2.0.1/grid
ORACLE_HOME=/u01/app/12.2.0.1/grid
Oracle Instance alive for sid “+ASM1”
[oracle@racnode-dc1-1 ~]$
[oracle@racnode-dc1-1 ~]$ export PREPATCH_LOG=$PATCH_TOP_DIR/`echo $ORACLE_HOME | awk -F/ ‘{print $NF}’`_prepatch_”$(hostname -s)”_lsinv.log
[oracle@racnode-dc1-1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail > $PREPATCH_LOG; echo $?
0
[oracle@racnode-dc1-1 ~]$ ls -l $PREPATCH_LOG
-rw-r–r– 1 oracle oinstall 205889 Feb 13 04:41 /u01/stage/patch/Jan2019/grid_prepatch_racnode-dc1-1_lsinv.log
[oracle@racnode-dc1-1 ~]$
[oracle@racnode-dc1-1 ~]$ . /media/patch/hawk.env
ORACLE_SID = [+ASM1] ? The Oracle base remains unchanged with value /u01/app/oracle
ORACLE_UNQNAME=hawk
ORACLE_SID=hawk1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/11.2.0.4/db1
Oracle Instance alive for sid “hawk1”
[oracle@racnode-dc1-1 ~]$
[oracle@racnode-dc1-1 ~]$ export PREPATCH_LOG=$PATCH_TOP_DIR/`echo $ORACLE_HOME | awk -F/ ‘{print $NF}’`_prepatch_”$(hostname -s)”_lsinv.log
[oracle@racnode-dc1-1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail > $PREPATCH_LOG; echo $?
0
[oracle@racnode-dc1-1 ~]$ ls -l $PREPATCH_LOG
-rw-r–r– 1 oracle oinstall 118180 Feb 13 04:41 /u01/stage/patch/Jan2019/db1_prepatch_racnode-dc1-1_lsinv.log
[oracle@racnode-dc1-1 ~]$

 

 

December 21, 2018

Find Database Growth Using OEM Repository

Filed under: 12.2,oracle — mdinh @ 2:20 pm

Typically, what as been done is to schedule job for each database to collect database growth.

This may be problematic as it can be forgotten when new databases are created versus the likelihood of forgetting to add database to monitoring for OEM.

EM12c, EM13c : Querying the Repository Database for Building Reports using Metric Information (Doc ID 2347253.1)

Those raw data are inserted in various tables like EM_METRIC_VALUES for example. 
EM aggregates those management data by hour and by day. 
Those raw data are kept 7 days; the one hour aggregated data are kept 31 days, while one day aggregated data are kept one year.

How to obtain the Historical Database Total Used and Allocated Size from OEM Repository

The above blog post provided a good starting point.

This post is using query to collect database size (metric_name=’DATABASE_SIZE’) vs tablespace size (metric_name=’tbspAllocation’) to avoid having to sum all tablespaces to determine database size.

OMS: 13.2.0 and EMREP DB: 12.2.0

Comparison for METRIC_COLUMN between DATABASE_SIZE and tbspAllocation.

For tbspAllocation, the size was not clear and did not research further but it does appear to be GB.

SQL> select distinct metric_name, METRIC_COLUMN from sysman.mgmt$metric_daily where metric_name='tbspAllocation' order by 1;

METRIC_NAME                                                      METRIC_COLUMN
---------------------------------------------------------------- ----------------------------------------------------------------
tbspAllocation                                                   spaceUsed
tbspAllocation                                                   spaceAllocated

SQL> select distinct METRIC_COLUMN from sysman.mgmt$metric_daily WHERE metric_name='DATABASE_SIZE';

METRIC_COLUMN
----------------------------------------------------------------
ALLOCATED_GB
USED_GB

TARGET_TYPE used (not all results presented):

SQL> select distinct target_type from sysman.mgmt$metric_daily order by 1;

TARGET_TYPE
----------------------------------------------------------------
oracle_database
oracle_pdb
rac_database

METRIC_NAME used (not all results presented):

SQL> select distinct metric_name from sysman.mgmt$metric_daily order by 1;

METRIC_NAME
----------------------------------------------------------------
DATABASE_SIZE
tbspAllocation

DEMO:

SQL> @dbsize.sql
SQL> -- Michael Dinh : Dec 20, 2018
SQL> set echo off
Enter value for 1: perf

TARGET_NAME                                        TARGET_TYPE     MONTH_DT  USED_GB ALLOCATED_GB PREVIOUS_MONTH DIFF_USED_GB
-------------------------------------------------- --------------- --------- ------- ------------ -------------- ------------
xxxxperf                                           rac_database    01-MAR-18  2698.6       3526.8
                                                   rac_database    01-APR-18  2709.9       3526.8         2698.6        11.31
                                                   rac_database    01-MAY-18  2728.8       3526.8         2709.9        18.86
                                                   rac_database    01-JUN-18  2735.4       3548.8         2728.8         6.61
                                                   rac_database    01-JUL-18  2746.4       3548.8         2735.4        11.01
                                                   rac_database    01-AUG-18  2758.7       3548.8         2746.4        12.27
                                                   rac_database    01-SEP-18  2772.5       3548.8         2758.7        13.82
                                                   rac_database    01-OCT-18  4888.8       6207.8         2772.5       2116.3
                                                   rac_database    01-NOV-18  4647.8       6207.8         4888.8         -241
                                                   rac_database    01-DEC-18  3383.2       6207.8         4647.8        -1265
yyyyperf                                           oracle_database 01-MAR-18   63.07       395.58
                                                   oracle_database 01-APR-18   63.19       395.58          63.07          .12
                                                   oracle_database 01-MAY-18   64.33       395.58          63.19         1.14
                                                   oracle_database 01-JUN-18   64.81       395.58          64.33          .48
                                                   oracle_database 01-JUL-18    65.1       395.58          64.81          .29
                                                   oracle_database 01-AUG-18   65.22       395.58           65.1          .12
                                                   oracle_database 01-SEP-18   65.79       395.58          65.22          .57
                                                   oracle_database 01-OCT-18   68.18       395.58          65.79         2.39
                                                   oracle_database 01-NOV-18   75.79       395.72          68.18         7.61
                                                   oracle_database 01-DEC-18    80.4       395.72          75.79         4.61

29 rows selected.

SQL> @dbsize
SQL> -- Michael Dinh : Dec 20, 2018
SQL> set echo off
Enter value for 1: *

TARGET_NAME                                        TARGET_TYPE     MONTH_DT  USED_GB ALLOCATED_GB PREVIOUS_MONTH DIFF_USED_GB
-------------------------------------------------- --------------- --------- ------- ------------ -------------- ------------
CDByyyy_xxxxxxxxxxxxxxxxxxxxxxxxxx_CDBROOT         oracle_pdb      01-MAR-18    7.96        94.73
                                                   oracle_pdb      01-APR-18    3.44        94.73           7.96        -4.52
                                                   oracle_pdb      01-MAY-18   12.26        95.07           3.44         8.82
                                                   oracle_pdb      01-JUN-18   76.18        95.12          12.26        63.92
                                                   oracle_pdb      01-JUL-18   70.87        95.15          76.18        -5.31
                                                   oracle_pdb      01-AUG-18   77.63        95.15          70.87         6.76
                                                   oracle_pdb      01-SEP-18     4.9        95.15          77.63       -72.73
                                                   oracle_pdb      01-OCT-18       4        95.15            4.9          -.9
                                                   oracle_pdb      01-NOV-18   41.34        95.15              4        37.34
                                                   oracle_pdb      01-DEC-18   33.52        95.15          41.34        -7.82
CDByyyy_xxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxPDB        oracle_pdb      01-MAR-18  1610.6         2571
                                                   oracle_pdb      01-APR-18  1644.9         2571         1610.6        34.27
                                                   oracle_pdb      01-MAY-18  1659.3       2571.3         1644.9        14.43
                                                   oracle_pdb      01-JUN-18  1694.7       2571.4         1659.3        35.32
                                                   oracle_pdb      01-JUL-18  1753.8       2571.4         1694.7        59.18
                                                   oracle_pdb      01-AUG-18  1827.9       2571.4         1753.8        74.06
                                                   oracle_pdb      01-SEP-18  1900.8       2571.4         1827.9        72.91
                                                   oracle_pdb      01-OCT-18  1977.2       2571.4         1900.8        76.43
                                                   oracle_pdb      01-NOV-18  2044.8       2571.4         1977.2         67.6
                                                   oracle_pdb      01-DEC-18  2144.5       2571.4         2044.8        99.64

Script:

set line 200 verify off trimspool off tab off pages 1000 numw 6 echo on
-- Michael Dinh : Dec 20, 2018
set echo off
/*
How to obtain the Historical Database Total Used and Allocated Size from OEM Repository
How to obtain the Historical Database Total Used and Allocated Size from OEM Repository
*/
col target_name for a50
col target_type for a15
undefine 1
break on target_name
WITH dbsz AS (
SELECT
target_name, target_type, month_dt,
SUM(DECODE(metric_column, 'USED_GB', maximum)) used_gb,
SUM(DECODE(metric_column, 'ALLOCATED_GB', maximum)) allocated_gb
FROM (
SELECT target_name, target_type, trunc(rollup_timestamp,'MONTH') month_dt, metric_column, MAX(maximum) maximum
FROM sysman.mgmt$metric_daily
WHERE target_type IN ('rac_database','oracle_database','oracle_pdb')
AND metric_name = 'DATABASE_SIZE'
AND metric_column IN ('ALLOCATED_GB','USED_GB')
AND REGEXP_LIKE(target_name,'&&1','i')
GROUP BY target_name, target_type, trunc(rollup_timestamp,'MONTH'), metric_column
)
GROUP BY target_name, target_type, month_dt
ORDER BY target_name, month_dt
)
SELECT target_name, target_type, month_dt, used_gb, allocated_gb,
LAG(used_gb,1) OVER (PARTITION BY target_name ORDER BY target_name) previous_month,
used_gb-LAG(used_gb,1) OVER (PARTITION BY target_name ORDER BY target_name) diff_used_gb
FROM dbsz
ORDER BY target_name, month_dt
;

UPDATED SQL SCRIPT:

SQL> @dbsize.sql xxxprod
SQL> -- Michael Dinh : Dec 20, 2018
SQL> set echo off

TARGET_NAME                                        TARGET_TYPE     MONTH_DT  USED_GB ALLOCATED_GB PREVIOUS_MONTH DIFF_USED_GB
-------------------------------------------------- --------------- --------- ------- ------------ -------------- ------------
xxxprod                                            rac_database    31-MAR-18   333.2       704.42
                                                   rac_database    30-APR-18  336.65       704.42          333.2         3.45
                                                   rac_database    31-MAY-18  350.48       704.42         336.65        13.83
                                                   rac_database    30-JUN-18  423.47        714.1         350.48        72.99
                                                   rac_database    31-JUL-18  397.42        714.1         423.47       -26.05
                                                   rac_database    31-AUG-18  415.61        714.1         397.42        18.19
                                                   rac_database    30-SEP-18   417.2       714.69         415.61         1.59
                                                   rac_database    31-OCT-18  421.04       714.69          417.2         3.84
                                                   rac_database    30-NOV-18  425.35       715.37         421.04         4.31
                                                   rac_database    20-DEC-18  428.44       723.11         425.35         3.09

10 rows selected.

SQL> !cat dbsize.sql



SQL> !cat dbsize.sql
set line 200 verify off trimspool off tab off pages 1000 numw 6 echo on
-- Michael Dinh : Dec 20, 2018
set echo off
/*
How to obtain the Historical Database Total Used and Allocated Size from OEM Repository
How to obtain the Historical Database Total Used and Allocated Size from OEM Repository
*/
col target_name for a50
col target_type for a15
break on target_name
WITH dbsz AS (
SELECT
target_name, target_type, month_dt,
SUM(DECODE(metric_column, 'USED_GB', maximum)) used_gb,
SUM(DECODE(metric_column, 'ALLOCATED_GB', maximum)) allocated_gb
FROM (
-- This shows LATEST date of month
SELECT target_name, target_type, MAX(rollup_timestamp) month_dt, metric_column, MAX(maximum) maximum
-- This shows FIRST date of month
-- SELECT target_name, target_type, TRUNC(rollup_timestamp,'MONTH') month_dt, metric_column, MAX(maximum) maximum
FROM sysman.mgmt$metric_daily
WHERE target_type IN ('rac_database','oracle_database','oracle_pdb')
AND metric_name = 'DATABASE_SIZE'
AND metric_column IN ('ALLOCATED_GB','USED_GB')
AND REGEXP_LIKE(target_name,'&1','i')
GROUP BY target_name, target_type, TRUNC(rollup_timestamp,'MONTH'), metric_column
)
GROUP BY target_name, target_type, month_dt
-- ORDER BY target_name, month_dt
)
SELECT target_name, target_type, month_dt, used_gb, allocated_gb,
LAG(used_gb,1) OVER (PARTITION BY target_name ORDER BY target_name) previous_month,
used_gb-LAG(used_gb,1) OVER (PARTITION BY target_name ORDER BY target_name) diff_used_gb
FROM dbsz
ORDER BY target_name, month_dt
;
undefine 1

May 3, 2018

Multiplex Redo Log

Filed under: 12c,oracle — mdinh @ 4:14 am

When db_create_online_log_dest_1 is defined, REDO log is not multiplexed which is good for creating STANDBY REDO.

REDO log is created at db_create_online_log_dest_1 ONLY.

However, when creating ONLINE REDO, db_create_online_log_dest_1 should NOT be defined to be multiplexed.

REDO logs are created at db_create_file_dest and db_recovery_file_dest.

[oracle@db-asm-1 sql]$ sqlplus / as sysdba @ logfile.sql

SQL*Plus: Release 12.2.0.1.0 Production on Thu May 3 05:56:30 2018

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

05:56:30 SYS @ owl:>show parameter db_create

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest                  string      +DATA
db_create_online_log_dest_1                      +FRA
db_create_online_log_dest_2
db_create_online_log_dest_3
db_create_online_log_dest_4
db_create_online_log_dest_5
05:56:30 SYS @ owl:>show parameter db_recovery_file

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      +FRA
db_recovery_file_dest_size           big integer 7116M

05:56:30 SYS @ owl:>select group#,member,type,status from v$logfile order by 1,2 asc;

    GROUP# MEMBER                                                                           TYPE    STATUS
---------- -------------------------------------------------------------------------------- ------- -------
         1 +DATA/OWL/ONLINELOG/group_1.261.962643743                                        ONLINE
         1 +FRA/OWL/ONLINELOG/group_1.257.962643743
         2 +DATA/OWL/ONLINELOG/group_2.262.962643743
         2 +FRA/OWL/ONLINELOG/group_2.258.962643743
         3 +DATA/OWL/ONLINELOG/group_3.263.962643745
         3 +FRA/OWL/ONLINELOG/group_3.259.962643745

6 rows selected.

05:56:30 SYS @ owl:>select group#,thread#,sequence#,bytes,status from v$log order by 1,2;

    GROUP#    THREAD#  SEQUENCE#      BYTES STATUS
---------- ---------- ---------- ---------- ----------------
         1          1         79  104857600 CURRENT
         2          1         77  104857600 INACTIVE
         3          1         78  104857600 INACTIVE

05:56:30 SYS @ owl:>select group#,thread#,sequence#,bytes,status from v$standby_log order by 1,2;

no rows selected

05:56:30 SYS @ owl:>alter database add standby logfile thread 1 group 11 size 104857600;

Database altered.

05:57:03 SYS @ owl:>alter system set db_create_online_log_dest_1='';

System altered.

05:57:39 SYS @ owl:>alter database add logfile thread 1 group 4 size 104857600;

Database altered.

05:58:01 SYS @ owl:>@logfile.sql
05:58:06 SYS @ owl:>set lines 200 tab off trimsp on pages 1000
05:58:06 SYS @ owl:>col member for a80
05:58:06 SYS @ owl:>break on TYPE
05:58:06 SYS @ owl:>set echo on
05:58:06 SYS @ owl:>show parameter db_create

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest                  string      +DATA
db_create_online_log_dest_1
db_create_online_log_dest_2
db_create_online_log_dest_3
db_create_online_log_dest_4
db_create_online_log_dest_5
05:58:06 SYS @ owl:>show parameter db_recovery_file

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      +FRA
db_recovery_file_dest_size           big integer 7116M
05:58:06 SYS @ owl:>select group#,member,type,status from v$logfile order by 1,2 asc;

    GROUP# MEMBER                                                                           TYPE    STATUS
---------- -------------------------------------------------------------------------------- ------- -------
         1 +DATA/OWL/ONLINELOG/group_1.261.962643743                                        ONLINE
         1 +FRA/OWL/ONLINELOG/group_1.257.962643743
         2 +DATA/OWL/ONLINELOG/group_2.262.962643743
         2 +FRA/OWL/ONLINELOG/group_2.258.962643743
         3 +DATA/OWL/ONLINELOG/group_3.263.962643745
         3 +FRA/OWL/ONLINELOG/group_3.259.962643745
         4 +DATA/OWL/ONLINELOG/group_4.267.975131881
         4 +FRA/OWL/ONLINELOG/group_4.295.975131881
        11 +FRA/OWL/ONLINELOG/group_11.296.975131823                                        STANDBY

9 rows selected.

05:58:07 SYS @ owl:>select group#,thread#,sequence#,bytes,status from v$log order by 1,2;

    GROUP#    THREAD#  SEQUENCE#      BYTES STATUS
---------- ---------- ---------- ---------- ----------------
         1          1         79  104857600 CURRENT
         2          1         77  104857600 INACTIVE
         3          1         78  104857600 INACTIVE
         4          1          0  104857600 UNUSED

05:58:07 SYS @ owl:>select group#,thread#,sequence#,bytes,status from v$standby_log order by 1,2;

    GROUP#    THREAD#  SEQUENCE#      BYTES STATUS
---------- ---------- ---------- ---------- ----------
        11          1          0  104857600 UNASSIGNED

05:58:07 SYS @ owl:>

December 2, 2017

Goldengate 12.3 Automatic CDR

Filed under: 12.2,GoldenGate,oracle — mdinh @ 11:51 pm

Automatic Conflict Detection and Resolution

Requirements: GoldenGate 12c (12.3.0.1) and Oracle Database 12c Release 2 (12.2) and later.

Automatic conflict detection and resolution does not require application changes for the following reasons:

  • Oracle Database automatically creates and maintains invisible timestamp columns.
  • Inserts, updates, and deletes use the delete tombstone log table to determine if a row was deleted.
  • LOB column conflicts can be detected.
  • Oracle Database automatically configures supplemental logging on required columns.

I have not had the chance to play with this yet and just only notice the documentation has been updated with details.

 

 

November 23, 2017

CRS-2674: Start of dbfs_mount failed

Filed under: 12c,GoldenGate,oracle,RAC — mdinh @ 1:04 am

$ crsctl start resource dbfs_mount
CRS-2672: Attempting to start ‘dbfs_mount’ on ‘node2’
CRS-2672: Attempting to start ‘dbfs_mount’ on ‘node1’
CRS-2674: Start of ‘dbfs_mount’ on ‘node1’ failed
CRS-2679: Attempting to clean ‘dbfs_mount’ on ‘node1’
CRS-2674: Start of ‘dbfs_mount’ on ‘node2’ failed
CRS-2679: Attempting to clean ‘dbfs_mount’ on ‘node2’
CRS-2681: Clean of ‘dbfs_mount’ on ‘node1’ succeeded
CRS-2681: Clean of ‘dbfs_mount’ on ‘node2’ succeeded
CRS-4000: Command Start failed, or completed with errors.

Check to make sure DBFS_USER password is not expired.

October 17, 2017

DB Starts with SQLPlus not SRVCTL

Filed under: 11g,oracle — mdinh @ 1:25 am

Reason why DB was able to be started using SQL*Plus and not srvctl because DB was configured incorrectly with srvctl.

$ srvctl start database -d DB01
PRCR-1079 : Failed to start resource ora.db01.db
CRS-5017: The resource action "ora.db01.db start" encountered the following error:
ORA-01078: failure in processing system parameters. 
For details refer to "(:CLSN00107:)" in "/u01/app/oracle/product/11.2.0/grid_2/log/host01/agent/ohasd/oraagent_oracle//oraagent_log".

CRS-2674: Start of 'ora.db01.db' on 'host01' failed

--- Spfile pointing to non-existing pfile.
$ srvctl config database -d DB01
Database unique name: DB01
Database name:
Oracle home: /u01/app/oracle/product/11.2.0/dbhome_2
Oracle user: oracle
Spfile: /oracle/product/11.2.0/dbhome_2/dbs/initDB01.ora
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Database instance: DB01
Disk Groups: DATA,FRA
Services:

SQL> show parameter spfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/db01/spfiledb01.ora

cat: /oracle/product/11.2.0/dbhome_2/dbs/initDB01.ora: No such file or directory

$ srvctl modify database -d DB01 -p +DATA/db01/spfiledb01.ora
$ srvctl config database -d DB01
Database unique name: DB01
Database name:
Oracle home: /u01/app/oracle/product/11.2.0/dbhome_2
Oracle user: oracle
Spfile: +DATA/db01/spfiledb01.ora
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Database instance: DB01
Disk Groups: DATA,FRA
Services:

October 9, 2017

No Guarantees with opatch -report or CheckConflict

Filed under: 11g,oracle — mdinh @ 8:13 pm

I have performed the following checks.

# $GRID_HOME/OPatch/opatch auto /media/swrepo/JUL2017PSU/26030799 -report -ocmrf /tmp/ocm.rsp
$ $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /media/swrepo/JUL2017PSU/26030799

Actual patching failed.

# $GRID_HOME/OPatch/opatch auto /media/swrepo/JUL2017PSU/26030799 -ocmrf /tmp/ocm.rsp
Executing /u01/app/oracle/product/11.2.0/grid/perl/bin/perl 
/u01/app/oracle/product/11.2.0/grid/OPatch/crs/patch11203.pl 
-patchdir /media/swrepo/JUL2017PSU -patchn 26030799 
-ocmrf /tmp/ocm.rsp -paramfile /u01/app/oracle/product/11.2.0/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/oracle/product/11.2.0/grid/cfgtoollogs/opatchauto2017-10-09_10-35-34.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/oracle/product/11.2.0/grid/cfgtoollogs/opatchauto2017-10-09_10-35-34.report.log

2017-10-09 10:35:34: Starting Oracle Restart Patch Setup
Using configuration parameter file: /u01/app/oracle/product/11.2.0/grid/crs/install/crsconfig_params

Stopping RAC /u01/app/oracle/product/11.2.0/dbhome_1 ...
Stopped RAC /u01/app/oracle/product/11.2.0/dbhome_1 successfully

patch /media/swrepo/JUL2017PSU/26030799/25869727  apply successful for home  /u01/app/oracle/product/11.2.0/dbhome_1
patch /media/swrepo/JUL2017PSU/26030799/25920335/custom/server/25920335  apply successful for home  /u01/app/oracle/product/11.2.0/dbhome_1

Stopping CRS...

Stopped CRS successfully

Error : The opatch Applicable check failed.  The patch /media/swrepo/JUL2017PSU/26030799/25920335 is not applicable to /u01/app/oracle/product/11.2.0/grid
Error:Patch Applicable check failed for /u01/app/oracle/product/11.2.0/grid

Starting CRS...

ERROR: Prereq checkApplicable failed. Refer log file for more details.


opatch auto failed.
#

Really useful info – ERROR: Prereq checkApplicable failed. Refer log file for more details.

I digress.

After some digging – search for ZOP-46 from /u01/app/oracle/product/11.2.0/grid/cfgtoollogs/opatch

$ grep -n "ZOP-46" opatch2017-10-09*.log
opatch2017-10-09_10-41-58AM_1.log:13:
[Oct 9, 2017 10:42:00 AM]    ZOP-46: 
The patch(es) are not applicable on the Oracle Home because some patch actions are not applicable. 
All required components, however, are installed.


$ head -25 opatch2017-10-09_10-41-58AM_1.log
[Oct 9, 2017 10:41:59 AM]    PREREQ session

[Oct 9, 2017 10:41:59 AM]    
OPatch invoked as follows: 'prereq CheckApplicable 
-ph /media/swrepo/JUL2017PSU/26030799/25920335 
-oh /u01/app/oracle/product/11.2.0/grid 
-invPtrLoc /u01/app/oracle/product/11.2.0/grid/oraInst.loc '

[Oct 9, 2017 10:41:59 AM]    OUI-67077:
                             Oracle Home       : /u01/app/oracle/product/11.2.0/grid
                             Central Inventory : /u01/app/oracle/oraInventory
                                from           : /u01/app/oracle/product/11.2.0/grid/oraInst.loc
                             OPatch version    : 11.2.0.3.6
                             OUI version       : 11.2.0.4.0
                             OUI location      : /u01/app/oracle/product/11.2.0/grid/oui
                             Log file location : /u01/app/oracle/product/11.2.0/grid/cfgtoollogs/opatch/opatch2017-10-09_10-41-58AM_1.log
[Oct 9, 2017 10:41:59 AM]    Patch history file: /u01/app/oracle/product/11.2.0/grid/cfgtoollogs/opatch/opatch_history.txt
[Oct 9, 2017 10:41:59 AM]    Invoking prereq "checkapplicable"

[Oct 9, 2017 10:42:00 AM]    
ZOP-46: The patch(es) are not applicable on the Oracle Home because some patch actions are not applicable. 
All required components, however, are installed.

[Oct 9, 2017 10:42:00 AM]    Patch 25920335:
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/bin/appvipcfg.pl" does not exists or is not readable
                             'oracle.crs, 11.2.0.4.0': Cannot copy file from 'appvipcfg.pl' to '/u01/app/oracle/product/11.2.0/grid/bin/appvipcfg.pl'
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/bin/oclumon.bin" does not exists or is not readable
                             'oracle.crs, 11.2.0.4.0': Cannot copy file from 'oclumon.bin' to '/u01/app/oracle/product/11.2.0/grid/bin/oclumon.bin'
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/bin/ologgerd" does not exists or is not readable
                             'oracle.crs, 11.2.0.4.0': Cannot copy file from 'ologgerd' to '/u01/app/oracle/product/11.2.0/grid/bin/ologgerd'
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/bin/osysmond.bin" does not exists or is not readable
                             'oracle.crs, 11.2.0.4.0': Cannot copy file from 'osysmond.bin' to '/u01/app/oracle/product/11.2.0/grid/bin/osysmond.bin'
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/crs/demo/coldfailover/act_db.pl" does not exists or is not readable
                             'oracle.crs, 11.2.0.4.0': Cannot copy file from 'act_db.pl' to '/u01/app/oracle/product/11.2.0/grid/crs/demo/coldfailover/act_db.pl'
                             Copy Action: Source File "/media/swrepo/JUL2017PSU/26030799/25920335/files/crs/demo/coldfailover/act_listener.pl" does not exists or is not readable
$ ls -l /media/swrepo/JUL2017PSU/26030799/25920335/files/bin/appvipcfg.pl
-rwxr-x--- 1 root root 9051 Jun 27 07:40 /media/swrepo/JUL2017PSU/26030799/25920335/files/bin/appvipcfg.pl

Please don’t ask me why.

Solution.

# cd /media/
# chmod -R 777 swrepo/
# chown -R oracle:dba patches/

opatch report “ERROR: Prereq checkApplicable failed.” when Applying Grid Infrastructure patch (Doc ID 1417268.1)

	A. Expected behaviour if GRID_HOME has not been unlocked
 	B. Bug 13575478
 	C. The patch is stored in a shared NFS location and there is a permission issue accessing the patch
 	D. The patch is not unzipped as grid user, often it is unzipped as root user
 	E. The patch is unzipped inside GRID_HOME

In summary, trust but verify!

September 29, 2017

Scheduler Jobs Do Not Run Automatically

Filed under: 12c,oracle — mdinh @ 2:09 am

After you have followed – IF: Jobs Do Not Run Automatically (Doc ID 2084527.1) – without any success,
then check to see if services have been created and are running.

RAC DB is 12.1.0.2.0 and was cloned from standby.

It just so happens, service as defined from the SQL below was not created:

select c.SERVICE
from dba_scheduler_jobs j, dba_scheduler_job_classes c
where j.JOB_CLASS=c.JOB_CLASS_NAME
and j.JOB_NAME=UPPER('&jobname')
;

To be honest, I was not able to find the issue and team mate did.

What I found very, very strange is manually running the job using exec dbms_scheduler.run_job is successful.

The manual job ran successfully without the service created and on the wrong node for where the service is defined
(the service is defined to run on node 2, while the manual run is from node 1).

Another unsolved mystery.

Next Page »

Create a free website or blog at WordPress.com.