Thinking Out Loud

February 20, 2015

Oracle Manual Standby – Applying Log

Filed under: 11g,oracle,RMAN — mdinh @ 7:49 pm

If you are running Oracle EE, there are many books on Data Guard. However, for Oracle SE and manual standby, have fun searching.

I wanted to get a better understanding of registering logfile.

Depending on how the standby environment is monitored, not registering logfile may yield incorrect results when checking standby lag.

When logfile is not registered, v$archived_log is not updated.

STANDBY: check status

[oracle@armor:hawk:/home/oracle]
$ sqlplus / as sysdba @stby.sql

SQL*Plus: Release 11.2.0.4.0 Production on Fri Feb 20 10:12:58 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Release 11.2.0.4.0 - 64bit Production

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied"
  2  from
  3  (
  4    select thread# thrd, max(sequence#) almax from v$archived_log
  5    where resetlogs_change#=(select resetlogs_change# from v$database) and applied='YES'
  6    group by thread#
  7  ) al,
  8  (
  9    select thread# thrd, max(sequence#) lhmax
 10    from v$log_history
 11    where first_time=(select max(first_time) from v$log_history)
 12    group by thread#
 13  ) lh
 14  where al.thrd = lh.thrd
 15  ;

          Thread Last Seq Received Last Seq Applied
---------------- ----------------- ----------------
               1                44               44

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select first_time,applied,archived,registrar,creator,thread#,sequence# from v$archived_log
  2  where resetlogs_change#=(select resetlogs_change# from v$database)
  3  and first_time>sysdate-3/24
  4  order by sequence# desc
  5  ;

NOTE: SRMN - RMAN at standby

FIRST_TIME          APPLIED   ARC REGISTR CREATOR          THREAD#        SEQUENCE#
------------------- --------- --- ------- ------- ---------------- ----------------
2015-02-20 10:09:09 YES       YES SRMN    SRMN                   1               44
2015-02-20 10:03:54 NO        YES SRMN    SRMN                   1               43
2015-02-20 09:58:33 YES       YES SRMN    SRMN                   1               42
2015-02-20 09:52:53 YES       YES SRMN    SRMN                   1               41
2015-02-20 09:47:17 YES       YES SRMN    SRMN                   1               40
2015-02-20 09:45:50 YES       YES SRMN    SRMN                   1               39
2015-02-20 09:41:36 YES       YES SRMN    SRMN                   1               38
2015-02-20 09:21:34 YES       YES SRMN    SRMN                   1               37
2015-02-20 09:01:33 YES       YES SRMN    SRMN                   1               36
2015-02-20 08:41:35 YES       YES SRMN    SRMN                   1               35
2015-02-20 08:21:34 YES       YES SRMN    SRMN                   1               34
2015-02-20 08:01:33 YES       YES SRMN    SRMN                   1               33
2015-02-20 07:41:30 YES       YES SRMN    SRMN                   1               32
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               31
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               30
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               29
2015-02-20 07:41:26 YES       YES SRMN    SRMN                   1               28
2015-02-20 07:41:23 YES       YES SRMN    SRMN                   1               27
2015-02-20 07:24:52 YES       YES SRMN    SRMN                   1               26

19 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY>

PRIMARY: create tablespace new1

ANGEL:(SYS@hawk):PRIMARY> create tablespace new1;

Tablespace created.

ANGEL:(SYS@hawk):PRIMARY> alter system archive log current;

System altered.

ANGEL:(SYS@hawk):PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /oradata/archivelog/hawk
Oldest online log sequence     44
Next log sequence to archive   46
Current log sequence           46
ANGEL:(SYS@hawk):PRIMARY>

PRIMARY: transfer archive log to STANDBY

[oracle@angel:hawk:/oradata/archivelog/hawk]
$ rsync -avh /oradata/archivelog/hawk/ armor:/oradata/archivelog/hawk/
sending incremental file list
./
hawk_45_1_872097259.arc

sent 170.68K bytes  received 34 bytes  341.42K bytes/sec
total size is 78.17M  speedup is 457.90
[oracle@angel:hawk:/oradata/archivelog/hawk]
$

STANDBY: apply log

[oracle@armor:hawk:/home/oracle]
$ sqlplus / as sysdba @recoverauto.sql

SQL*Plus: Release 11.2.0.4.0 Production on Fri Feb 20 10:14:43 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Release 11.2.0.4.0 - 64bit Production

ARMOR:(SYS@hawk):PHYSICAL STANDBY> recover automatic standby database;
ORA-00279: change 216448 generated at 02/20/2015 10:14:03 needed for thread 1
ORA-00289: suggestion : /oradata/archivelog/hawk/hawk_46_1_872097259.arc
ORA-00280: change 216448 for thread 1 is in sequence #46
ORA-00278: log file '/oradata/archivelog/hawk/hawk_46_1_872097259.arc' no longer needed for this recovery
ORA-00308: cannot open archived log '/oradata/archivelog/hawk/hawk_46_1_872097259.arc'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3


Specify log: {=suggested | filename | AUTO | CANCEL}
cancel
Media recovery cancelled.
ARMOR:(SYS@hawk):PHYSICAL STANDBY> @@stby.sql
ARMOR:(SYS@hawk):PHYSICAL STANDBY> set echo on numw 16 lines 200 pages 1000
ARMOR:(SYS@hawk):PHYSICAL STANDBY> select al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied"
  2  from
  3  (
  4    select thread# thrd, max(sequence#) almax from v$archived_log
  5    where resetlogs_change#=(select resetlogs_change# from v$database) and applied='YES'
  6    group by thread#
  7  ) al,
  8  (
  9    select thread# thrd, max(sequence#) lhmax
 10    from v$log_history
 11    where first_time=(select max(first_time) from v$log_history)
 12    group by thread#
 13  ) lh
 14  where al.thrd = lh.thrd
 15  ;

          Thread Last Seq Received Last Seq Applied
---------------- ----------------- ----------------
               1                44               45

NOTE: sequence 45 is not registered with v$archived_log but has been applied

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select first_time,applied,archived,registrar,creator,thread#,sequence# from v$archived_log
  2  where resetlogs_change#=(select resetlogs_change# from v$database)
  3  and first_time>sysdate-3/24
  4  order by sequence# desc
  5  ;

FIRST_TIME          APPLIED   ARC REGISTR CREATOR          THREAD#        SEQUENCE#
------------------- --------- --- ------- ------- ---------------- ----------------
2015-02-20 10:09:09 YES       YES SRMN    SRMN                   1               44
2015-02-20 10:03:54 NO        YES SRMN    SRMN                   1               43
2015-02-20 09:58:33 YES       YES SRMN    SRMN                   1               42
2015-02-20 09:52:53 YES       YES SRMN    SRMN                   1               41
2015-02-20 09:47:17 YES       YES SRMN    SRMN                   1               40
2015-02-20 09:45:50 YES       YES SRMN    SRMN                   1               39
2015-02-20 09:41:36 YES       YES SRMN    SRMN                   1               38
2015-02-20 09:21:34 YES       YES SRMN    SRMN                   1               37
2015-02-20 09:01:33 YES       YES SRMN    SRMN                   1               36
2015-02-20 08:41:35 YES       YES SRMN    SRMN                   1               35
2015-02-20 08:21:34 YES       YES SRMN    SRMN                   1               34
2015-02-20 08:01:33 YES       YES SRMN    SRMN                   1               33
2015-02-20 07:41:30 YES       YES SRMN    SRMN                   1               32
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               31
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               30
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               29
2015-02-20 07:41:26 YES       YES SRMN    SRMN                   1               28
2015-02-20 07:41:23 YES       YES SRMN    SRMN                   1               27
2015-02-20 07:24:52 YES       YES SRMN    SRMN                   1               26

19 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select name from v$tablespace;

NAME
------------------------------
SYSTEM
SYSAUX
UNDOTBS1
TEMP
USERS
NEW1

6 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY>

PRIMARY: create tablespace new2

ANGEL:(SYS@hawk):PRIMARY> create tablespace new2;

Tablespace created.

ANGEL:(SYS@hawk):PRIMARY> alter system archive log current;

System altered.

ANGEL:(SYS@hawk):PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /oradata/archivelog/hawk
Oldest online log sequence     45
Next log sequence to archive   47
Current log sequence           47
ANGEL:(SYS@hawk):PRIMARY>

PRIMARY: transfer archive log to STANDBY

[oracle@angel:hawk:/oradata/archivelog/hawk]
$ rsync -avh /oradata/archivelog/hawk/ armor:/oradata/archivelog/hawk/
sending incremental file list
./
hawk_46_1_872097259.arc

sent 144.59K bytes  received 34 bytes  289.24K bytes/sec
total size is 78.31M  speedup is 541.50
[oracle@angel:hawk:/oradata/archivelog/hawk]
$

STANDBY: register archive log using RMAN

[oracle@armor:hawk:/home/oracle]
$ rman @catalog_arc.rman

Recovery Manager: Release 11.2.0.4.0 - Production on Fri Feb 20 10:16:20 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

RMAN> set echo on
2> connect target;
3> catalog start with '/oradata/archivelog/hawk/' noprompt;
4> exit
echo set on

connected to target database: HAWK (DBID=3130795691, not open)

using target database control file instead of recovery catalog
searching for all files that match the pattern /oradata/archivelog/hawk/

List of Files Unknown to the Database
=====================================
File Name: /oradata/archivelog/hawk/hawk_46_1_872097259.arc
File Name: /oradata/archivelog/hawk/hawk_45_1_872097259.arc
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /oradata/archivelog/hawk/hawk_46_1_872097259.arc
File Name: /oradata/archivelog/hawk/hawk_45_1_872097259.arc

Recovery Manager complete.

STANDBY: apply log

[oracle@armor:hawk:/home/oracle]
$ sqlplus / as sysdba @recoverauto.sql

SQL*Plus: Release 11.2.0.4.0 Production on Fri Feb 20 10:16:34 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Release 11.2.0.4.0 - 64bit Production

ARMOR:(SYS@hawk):PHYSICAL STANDBY> recover automatic standby database;
ORA-00279: change 216754 generated at 02/20/2015 10:15:39 needed for thread 1
ORA-00289: suggestion : /oradata/archivelog/hawk/hawk_47_1_872097259.arc
ORA-00280: change 216754 for thread 1 is in sequence #47
ORA-00278: log file '/oradata/archivelog/hawk/hawk_47_1_872097259.arc' no longer needed for this recovery
ORA-00308: cannot open archived log '/oradata/archivelog/hawk/hawk_47_1_872097259.arc'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3


Specify log: {=suggested | filename | AUTO | CANCEL}
cancel
Media recovery cancelled.
ARMOR:(SYS@hawk):PHYSICAL STANDBY> @@stby.sql
ARMOR:(SYS@hawk):PHYSICAL STANDBY> set echo on numw 16 lines 200 pages 1000
ARMOR:(SYS@hawk):PHYSICAL STANDBY> select al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied"
  2  from
  3  (
  4    select thread# thrd, max(sequence#) almax from v$archived_log
  5    where resetlogs_change#=(select resetlogs_change# from v$database) and applied='YES'
  6    group by thread#
  7  ) al,
  8  (
  9    select thread# thrd, max(sequence#) lhmax
 10    from v$log_history
 11    where first_time=(select max(first_time) from v$log_history)
 12    group by thread#
 13  ) lh
 14  where al.thrd = lh.thrd
 15  ;

          Thread Last Seq Received Last Seq Applied
---------------- ----------------- ----------------
               1                46               46

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select first_time,applied,archived,registrar,creator,thread#,sequence# from v$archived_log
  2  where resetlogs_change#=(select resetlogs_change# from v$database)
  3  and first_time>sysdate-3/24
  4  order by sequence# desc
  5  ;

FIRST_TIME          APPLIED   ARC REGISTR CREATOR          THREAD#        SEQUENCE#
------------------- --------- --- ------- ------- ---------------- ----------------
2015-02-20 10:14:03 YES       YES SRMN    SRMN                   1               46
2015-02-20 10:11:03 NO        YES SRMN    SRMN                   1               45
2015-02-20 10:09:09 YES       YES SRMN    SRMN                   1               44
2015-02-20 10:03:54 NO        YES SRMN    SRMN                   1               43
2015-02-20 09:58:33 YES       YES SRMN    SRMN                   1               42
2015-02-20 09:52:53 YES       YES SRMN    SRMN                   1               41
2015-02-20 09:47:17 YES       YES SRMN    SRMN                   1               40
2015-02-20 09:45:50 YES       YES SRMN    SRMN                   1               39
2015-02-20 09:41:36 YES       YES SRMN    SRMN                   1               38
2015-02-20 09:21:34 YES       YES SRMN    SRMN                   1               37
2015-02-20 09:01:33 YES       YES SRMN    SRMN                   1               36
2015-02-20 08:41:35 YES       YES SRMN    SRMN                   1               35
2015-02-20 08:21:34 YES       YES SRMN    SRMN                   1               34
2015-02-20 08:01:33 YES       YES SRMN    SRMN                   1               33
2015-02-20 07:41:30 YES       YES SRMN    SRMN                   1               32
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               31
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               30
2015-02-20 07:41:29 YES       YES SRMN    SRMN                   1               29
2015-02-20 07:41:26 YES       YES SRMN    SRMN                   1               28
2015-02-20 07:41:23 YES       YES SRMN    SRMN                   1               27
2015-02-20 07:24:52 YES       YES SRMN    SRMN                   1               26

21 rows selected.

STANDBY: open read only test

ARMOR:(SYS@hawk):PHYSICAL STANDBY> @openreadonly
ARMOR:(SYS@hawk):PHYSICAL STANDBY> set echo on numw 16 lines 200
ARMOR:(SYS@hawk):PHYSICAL STANDBY> alter database open read only;

Database altered.

ARMOR:(SYS@hawk):PHYSICAL STANDBY> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
ARMOR:(SYS@hawk):PHYSICAL STANDBY> startup nomount;
ORACLE instance started.

Total System Global Area       1068937216 bytes
Fixed Size                        2260088 bytes
Variable Size                   281019272 bytes
Database Buffers                780140544 bytes
Redo Buffers                      5517312 bytes
ARMOR:(SYS@hawk):PHYSICAL STANDBY> alter database mount standby database;

Database altered.

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select controlfile_type,open_mode,database_role,db_unique_name,standby_became_primary_scn from v$database
  2  ;

CONTROL OPEN_MODE            DATABASE_ROLE    DB_UNIQUE_NAME                 STANDBY_BECAME_PRIMARY_SCN
------- -------------------- ---------------- ------------------------------ --------------------------
STANDBY MOUNTED              PHYSICAL STANDBY hawk_sfo                                                0

ARMOR:(SYS@hawk):PHYSICAL STANDBY> @@stby.sql
ARMOR:(SYS@hawk):PHYSICAL STANDBY> set echo on numw 16 lines 200 pages 1000
ARMOR:(SYS@hawk):PHYSICAL STANDBY> select al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied"
  2  from
  3  (
  4    select thread# thrd, max(sequence#) almax from v$archived_log
  5    where resetlogs_change#=(select resetlogs_change# from v$database) and applied='YES'
  6    group by thread#
  7  ) al,
  8  (
  9    select thread# thrd, max(sequence#) lhmax
 10    from v$log_history
 11    where first_time=(select max(first_time) from v$log_history)
 12    group by thread#
 13  ) lh
 14  where al.thrd = lh.thrd
 15  ;

          Thread Last Seq Received Last Seq Applied
---------------- ----------------- ----------------
               1                46               46

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select first_time,applied,archived,registrar,creator,thread#,sequence# from v$archived_log
  2  where resetlogs_change#=(select resetlogs_change# from v$database)
  3  and first_time>sysdate-3/24
  4  order by sequence# desc
  5  ;

FIRST_TIM APPLIED   ARC REGISTR CREATOR          THREAD#        SEQUENCE#
--------- --------- --- ------- ------- ---------------- ----------------
20-FEB-15 YES       YES SRMN    SRMN                   1               46
20-FEB-15 YES       YES SRMN    SRMN                   1               45
20-FEB-15 YES       YES SRMN    SRMN                   1               44
20-FEB-15 YES       YES SRMN    SRMN                   1               43
20-FEB-15 YES       YES SRMN    SRMN                   1               42
20-FEB-15 YES       YES SRMN    SRMN                   1               41
20-FEB-15 YES       YES SRMN    SRMN                   1               40
20-FEB-15 YES       YES SRMN    SRMN                   1               39
20-FEB-15 YES       YES SRMN    SRMN                   1               38
20-FEB-15 YES       YES SRMN    SRMN                   1               37
20-FEB-15 YES       YES SRMN    SRMN                   1               36
20-FEB-15 YES       YES SRMN    SRMN                   1               35
20-FEB-15 YES       YES SRMN    SRMN                   1               34
20-FEB-15 YES       YES SRMN    SRMN                   1               33
20-FEB-15 YES       YES SRMN    SRMN                   1               32
20-FEB-15 YES       YES SRMN    SRMN                   1               31
20-FEB-15 YES       YES SRMN    SRMN                   1               30
20-FEB-15 YES       YES SRMN    SRMN                   1               29
20-FEB-15 YES       YES SRMN    SRMN                   1               28
20-FEB-15 YES       YES SRMN    SRMN                   1               27
20-FEB-15 YES       YES SRMN    SRMN                   1               26

21 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY> select name from v$tablespace;

NAME
------------------------------
SYSTEM
SYSAUX
UNDOTBS1
TEMP
USERS
NEW1
NEW2

7 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY>

PRIMARY: configuration

ANGEL:(SYS@hawk):PRIMARY> show parameter name

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
cell_offloadgroup_name    string
db_file_name_convert      string
db_name                   string      hawk
db_unique_name            string      hawk
global_names              boolean     FALSE
instance_name             string      hawk
lock_name_space           string
log_file_name_convert     string
processor_group_name      string
service_names             string      hawk
ANGEL:(SYS@hawk):PRIMARY> show parameter convert

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
db_file_name_convert      string
log_file_name_convert     string
ANGEL:(SYS@hawk):PRIMARY> show parameter standby

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
standby_archive_dest      string      ?/dbs/arch
standby_file_management   string      AUTO
ANGEL:(SYS@hawk):PRIMARY> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/oradata/HAWK/datafile/o1_mf_system_bgf2mflo_.dbf
/oradata/HAWK/datafile/o1_mf_sysaux_bgf2mj87_.dbf
/oradata/HAWK/datafile/o1_mf_undotbs1_bgf2mkds_.dbf
/oradata/HAWK/datafile/o1_mf_users_bgf2mlf8_.dbf
/oradata/HAWK/datafile/o1_mf_new1_bggyc50z_.dbf
/oradata/HAWK/datafile/o1_mf_new2_bggyg5ky_.dbf

6 rows selected.

ANGEL:(SYS@hawk):PRIMARY>

STANDBY: configuration

ARMOR:(SYS@hawk):PHYSICAL STANDBY> show parameter name

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
cell_offloadgroup_name    string
db_file_name_convert      string
db_name                   string      hawk
db_unique_name            string      hawk_sfo
global_names              boolean     FALSE
instance_name             string      hawk
lock_name_space           string
log_file_name_convert     string
processor_group_name      string
service_names             string      hawk
ARMOR:(SYS@hawk):PHYSICAL STANDBY> show parameter convert

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
db_file_name_convert      string
log_file_name_convert     string
ARMOR:(SYS@hawk):PHYSICAL STANDBY> show parameter standby

NAME                      TYPE        VALUE
------------------------- ----------- ----------------------------------------
standby_archive_dest      string      ?/dbs/arch
standby_file_management   string      AUTO
ARMOR:(SYS@hawk):PHYSICAL STANDBY> select name from v$datafile;

NAME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/oradata/HAWK_SFO/datafile/o1_mf_system_bgf4o5x7_.dbf
/oradata/HAWK_SFO/datafile/o1_mf_sysaux_bgf4on2c_.dbf
/oradata/HAWK_SFO/datafile/o1_mf_undotbs1_bgf4of05_.dbf
/oradata/HAWK_SFO/datafile/o1_mf_users_bgf4oq4b_.dbf
/oradata/HAWK_SFO/datafile/o1_mf_new1_bggydmsq_.dbf
/oradata/HAWK_SFO/datafile/o1_mf_new2_bggyj2n3_.dbf

6 rows selected.

ARMOR:(SYS@hawk):PHYSICAL STANDBY>

PRIMARY: drop tablespace and register/apply log to STANDBY – review alert log

[oracle@armor:hawk:/u01/app/oracle/diag/rdbms/hawk_sfo/hawk/trace]
$ cat alert_hawk.log
Fri Feb 20 11:23:18 2015
ALTER DATABASE RECOVER  automatic standby database
Media Recovery Start
 started logmerger process
Fri Feb 20 11:23:18 2015
Managed Standby Recovery not using Real Time Apply
Parallel Media Recovery started with 2 slaves
Media Recovery Log /oradata/archivelog/hawk/hawk_55_1_872097259.arc
Recovery deleting file #5:'/oradata/HAWK_SFO/datafile/o1_mf_new1_bggydmsq_.dbf' from controlfile.
Deleted Oracle managed file /oradata/HAWK_SFO/datafile/o1_mf_new1_bggydmsq_.dbf
Recovery dropped tablespace 'NEW1'
Recovery deleting file #6:'/oradata/HAWK_SFO/datafile/o1_mf_new2_bggyj2n3_.dbf' from controlfile.
Deleted Oracle managed file /oradata/HAWK_SFO/datafile/o1_mf_new2_bggyj2n3_.dbf
Recovery dropped tablespace 'NEW2'
Media Recovery Log /oradata/archivelog/hawk/hawk_56_1_872097259.arc
Errors with log /oradata/archivelog/hawk/hawk_56_1_872097259.arc
Errors in file /u01/app/oracle/diag/rdbms/hawk_sfo/hawk/trace/hawk_pr00_13085.trc:
ORA-00308: cannot open archived log '/oradata/archivelog/hawk/hawk_56_1_872097259.arc'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-279 signalled during: ALTER DATABASE RECOVER  automatic standby database  ...
ALTER DATABASE RECOVER    CANCEL
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER    CANCEL
[oracle@armor:hawk:/u01/app/oracle/diag/rdbms/hawk_sfo/hawk/trace]
$

February 16, 2015

Using rsync to clone Goldengate installation

Filed under: GoldenGate — mdinh @ 3:15 pm

You may be thinking, why clone Goldengate and why now just download it?
The exact version and patch level might not be available.
Too lazy to search for it and many other reasons you can come up with.

Why use rsync and not tar – scp? I wanted to refresh memory of using rsync.

Commands used:

local source /u01/app/ggs01/ and remote target arrow:/u01/app/ggs03/

rsync -avh --delete --dry-run --exclude 'dirdatold' /u01/app/ggs01/ arrow:/u01/app/ggs03/
rsync -avh --delete --exclude 'dirdatold' /u01/app/ggs01/ arrow:/u01/app/ggs03/

Note:
/u01/app/ggs01/ means synch contents of directory to target
/u01/app/ggs01 means create ggs01 directory and sync contents to target

Demo:

Source: /u01/app/ggs01 and dirdata is symbolic link

oracle@arrow:las:/u01/app/ggs01
$ ls -ld dir*
drwxr-x---. 2 oracle oinstall 4096 Jan 13 13:12 dirchk
lrwxrwxrwx. 1 oracle oinstall   15 Feb 15 06:20 dirdat -> /oradata/backup
drwxr-x---. 2 oracle oinstall 4096 Jul 22  2014 dirdatold
drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 dirdef
drwxr-x---. 2 oracle oinstall 4096 Apr  4  2014 dirjar
drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 dirout
drwxr-x---. 2 oracle oinstall 4096 Feb 12 15:35 dirpcs
drwxr-x---. 2 oracle oinstall 4096 Jan 13 12:55 dirprm
drwxr-x---. 2 oracle oinstall 4096 Feb 12 15:36 dirrpt
drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 dirsql
drwxr-x---. 2 oracle oinstall 4096 Sep 25 08:56 dirtmp

Target: arrow:/u01/app/ggs03/

oracle@arrow:las:/u01/app/ggs01
$ ls -l /u01/app/ggs03/
total 0

Let’s do a dry run first.

oracle@arrow:las:/u01/app/ggs01

$ rsync -avh --delete --dry-run --exclude 'dirdatold' /u01/app/ggs01/ arrow:/u01/app/ggs03/
oracle@arrow's password:

sending incremental file list
./

.....

output ommited for brevity

dirout/
dirpcs/
dirprm/
dirprm/esan.prm
dirprm/jagent.prm
dirprm/mgr.prm
dirrpt/
dirrpt/ESAN.rpt
dirrpt/ESAN0.rpt
dirrpt/ESAN1.rpt
dirrpt/ESAN2.rpt
dirrpt/ESAN3.rpt
dirrpt/ESAN4.rpt
dirrpt/ESAN5.rpt
dirrpt/ESAN6.rpt
dirrpt/ESAN7.rpt
dirrpt/ESAN8.rpt
dirrpt/ESAN9.rpt
dirrpt/MGR.rpt
dirrpt/MGR0.rpt
dirrpt/MGR1.rpt
dirrpt/MGR2.rpt
dirrpt/MGR3.rpt
dirrpt/MGR4.rpt
dirrpt/MGR5.rpt
dirrpt/MGR6.rpt
dirrpt/MGR7.rpt
dirrpt/MGR8.rpt
dirrpt/MGR9.rpt
dirsql/
dirtmp/


sent 6.96K bytes  received 767 bytes  15.44K bytes/sec
total size is 237.10M  speedup is 30704.36 (DRY RUN)

oracle@arrow:las:/u01/app/ggs01
$ ls -l /u01/app/ggs03/
total 0

Perform actual rsync

oracle@arrow:las:/u01/app/ggs01

$ rsync -avh --delete --exclude 'dirdatold' /u01/app/ggs01/ arrow:/u01/app/ggs03/
oracle@arrow's password:

sending incremental file list
./

.....

output ommited for brevity

dirout/
dirpcs/
dirprm/
dirprm/esan.prm
dirprm/jagent.prm
dirprm/mgr.prm
dirrpt/
dirrpt/ESAN.rpt
dirrpt/ESAN0.rpt
dirrpt/ESAN1.rpt
dirrpt/ESAN2.rpt
dirrpt/ESAN3.rpt
dirrpt/ESAN4.rpt
dirrpt/ESAN5.rpt
dirrpt/ESAN6.rpt
dirrpt/ESAN7.rpt
dirrpt/ESAN8.rpt
dirrpt/ESAN9.rpt
dirrpt/MGR.rpt
dirrpt/MGR0.rpt
dirrpt/MGR1.rpt
dirrpt/MGR2.rpt
dirrpt/MGR3.rpt
dirrpt/MGR4.rpt
dirrpt/MGR5.rpt
dirrpt/MGR6.rpt
dirrpt/MGR7.rpt
dirrpt/MGR8.rpt
dirrpt/MGR9.rpt
dirsql/
dirtmp/

sent 237.14M bytes  received 4.40K bytes  31.62M bytes/sec
total size is 237.10M  speedup is 1.00

oracle@arrow:las:/u01/app/ggs01
$ ls -ld /u01/app/ggs03/dir*
drwxr-x---. 2 oracle oinstall 4096 Jan 13 13:12 /u01/app/ggs03/dirchk

lrwxrwxrwx. 1 oracle oinstall   15 Feb 15 06:20 /u01/app/ggs03/dirdat -> /oradata/backup

drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 /u01/app/ggs03/dirdef
drwxr-x---. 2 oracle oinstall 4096 Apr  4  2014 /u01/app/ggs03/dirjar
drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 /u01/app/ggs03/dirout
drwxr-x---. 2 oracle oinstall 4096 Feb 12 15:35 /u01/app/ggs03/dirpcs
drwxr-x---. 2 oracle oinstall 4096 Jan 13 12:55 /u01/app/ggs03/dirprm
drwxr-x---. 2 oracle oinstall 4096 Feb 12 15:36 /u01/app/ggs03/dirrpt
drwxr-x---. 2 oracle oinstall 4096 Apr 26  2014 /u01/app/ggs03/dirsql
drwxr-x---. 2 oracle oinstall 4096 Sep 25 08:56 /u01/app/ggs03/dirtmp
oracle@arrow:las:/u01/app/ggs01
$

Did it work?

oracle@arrow:las:/u01/app/ggs01
$ cd /u01/app/ggs03/
oracle@arrow:las:/u01/app/ggs03
$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.21 18343248 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_140404.1029_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr  4 2014 15:18:36

Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.



GGSCI (arrow.localdomain) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED
EXTRACT     STOPPED     ESAN        00:01:02      788:32:31
REPLICAT    STOPPED     RLAS_SAN    00:00:00      4989:14:29


GGSCI (arrow.localdomain) 2> exit

The above was from a neglected test environment
Delete details for extract/replicat at dirchk

oracle@arrow:las:/u01/app/ggs03
$ cd dirchk/
oracle@arrow:las:/u01/app/ggs03/dirchk
$ ll
total 8
-rw-r-----. 1 oracle oinstall 2048 Jan 13 13:12 ESAN.cpe
-rw-r-----. 1 oracle oinstall 2048 Jul 22  2014 RLAS_SAN.cpr
oracle@arrow:las:/u01/app/ggs03/dirchk
$ rm *
oracle@arrow:las:/u01/app/ggs03/dirchk
$ cd ../dirpcs/
oracle@arrow:las:/u01/app/ggs03/dirpcs
$ ll
total 0
oracle@arrow:las:/u01/app/ggs03/dirpcs
$ cd ..
oracle@arrow:las:/u01/app/ggs03
$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.21 18343248 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_140404.1029_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr  4 2014 15:18:36

Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.



GGSCI (arrow.localdomain) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED


GGSCI (arrow.localdomain) 2> start mgr

Manager started.


GGSCI (arrow.localdomain) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED


GGSCI (arrow.localdomain) 4> exit

Now, what’s wrong?

oracle@arrow:las:/u01/app/ggs03
$ tail ggserr.log
2015-02-12 15:36:02  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): start mgr.
2015-02-12 15:36:02  ERROR   OGG-00664  Oracle GoldenGate Manager for Oracle, mgr.prm:  OCI Error during OCIServerAttach (status = 12541-ORA-12541: TNS:no listener).
2015-02-12 15:36:02  ERROR   OGG-01668  Oracle GoldenGate Manager for Oracle, mgr.prm:  PROCESS ABENDING.
2015-02-12 15:36:04  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): info all.
2015-02-15 09:44:51  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): info all.
2015-02-15 09:45:31  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): info all.
2015-02-15 09:45:41  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): start mgr.
2015-02-15 09:45:43  ERROR   OGG-00664  Oracle GoldenGate Manager for Oracle, mgr.prm:  OCI Error during OCIServerAttach (status = 12541-ORA-12541: TNS:no listener).
2015-02-15 09:45:43  ERROR   OGG-01668  Oracle GoldenGate Manager for Oracle, mgr.prm:  PROCESS ABENDING.
2015-02-15 09:45:44  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): info all.

oracle@arrow:las:/u01/app/ggs03
$ cat dirprm/mgr.prm
PORT 7901
DYNAMICPORTLIST 15100-15120

USERID ggs@san, PASSWORD *****

PURGEOLDEXTRACTS /u01/app/ggs01/dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3
PURGEMARKERHISTORY MINKEEPDAYS 5, MAXKEEPDAYS 7, FREQUENCYHOURS 24
PURGEDDLHISTORY MINKEEPDAYS 5, MAXKEEPDAYS 7, FREQUENCYHOURS 24

AUTOSTART ER *
AUTORESTART ER *, RETRIES 5, WAITMINUTES 2, RESETMINUTES 60

CHECKMINUTES 1
LAGINFOMINUTES 0
LAGCRITICALMINUTES 1

oracle@arrow:las:/u01/app/ggs03
$ vi dirprm/mgr.prm

oracle@arrow:las:/u01/app/ggs03
$ cat dirprm/mgr.prm
PORT 7901
DYNAMICPORTLIST 15100-15120

-- USERID ggs@san, PASSWORD 888

PURGEOLDEXTRACTS /u01/app/ggs01/dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3
PURGEMARKERHISTORY MINKEEPDAYS 5, MAXKEEPDAYS 7, FREQUENCYHOURS 24
PURGEDDLHISTORY MINKEEPDAYS 5, MAXKEEPDAYS 7, FREQUENCYHOURS 24

AUTOSTART ER *
AUTORESTART ER *, RETRIES 5, WAITMINUTES 2, RESETMINUTES 60

CHECKMINUTES 1
LAGINFOMINUTES 0
LAGCRITICALMINUTES 1
oracle@arrow:las:/u01/app/ggs03
$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.21 18343248 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_140404.1029_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr  4 2014 15:18:36

Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.



GGSCI (arrow.localdomain) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED


GGSCI (arrow.localdomain) 2> start mgr

Manager started.


GGSCI (arrow.localdomain) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING


GGSCI (arrow.localdomain) 4> exit
oracle@arrow:las:/u01/app/ggs03
$

Don’t forget Oracle libraries are required to run Goldengate

oracle@arrow:las:/u01/app/ggs03
$ ldd ggsci
        linux-vdso.so.1 =>  (0x00007fff95ffa000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000039e6000000)
        libgglog.so => /u01/app/ggs03/./libgglog.so (0x00007f862ca8d000)
        libggrepo.so => /u01/app/ggs03/./libggrepo.so (0x00007f862c923000)
        libdb-5.2.so => /u01/app/ggs03/./libdb-5.2.so (0x00007f862c688000)
        libicui18n.so.38 => /u01/app/ggs03/./libicui18n.so.38 (0x00007f862c327000)
        libicuuc.so.38 => /u01/app/ggs03/./libicuuc.so.38 (0x00007f862bfee000)
        libicudata.so.38 => /u01/app/ggs03/./libicudata.so.38 (0x00007f862b012000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000039e6800000)
        libxerces-c.so.28 => /u01/app/ggs03/./libxerces-c.so.28 (0x00007f862aafa000)
        libantlr3c.so => /u01/app/ggs03/./libantlr3c.so (0x00007f862a9e4000)

        libnnz11.so => /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so (0x00007f862a616000)
        libclntsh.so.11.1 => /u01/app/oracle/product/11.2.0/dbhome_1/lib/libclntsh.so.11.1 (0x00007f8627ba0000)

        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000039f1c00000)
        libm.so.6 => /lib64/libm.so.6 (0x00000039e7400000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000039f1800000)
        libc.so.6 => /lib64/libc.so.6 (0x00000039e6400000)
        /lib64/ld-linux-x86-64.so.2 (0x00000039e5c00000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00000039f3400000)
        libaio.so.1 => /lib64/libaio.so.1 (0x00007f862799d000)

oracle@arrow:las:/u01/app/ggs03
$ env |egrep 'HOME|LD'
OLDPWD=/home/oracle
LD_LIBRARY_PATH=/u01/app/oracle/product/11.2.0/dbhome_1/lib:/lib:/usr/lib
HOME=/home/oracle
ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
oracle@arrow:las:/u01/app/ggs03
$ unset ORACLE_HOME
oracle@arrow:las:/u01/app/ggs03
$ export LD_LIBRARY_PATH=/lib:/usr/lib
oracle@arrow:las:/u01/app/ggs03
$ ./ggsci
./ggsci: error while loading shared libraries: libnnz11.so: cannot open shared object file: No such file or directory
oracle@arrow:las:/u01/app/ggs03
$

February 10, 2015

Goldengate – start replicat ATSCN or AFTERSCN ?

Filed under: DataPump,GoldenGate,RMAN — mdinh @ 3:36 am

When using Goldengate to instantiate target database from an Oracle source database, replicat process can be started to concide with extract based the method used for instantiation, e.g. RMAN or datapump.

ATSCN is used to start replicat if RMAN is used to instantiate target.
From Database Backup and Recovery Reference, UNTIL SCN specifies an SCN as an upper limit.
RMAN restore or recover up to but not including the specified SCN.

AFTERSCN is used to start replicat if datapump is used to instantiate target.
The export operation performed is consistent as of FLASHBACK_SCN.

Hope this helps to clear up when to use ATSCN versus AFTERSCN.

Referemce:
Oracle GoldenGate Best Practices: Instantiation from an Oracle Source Database –  Doc ID 1276058.1

December 14, 2014

GOLDENDATE HA MAA RAC ACFS XAG

Filed under: 11g,GoldenGate,RAC — mdinh @ 11:39 pm

Purpose is to demonstrate how to create HA for Bi-Directional Replication Goldengate installed on ACFS with RAC cluster using XAG.

XAG simplifies the process since there are no requirements to create action scripts.

Please review REFERENCE section for versions used in test case and versions requirements.

Goldengate is installed on ACFS for simplicity; otherwise, at a minimum the following directories br, dirchk, dirdat, dirtmp will need to be on shared storage with symbolic links if installed on local storage. Keyword is minimum until you find out more directories are required.

Role separation was a huge PITA and do not attempt to perform chmod -R 775 /u01 as it will break since the setuid get unset.
Even with chmod 6751 oracle may prove to be ineffective and relink was done.

# id grid
uid=54322(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),493(vboxsf),54323(asmadmin),54324(asmdba)

# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),493(vboxsf),54324(asmdba)

# id gguser
uid=54323(gguser) gid=54321(oinstall) groups=54321(oinstall),54322(dba),493(vboxsf)

$ ls -l $ORACLE_HOME/bin/oracle
-rwsr-s--x. 1 grid oinstall 209914519 Dec  7 20:57 /u01/app/11.2.0.4/grid/bin/oracle

Last, DO NOT stop Goldengate using ggsci. For cluster-aware Goldengate, use $XAG_HOME/bin/agctl.

The configuration shown is for one cluster only and the same would have to be performed on the other clusters.

AS ROOT, CONFIGURE GOLDENGATE VIP (lax-ggate1-vip)
1. Determine the network number

# crsctl stat res -p|grep -ie .network -ie subnet|grep -ie name -ie ora_subnet
NAME=ora.net1.network
USR_ORA_SUBNET=192.168.56.0

2. Create GoldenGate VIP with naming Convention: LAX (closest airport code to data center), ggate1 (for Goldengate on network number 1), vip
Note IP address provided is 192.168.56.41

# appvipcfg create -network=1 -ip=192.168.56.41 -vipname=lax-ggate1-vip -user=root -group=oinstall
Production Copyright 2007, 2008, Oracle.All rights reserved
2014-12-14 10:58:41: Creating Resource Type
2014-12-14 10:58:41: Executing /u01/app/11.2.0.4/grid/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /u01/app/11.2.0.4/grid/crs/template/appvip.type
2014-12-14 10:58:41: Executing cmd: /u01/app/11.2.0.4/grid/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /u01/app/11.2.0.4/grid/crs/template/appvip.type
2014-12-14 10:58:42: Create the Resource
2014-12-14 10:58:42: Executing /u01/app/11.2.0.4/grid/bin/crsctl add resource lax-ggate1-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=192.168.56.41,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:root:rwx,pgrp:root:r-x,other::r--,group:oinstall:r-x,user:root:r-x',HOSTING_MEMBERS=rac01.localdomain,APPSVIP_FAILBACK="
2014-12-14 10:58:42: Executing cmd: /u01/app/11.2.0.4/grid/bin/crsctl add resource lax-ggate1-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=192.168.56.41,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:root:rwx,pgrp:root:r-x,other::r--,group:oinstall:r-x,user:root:r-x',HOSTING_MEMBERS=rac01.localdomain,APPSVIP_FAILBACK="

3. Set permission for users to start/stop VIP

crsctl setperm resource lax-ggate1-vip -u user:oracle:r-x
# crsctl setperm resource lax-ggate1-vip -u user:grid:r-x
# crsctl setperm resource lax-ggate1-vip -u user:gguser:r-x
# crsctl start resource lax-ggate1-vip
CRS-2672: Attempting to start 'lax-ggate1-vip' on 'rac02'
CRS-2676: Start of 'lax-ggate1-vip' on 'rac02' succeeded

RETRIEVE INFORMATIONS TO CREATE GOLDENGATE AGENT
–nodes

$ olsnodes
rac01
rac02

–filesystems

$ crsctl stat res -w "TYPE = ora.acfs.type" -p|grep '^NAME'
NAME=ora.dg_acfs.vg_acfs.acfs
NAME=ora.dg_acfs.vg_acfs.acfs

–databases

$ crsctl stat res -w "TYPE = ora.database.type"
NAME=ora.emu.db
TYPE=ora.database.type
TARGET=ONLINE         , ONLINE
STATE=ONLINE on rac01, ONLINE on rac02

— extracts and replicats

GGSCI (rac02.localdomain) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     ELAX        00:00:00      00:00:02
EXTRACT     RUNNING     PLAX_DEN    00:00:00      00:00:03
REPLICAT    RUNNING     RDEN_LAX    00:00:00      00:00:06

AS GRID, CONFIGURE GOLDENGATE AGENT (lax_ggate)
Options for –instance_type is source|target|dual and I am interpreting dual is bi-directional.

$XAG_HOME/bin/agctl add goldengate lax_ggate \
--gg_home /acfsmount/ggs112 \
--instance_type dual \
--nodes rac01,rac02 \
--vip_name lax-ggate1-vip \
--filesystems ora.dg_acfs.vg_acfs.acfs \
--databases ora.emu.db \
--oracle_home /u01/app/oracle/product/11.2.0.4/db_1 \
--monitor_extracts ELAX,PLAX_DEN \
--critical_extracts ELAX,PLAX_DEN \
--monitor_replicats RDEN_LAX \
--critical_replicats RDEN_LAX

VERIFY RESULTS

$ $XAG_HOME/bin/agctl config goldengate lax_ggate
GoldenGate location is: /acfsmount/ggs112
GoldenGate instance type is: dual
Configured to run on Nodes: rac01 rac02
ORACLE_HOME location is: /u01/app/oracle/product/11.2.0.4/db_1
Databases needed: ora.emu.db
File System resources needed: ora.dg_acfs.vg_acfs.acfs
Extracts to monitor: ELAX,PLAX_DEN
Replicats to monitor: RDEN_LAX
Critical extracts: ELAX,PLAX_DEN
Critical replicats: RDEN_LAX
Autostart on DataGuard role transition to PRIMARY: no
Autostart JAgent: no

START GOLDENDATE

$XAG_HOME/bin/agctl start goldengate lax_ggate

CHECK GOLDENGATE STATUS

$XAG_HOME/bin/agctl status goldengate lax_ggate
Goldengate  instance 'lax_ggate' INTERMEDIATE on rac02

$XAG_HOME/bin/agctl status goldengate lax_ggate
Goldengate  instance 'lax_ggate' is running on rac02

$ crsstat

Resource Name                            Resource Type  Target       State        Node            State Details
---------------------------------------- -------------- ------------ ------------ --------------- ---------------
lax-ggate1-vip                           appvip_net1    ONLINE       ONLINE       rac02
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac02
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac01
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac02
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac01
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac02
ora.LISTENER_SCAN1.lsnr                  SCAN Listener  ONLINE       ONLINE       rac01
ora.LISTENER_SCAN2.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.LISTENER_SCAN3.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.asm                                  ASM            ONLINE       ONLINE       rac01           Started
ora.asm                                  ASM            ONLINE       ONLINE       rac02           Started
ora.cvu                                  cvu            ONLINE       ONLINE       rac02
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac01           mounted on /acfsmount
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac02           mounted on /acfsmount
ora.emu.db                               database       ONLINE       ONLINE       rac01           Open
ora.emu.db                               database       ONLINE       ONLINE       rac02           Open
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac01
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac02
ora.oc4j                                 OC4J           ONLINE       ONLINE       rac02
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac01
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac02
ora.rac01.vip                            Cluster VIP    ONLINE       ONLINE       rac01
ora.rac02.vip                            Cluster VIP    ONLINE       ONLINE       rac02
ora.registry.acfs                        registry       ONLINE       ONLINE       rac01
ora.registry.acfs                        registry       ONLINE       ONLINE       rac02
ora.scan1.vip                            SCAN VIP       ONLINE       ONLINE       rac01
ora.scan2.vip                            SCAN VIP       ONLINE       ONLINE       rac02
ora.scan3.vip                            SCAN VIP       ONLINE       ONLINE       rac02
xag.lax_ggate.goldengate                 goldengate     ONLINE       ONLINE       rac02

CONNECT TO THE CORRECT NODE FOR GOLDENGATE

$ su - gguser
Password:
0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!!!!! PLEASE CONNECT TO NODE WHERE GOLDENGATE IS RUNNING !!!!!
!!!!!                                                    !!!!!
--+ Goldengate  instance 'lax_ggate' is running on rac02 +--
!!!!!                                                    !!!!!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[gguser@rac01:/home/gguser]
$ ssh rac02
Last login: Sun Dec 14 13:23:48 2014 from rac01
1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~ GOLDENGATE IS RUNNING ON THIS NODE - GOOD TO GO    ~~~~~
--+ Goldengate  instance 'lax_ggate' is running on rac02 +--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[gguser@rac02:/home/gguser]
$

.bash_profile
# User specific environment and startup programs
ogg=`/u01/app/grid/xag/bin/agctl status goldengate lax_ggate`
/u01/app/grid/xag/bin/agctl status goldengate lax_ggate|grep -ic `hostname -f`
if [ "$?" != "0" ]; then
  clear
  echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
  echo "!!!!! PLEASE CONNECT TO NODE WHERE GOLDENGATE IS RUNNING !!!!!"
  echo "!!!!!                                                    !!!!!"
  echo "--+ $ogg +--"
  echo "!!!!!                                                    !!!!!"
  echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
else
  clear
  echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
  echo "~~~~~ GOLDENGATE IS RUNNING ON THIS NODE - GOOD TO GO    ~~~~~"
  echo "--+ $ogg +--"
  echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
fi
umask 0002

RELOCATE GOLDENGATE TO DIFFERENT NODE

[grid@rac01:+ASM1:/home/grid]
$ crsstat


Resource Name                            Resource Type  Target       State        Node            State Details
---------------------------------------- -------------- ------------ ------------ --------------- ---------------
lax-ggate1-vip                           appvip_net1    ONLINE       ONLINE       rac02
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac02
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac01
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac02
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac01
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac02
ora.LISTENER_SCAN1.lsnr                  SCAN Listener  ONLINE       ONLINE       rac01
ora.LISTENER_SCAN2.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.LISTENER_SCAN3.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.asm                                  ASM            ONLINE       ONLINE       rac01           Started
ora.asm                                  ASM            ONLINE       ONLINE       rac02           Started
ora.cvu                                  cvu            ONLINE       ONLINE       rac02
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac01           mounted on /acfsmount
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac02           mounted on /acfsmount
ora.emu.db                               database       ONLINE       ONLINE       rac01           Open
ora.emu.db                               database       ONLINE       ONLINE       rac02           Open
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac01
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac02
ora.oc4j                                 OC4J           ONLINE       ONLINE       rac02
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac01
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac02
ora.rac01.vip                            Cluster VIP    ONLINE       ONLINE       rac01
ora.rac02.vip                            Cluster VIP    ONLINE       ONLINE       rac02
ora.registry.acfs                        registry       ONLINE       ONLINE       rac01
ora.registry.acfs                        registry       ONLINE       ONLINE       rac02
ora.scan1.vip                            SCAN VIP       ONLINE       ONLINE       rac01
ora.scan2.vip                            SCAN VIP       ONLINE       ONLINE       rac02
ora.scan3.vip                            SCAN VIP       ONLINE       ONLINE       rac02
xag.lax_ggate.goldengate                 goldengate     ONLINE       ONLINE       rac02

[grid@rac01:+ASM1:/home/grid]
$ $XAG_HOME/bin/agctl relocate goldengate lax_ggate --node rac01
[grid@rac01:+ASM1:/home/grid]
$ crsstat


Resource Name                            Resource Type  Target       State        Node            State Details
---------------------------------------- -------------- ------------ ------------ --------------- ---------------
lax-ggate1-vip                           appvip_net1    ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac02
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac01
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac02
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac01
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac02
ora.LISTENER_SCAN1.lsnr                  SCAN Listener  ONLINE       ONLINE       rac01
ora.LISTENER_SCAN2.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.LISTENER_SCAN3.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.asm                                  ASM            ONLINE       ONLINE       rac01           Started
ora.asm                                  ASM            ONLINE       ONLINE       rac02           Started
ora.cvu                                  cvu            ONLINE       ONLINE       rac02
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac01           mounted on /acfsmount
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac02           mounted on /acfsmount
ora.emu.db                               database       ONLINE       ONLINE       rac01           Open
ora.emu.db                               database       ONLINE       ONLINE       rac02           Open
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac01
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac02
ora.oc4j                                 OC4J           ONLINE       ONLINE       rac02
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac01
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac02
ora.rac01.vip                            Cluster VIP    ONLINE       ONLINE       rac01
ora.rac02.vip                            Cluster VIP    ONLINE       ONLINE       rac02
ora.registry.acfs                        registry       ONLINE       ONLINE       rac01
ora.registry.acfs                        registry       ONLINE       ONLINE       rac02
ora.scan1.vip                            SCAN VIP       ONLINE       ONLINE       rac01
ora.scan2.vip                            SCAN VIP       ONLINE       ONLINE       rac02
ora.scan3.vip                            SCAN VIP       ONLINE       ONLINE       rac02
xag.lax_ggate.goldengate                 goldengate     ONLINE       INTERMEDIATE rac01           ER(s) not running : ELAX,ELAX,RDEN_LAX

[grid@rac01:+ASM1:/home/grid]
$ crsstat


Resource Name                            Resource Type  Target       State        Node            State Details
---------------------------------------- -------------- ------------ ------------ --------------- ---------------
lax-ggate1-vip                           appvip_net1    ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac01
ora.DATA2.dg                             diskgroup      ONLINE       ONLINE       rac02
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac01
ora.DG_ACFS.dg                           diskgroup      ONLINE       ONLINE       rac02
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac01
ora.LISTENER.lsnr                        Listener       ONLINE       ONLINE       rac02
ora.LISTENER_SCAN1.lsnr                  SCAN Listener  ONLINE       ONLINE       rac01
ora.LISTENER_SCAN2.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.LISTENER_SCAN3.lsnr                  SCAN Listener  ONLINE       ONLINE       rac02
ora.asm                                  ASM            ONLINE       ONLINE       rac01           Started
ora.asm                                  ASM            ONLINE       ONLINE       rac02           Started
ora.cvu                                  cvu            ONLINE       ONLINE       rac02
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac01           mounted on /acfsmount
ora.dg_acfs.vg_acfs.acfs                 acfs           ONLINE       ONLINE       rac02           mounted on /acfsmount
ora.emu.db                               database       ONLINE       ONLINE       rac01           Open
ora.emu.db                               database       ONLINE       ONLINE       rac02           Open
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.gsd                                  Gbl Svc Daemon OFFLINE      OFFLINE
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac01
ora.net1.network                         Network (VIP)  ONLINE       ONLINE       rac02
ora.oc4j                                 OC4J           ONLINE       ONLINE       rac02
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac01
ora.ons                                  Ora Notif Svc  ONLINE       ONLINE       rac02
ora.rac01.vip                            Cluster VIP    ONLINE       ONLINE       rac01
ora.rac02.vip                            Cluster VIP    ONLINE       ONLINE       rac02
ora.registry.acfs                        registry       ONLINE       ONLINE       rac01
ora.registry.acfs                        registry       ONLINE       ONLINE       rac02
ora.scan1.vip                            SCAN VIP       ONLINE       ONLINE       rac01
ora.scan2.vip                            SCAN VIP       ONLINE       ONLINE       rac02
ora.scan3.vip                            SCAN VIP       ONLINE       ONLINE       rac02
xag.lax_ggate.goldengate                 goldengate     ONLINE       ONLINE       rac01

STOP GOLDENDATE

[grid@rac01:+ASM1:/home/grid]
$ $XAG_HOME/bin/agctl stop goldengate lax_ggate
[grid@rac01:+ASM1:/home/grid]

DISCOVERY
It’s easy to know where everything when you are the architect, but what happens when you are not?
How to find the Goldengate VIP and is it in DNS?

$ crsctl stat res lax-ggate1-vip -p|grep USR_ORA_VIP
GEN_USR_ORA_VIP=
USR_ORA_VIP=192.168.56.41

$ nslookup 192.168.56.41
Server:         127.0.0.1
Address:        127.0.0.1#53

41.56.168.192.in-addr.arpa      name = lax-ggate1-vip.localdomain.

$ crsctl stat res -w "TYPE = ora.acfs.type" -p|grep '^MOUNT'
MOUNTPOINT_PATH=/acfsmount
MOUNTPOINT_PATH=/acfsmount

REFERENCE:

Oracle GoldenGate Best Practices: 
Configuring Oracle GoldenGate with Oracle Grid Infrastructure Bundled Agents (XAG) - (Document 1527310.1) 

Oracle Clusterware: 

http://oracle.com/goto/clusterware

Oracle Grid Infrastructure (Bundled) Agents: 

http://www.oracle.com/technetwork/database/database-technologies/clusterware/downloads/index.html

Oracle Grid Infrastructure Agents Version 5.1
11.2.0.3 and later 
12.1.0.1 and later 

http://www.oracle.com/technetwork/database/database-technologies/clusterware/downloads/ogiba-2189738.pdf

SRDC - Data to Collect on GoldenGate Issues Related to XAG (Doc ID 1913048.1)	

Patch 16762459: ORACLE GOLDENGATE V11.2.1.0.7 FOR ORACLE 11G

[gguser@angel:/u01/app/ggs112]
$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.7 16934304 OGGCORE_11.2.1.0.7_PLATFORMS_130709.1600.1_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Jul 18 2013 07:04:28

Copyright (C) 1995, 2013, Oracle and/or its affiliates. All rights reserved.

GGSCI (angel.local) 1> dblogin userid ggadmin, password ggadmin
Successfully logged into database.

GGSCI (angel.local) 2> versions
Operating System:
Linux
Version #1 SMP Fri Feb 22 18:16:18 PST 2013, Release 2.6.39-400.17.1.el6uek.x86_64
Node: angel.local
Machine: x86_64

Database:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production

December 5, 2014

GUI Be Gone

Filed under: oracle — mdinh @ 12:08 am

I am losing the luxury of using GUI since clients typically may not have X-Windows or VNC Server installed

Adapt or die and re-learning command line again.

Better to verify before installation than to clean up a problematic install.

Use runcluvfy to verify DB install:
runcluvfy.sh stage -pre dbinst -n rac01,rac02 -r 11gR2 -d /u01/app/oracle/product/11.2.0.4/db_1 -osdba dba -fixup -fixupdir /tmp -verbose

Use runInstaller -executePrereqs to verify responseFile for silent install and detect issues:
runInstaller -silent -executePrereqs -waitforcompletion -force -responseFile /media/sf_Linux/11.2.0.4/database/rac_db_swonly.rsp

Use grep to find results from installActions log:
grep -e ‘[[:upper:]]:’ installActions2014-12-04_02-49-27PM.log|cut -d “:” -f1|sort -u

DEMO:

[oracle@rac01:/media/sf_Linux/11.2.0.4/grid]
$ ./runcluvfy.sh stage -pre dbinst -n rac01,rac02 -r 11gR2 -d /u01/app/oracle/product/11.2.0.4/db_1 -osdba dba -fixup -fixupdir /tmp -verbose

Performing pre-checks for database installation

Checking node reachability...

Check: Node reachability from node "rac01"
  Destination Node                      Reachable?
  ------------------------------------  ------------------------
  rac02                                 yes
  rac01                                 yes
Result: Node reachability check passed from node "rac01"


Checking user equivalence...

Check: User equivalence for user "oracle"
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
Result: User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed

Verification of the hosts config file successful


Interface information for node "rac02"
 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   10.0.2.15       10.0.2.0        0.0.0.0         10.0.2.2        08:00:27:87:91:11 1500
 eth1   192.168.56.12   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:4A:7B:27 1500
 eth1   192.168.56.33   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:4A:7B:27 1500
 eth1   192.168.56.32   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:4A:7B:27 1500
 eth1   192.168.56.22   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:4A:7B:27 1500
 eth2   10.10.10.12     10.0.0.0        0.0.0.0         10.0.2.2        08:00:27:E8:D6:21 1500
 eth2   169.254.82.236  169.254.0.0     0.0.0.0         10.0.2.2        08:00:27:E8:D6:21 1500


Interface information for node "rac01"
 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   10.0.2.15       10.0.2.0        0.0.0.0         10.0.2.2        08:00:27:02:B1:57 1500
 eth1   192.168.56.11   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:BD:66:A4 1500
 eth1   192.168.56.21   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:BD:66:A4 1500
 eth1   192.168.56.31   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:BD:66:A4 1500
 eth2   10.10.10.11     10.0.0.0        0.0.0.0         10.0.2.2        08:00:27:60:79:0F 1500
 eth2   169.254.34.109  169.254.0.0     0.0.0.0         10.0.2.2        08:00:27:60:79:0F 1500


Check: Node connectivity for interface "eth1"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac02[192.168.56.12]            rac02[192.168.56.33]            yes
  rac02[192.168.56.12]            rac02[192.168.56.32]            yes
  rac02[192.168.56.12]            rac02[192.168.56.22]            yes
  rac02[192.168.56.12]            rac01[192.168.56.11]            yes
  rac02[192.168.56.12]            rac01[192.168.56.21]            yes
  rac02[192.168.56.12]            rac01[192.168.56.31]            yes
  rac02[192.168.56.33]            rac02[192.168.56.32]            yes
  rac02[192.168.56.33]            rac02[192.168.56.22]            yes
  rac02[192.168.56.33]            rac01[192.168.56.11]            yes
  rac02[192.168.56.33]            rac01[192.168.56.21]            yes
  rac02[192.168.56.33]            rac01[192.168.56.31]            yes
  rac02[192.168.56.32]            rac02[192.168.56.22]            yes
  rac02[192.168.56.32]            rac01[192.168.56.11]            yes
  rac02[192.168.56.32]            rac01[192.168.56.21]            yes
  rac02[192.168.56.32]            rac01[192.168.56.31]            yes
  rac02[192.168.56.22]            rac01[192.168.56.11]            yes
  rac02[192.168.56.22]            rac01[192.168.56.21]            yes
  rac02[192.168.56.22]            rac01[192.168.56.31]            yes
  rac01[192.168.56.11]            rac01[192.168.56.21]            yes
  rac01[192.168.56.11]            rac01[192.168.56.31]            yes
  rac01[192.168.56.21]            rac01[192.168.56.31]            yes
Result: Node connectivity passed for interface "eth1"


Check: TCP connectivity of subnet "192.168.56.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac01:192.168.56.11             rac02:192.168.56.12             passed
  rac01:192.168.56.11             rac02:192.168.56.33             passed
  rac01:192.168.56.11             rac02:192.168.56.32             passed
  rac01:192.168.56.11             rac02:192.168.56.22             passed
  rac01:192.168.56.11             rac01:192.168.56.21             passed
  rac01:192.168.56.11             rac01:192.168.56.31             passed
Result: TCP connectivity check passed for subnet "192.168.56.0"


Check: Node connectivity for interface "eth2"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac02[10.10.10.12]              rac01[10.10.10.11]              yes
Result: Node connectivity passed for interface "eth2"


Check: TCP connectivity of subnet "10.0.0.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac01:10.10.10.11               rac02:10.10.10.12               passed
Result: TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.56.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Result: Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.56.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.56.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Check: Total memory
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         3.8674GB (4055296.0KB)    1GB (1048576.0KB)         passed
  rac01         3.8674GB (4055296.0KB)    1GB (1048576.0KB)         passed
Result: Total memory check passed

Check: Available memory
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         2.6666GB (2796104.0KB)    50MB (51200.0KB)          passed
  rac01         2.7855GB (2920820.0KB)    50MB (51200.0KB)          passed
Result: Available memory check passed

Check: Swap space
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         8GB (8388604.0KB)         3.8674GB (4055296.0KB)    passed
  rac01         8GB (8388604.0KB)         3.8674GB (4055296.0KB)    passed
Result: Swap space check passed

Check: Free disk space for "rac02:/u01/app/oracle/product/11.2.0.4/db_1,rac02:/tmp"
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /u01/app/oracle/product/11.2.0.4/db_1  rac02         /             45.459GB      6.7GB         passed
  /tmp              rac02         /             45.459GB      6.7GB         passed
Result: Free disk space check passed for "rac02:/u01/app/oracle/product/11.2.0.4/db_1,rac02:/tmp"

Check: Free disk space for "rac01:/u01/app/oracle/product/11.2.0.4/db_1,rac01:/tmp"
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /u01/app/oracle/product/11.2.0.4/db_1  rac01         /             45.4215GB     6.7GB         passed
  /tmp              rac01         /             45.4215GB     6.7GB         passed
Result: Free disk space check passed for "rac01:/u01/app/oracle/product/11.2.0.4/db_1,rac01:/tmp"

Check: User existence for "oracle"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists(54321)
  rac01         passed                    exists(54321)

Checking for multiple users with UID value 54321
Result: Check for multiple users with UID value 54321 passed
Result: User existence check passed for "oracle"

Check: Group existence for "oinstall"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists
  rac01         passed                    exists
Result: Group existence check passed for "oinstall"

Check: Group existence for "dba"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists
  rac01         passed                    exists
Result: Group existence check passed for "dba"

Check: Membership of user "oracle" in group "oinstall" [as Primary]
  Node Name         User Exists   Group Exists  User in Group  Primary       Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             yes           yes           yes           yes           passed
  rac01             yes           yes           yes           yes           passed
Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed

Check: Membership of user "oracle" in group "dba"
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             yes           yes           yes           passed
  rac01             yes           yes           yes           passed
Result: Membership check for user "oracle" in group "dba" passed

Check: Run level
  Node Name     run level                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         5                         3,5                       passed
  rac01         5                         3,5                       passed
Result: Run level check passed

Check: Hard limits for "maximum open file descriptors"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             hard          65536         65536         passed
  rac01             hard          65536         65536         passed
Result: Hard limits check passed for "maximum open file descriptors"

Check: Soft limits for "maximum open file descriptors"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             soft          1024          1024          passed
  rac01             soft          1024          1024          passed
Result: Soft limits check passed for "maximum open file descriptors"

Check: Hard limits for "maximum user processes"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             hard          16384         16384         passed
  rac01             hard          16384         16384         passed
Result: Hard limits check passed for "maximum user processes"

Check: Soft limits for "maximum user processes"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             soft          16384         2047          passed
  rac01             soft          16384         2047          passed
Result: Soft limits check passed for "maximum user processes"

Check: System architecture
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         x86_64                    x86_64                    passed
  rac01         x86_64                    x86_64                    passed
Result: System architecture check passed

Check: Kernel version
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         2.6.39-400.17.1.el6uek.x86_64  2.6.32                    passed
  rac01         2.6.39-400.17.1.el6uek.x86_64  2.6.32                    passed
Result: Kernel version check passed

Check: Kernel parameter for "semmsl"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             250           250           250           passed
  rac01             250           250           250           passed
Result: Kernel parameter check passed for "semmsl"

Check: Kernel parameter for "semmns"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             32000         32000         32000         passed
  rac01             32000         32000         32000         passed
Result: Kernel parameter check passed for "semmns"

Check: Kernel parameter for "semopm"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             100           100           100           passed
  rac01             100           100           100           passed
Result: Kernel parameter check passed for "semopm"

Check: Kernel parameter for "semmni"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             128           128           128           passed
  rac01             128           128           128           passed
Result: Kernel parameter check passed for "semmni"

Check: Kernel parameter for "shmmax"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4398046511104  4398046511104  2076311552    passed
  rac01             4398046511104  4398046511104  2076311552    passed
Result: Kernel parameter check passed for "shmmax"

Check: Kernel parameter for "shmmni"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4096          4096          4096          passed
  rac01             4096          4096          4096          passed
Result: Kernel parameter check passed for "shmmni"

Check: Kernel parameter for "shmall"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4294967296    4294967296    2097152       passed
  rac01             4294967296    4294967296    2097152       passed
Result: Kernel parameter check passed for "shmall"

Check: Kernel parameter for "file-max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             6815744       6815744       6815744       passed
  rac01             6815744       6815744       6815744       passed
Result: Kernel parameter check passed for "file-max"

Check: Kernel parameter for "ip_local_port_range"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed
  rac01             between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed
Result: Kernel parameter check passed for "ip_local_port_range"

Check: Kernel parameter for "rmem_default"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             262144        262144        262144        passed
  rac01             262144        262144        262144        passed
Result: Kernel parameter check passed for "rmem_default"

Check: Kernel parameter for "rmem_max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4194304       4194304       4194304       passed
  rac01             4194304       4194304       4194304       passed
Result: Kernel parameter check passed for "rmem_max"

Check: Kernel parameter for "wmem_default"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             262144        262144        262144        passed
  rac01             262144        262144        262144        passed
Result: Kernel parameter check passed for "wmem_default"

Check: Kernel parameter for "wmem_max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             1048576       1048576       1048576       passed
  rac01             1048576       1048576       1048576       passed
Result: Kernel parameter check passed for "wmem_max"

Check: Kernel parameter for "aio-max-nr"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             1048576       1048576       1048576       passed
  rac01             1048576       1048576       1048576       passed
Result: Kernel parameter check passed for "aio-max-nr"

Check: Package existence for "binutils"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
  rac01         binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
Result: Package existence check passed for "binutils"

Check: Package existence for "compat-libcap1"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         compat-libcap1-1.10-1     compat-libcap1-1.10       passed
  rac01         compat-libcap1-1.10-1     compat-libcap1-1.10       passed
Result: Package existence check passed for "compat-libcap1"

Check: Package existence for "compat-libstdc++-33(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
  rac01         compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
Result: Package existence check passed for "compat-libstdc++-33(x86_64)"

Check: Package existence for "libgcc(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libgcc(x86_64)-4.4.7-11.el6  libgcc(x86_64)-4.4.4      passed
  rac01         libgcc(x86_64)-4.4.7-11.el6  libgcc(x86_64)-4.4.4      passed
Result: Package existence check passed for "libgcc(x86_64)"

Check: Package existence for "libstdc++(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libstdc++(x86_64)-4.4.7-11.el6  libstdc++(x86_64)-4.4.4   passed
  rac01         libstdc++(x86_64)-4.4.7-11.el6  libstdc++(x86_64)-4.4.4   passed
Result: Package existence check passed for "libstdc++(x86_64)"

Check: Package existence for "libstdc++-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libstdc++-devel(x86_64)-4.4.7-11.el6  libstdc++-devel(x86_64)-4.4.4  passed
  rac01         libstdc++-devel(x86_64)-4.4.7-11.el6  libstdc++-devel(x86_64)-4.4.4  passed
Result: Package existence check passed for "libstdc++-devel(x86_64)"

Check: Package existence for "sysstat"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
  rac01         sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
Result: Package existence check passed for "sysstat"

Check: Package existence for "gcc"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         gcc-4.4.7-11.el6          gcc-4.4.4                 passed
  rac01         gcc-4.4.7-11.el6          gcc-4.4.4                 passed
Result: Package existence check passed for "gcc"

Check: Package existence for "gcc-c++"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         gcc-c++-4.4.7-11.el6      gcc-c++-4.4.4             passed
  rac01         gcc-c++-4.4.7-11.el6      gcc-c++-4.4.4             passed
Result: Package existence check passed for "gcc-c++"

Check: Package existence for "ksh"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         ksh-20120801-21.el6.1     ksh-20100621              passed
  rac01         ksh-20120801-21.el6.1     ksh-20100621              passed
Result: Package existence check passed for "ksh"

Check: Package existence for "make"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         make-3.81-20.el6          make-3.81                 passed
  rac01         make-3.81-20.el6          make-3.81                 passed
Result: Package existence check passed for "make"

Check: Package existence for "glibc(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         glibc(x86_64)-2.12-1.149.el6  glibc(x86_64)-2.12        passed
  rac01         glibc(x86_64)-2.12-1.149.el6  glibc(x86_64)-2.12        passed
Result: Package existence check passed for "glibc(x86_64)"

Check: Package existence for "glibc-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         glibc-devel(x86_64)-2.12-1.149.el6  glibc-devel(x86_64)-2.12  passed
  rac01         glibc-devel(x86_64)-2.12-1.149.el6  glibc-devel(x86_64)-2.12  passed
Result: Package existence check passed for "glibc-devel(x86_64)"

Check: Package existence for "libaio(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
  rac01         libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
Result: Package existence check passed for "libaio(x86_64)"

Check: Package existence for "libaio-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
  rac01         libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
Result: Package existence check passed for "libaio-devel(x86_64)"

Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed

Check: Current group ID
Result: Current group ID check passed

Starting check for consistency of primary group of root user
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed

Check for consistency of root user's primary group passed

Check default user file creation mask
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         0022                      0022                      passed
  rac01         0022                      0022                      passed
Result: Default user file creation mask check passed

Checking CRS integrity...

Clusterware version consistency passed
The Oracle Clusterware is healthy on node "rac02"
The Oracle Clusterware is healthy on node "rac01"

CRS integrity check passed

Checking Cluster manager integrity...


Checking CSS daemon...

  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 running
  rac01                                 running

Oracle Cluster Synchronization Services appear to be online.

Cluster manager integrity check passed


Checking node application existence...

Checking existence of VIP node application (required)
  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         yes                       yes                       passed
  rac01         yes                       yes                       passed
VIP node application check passed

Checking existence of NETWORK node application (required)
  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         yes                       yes                       passed
  rac01         yes                       yes                       passed
NETWORK node application check passed

Checking existence of GSD node application (optional)
  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         no                        no                        exists
  rac01         no                        no                        exists
GSD node application is offline on nodes "rac02,rac01"

Checking existence of ONS node application (optional)
  Node Name     Required                  Running?                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         no                        yes                       passed
  rac01         no                        yes                       passed
ONS node application check passed


Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed

Checking if CTSS Resource is running on all nodes...
Check: CTSS Resource running on all nodes
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
Result: CTSS resource check passed


Querying CTSS for time offset on all nodes...
Result: Query of CTSS for time offset passed

Check CTSS state started...
Check: CTSS state
  Node Name                             State
  ------------------------------------  ------------------------
  rac02                                 Active
  rac01                                 Active
CTSS is in Active state. Proceeding with check of clock time offsets on all nodes...
Reference Time Offset Limit: 1000.0 msecs
Check: Reference Time Offset
  Node Name     Time Offset               Status
  ------------  ------------------------  ------------------------
  rac02         0.0                       passed
  rac01         0.0                       passed

Time offset is within the specified limits on the following set of nodes:
"[rac02, rac01]"
Result: Check of clock time offsets passed


Oracle Cluster Time Synchronization Services check passed
Checking consistency of file "/etc/resolv.conf" across nodes

Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined
File "/etc/resolv.conf" does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...
domain entry in file "/etc/resolv.conf" is consistent across nodes
Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes...
search entry in file "/etc/resolv.conf" is consistent across nodes
Checking file "/etc/resolv.conf" to make sure that only one search entry is defined
All nodes have one search entry defined in file "/etc/resolv.conf"
Checking all nodes to make sure that search entry is "localdomain" as found on node "rac02"
All nodes of the cluster have same value for 'search'
Checking DNS response time for an unreachable node
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
The DNS response time for an unreachable node is within acceptable limit on all nodes

File "/etc/resolv.conf" is consistent across nodes

Check: Time zone consistency
Result: Time zone consistency check passed

Checking Single Client Access Name (SCAN)...
  SCAN Name         Node          Running?      ListenerName  Port          Running?
  ----------------  ------------  ------------  ------------  ------------  ------------
  dinh-scan         rac01         true          LISTENER_SCAN1  1521          true
  dinh-scan         rac02         true          LISTENER_SCAN2  1521          true
  dinh-scan         rac02         true          LISTENER_SCAN3  1521          true

Checking TCP connectivity to SCAN Listeners...
  Node          ListenerName              TCP connectivity?
  ------------  ------------------------  ------------------------
  rac01         LISTENER_SCAN1            yes
  rac01         LISTENER_SCAN2            yes
  rac01         LISTENER_SCAN3            yes
TCP connectivity to SCAN Listeners exists on all cluster nodes

Checking name resolution setup for "dinh-scan"...

Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes...
Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is defined
More than one "hosts" entry does not exist in any "/etc/nsswitch.conf" file
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed

  SCAN Name     IP Address                Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  dinh-scan     192.168.56.33             passed
  dinh-scan     192.168.56.31             passed
  dinh-scan     192.168.56.32             passed

Verification of SCAN VIP and Listener setup passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Database and Clusterware version compatibility


Checking ASM and CRS version compatibility
ASM and CRS versions are compatible
Database version "11.2" is compatible with the Clusterware version "11.2.0.4.0".
Database Clusterware version compatibility passed

Pre-check for database installation was successful.
[oracle@rac01:/media/sf_Linux/11.2.0.4/grid]
$

[oracle@rac01:/media/sf_Linux/11.2.0.4/database]
$ ./runInstaller -silent -executePrereqs -showProgress -waitforcompletion -force -responseFile /media/sf_Linux/11.2.0.4/database/rac_db_swonly.rsp

Starting Oracle Universal Installer...

Checking Temp space: must be greater than 120 MB.   Actual 44381 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 8191 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2014-12-04_02-49-27PM. Please wait ...
[oracle@rac01:/media/sf_Linux/11.2.0.4/database]
$ cd /u01/app/oraInventory/logs/
[oracle@rac01:/u01/app/oraInventory/logs]
$ ls -lrt
total 1068
-rw-r-----. 1 grid   oinstall     47 Nov 30 19:40 time2014-11-30_07-40-41PM.log
-rw-r-----. 1 grid   oinstall      0 Nov 30 19:42 oraInstall2014-11-30_07-40-41PM.err
-rw-r-----. 1 grid   oinstall      0 Nov 30 19:45 oraInstall2014-11-30_07-40-41PM.err.rac02
-rw-r-----. 1 grid   oinstall    121 Nov 30 19:46 oraInstall2014-11-30_07-40-41PM.out.rac02
-rw-r-----. 1 grid   oinstall   7650 Nov 30 19:46 AttachHome2014-11-30_07-40-41PM.log.rac02
-rw-r-----. 1 grid   oinstall    348 Nov 30 19:46 silentInstall2014-11-30_07-40-41PM.log
-rw-r-----. 1 grid   oinstall   1968 Nov 30 19:46 oraInstall2014-11-30_07-40-41PM.out
-rw-r-----. 1 grid   oinstall 975962 Nov 30 19:46 installActions2014-11-30_07-40-41PM.log
-rw-r-----. 1 grid   oinstall      0 Nov 30 20:12 oraInstall2014-11-30_08-12-19PM.err
-rw-r-----. 1 grid   oinstall      0 Nov 30 20:12 oraInstall2014-11-30_08-12-19PM.err.rac02
-rw-r-----. 1 grid   oinstall   7357 Nov 30 20:13 UpdateNodeList2014-11-30_08-12-19PM.log.rac02
-rw-r-----. 1 grid   oinstall     33 Nov 30 20:13 oraInstall2014-11-30_08-12-19PM.out.rac02
-rw-r-----. 1 grid   oinstall  11305 Nov 30 20:13 UpdateNodeList2014-11-30_08-12-19PM.log
-rw-r-----. 1 grid   oinstall     33 Nov 30 20:13 oraInstall2014-11-30_08-12-19PM.out
-rw-r--r--. 1 oracle oinstall     47 Dec  4 14:49 time2014-12-04_02-49-27PM.log
-rw-rw----. 1 oracle oinstall  56317 Dec  4 14:49 installActions2014-12-04_02-49-27PM.log
-rw-r--r--. 1 oracle oinstall      0 Dec  4 14:49 oraInstall2014-12-04_02-49-27PM.out
-rw-r--r--. 1 oracle oinstall      0 Dec  4 14:49 oraInstall2014-12-04_02-49-27PM.err

[oracle@rac01:/u01/app/oraInventory/logs]
$ tail -20 installActions2014-12-04_02-49-27PM.log

INFO: Actual Value:libaio-devel(x86_64)-0.3.107-10.el6
INFO: -----------------------------------------------
INFO: *********************************************
INFO: Users With Same UID: This test checks that multiple users do not exist with user id as "0".
INFO: Severity:CRITICAL
INFO: OverallStatus:SUCCESSFUL
INFO: -----------------------------------------------
INFO: Verification Result for Node:rac01
WARNING: Result values are not available for this verification task
INFO: *********************************************
INFO: Root user consistency: This test checks the consistency of the primary group of the root user across the cluster nodes
INFO: Severity:IGNORABLE
INFO: OverallStatus:SUCCESSFUL
INFO: -----------------------------------------------
INFO: Verification Result for Node:rac01
WARNING: Result values are not available for this verification task
INFO: All forked task are completed at state prepInstall
INFO: Exit Status is 0
INFO: Shutdown Oracle Database 11g Release 2 Installer
INFO: Unloading Setup Driver

[oracle@rac01:/u01/app/oraInventory/logs]
$ grep -e ‘[[:upper:]]:’ installActions2014-12-04_02-49-27PM.log|cut -d “:” -f1|sort -u

INFO
/tmp/OraInstall2014-12-04_02-49-27PM
WARNING
[oracle@rac01:/u01/app/oraInventory/logs]
$

December 1, 2014

Using runInstaller to check Prereqs with responseFile

Filed under: 11g,RAC — mdinh @ 3:38 am

The SILENT method

[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$ ./runInstaller -silent -executePrereqs -showProgress -waitforcompletion -force -responseFile /media/sf_Linux/11.2.0.4/grid/grid_crs_config.rsp
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 120 MB.   Actual 48807 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 8191 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2014-11-30_06-40-05PM. Please wait ...[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$ cd /u01/app/oraInventory/logs/
[grid@rac01:/u01/app/oraInventory/logs]
$ ls -lrt
total 84
-rw-r--r--. 1 grid oinstall    47 Nov 30 18:40 time2014-11-30_06-40-05PM.log
-rw-rw----. 1 grid oinstall 80257 Nov 30 18:41 installActions2014-11-30_06-40-05PM.log
-rw-r--r--. 1 grid oinstall     0 Nov 30 18:41 oraInstall2014-11-30_06-40-05PM.out
-rw-r--r--. 1 grid oinstall     0 Nov 30 18:41 oraInstall2014-11-30_06-40-05PM.err
[grid@rac01:/u01/app/oraInventory/logs]
$ tail installActions2014-11-30_06-40-05PM.log
INFO: Actual Value:0|2
INFO: -----------------------------------------------
INFO: -----------------------------------------------
INFO: Verification Result for Node:rac01
INFO: Expected Value:0|2
INFO: Actual Value:0|2
INFO: -----------------------------------------------
INFO: All forked task are completed at state init
INFO: Exit Status is 0
INFO: Shutdown Oracle Grid Infrastructure

The NOT SO SILENT method
executePrereqs

Avoid false errors with runcluvfy.sh for VirtualBox

Filed under: 11g,RAC — mdinh @ 2:32 am

My typical VirtualBox network configuration is: eth0 (NAT), eth1 (Host Only), eth2(Internal)

runcluvfy.sh stage -pre crsinst -n rac01,rac02 -r 11gR2 -fixup -fixupdir /tmp – FAILED

Node connectivity passed for subnet "10.0.2.0" with node(s) rac02,rac01

ERROR:
PRVF-7617 : Node connectivity between "rac01 : 10.0.2.15" and "rac02 : 10.0.2.15" failed
TCP connectivity check failed for subnet "10.0.2.0"

Node connectivity passed for subnet "192.168.56.0" with node(s) rac02,rac01
TCP connectivity check passed for subnet "192.168.56.0"

Node connectivity passed for subnet "10.0.0.0" with node(s) rac02,rac01
TCP connectivity check passed for subnet "10.0.0.0"

Pre-check for cluster services setup was unsuccessful.
Checks did not pass for the following node(s):
        rac01
[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$

I understand the error is ignorable but why create an ignorable error in the first place?
Next, I would like to look at the bottom line to know if the check passed for failed and if the failure is false positive, then it would require wasted time for investigation.

A better alternative is to exclude eth0 from the check using -networks eth1,eth2 to have successful setup.

Pre-check for cluster services setup was successful.

runcluvfy.sh stage -pre crsinst -n rac01,rac02 -r 11gR2 -fixup -fixupdir /tmp -networks eth1,eth2 -verbose

[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:02:B1:57
          inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe02:b157/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:578823 errors:0 dropped:0 overruns:0 frame:0
          TX packets:292879 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:477712790 (455.5 MiB)  TX bytes:15902513 (15.1 MiB)
[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$ ./runcluvfy.sh stage -pre crsinst -n rac01,rac02 -r 11gR2 -fixup -fixupdir /tmp -networks eth1,eth2 -verbose

Performing pre-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "rac01"
  Destination Node                      Reachable?
  ------------------------------------  ------------------------
  rac02                                 yes
  rac01                                 yes
Result: Node reachability check passed from node "rac01"


Checking user equivalence...

Check: User equivalence for user "grid"
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
Result: User equivalence check passed for user "grid"

Checking node connectivity...

Checking hosts config file...
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed

Verification of the hosts config file successful


Interface information for node "rac02"
 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   10.0.2.15       10.0.2.0        0.0.0.0         10.0.2.2        08:00:27:87:91:11 1500
 eth1   192.168.56.12   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:4A:7B:27 1500
 eth2   10.10.10.12     10.0.0.0        0.0.0.0         10.0.2.2        08:00:27:E8:D6:21 1500


Interface information for node "rac01"
 Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU
 ------ --------------- --------------- --------------- --------------- ----------------- ------
 eth0   10.0.2.15       10.0.2.0        0.0.0.0         10.0.2.2        08:00:27:02:B1:57 1500
 eth1   192.168.56.11   192.168.56.0    0.0.0.0         10.0.2.2        08:00:27:BD:66:A4 1500
 eth2   10.10.10.11     10.0.0.0        0.0.0.0         10.0.2.2        08:00:27:60:79:0F 1500


Check: Node connectivity for interface "eth1"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac02[192.168.56.12]            rac01[192.168.56.11]            yes
Result: Node connectivity passed for interface "eth1"


Check: TCP connectivity of subnet "192.168.56.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac01:192.168.56.11             rac02:192.168.56.12             passed
Result: TCP connectivity check passed for subnet "192.168.56.0"


Check: Node connectivity for interface "eth2"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac02[10.10.10.12]              rac01[10.10.10.11]              yes
Result: Node connectivity passed for interface "eth2"


Check: TCP connectivity of subnet "10.0.0.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  rac01:10.10.10.11               rac02:10.10.10.12               passed
Result: TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.56.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Result: Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.56.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.56.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking ASMLib configuration.
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
Result: Check for ASMLib configuration passed.

Check: Total memory
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         3.8674GB (4055296.0KB)    1.5GB (1572864.0KB)       passed
  rac01         3.8674GB (4055296.0KB)    1.5GB (1572864.0KB)       passed
Result: Total memory check passed

Check: Available memory
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         3.5583GB (3731180.0KB)    50MB (51200.0KB)          passed
  rac01         3.269GB (3427772.0KB)     50MB (51200.0KB)          passed
Result: Available memory check passed

Check: Swap space
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         8GB (8388604.0KB)         3.8674GB (4055296.0KB)    passed
  rac01         8GB (8388604.0KB)         3.8674GB (4055296.0KB)    passed
Result: Swap space check passed

Check: Free disk space for "rac02:/tmp"
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /tmp              rac02         /             52.2988GB     1GB           passed
Result: Free disk space check passed for "rac02:/tmp"

Check: Free disk space for "rac01:/tmp"
  Path              Node Name     Mount point   Available     Required      Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  /tmp              rac01         /             49.9507GB     1GB           passed
Result: Free disk space check passed for "rac01:/tmp"

Check: User existence for "grid"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists(54322)
  rac01         passed                    exists(54322)

Checking for multiple users with UID value 54322
Result: Check for multiple users with UID value 54322 passed
Result: User existence check passed for "grid"

Check: Group existence for "oinstall"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists
  rac01         passed                    exists
Result: Group existence check passed for "oinstall"

Check: Group existence for "dba"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    exists
  rac01         passed                    exists
Result: Group existence check passed for "dba"

Check: Membership of user "grid" in group "oinstall" [as Primary]
  Node Name         User Exists   Group Exists  User in Group  Primary       Status
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             yes           yes           yes           yes           passed
  rac01             yes           yes           yes           yes           passed
Result: Membership check for user "grid" in group "oinstall" [as Primary] passed

Check: Membership of user "grid" in group "dba"
  Node Name         User Exists   Group Exists  User in Group  Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             yes           yes           yes           passed
  rac01             yes           yes           yes           passed
Result: Membership check for user "grid" in group "dba" passed

Check: Run level
  Node Name     run level                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         5                         3,5                       passed
  rac01         5                         3,5                       passed
Result: Run level check passed

Check: Hard limits for "maximum open file descriptors"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             hard          65536         65536         passed
  rac01             hard          65536         65536         passed
Result: Hard limits check passed for "maximum open file descriptors"

Check: Soft limits for "maximum open file descriptors"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             soft          1024          1024          passed
  rac01             soft          1024          1024          passed
Result: Soft limits check passed for "maximum open file descriptors"

Check: Hard limits for "maximum user processes"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             hard          16384         16384         passed
  rac01             hard          16384         16384         passed
Result: Hard limits check passed for "maximum user processes"

Check: Soft limits for "maximum user processes"
  Node Name         Type          Available     Required      Status
  ----------------  ------------  ------------  ------------  ----------------
  rac02             soft          16384         2047          passed
  rac01             soft          16384         2047          passed
Result: Soft limits check passed for "maximum user processes"

Check: System architecture
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         x86_64                    x86_64                    passed
  rac01         x86_64                    x86_64                    passed
Result: System architecture check passed

Check: Kernel version
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         2.6.39-400.17.1.el6uek.x86_64  2.6.32                    passed
  rac01         2.6.39-400.17.1.el6uek.x86_64  2.6.32                    passed
Result: Kernel version check passed

Check: Kernel parameter for "semmsl"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             250           250           250           passed
  rac01             250           250           250           passed
Result: Kernel parameter check passed for "semmsl"

Check: Kernel parameter for "semmns"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             32000         32000         32000         passed
  rac01             32000         32000         32000         passed
Result: Kernel parameter check passed for "semmns"

Check: Kernel parameter for "semopm"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             100           100           100           passed
  rac01             100           100           100           passed
Result: Kernel parameter check passed for "semopm"

Check: Kernel parameter for "semmni"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             128           128           128           passed
  rac01             128           128           128           passed
Result: Kernel parameter check passed for "semmni"

Check: Kernel parameter for "shmmax"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4398046511104  4398046511104  2076311552    passed
  rac01             4398046511104  4398046511104  2076311552    passed
Result: Kernel parameter check passed for "shmmax"

Check: Kernel parameter for "shmmni"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4096          4096          4096          passed
  rac01             4096          4096          4096          passed
Result: Kernel parameter check passed for "shmmni"

Check: Kernel parameter for "shmall"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4294967296    4294967296    2097152       passed
  rac01             4294967296    4294967296    2097152       passed
Result: Kernel parameter check passed for "shmall"

Check: Kernel parameter for "file-max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             6815744       6815744       6815744       passed
  rac01             6815744       6815744       6815744       passed
Result: Kernel parameter check passed for "file-max"

Check: Kernel parameter for "ip_local_port_range"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed
  rac01             between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed
Result: Kernel parameter check passed for "ip_local_port_range"

Check: Kernel parameter for "rmem_default"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             262144        262144        262144        passed
  rac01             262144        262144        262144        passed
Result: Kernel parameter check passed for "rmem_default"

Check: Kernel parameter for "rmem_max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             4194304       4194304       4194304       passed
  rac01             4194304       4194304       4194304       passed
Result: Kernel parameter check passed for "rmem_max"

Check: Kernel parameter for "wmem_default"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             262144        262144        262144        passed
  rac01             262144        262144        262144        passed
Result: Kernel parameter check passed for "wmem_default"

Check: Kernel parameter for "wmem_max"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             1048576       1048576       1048576       passed
  rac01             1048576       1048576       1048576       passed
Result: Kernel parameter check passed for "wmem_max"

Check: Kernel parameter for "aio-max-nr"
  Node Name         Current       Configured    Required      Status        Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  rac02             1048576       1048576       1048576       passed
  rac01             1048576       1048576       1048576       passed
Result: Kernel parameter check passed for "aio-max-nr"

Check: Package existence for "binutils"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
  rac01         binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed
Result: Package existence check passed for "binutils"

Check: Package existence for "compat-libcap1"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         compat-libcap1-1.10-1     compat-libcap1-1.10       passed
  rac01         compat-libcap1-1.10-1     compat-libcap1-1.10       passed
Result: Package existence check passed for "compat-libcap1"

Check: Package existence for "compat-libstdc++-33(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
  rac01         compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed
Result: Package existence check passed for "compat-libstdc++-33(x86_64)"

Check: Package existence for "libgcc(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libgcc(x86_64)-4.4.7-11.el6  libgcc(x86_64)-4.4.4      passed
  rac01         libgcc(x86_64)-4.4.7-11.el6  libgcc(x86_64)-4.4.4      passed
Result: Package existence check passed for "libgcc(x86_64)"

Check: Package existence for "libstdc++(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libstdc++(x86_64)-4.4.7-11.el6  libstdc++(x86_64)-4.4.4   passed
  rac01         libstdc++(x86_64)-4.4.7-11.el6  libstdc++(x86_64)-4.4.4   passed
Result: Package existence check passed for "libstdc++(x86_64)"

Check: Package existence for "libstdc++-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libstdc++-devel(x86_64)-4.4.7-11.el6  libstdc++-devel(x86_64)-4.4.4  passed
  rac01         libstdc++-devel(x86_64)-4.4.7-11.el6  libstdc++-devel(x86_64)-4.4.4  passed
Result: Package existence check passed for "libstdc++-devel(x86_64)"

Check: Package existence for "sysstat"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
  rac01         sysstat-9.0.4-20.el6      sysstat-9.0.4             passed
Result: Package existence check passed for "sysstat"

Check: Package existence for "gcc"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         gcc-4.4.7-11.el6          gcc-4.4.4                 passed
  rac01         gcc-4.4.7-11.el6          gcc-4.4.4                 passed
Result: Package existence check passed for "gcc"

Check: Package existence for "gcc-c++"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         gcc-c++-4.4.7-11.el6      gcc-c++-4.4.4             passed
  rac01         gcc-c++-4.4.7-11.el6      gcc-c++-4.4.4             passed
Result: Package existence check passed for "gcc-c++"

Check: Package existence for "ksh"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         ksh-20120801-21.el6.1     ksh-20100621              passed
  rac01         ksh-20120801-21.el6.1     ksh-20100621              passed
Result: Package existence check passed for "ksh"

Check: Package existence for "make"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         make-3.81-20.el6          make-3.81                 passed
  rac01         make-3.81-20.el6          make-3.81                 passed
Result: Package existence check passed for "make"

Check: Package existence for "glibc(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         glibc(x86_64)-2.12-1.149.el6  glibc(x86_64)-2.12        passed
  rac01         glibc(x86_64)-2.12-1.149.el6  glibc(x86_64)-2.12        passed
Result: Package existence check passed for "glibc(x86_64)"

Check: Package existence for "glibc-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         glibc-devel(x86_64)-2.12-1.149.el6  glibc-devel(x86_64)-2.12  passed
  rac01         glibc-devel(x86_64)-2.12-1.149.el6  glibc-devel(x86_64)-2.12  passed
Result: Package existence check passed for "glibc-devel(x86_64)"

Check: Package existence for "libaio(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
  rac01         libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed
Result: Package existence check passed for "libaio(x86_64)"

Check: Package existence for "libaio-devel(x86_64)"
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  rac02         libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
  rac01         libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed
Result: Package existence check passed for "libaio-devel(x86_64)"

Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed

Check: Current group ID
Result: Current group ID check passed

Starting check for consistency of primary group of root user
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed

Check for consistency of root user's primary group passed

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
No NTP Daemons or Services were found to be running

Result: Clock synchronization check using Network Time Protocol(NTP) passed

Checking Core file name pattern consistency...
Core file name pattern consistency check passed.

Checking to make sure user "grid" is not in "root" group
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  rac02         passed                    does not exist
  rac01         passed                    does not exist
Result: User "grid" is not part of "root" group. Check passed

Check default user file creation mask
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  rac02         0022                      0022                      passed
  rac01         0022                      0022                      passed
Result: Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined
File "/etc/resolv.conf" does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...
domain entry in file "/etc/resolv.conf" is consistent across nodes
Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes...
search entry in file "/etc/resolv.conf" is consistent across nodes
Checking file "/etc/resolv.conf" to make sure that only one search entry is defined
All nodes have one search entry defined in file "/etc/resolv.conf"
Checking all nodes to make sure that search entry is "localdomain" as found on node "rac02"
All nodes of the cluster have same value for 'search'
Checking DNS response time for an unreachable node
  Node Name                             Status
  ------------------------------------  ------------------------
  rac02                                 passed
  rac01                                 passed
The DNS response time for an unreachable node is within acceptable limit on all nodes

File "/etc/resolv.conf" is consistent across nodes

Check: Time zone consistency
Result: Time zone consistency check passed

Starting check for Reverse path filter setting ...
Reverse path filter setting is correct for all private interconnect network interfaces on node "rac02.localdomain".
Reverse path filter setting is correct for all private interconnect network interfaces on node "rac01.localdomain".

Check for Reverse path filter setting passed

Pre-check for cluster services setup was successful.
[grid@rac01:/media/sf_Linux/11.2.0.4/grid]
$

October 31, 2014

Better Communication

Filed under: Uncategorized — mdinh @ 1:09 am

Sometimes I get excited over simple things. Today, I learned something new, something about human behaviors.

Basically, there are 4 quadrants for human behaviors.

If one does not understand the other person’s behavior, then communication can be very difficult.

To find more about human behaviors, read The DISC Model of Human Behavior – A Quick Overview.

Which quadrant are you in, versus your manager?

October 24, 2014

Oracle Real World Performance

Filed under: oracle,performance — mdinh @ 4:39 am

Great videos on Oracle Real World Performance.

August 2, 2014

RMAN Pet Peeves

Filed under: 11g,oracle,RMAN — mdinh @ 6:38 pm

Do you validate your backup and what command do you use?

Lately, I have been using restore database validate preview summary to kill 2 birds with 1 stone.

The issue is RMAN will skip validation of archived log backupset when archived log exists.

Does this seem wrong to you?

Please take a look at a test case here

What do you think?

Next Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 332 other followers