cluvfy „cheat sheet“ for RAC-Setup

While doing new RAC Cluster setup’s I was looking in parallel for an easy way to check an existing or new Oracle RAC Installation.

Yes, the tool to do most of the tasks is „cluvfy“, the Cluster Verification Utility. cluvfy can de downloaded and installed as standalone version.

cluvfy will be updated in a regular manner up to the actual Oracle Release Updates.

So please check My Oracle Support Patch 30839369. 

 

Download latest CVU Version and do the Setup 

Patch 30839369: Standalone CVU version 19.9 October 2020

 

check the Setup and Version

cluvfy –version

 

Here is my cluvfy „cheat sheet“

 

Post Hardware & OS Setup

    • cluvfy stage -post hwos -n all

 

Post Network Installation

    • cluvfy comp nodecon -n all -verbose

 

Post CRS Software Setup

    • cluvfy stage -post crsinst -n all

 

Clustermanager integrity

    • cluvfy comp clumgr –allnodes

 

Check OCR integrity

    • cluvfy comp ocr -allnodes

 

Check ASM integrity (if used)

    • cluvfy comp asm -allnodes

 

Post check Oracle Home Setup and all your other Oracle Homes

    • cluvfy comp software -allnodes -d /u01/app/oracle/product/19.0.0/dbhome_1 -allfiles

 

I do not post the output from all these commands because this depends on your RAC Setup. 

What I can say is the checks and the output is very detailed and helpful.

If you get error messages during the runtime fix these errors because then you can be sure that your RAC setup is healthy and functioning. 

 

 

 

 

Oracle Exadata Deployment Assistant (OEDA) new Update Dec 2020

The configuration of an Exadata Systems without the OEDA tool is not possible. Oracle is continuously improving this tool. In the meantime there are monthly updates available. This ensures that every hardware or software change can be implemented on customer site via OEDA.

What has to be done to get the OEDA up and running?

Download the patch from My Oracle Support  Patch 30640393

The tool is available for many platforms including Windows, Mac OS, Linux, etc.

The actual monthly build is

Update 32241522 for Release 19.9 and Earlier

from December 2020

After the download unzip the file. In my environment a directory „macosx-x64″ was created.

The Setup is very easy start on the command line

./installOedaServer.sh -p 7072

At the end of the installation the Webserver starts on the localhost

the OEDA tool automatically in your browser.

http://localhost:7072/oeda

 

 

Now you are ready to start the configuration for the new Exadata Project.

 

 

 

 

 

 

 

How to check the Oracle Patch history

(new Update see below)

Patching is a task which is really often done and if you sit in front of a Oracle System which

you know while doing regularly maintenance the Patch status is more or less known to you

 

But what if your college ask:

„Please check an Oracle System from a new customer and let me know if they had patched

the System in the last six month.“

 

Now most of us do a Login set the environment for example Oracle_Home or Grid_Home

and start an „opatch“ or checked via the database and registry.

 

To get a complete overview what happen on the System is more easy from my point of view

while taking a look in the opatch_history.txt file.

This textfile exist for every Oracle Home on the server and shows the complete history

 

ssh host1

su - oracle

cat /etc/oratab

db1:/u01/app/oracle/product/12.2.0.1/dbhome_1:Y

. oraenv

cd /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatch

vi opatch_history.txt


You saw exactly at which time which command was entered against the Oracle Home.

So you can very easy check when and which Patches where applied in the last time.

 

Date & Time : Mon Aug 03 02:05:45 CEST 2020

Oracle Home : /u01/app/oracle/product/12.2.0.1/dbhome_1

OPatch Ver. : 12.2.0.1.19

Current Dir : /u01/app/oracle/product/12.2.0.1/dbhome_1

Command     : lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 -local -invPtrLoc /u01/app/oracle/product/12.2.0.1/dbhome_1/oraInst.loc

Log File    : /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatch/opatch2020-08-03_02-05-45AM_1.log


Another example for an Oracle Grid Home here Release 19c.

 

Grid Home is „/u01/app/19.0.0/grid“

/u01/app/19.0.0/grid/cfgtoollogs/opatch/opatch_history.txt

 

Date & Time : Mon Aug 03 02:26:04 CEST 2020

Oracle Home : /u01/app/19.0.0/grid

OPatch Ver. : 12.2.0.1.17

Current Dir : /u01/app/19.0.0/grid

Command     : lsinventory -oh /u01/app/19.0.0/grid -local -invPtrLoc /u01/app/19.0.0/grid/oraInst.loc

Log File    : /u01/app/19.0.0/grid/cfgtoollogs/opatch/opatch2020-08-03_02-26-04AM_1.log
Try it and check your opatch_history.txt file. It is really helpful.

 

Update on this Post after Roy (Swonger) VP from Oracle ask me if I can add an additional information I do of course.

Roy says:

One of the most common issues I see is when customers run OPatch but forget to run Datapatch to apply the SQL changes to the database. Mike had a good blog post about the dba_registry_history and dba_registry_sqlpatch views that explains which to use depending on the version you are running: https://mikedietrichde.com/2016/11/30/dba_registry_history-vs-dba_registry_sqlpatch/ 

Yes, please take a look to Mike’s Blog which is very helpful (as all his posts 👍)

In my special case the main topic of my college was the question „Can you check if the customer has tried to install Patches in the Grid Home during the last month?“. This question could be easy answered via the opatch_history.txt file. 

So have fun during the next patching action and not forget the datapatch

 

 

 

 

Clone Oracle Home in Release 12.2 (RAC with 3 Nodes)

I do many patching activities in the moment and for this I checked out for an easy way to clone the

actual Oracle Home of the 12.2 installation

Okay, that’s easy I had written a blog post for 12.1 so let’s go

But then I found out that this procedure with the „clone.pl“ script will not work in 12.2

 

In Oracle Release 12.2 the clone of an Oracle Home will work in the following way:

Source Home   „/u01/app/oracle/product/12.2.0.1/dbhome_1“

Target Home    „/u01/app/oracle/product/12.2.0.1/dbhome_2“

The target home I named it „dbhome_2“ is 12.2 + RU Jan 2020

All steps need to be done as os user „oracle“


 

cp -r /u01/app/oracle/product/12.2.0.1/dbhome_1 /u01/app/oracle/product/12.2.0.1/dbhome_2 

cd /u01/app/oracle/product/12.2.0.1/dbhome_2 

./oui/bin/runInstaller -clone -waitForCompletion ORACLE_HOME="/u01/app/oracle/product/12.2.0.1/dbhome_2" ORACLE_HOME_NAME="RUJAN2020" ORACLE_BASE="/u01/app/oracle" CLUSTER_NODES="{node01,node02,node03}" LOCAL_NODE="node01" -silent 

 

Download your platform specific RU Jan 2020 from My Oracle Support

This RU Jan 2020 will be applied to the new „dbhome_2“ with the opatch tool not described here.

After this is finished clone the new created „dbhome_2“ on all other nodes

 




$ cd /u01/app/oracle/product/12.2.0.1

$ tar czf home_2.tgz dbhome_2

$ scp home_2.tgz oracle@node02:/u01/app/oracle/product/12.2.0.1/

$ ssh node02

$ cd /u01/app/oracle/product/12.2.0.1

$ tar xf home_2.tgz

$ cd dbhome_2/oui/bin

$ ./runInstaller -clone -waitForCompletion ORACLE_HOME="/u01/app/oracle/product/12.2.0.1/dbhome_2" ORACLE_HOME_NAME="RUJAN2020" ORACLE_BASE="/u01/app/oracle" CLUSTER_NODES="{node01,node02,node03}" LOCAL_NODE="node02" –silent


 



$scp home_2.tgz oracle@node03:/u01/app/oracle/product/12.2.0.1/

$ ssh node03

$ cd /u01/app/oracle/product/12.2.0.1

$ tar xf home_2.tgz

$ cd dbhome_2/oui/bin

$ ./runInstaller -clone -waitForCompletion ORACLE_HOME="/u01/app/oracle/product/12.2.0.1/dbhome_2" ORACLE_HOME_NAME="RUJAN2020" ORACLE_BASE="/u01/app/oracle" CLUSTER_NODES="{node01,node02,node03}" LOCAL_NODE="node03" –silent


 

Ready is the 12.2 clone + RU Jan 2020 👍  🙂

 

 

 

 

 

 

 

tfactl summary as HTML (really tricky)

I read an interesting article from Michael Schulze, Optiz Consulting in the actual „Red Stack Magazin / April 2020“, about Tools for the daily Exadata maintenance but it is not so easy as described.

Since some time ago AHF (Autonomous Health Framework) is the tool for checks of any kind on the Exadata.

I have installed and used here Version 19.3 which is an old one. I will update in a few weeks the whole environment to Version 20.1.

So I try to create the „summary HTML“ Report but it will not work as described in Michaels article.

Why?

 



tfactl summary -overview -html


 

Sometimes you need to start the command twice and you NEED to enter a „q

after you saw the „tfactl_summary>“ this is important otherwise the HTML report will not be created.

 

Example

tfactl summary -overview -html

WARNING – TFA Software is older than 180 days. Please consider upgrading TFA to the latest version.

  Executing Summary in Parallel on Following Nodes:

    Node : exa01                             

    Node : exa02                             

    Node : exa03                             

    Node : exa04                             

LOGFILE LOCATION : /opt/oracle.ahf/data/repository/suptools/exa01/summary/root/20200502174852/log/summary_command_20200502174852_exa01_177655.log

  Component Specific Summary collection :

    – Collecting CRS details … Done.   

    – Collecting ASM details … Done.   

    – Collecting ACFS details … Done.

    – Collecting DATABASE details … Done.

    – Collecting EXADATA details … Done.

    – Collecting PATCH details … Done.   

    – Collecting LISTENER details … Done.

    – Collecting NETWORK details … Done.

    – Collecting OS details … Done.      

    – Collecting TFA details … Done.     

    – Collecting SUMMARY details … Done.

  Remote Summary Data Collection : In-Progress – Please wait …

  – Data Collection From Node – exa02 .. Done.           

  – Data Collection From Node – exa03 .. Done.           

  – Data Collection From Node – exa04 .. Done.           

  Prepare Clusterwide Summary Overview … Done

      cluster_status_summary                   

                                                                                                                           

  DETAILS                                                                                             STATUS    COMPONENT 

+---------------------------------------------------------------------------------------------------+---------+-----------+

  .-----------------------------------------------.                                                   PROBLEM   CRS        

  | CRS_SERVER_STATUS   : ONLINE                  |                                                                        

  | CRS_STATE           : ONLINE                  |                                                                        

  | CRS_INTEGRITY_CHECK : FAIL                    |                                                                        

  | CRS_RESOURCE_STATUS : OFFLINE Resources Found |                                                                        

  '-----------------------------------------------'                                                                        

  .-------------------------------------------------------.                                           PROBLEM   ASM        

  | ASM_DISK_SIZE_STATUS : WARNING - Available Size < 20% |                                                                

  | ASM_BLOCK_STATUS     : PASS                           |                                                                

  | ASM_CHAIN_STATUS     : PASS                           |                                                                

  | ASM_INCIDENTS        : FAIL                           |                                                                

  | ASM_PROBLEMS         : FAIL                           |                                                                

  '-------------------------------------------------------'                                                                

  .-----------------------.                                                                           OFFLINE   ACFS       

  | ACFS_STATUS : OFFLINE |                                                                                                

  '-----------------------'                                                                                                

  .-----------------------------------------------------------------------------------------------.   PROBLEM   DATABASE   

  | ORACLE_HOME_DETAILS                                                        | ORACLE_HOME_NAME |                        

  +----------------------------------------------------------------------------+------------------+                        

  | .------------------------------------------------------------------------. | OraDB12Home1     |                        

  | | DB_CHAINS | DB_BLOCKS | INCIDENTS | PROBLEMS | DATABASE_NAME | STATUS  | |                  |                        

  | +-----------+-----------+-----------+----------+---------------+---------+ |                  |                        

  | | PROBLEM   | PASS      | PROBLEM   | PROBLEM  | i10   | PROBLEM | |                  |                        

  | | PROBLEM   | PROBLEM   | PROBLEM   | PROBLEM  | p10     | PROBLEM | |                  |                        

  | | PASS      | PASS      | PROBLEM   | PROBLEM  | i20    | PROBLEM | |                  |                        

  | '-----------+-----------+-----------+----------+---------------+---------' |                  |                        

  '----------------------------------------------------------------------------+------------------'                        

  .--------------------------------.                                                                  PROBLEM   EXADATA    

  | SWITCH_SSH_STATUS : CONFIGURED |                                                                                       

  | CELL_SSH_STATUS   : CONFIGURED |                                                                                       

  | ENVIRONMENT_TEST  : PASS       |                                                                                       

  | LINKUP            : PASS       |                                                                                       

  | LUN_STATUS        : NORMAL     |                                                                                       

  | RS_STATUS         : RUNNING    |                                                                                       

  | CELLSRV_STATUS    : RUNNING    |                                                                                       

  | MS_STATUS         : RUNNING    |                                                                                       

  '--------------------------------'                                                                                       

  .----------------------------------------------.                                                    OK        PATCH      

  | CRS_PATCH_CONSISTENCY_ACROSS_NODES      : OK |                                                                         

  | DATABASE_PATCH_CONSISTENCY_ACROSS_NODES : OK |                                                                         

  '----------------------------------------------'                                                                         

  .-----------------------.                                                                           OK        LISTENER

  | LISTNER_STATUS   : OK |

  '-----------------------'

  .---------------------------.                                                                       OK        NETWORK

  | CLUSTER_NETWORK_STATUS :  |

  '---------------------------'

  .-----------------------.                                                                           OK        OS

  | MEM_USAGE_STATUS : OK |

  '-----------------------'

  .----------------------.                                                                            OK        TFA

  | TFA_STATUS : RUNNING |

  '----------------------'

  .------------------------------------.                                                              OK        SUMMARY

  | SUMMARY_EXECUTION_TIME : 0H:2M:31S |

  '------------------------------------'

+---------------------------------------------------------------------------------------------------+---------+-----------+

        ### Entering in to SUMMARY Command-Line Interface ###

tfactl_summary>list

  Components : Select Component - select [component_number|component_name]

        1 => overview

        2 => crs_overview

        3 => asm_overview

        4 => acfs_overview

        5 => database_overview

        6 => exadata_overview

        7 => patch_overview

        8 => listener_overview

        9 => network_overview

        10 => os_overview

        11 => tfa_overview

        12 => summary_overview

tfactl_summary>q

        ### Exited From SUMMARY Command-Line Interface ###

--------------------------------------------------------------------

REPOSITORY  : /opt/oracle.ahf/data/repository/suptools/exa01/summary/root/20200502174852/exa01

HTML REPORT : <REPOSITORY>/report/Consolidated_Summary_Report_20200502174852.html

--------------------------------------------------------------------

 

So enter „q“ and the HTML Report is created. Then start a browser and you the Report which is really helpful.

 

 

Have fun :-) and if possible use directly AHF 20.1, or do an Update like me …..

 

 

 

 

 

RU Apr 2020 installed

RU Update 19.7 done.

Environment : Exadata, X7-2, 4 node, RAC Cluster

 

opatch lspatches

30805684;OJVM RELEASE UPDATE: 19.7.0.0.200414 (30805684)

30869156;Database Release Update : 19.7.0.0.200414 (30869156)

29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)

 

No error, works perfect. 👍

 

So we can go on to test Upgrades with „autoupgrade“ for the RAC DB’s …..