Database Out of Place Patching Through runInstaller #JoelKallmanDay

Step 2 - Patch My Breath Away. (DB OOP Patching)

I’ve moved my blog from https://insanedba.blogspot.com to https://dincosman.com Please update your bookmarks and follow/subscribe at the new address for all the latest updates and content. More up-to-date content of this post may be available there.

I will patch all my database servers database software from 19.16 to 19.20. As there are more than 20 servers to patch, we will use fleet patching. First, I will patch a 2-node cluster database homes by using out-of-place (OOP) patching methodology through runInstaller in silent mode.

Later, I will use patched database homes to create a gold image for fleet patching of all database homes.

Setup List:

 * Database 19.3 Base Release (LINUX.X64_193000_db_home.zip)

 * Database 19.20 RU (p35320081_190000_Linux-x86-64.zip)

 * Grid 19.20 August MRP involves DB August MRP. (p35656840_1920000DBRU_Linux-x86-64.zip)

 * OJVM 19.20 Patch (p35354406_190000_Linux-x86-64.zip)

 * DPBP 19.20 Datapump Bundle Patch (p35512813_1920000DBRU_Generic.zip)

 * Spatial 19.20 Bundle Patch (p35602358_1920000DBRU_Linux-x86-64.zip)

 * DB 555.1 Recommended One-off (p35372179_1920000DBRU_Linux-x86-64.zip)

 * DB 555.1 Recommended One-off (p30787757_1920000DBRU_Linux-x86-64.zip)

 * Opatch Latest Version (p6880880_122010_Linux-x86-64.zip)

Preparation Phase:

	[oracle@oravt01 product]$ cd /u01/app/oracle/product
	[oracle@oravt01 product]$ mkdir -p 19.20/dbhome_1
	[oracle@oravt01 ~]$ cd /u01/app/Setup/
	[oracle@oravt01 Setup]$ mkdir DBRU
	[oracle@oravt01 Setup]$ mkdir OJVM
	[oracle@oravt01 Setup]$ mkdir DPBP
	[oracle@oravt01 Setup]$ mkdir DBRU_Oneoff
	[oracle@oravt01 Setup]$ mkdir GIMRP
	[oracle@oravt01 Setup]$ mkdir Spatial
	[oracle@oravt01 Setup]$ cd DBRU
	[oracle@oravt01 DBRU]$ unzip p35320081_190000_Linux-x86-64.zip
	[oracle@oravt01 DBRU]$ cd ../GIMRP
	[oracle@oravt01 GIMRP]$ unzip p35656840_1920000DBRU_Linux-x86-64.zip
	[oracle@oravt01 GIMRP]$ cd ../Spatial
	[oracle@oravt01 Spatial]$ unzip p35602358_1920000DBRU_Linux-x86-64.zip
	[oracle@oravt01 Spatial]$ cd /u01/app/Setup/DBRU_Oneoff/
	[oracle@oravt01 DBRU_Oneoff]$ unzip p30787757_1920000DBRU_Linux-x86-64.zip
	[oracle@oravt01 DBRU_Oneoff]$ unzip p35372179_1920000DBRU_Linux-x86-64.zip
	[oracle@oravt01 DBRU_Oneoff]$ cd /u01/app/Setup/DPBP/
	[oracle@oravt01 DPBP]$ unzip p35512813_1920000DBRU_Generic.zip
	[oracle@oravt01 DPBP]$ cd /u01/app/Setup/OJVM/
	[oracle@oravt01 OJVM]$ unzip p35354406_190000_Linux-x86-64.zip
	[oracle@oravt01 OJVM]$ cd /u01/app/oracle/product/19.20/dbhome_1/
	[oracle@oravt01 dbhome_1]$ unzip -qo /u01/app/Setup/LINUX.X64_193000_db_home.zip
	[oracle@oravt01 dbhome_1]$ rm -rf OPatch
	[oracle@oravt01 dbhome_1]$ unzip -qo /u01/app/Setup/p6880880_122010_Linux-x86-64.zip
	[oracle@oravt01 dbhome_1]$ export ORACLE_HOME=/u01/app/oracle/product/19.20/dbhome_1
	[oracle@oravt01 dbhome_1]$ cd $ORACLE_HOME

view raw db_oop_prepare.txt hosted with ❤ by GitHub

We replaced scp binary with the below commands not to hit "[FATAL] [INS-06006] Passwordless SSH connectivity not set up between the following node(s):" error related to Openssh_8 shipped with Oracle Linux 8 according to INS-06006 GI RunInstaller Fails If OpenSSH Is Upgraded to 8.x (Doc ID 2555697.1)

	[root@oravt01 ~]# ssh -V
	OpenSSH_8.0p1, OpenSSL 1.1.1k FIPS 25 Mar 2021
	-- rename original scp binary.
	[root@oravt01 ~]# mv /usr/bin/scp /usr/bin/scp.orig
	--create new scp binary referring to original one
	[root@oravt01 ~]# vi /usr/bin/scp
	/usr/bin/scp.orig -T $*
	--chmod to executable
	[root@oravt01 ~]# chmod 555 /usr/bin/scp

view raw replace_scp.txt hosted with ❤ by GitHub

Deploying new software to all nodes with a single command. (DBRU + DB MRP + OJVM + DPBP + Spatial Bundle + One-offs)

[oracle@oravt01 grid]$ export CV_ASSUME_DISTID=OL7 ./runInstaller -applyRU /u01/app/Setup/DBRU/35320081
-applyOneOffs /u01/app/Setup/OJVM/35354406,/u01/app/Setup/DPBP/35512813,/u01/app/Setup/Spatial/35602358,/u01/app/Setup/DBRU_Oneoff/30787757,/u01/app/Setup/DBRU_Oneoff/35372179,/u01/app/Setup/GIMRP/35656840/35571653,/u01/app/Setup/GIMRP/35656840/35156218,/u01/app/Setup/GIMRP/35656840/35395648,/u01/app/Setup/GIMRP/35656840/35499173,/u01/app/Setup/GIMRP/35656840/35525143,/u01/app/Setup/GIMRP/35656840/35585502,/u01/app/Setup/GIMRP/35656840/35655714
-silent -ignorePrereqFailure -waitforcompletion
oracle.install.option=INSTALL_DB_SWONLY
UNIX_GROUP_NAME=oinstall
INVENTORY_LOCATION=/u01/app/oraInventory
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/19.20/dbhome_1
oracle.install.db.InstallEdition=EE
oracle.install.db.OSDBA_GROUP=dba
oracle.install.db.OSOPER_GROUP=oper
oracle.install.db.OSBACKUPDBA_GROUP=backupdba
oracle.install.db.OSDGDBA_GROUP=dgdba
oracle.install.db.OSKMDBA_GROUP=kmdba
oracle.install.db.OSRACDBA_GROUP=racdba
oracle.install.db.CLUSTER_NODES=oravt01,oravt02
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false
DECLINE_SECURITY_UPDATES=true

view raw deploy_dbhome_oop.md hosted with ❤ by GitHub

Now, new database home software is ready to go. We will just execute root.sh scripts on all nodes.

	[root@oravt01 dbhome_1]# /u01/app/oracle/product/19.20/dbhome_1/root.sh
	[root@oravt02 dbhome_1]# /u01/app/oracle/product/19.20/dbhome_1/root.sh

view raw execute_rootsh_db.txt hosted with ❤ by GitHub

As we make use of unified auditing on all databases, we will relink unified auditing with the on option with the below command before switching to the new home.

	[oracle@oravt01 lib]$ cd /u01/app/oracle/product/19.20/dbhome_1/rdbms/lib
	[oracle@oravt01 lib]$ export ORACLE_HOME=/u01/app/oracle/product/19.20/dbhome_1
	[oracle@oravt01 lib]$ make -f ins_rdbms.mk uniaud_on ioracle
	[oracle@oravt02 lib]$ cd /u01/app/oracle/product/19.20/dbhome_1/rdbms/lib
	[oracle@oravt02 lib]$ export ORACLE_HOME=/u01/app/oracle/product/19.20/dbhome_1
	[oracle@oravt02 lib]$ make -f ins_rdbms.mk uniaud_on ioracle

view raw relink_unified_audit_on.txt hosted with ❤ by GitHub

Now it is time to switch to the new homes. Just before the switch;

    * Do not forget to update all your shell scripts, .bashrc etc files to refer to the new db home.

    * Copy tnsnames.ora and sqlnet.ora from the old db home to the new db home.

    * Copy glogin.sqlfrom the old db home to the new db home.

   * Make sure that your wallet path is not dependent on your old db home, if it is, just copy to a different path and update your sqlnet.ora.

Final step, modify your database homes with srvctl command and startup your databases from the new home node by node. Below is a sample.

	[oracle@oravt01 dbhome_1]$ srvctl modify database -d clouddb -oraclehome /u01/app/oracle/product/19.20/dbhome_1
	[oracle@oravt01 ~]$ srvctl stop instance -d clouddb -i clouddb1
	[oracle@oravt01 ~]$ srvctl start instance -d clouddb -i clouddb1
	[oracle@oravt02 ~]$ srvctl stop instance -d clouddb -i clouddb2
	[oracle@oravt02 ~]$ srvctl start instance -d clouddb -i clouddb2

view raw srvctl_modify_db.txt hosted with ❤ by GitHub

Hope it helps.

See you on Step 3 - Patching in the Deep. (Grid Patching using Fleet Maintenance).