RAC Node Deletion in 12c
In this post, I shall be discussing on the steps involved in removing a node from RAC cluster in oracle 12c.
Environment:
Cluster Nodes: 12cnode1, 12cnode2, 12cnode3 Database instances on the above nodes: srprim1 on 12cnode1 srprim2 on 12cnode2 srprim3 on 12cnode3
Grid Infrastructure version: 12.1.0.2 RDBMS version: 12.1.0.2 Node to be removed: 12cnode3
Here is a brief detail on the environment. The list of nodes captured using “olsnodes”.
[oracle@12cnode3 ~]$ olsnodes 12cnode1 12cnode2 12cnode3 [oracle@12cnode3 ~]$
Also this environment has Flex ASM enabled.
[oracle@12cnode3 ~]$ asmcmd showclustermode ASM cluster : Flex mode enabled [oracle@12cnode3 ~]$ [oracle@12cnode3 ~]$ asmcmd showclusterstate Normal [oracle@12cnode3 ~]$
Below is the list of the resources managed by this cluster.
[oracle@12cnode3 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ONLINE ONLINE 12cnode3 STABLE ora.DATA.dg ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE OFFLINE OFFLINE 12cnode3 STABLE ora.FRA.dg ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE OFFLINE OFFLINE 12cnode3 STABLE ora.LISTENER.lsnr ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE OFFLINE OFFLINE 12cnode3 STABLE ora.net1.network ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ONLINE ONLINE 12cnode3 STABLE ora.ons ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ONLINE ONLINE 12cnode3 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.12cnode1.vip 1 ONLINE ONLINE 12cnode1 STABLE ora.12cnode2.vip 1 ONLINE ONLINE 12cnode2 STABLE ora.12cnode3.vip 1 ONLINE ONLINE 12cnode3 STABLE ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE 12cnode1 STABLE ora.MGMTLSNR 1 ONLINE ONLINE 12cnode1 169.254.3.189 192.16 8.1.107,STABLE ora.asm 1 ONLINE ONLINE 12cnode1 Started,STABLE 3 ONLINE ONLINE 12cnode2 Started,STABLE ora.cvu 1 ONLINE ONLINE 12cnode1 STABLE ora.mgmtdb 1 ONLINE ONLINE 12cnode1 Open,STABLE ora.oc4j 1 ONLINE ONLINE 12cnode1 STABLE ora.scan1.vip 1 ONLINE ONLINE 12cnode1 STABLE ora.srprim.db 1 ONLINE ONLINE 12cnode1 Open,STABLE 2 ONLINE ONLINE 12cnode2 Open,STABLE 3 ONLINE ONLINE 12cnode3 Open,STABLE ora.srprim.srprim_i03.svc 1 ONLINE ONLINE 12cnode1 STABLE --------------------------------------------------------------------------------
Since Flex ASM is enabled, I see that all my oracle database instances are running on all the nodes even though ASM is not currently running on node 12cnode3. But that should not be a show stopper for us.
[oracle@12cnode3 ~]$ srvctl status asm ASM is running on 12cnode2,12cnode1
[oracle@12cnode3 ~]$ crsctl get node role config -all Node '12cnode1' configured role is 'hub' Node '12cnode2' configured role is 'hub' Node '12cnode3' configured role is 'hub'
[oracle@12cnode3 ~]$ crsctl get node role status -all Node '12cnode1' active role is 'hub' Node '12cnode2' active role is 'hub' Node '12cnode3' active role is 'hub'
Below is the detail of the votedisk and OCR files.
[oracle@12cnode3 ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE eba1a2f1e10b4f60bfbe59bd50be0ae6 (/dev/DSK1) [DATA] Located 1 voting disk(s).
[oracle@12cnode3 ~]$ ocrcheck Status of Oracle Cluster Registry is as follows : Version : 4 Total space (kbytes) : 409568 Used space (kbytes) : 1740 Available space (kbytes) : 407828 ID : 553047649 Device/File Name : +DATA Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged user [oracle@12cnode3 ~]$
Status of the database instances and the services currently running on the nodes.
[oracle@12cnode3 ~]$ srvctl status database -db srprim -v -f Instance srprim1 is running on node 12cnode1 with online services srprim_i03. Instance status: Open. Instance srprim2 is running on node 12cnode2. Instance status: Open. Instance srprim3 is running on node 12cnode3. Instance status: Open. [oracle@12cnode3 ~]$
Configuration of the database that is currently running on the cluster nodes.
[oracle@12cnode3 ~]$ srvctl config database -db srprim Database unique name: srprim Database name: srprim Oracle home: /u01/app/oracle/product/12.1.0.2/db_1 Oracle user: oracle Spfile: +DATA/SRPRIM/PARAMETERFILE/spfile.291.904396381 Password file: +DATA/SRPRIM/PASSWORD/pwdsrprim.276.904395317 Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: Disk Groups: FRA,DATA Mount point paths: Services: srprim_i03 Type: RAC Start concurrency: Stop concurrency: OSDBA group: oinstall OSOPER group: oinstall Database instances: srprim1,srprim2,srprim3 Configured nodes: 12cnode1,12cnode2,12cnode3 Database is administrator managed
Below is the configuration of the service configured for the “srprim” database.
[oracle@12cnode3 ~]$ srvctl config service -s srprim_i03 -db srprim Service name: srprim_i03 Server pool: Cardinality: 1 Disconnect: false Service role: PRIMARY Management policy: AUTOMATIC DTP transaction: false AQ HA notifications: false Global: false Commit Outcome: false Failover type: Failover method: TAF failover retries: TAF failover delay: Connection Load Balancing Goal: LONG Runtime Load Balancing Goal: NONE TAF policy specification: NONE Edition: Pluggable database name: srpdb1 Maximum lag time: ANY SQL Translation Profile: Retention: 86400 seconds Replay Initiation Time: 300 seconds Session State Consistency: GSM Flags: 0 Service is enabled Preferred instances: srprim3 Available instances: srprim1,srprim2
If you notice above, the service srprim_i03 is configured to run preferably on srprim3 instance (node 12cnode3). Hence we need to modify the “preferred instances” for service srprim_i03 else we would face error with message stating that “Cannot delete selected instance since it belongs to following services ‘srprim_i03’ as only preferred instance. Modify the services and try again.”
So, let’s modify the preferrend instances list for “srprim_i03” service as “srprim1”, and “srprim2”.
[oracle@12cnode3 ~]$ srvctl modify service -service srprim_i03 -db srprim -n -i srprim1,srprim2 [oracle@12cnode3 ~]$ srvctl status service -s srprim_i03 -db srprim Service srprim_i03 is running on instance(s) srprim1
Stop and start the service to get the new configuration in place.
[oracle@12cnode3 ~]$ srvctl stop service -s srprim_i03 -db srprim [oracle@12cnode3 ~]$ srvctl start service -s srprim_i03 -db srprim [oracle@12cnode3 ~]$ srvctl status service -s srprim_i03 -db srprim Service srprim_i03 is running on instance(s) srprim1,srprim2
[oracle@12cnode1 ~]$ srvctl status database -db srprim -v -f Instance srprim1 is running on node 12cnode1 with online services srprim_i03. Instance status: Open. Instance srprim2 is running on node 12cnode2 with online services srprim_i03. Instance status: Open. Instance srprim3 is running on node 12cnode3. Instance status: Open.
Now all looks good, so let’s move on with the node deletion. Removing node from a cluster involves 2 steps:
1. Removing any database instance residing on the node that needs to be deleted.
2. Removing the node from the cluster.
Now, first, let’s remove the database instance srprim3 residing on the node to be removed 12cnode3.
Run dbca from different node of the cluster and not on the node which you are trying to remove. I’m running “dbca” in silent mode here to delete the instance on the node which will be removed (srprim3).
[oracle@12cnode1 ~]$ dbca -silent -deleteInstance -gdbName srprim -instanceName srprim3 -nodeList 12cnode3 -sysDBAUserName sys -sysDBAPassword oracle Deleting instance 1% complete 2% complete 6% complete 13% complete 20% complete 26% complete 33% complete 40% complete 46% complete 53% complete 60% complete 66% complete Completing instance management. 100% complete Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/srprim.log" for further details. [oracle@12cnode1 ~]$
Let’s check the database and it’s instances’s status. Instance srprim3 has now been removed from node 12cnode3.
[oracle@12cnode1 ~]$ srvctl status database -db srprim -v -f Instance srprim1 is running on node 12cnode1 with online services srprim_i03. Instance status: Open. Instance srprim2 is running on node 12cnode2 with online services srprim_i03. Instance status: Open. [oracle@12cnode1 ~]$
From the database level, it’s clear below that the thread information and the redo logs associated with srprim3 too have been removed.
[oracle@12cnode1 ~]$ sqlplus / as sysdba SQL*Plus: Release 12.1.0.2.0 Production on Sun Sep 3 19:19:40 2017 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Advanced Analytics and Real Application Testing options SYS@srprim1>select group#,thread#,bytes/1024/1024 from v$log; GROUP# THREAD# BYTES/1024/1024 ---------- ---------- --------------- 1 1 50 2 1 50 3 2 50 4 2 50
SYS@srprim1>select group#,member from v$logfile order by group#; GROUP# MEMBER ---------- ------------------------------------------------------------ 1 +FRA/SRPRIM/ONLINELOG/group_1.257.904395571 1 +DATA/SRPRIM/ONLINELOG/group_1.282.904395571 2 +FRA/SRPRIM/ONLINELOG/group_2.258.904395577 2 +DATA/SRPRIM/ONLINELOG/group_2.283.904395575 3 +DATA/SRPRIM/ONLINELOG/group_3.289.904396373 3 +FRA/SRPRIM/ONLINELOG/group_3.259.904396375 4 +DATA/SRPRIM/ONLINELOG/group_4.290.904396377 4 +FRA/SRPRIM/ONLINELOG/group_4.260.904396379 8 rows selected.
SYS@srprim1>select name,value,inst_id from gv$parameter where name='undo_tablespace'; NAME VALUE INST_ID ------------------------------ -------------------------------------------------- ---------- undo_tablespace UNDOTBS2 2 undo_tablespace UNDOTBS1 1
Now, on the node to be removed, we need to update the inventory file. So, on 12cnode3, we update the inventory file with the database home listing only the node to be removed (12cnode3). This needs to be done only on the node to be removed and is done by running “oui” utility.
Current content of the inventory file on 12cnode3:
[oracle@12cnode3 ContentsXML]$ cat inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> <HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.1.0.2/db_1" TYPE="O" IDX="2"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode3 ContentsXML]$
Updating the database home node list on 12cnode3. Please note that we need to specify the “-local” option to “oui” in order to update the inventory locally.
[oracle@12cnode3 ~]$ cd /u01/app/oracle/product/12.1.0.2/db_1/oui/bin/ [oracle@12cnode3 bin]$ [oracle@12cnode3 bin]$ ./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={12cnode3}" -local Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 10237 MB Passed The inventory pointer is located at /etc/oraInst.loc 'UpdateNodeList' was successful. [oracle@12cnode3 bin]$
Below is the latest inventory file on 12cnode3.
[oracle@12cnode3 ContentsXML]$ cat inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> <HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.1.0.2/db_1" TYPE="O" IDX="2"> <NODE_LIST> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode3 ContentsXML]$
Once the inventory is updated with the database home nodelist being only the node to be removed on 12cnode3, let’s deinstall RDBMS home on 12cnode3.
Again please note that we need to specify the “-local” option to the “deinstall” utility to remove the RDBMS binary on 12cnode3.
[oracle@12cnode3 ~]$ cd /u01/app/oracle/product/12.1.0.2/db_1/deinstall/ [oracle@12cnode3 deinstall]$ ./deinstall -local Checking for required files and bootstrapping ... Please wait ... Location of logs /u01/app/oraInventory/logs/ ############ ORACLE DECONFIG TOOL START ############ ######################### DECONFIG CHECK OPERATION START ######################### ## [START] Install check configuration ## Checking for existence of the Oracle home location /u01/app/oracle/product/12.1.0.2/db_1 Oracle Home type selected for deinstall is: Oracle Real Application Cluster Database Oracle Base selected for deinstall is: /u01/app/oracle Checking for existence of central inventory location /u01/app/oraInventory Checking for existence of the Oracle Grid Infrastructure home /u01/app/12.1.0.2/grid The following nodes are part of this cluster: 12cnode3,12cnode2,12cnode1 Checking for sufficient temp space availability on node(s) : '12cnode3' ## [END] Install check configuration ## Network Configuration check config START Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_check2017-09-03_07-37-06-PM.log Network Configuration check config END Database Check Configuration START Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_check2017-09-03_07-37-16-PM.log Use comma as separator when specifying list of values as input Specify the list of database names that are configured locally on this node for this Oracle home. Local configurations of the discovered databases will be removed [srprim3,srprim3]: Database Check Configuration END Oracle Configuration Manager check START OCM check log file location : /u01/app/oraInventory/logs//ocm_check936.log Oracle Configuration Manager check END ######################### DECONFIG CHECK OPERATION END ######################### ####################### DECONFIG CHECK OPERATION SUMMARY ####################### Oracle Grid Infrastructure Home is: /u01/app/12.1.0.2/grid The following nodes are part of this cluster: 12cnode3,12cnode2,12cnode1 The cluster node(s) on which the Oracle home deinstallation will be performed are:12cnode3 Oracle Home selected for deinstall is: /u01/app/oracle/product/12.1.0.2/db_1 Inventory Location where the Oracle home registered is: /u01/app/oraInventory The option -local will not modify any database configuration for this Oracle home. Checking the config status for CCR Oracle Home exists with CCR directory, but CCR is not configured CCR check is finished Do you want to continue (y - yes, n - no)? [n]: y A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-09-03_07-36-41-PM.out' Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-09-03_07-36-41-PM.err' ######################## DECONFIG CLEAN OPERATION START ######################## Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_clean2017-09-03_07-38-17-PM.log Network Configuration clean config START Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_clean2017-09-03_07-38-17-PM.log Network Configuration clean config END Oracle Configuration Manager clean START OCM clean log file location : /u01/app/oraInventory/logs//ocm_clean936.log Oracle Configuration Manager clean END ######################### DECONFIG CLEAN OPERATION END ######################### ####################### DECONFIG CLEAN OPERATION SUMMARY ####################### Cleaning the config for CCR As CCR is not configured, so skipping the cleaning of CCR configuration CCR clean is finished ####################################################################### ############# ORACLE DECONFIG TOOL END ############# Using properties file /tmp/deinstall2017-09-03_07-32-35PM/response/deinstall_2017-09-03_07-36-41-PM.rsp Location of logs /u01/app/oraInventory/logs/ ############ ORACLE DEINSTALL TOOL START ############ ####################### DEINSTALL CHECK OPERATION SUMMARY ####################### A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-09-03_07-36-41-PM.out' Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-09-03_07-36-41-PM.err' ######################## DEINSTALL CLEAN OPERATION START ######################## ## [START] Preparing for Deinstall ## Setting LOCAL_NODE to 12cnode3 Setting CLUSTER_NODES to 12cnode3 Setting CRS_HOME to false Setting oracle.installer.invPtrLoc to /tmp/deinstall2017-09-03_07-32-35PM/oraInst.loc Setting oracle.installer.local to true ## [END] Preparing for Deinstall ## Setting the force flag to false Setting the force flag to cleanup the Oracle Base Oracle Universal Installer clean START Detach Oracle home '/u01/app/oracle/product/12.1.0.2/db_1' from the central inventory on the local node : Done Delete directory '/u01/app/oracle/product/12.1.0.2/db_1' on the local node : Done The Oracle Base directory '/u01/app/oracle' will not be removed on local node. The directory is in use by Oracle Home '/u01/app/12.1.0.2/grid'. Oracle Universal Installer cleanup was successful. Oracle Universal Installer clean END ## [START] Oracle install clean ## Clean install operation removing temporary directory '/tmp/deinstall2017-09-03_07-32-35PM' on node '12cnode3' ## [END] Oracle install clean ## ######################### DEINSTALL CLEAN OPERATION END ######################### ####################### DEINSTALL CLEAN OPERATION SUMMARY ####################### Successfully detached Oracle home '/u01/app/oracle/product/12.1.0.2/db_1' from the central inventory on the local node. Successfully deleted directory '/u01/app/oracle/product/12.1.0.2/db_1' on the local node. Oracle Universal Installer cleanup was successful. Oracle deinstall tool successfully cleaned up temporary directories. ####################################################################### ############# ORACLE DEINSTALL TOOL END ############# [oracle@12cnode3 deinstall]$
On the remaining nodes of the cluster, if we would check the inventory file, the database home would still list all the nodes. So, we need to update the inventory file on the remaining nodes with the node list under the Database home listing only the remaining nodes.
On 12cnode1, let me update the node list under the Database Home using OUI utility.
Current content of the inventory file on 12cnode1.
[oracle@12cnode1 ~]$ cd /u01/app/oraInventory/ContentsXML/ [oracle@12cnode1 ContentsXML]$ ls -lrt total 12 -rw-rw----. 1 oracle oinstall 817 Mar 24 2016 inventory.xml -rw-rw----. 1 oracle oinstall 292 Mar 24 2016 libs.xml -rw-rw----. 1 oracle oinstall 329 Mar 24 2016 comps.xml [oracle@12cnode1 ContentsXML]$ cat inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> <HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.1.0.2/db_1" TYPE="O" IDX="2"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode1 ContentsXML]$
Update the Database Home nodelist using oui with the cluster_nodes option listing the remaining nodes of the cluster.
[oracle@12cnode1 ~]$ . oraenv ORACLE_SID = [srprim1] ? The Oracle base remains unchanged with value /u01/app/oracle [oracle@12cnode1 ~]$ cd $ORACLE_HOME/oui/bin [oracle@12cnode1 bin]$ ./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={12cnode1,12cnode2}" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 8125 MB Passed The inventory pointer is located at /etc/oraInst.loc 'UpdateNodeList' was successful. [oracle@12cnode1 bin]$
Once the inventory is updated, let’s verify the contents of the inventory file.
[oracle@12cnode1 bin]$ cat /u01/app/oraInventory/ContentsXML/inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> <HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.1.0.2/db_1" TYPE="O" IDX="2"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode1 bin]$
So, we have now successfully removed the database instance and the RDBMS home from 12cnode3. Let me now proceed with removing the node from the cluster.
Just briefing up, the “olsnodes” is listing out node 12cnode3. Further actions involve deconfiguring the cluster resources on 12cnode3 and then removing the GI home.
[oracle@12cnode1 ~]$ . oraenv ORACLE_SID = [srprim1] ? +ASM1 The Oracle base remains unchanged with value /u01/app/oracle [oracle@12cnode1 ~]$ [oracle@12cnode1 ~]$ olsnodes -s 12cnode1 Active 12cnode2 Active 12cnode3 Active [oracle@12cnode1 ~]$
[oracle@12cnode3 ~]$ olsnodes -s 12cnode1 Active 12cnode2 Active 12cnode3 Active [oracle@12cnode3 ~]$
On 12cnode3, deconfigure the cluster resources by running “rootcrs.pl” script with “deconfig” option from the GI home.
[root@12cnode3 ~]# . oraenv ORACLE_SID = [root] ? +ASM3 The Oracle base has been set to /u01/app/oracle [root@12cnode3 ~]# [root@12cnode3 ~]# cd /u01/app/12.1.0.2/grid/crs/install [root@12cnode3 install]# ./rootcrs.pl -deconfig -force Using configuration parameter file: ./crsconfig_params Network 1 exists Subnet IPv4: 192.168.0.0/255.255.255.0/eth0, static Subnet IPv6: Ping Targets: Network is enabled Network is individually enabled on nodes: Network is individually disabled on nodes: VIP exists: network number 1, hosting node 12cnode1 VIP Name: 12cnode1-vip.mydomain VIP IPv4 Address: 192.168.0.117 VIP IPv6 Address: VIP is enabled. VIP is individually enabled on nodes: VIP is individually disabled on nodes: VIP exists: network number 1, hosting node 12cnode2 VIP Name: 12cnode2-vip.mydomain VIP IPv4 Address: 192.168.0.118 VIP IPv6 Address: VIP is enabled. VIP is individually enabled on nodes: VIP is individually disabled on nodes: VIP exists: network number 1, hosting node 12cnode3 VIP Name: 12cnode3-vip VIP IPv4 Address: 192.168.0.121 VIP IPv6 Address: VIP is enabled. VIP is individually enabled on nodes: VIP is individually disabled on nodes: ONS exists: Local port 6100, remote port 6200, EM port 2016, Uses SSL false ONS is enabled ONS is individually enabled on nodes: ONS is individually disabled on nodes: CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '12cnode3' CRS-2673: Attempting to stop 'ora.crsd' on '12cnode3' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on '12cnode3' CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '12cnode3' CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '12cnode3' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on '12cnode3' has completed CRS-2677: Stop of 'ora.crsd' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.evmd' on '12cnode3' CRS-2673: Attempting to stop 'ora.storage' on '12cnode3' CRS-2673: Attempting to stop 'ora.ctssd' on '12cnode3' CRS-2673: Attempting to stop 'ora.mdnsd' on '12cnode3' CRS-2673: Attempting to stop 'ora.gpnpd' on '12cnode3' CRS-2673: Attempting to stop 'ora.drivers.acfs' on '12cnode3' CRS-2677: Stop of 'ora.storage' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.asm' on '12cnode3' CRS-2677: Stop of 'ora.drivers.acfs' on '12cnode3' succeeded CRS-2677: Stop of 'ora.asm' on '12cnode3' succeeded CRS-2677: Stop of 'ora.ctssd' on '12cnode3' succeeded CRS-2677: Stop of 'ora.evmd' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '12cnode3' CRS-2677: Stop of 'ora.mdnsd' on '12cnode3' succeeded CRS-2677: Stop of 'ora.gpnpd' on '12cnode3' succeeded CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.cssd' on '12cnode3' CRS-2677: Stop of 'ora.cssd' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.crf' on '12cnode3' CRS-2677: Stop of 'ora.crf' on '12cnode3' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on '12cnode3' CRS-2677: Stop of 'ora.gipcd' on '12cnode3' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '12cnode3' has completed CRS-4133: Oracle High Availability Services has been stopped. 2017/09/03 20:05:03 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector. 2017/09/03 20:05:37 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector. 2017/09/03 20:05:40 CLSRSC-336: Successfully deconfigured Oracle Clusterware stack on this node [root@12cnode3 install]#
Now, remove the node from the cluster using the “crsctl” utility.
[root@12cnode1 ~]# . oraenv ORACLE_SID = [root] ? +ASM1 The Oracle base has been set to /u01/app/oracle [root@12cnode1 ~]# [root@12cnode1 ~]# crsctl delete node -n 12cnode3 CRS-4661: Node 12cnode3 successfully deleted. [root@12cnode1 ~]#
The following steps involve removing the GI home which is similar to what we followed while removing the RDBMS home.
Firstly, update the inventory file on 12cnode3 with the information that this node is lonely and not part of the cluster. This is done by running the OUI utility from GI home.
Options passed to OUI are as below:
ORACLE_HOME: GI home
CLUSTER_NODES: 12cnode3 (The node to be removed)
CRS=TRUE: as we are dealing with GI home and not RDBMS home
local: which defines that the update needs to be done only on this node.
[oracle@12cnode3 ~]$ id uid=501(oracle) gid=502(oinstall) groups=502(oinstall),501(dba) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [oracle@12cnode3 ~]$ [oracle@12cnode3 ~]$ . oraenv ORACLE_SID = [+ASM3] ? The Oracle base remains unchanged with value /u01/app/oracle [oracle@12cnode3 ~]$ [oracle@12cnode3 ~]$ cd $ORACLE_HOME/oui/bin [oracle@12cnode3 bin]$ ./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={12cnode3}" CRS=TRUE -local Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 10237 MB Passed The inventory pointer is located at /etc/oraInst.loc 'UpdateNodeList' was successful.
Verfiy the contents of the inventory file on 12cnode3.
[oracle@12cnode3 bin]$ [oracle@12cnode3 bin]$ [oracle@12cnode3 bin]$ cat /u01/app/oraInventory/ContentsXML/inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode3"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode3 bin]$
Now, I can deinstall the GI home on 12cnode3 by running the “deinstall” script from the GI home.
[oracle@12cnode3 ~]$ echo $ORACLE_SID +ASM3 [oracle@12cnode3 ~]$ cd /u01/app/12.1.0.2/grid/deinstall/ [oracle@12cnode3 deinstall]$ ./deinstall -local Checking for required files and bootstrapping ... Please wait ... Location of logs /tmp/deinstall2017-09-03_08-13-07PM/logs/ ############ ORACLE DECONFIG TOOL START ############ ######################### DECONFIG CHECK OPERATION START ######################### ## [START] Install check configuration ## Checking for existence of the Oracle home location /u01/app/12.1.0.2/grid Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster Oracle Base selected for deinstall is: /u01/app/oracle Checking for existence of central inventory location /u01/app/oraInventory Checking for existence of the Oracle Grid Infrastructure home /u01/app/12.1.0.2/grid The following nodes are part of this cluster: 12cnode3 Checking for sufficient temp space availability on node(s) : '12cnode3' ## [END] Install check configuration ## Traces log file: /tmp/deinstall2017-09-03_08-13-07PM/logs//crsdc_2017-09-03_08-17-00PM.log Network Configuration check config START Network de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/netdc_check2017-09-03_08-17-02-PM.log Network Configuration check config END Asm Check Configuration START ASM de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/asmcadc_check2017-09-03_08-17-02-PM.log Database Check Configuration START Database de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/databasedc_check2017-09-03_08-17-02-PM.log Database Check Configuration END ######################### DECONFIG CHECK OPERATION END ######################### ####################### DECONFIG CHECK OPERATION SUMMARY ####################### Oracle Grid Infrastructure Home is: /u01/app/12.1.0.2/grid The following nodes are part of this cluster: 12cnode3 The cluster node(s) on which the Oracle home deinstallation will be performed are:12cnode3 Oracle Home selected for deinstall is: /u01/app/12.1.0.2/grid Inventory Location where the Oracle home registered is: /u01/app/oraInventory Option -local will not modify any ASM configuration. Do you want to continue (y - yes, n - no)? [n]: y A log of this session will be written to: '/tmp/deinstall2017-09-03_08-13-07PM/logs/deinstall_deconfig2017-09-03_08-16-54-PM.out' Any error messages from this session will be written to: '/tmp/deinstall2017-09-03_08-13-07PM/logs/deinstall_deconfig2017-09-03_08-16-54-PM.err' ######################## DECONFIG CLEAN OPERATION START ######################## Database de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/databasedc_clean2017-09-03_08-17-55-PM.log ASM de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/asmcadc_clean2017-09-03_08-17-55-PM.log ASM Clean Configuration END Network Configuration clean config START Network de-configuration trace file location: /tmp/deinstall2017-09-03_08-13-07PM/logs/netdc_clean2017-09-03_08-17-55-PM.log Network Configuration clean config END ######################### DECONFIG CLEAN OPERATION END ######################### ####################### DECONFIG CLEAN OPERATION SUMMARY ####################### Oracle Clusterware was already stopped and de-configured on node "12cnode3" Oracle Clusterware is stopped and de-configured successfully. ####################################################################### ############# ORACLE DECONFIG TOOL END ############# Using properties file /tmp/deinstall2017-09-03_08-13-07PM/response/deinstall_2017-09-03_08-16-54-PM.rsp Location of logs /tmp/deinstall2017-09-03_08-13-07PM/logs/ ############ ORACLE DEINSTALL TOOL START ############ ####################### DEINSTALL CHECK OPERATION SUMMARY ####################### A log of this session will be written to: '/tmp/deinstall2017-09-03_08-13-07PM/logs/deinstall_deconfig2017-09-03_08-16-54-PM.out' Any error messages from this session will be written to: '/tmp/deinstall2017-09-03_08-13-07PM/logs/deinstall_deconfig2017-09-03_08-16-54-PM.err' ######################## DEINSTALL CLEAN OPERATION START ######################## ## [START] Preparing for Deinstall ## Setting LOCAL_NODE to 12cnode3 Setting CLUSTER_NODES to 12cnode3 Setting CRS_HOME to true Setting oracle.installer.invPtrLoc to /tmp/deinstall2017-09-03_08-13-07PM/oraInst.loc Setting oracle.installer.local to true ## [END] Preparing for Deinstall ## Setting the force flag to false Setting the force flag to cleanup the Oracle Base Oracle Universal Installer clean START Detach Oracle home '/u01/app/12.1.0.2/grid' from the central inventory on the local node : Done Failed to delete the directory '/u01/app/12.1.0.2/grid/dc_ocm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/has'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/ohasd/12cnode3'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/ohasd'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/crs/12cnode3'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/crs'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/css/12cnode3'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/css'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/evm/12cnode3'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth/evm'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/auth'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/wwg'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/diagnostics'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/sqlplus'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/demo'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/jdk'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/jlib'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ord'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/ologgerd/init/12cnode3.pid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/ologgerd/init/12cnode3'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ologgerd/init'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/ologgerd'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/cdata'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocssd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_start.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclsvmon'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/scrctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sysresv'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orald'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dg4pwdO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/patchgen'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_register.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraenv'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clsecho'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsallociO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crsd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmsort'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gsd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/asmcmdcore'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/adrci'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/osysmond.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapbind'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extjob'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapmodify'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afdtool'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rmanO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clscfg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrdump'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/bndlchk'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/scriptagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/appagent.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsdepxaO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocssdrim'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extjobO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/appvipcfg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sqlplus'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrconfig.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crstmpl.scr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rman'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tkprofO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/setasmgid0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/skgxpinfo'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agtctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssvfupgd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbgeu_run_action.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrcheck.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcroute'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ghappctl.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/netmgr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/diskmon'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kfod'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbshut'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mkpatch'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcasst'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/aqxmlctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/renamedg0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/emcrsp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crsdiag.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/loadpsp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/asmca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cluutil'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/impO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gnsd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dg4pwd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orapki'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oprocd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/expO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmshow.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sysresv0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrcheck'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crsctl.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrpatch'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/nidO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ologgerd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_relocate'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sclsspawn'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapmoddn'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbstart'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sqlldrO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssdmonitor'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/racgwrap'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clscfg.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agapacheas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sbttestO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okadriverstate'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tnsping'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsdriverstate'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ologdbg.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tkprof'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/octssd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmwatch'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfssinglefsmount'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clsid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/osdbagrp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/racgvip'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/setasmgidwrap'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsnrctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okinit0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmshow'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gpnptool.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clsid.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/nid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evt.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmlogger'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cursize'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mkstore'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclumon.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/impdpO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_preapply'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_start'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocssd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ojvmtc'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/jssu'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclsomon'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_apply'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssvfupgd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/racgeut'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmmklib.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsalloci'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapcompare'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gipcd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crswrapexece.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agsiebsrvras'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agtomcatas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oradnssd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oracle'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/osysmond'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ologdbg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclumon'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afdroot'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oranetmonitor.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/olsnodes'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crsd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssdagent.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agsiebgtwyas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cemutls'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/impdp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsnodes'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mapsga'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/wrap'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmpost.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dsml2ldif'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tstshm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/loadjava'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tnsping0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/amdu'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lbuilder'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_monitor'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oerr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsotsO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/appagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/uidrvciO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcldr0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/emdwgrd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extprocO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oc4jctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dropjava'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmmkbin.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ghappctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/symfind'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/setasmgid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/maxmem'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsdb.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/adapters'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ndfnceca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/odnsd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gensyslib'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okdstry0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evminfo'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orapwdO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmmkbin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/fmputlhp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ohasd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cursizeO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/unzip'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extproc'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/schema'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genorasdksh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/odnsd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kgmgrO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tstshmO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/usrvip'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sbttest'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genoccish'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oc4jctl.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agpsappas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orapki.bat'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_apply.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclumon.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gpnpd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/aggoldengateas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genclntsh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orapwd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cluvfy'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mapsga0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/onsctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mkstore.bat'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/asmproxy'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oifcfg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapdelete'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsregistrymount'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldifmigrator'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tnnfg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genclntst'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbua'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/diagsetup'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cemutlo.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_profile'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_getperm.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mdnsd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mkpatchO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oerr.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ncomp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_stat'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraagent.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/schemasync'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rawutl0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/coraenv'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/expdpO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/srvctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oifcfg.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kfod.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extusrupgrade'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbhome'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbfsizeO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/uidrvci'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cemutlo'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_unregister.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okinit'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsdb'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapaddmt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmwatch.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orabase'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/linkshlib'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okdstry'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/owm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agtctlO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crfsetenv'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_unregister'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agpsbatchas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/odig.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/agpspiaas'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/statusnc'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lxchknlb'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/expdp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/olsnodes.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orarootagent.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_getperm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orion'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clsecho.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/qosctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsots'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extjoboO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/emdwgrd.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/xmlwf'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_relocate.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mdnsd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gpnpd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oradaemonagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/umu'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gnsd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kgmgr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/diagcollection.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oradism'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gennttab'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dumpsga'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/loadpspO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genagtsh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraxsl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/netca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/octssd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clssproxy.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/deploync'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/appvipcfg.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcroute0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/aqxmlctl.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_profile.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genezi'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssdagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rconfig'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dgmgrlO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/renamedg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/wrcO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclskd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmsort.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lxegen'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/odisrvreg'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cssdmonitor.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evminfo.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cemutls.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrconfig'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kfed'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/wrc'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orionO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/eusm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ohasd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcsess'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/osh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afdtool.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rhpctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_register'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/sqlldr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rdtool'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/echodo'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/hsdepxa'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ojvmjava'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/vipca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/relink'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapsearch'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/exp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/adrciO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/amduO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/asmcmd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmpost'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oidca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbfs_client'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afdload'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmlogger.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oracleO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/genksms'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/emca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oranetmonitor'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/cluvfyrac.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbfsize'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsnodes.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/trcldr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tnslsnr0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/diskmon.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_stop.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/plshprofO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/wrapO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/emcrsp.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oclskd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsreplcrs'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraping'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lmsgen'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/netca_deinst.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gipcd.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oklist0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orajaxb'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/scriptagent.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dumpsga0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oidprovtool'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_setperm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/clssproxy'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/racgevtf'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/imp'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gennfgt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/racgmain'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crsctl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oklist'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afddriverstate'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/tnslsnr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocssdrim.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/maxmemO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lcsscan'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/orarootagent'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsroot'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/osdbagrp0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ocrdump.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kfedO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oradnssd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dgmgrl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/extjobo'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/oraxml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_initializer'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/diagcollection.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/kfodO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_stat.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/zip'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsrepl_transport'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/acfsload'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/platform_common'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapadd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/plshprof'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbv'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/evmmklib'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/gpnptool'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/ldapmodifymt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lxinst'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okaload'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/fmputl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/odig'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/srvconfig'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/skgxpinfoO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_setperm.bin'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/geneziO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/rawutl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/mgmtca'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/dbvO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/crs_stop'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/okaroot'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/afdboot'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/bin/lsnrctl0'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/bin'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/javavm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/xdk'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/oracore'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ucp'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/ctss/init/12cnode3.pid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/ctss/init/12cnode3'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ctss/init'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/ctss'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/osysmond/init/12cnode3.pid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/osysmond/init/12cnode3'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/osysmond/init'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/osysmond'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/install'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/assistants'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/profiles/peer'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/profiles'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/wallets/pa'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/wallets/root'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/wallets/prdr'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/wallets/peer'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1/wallets'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/new__12cnode1'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/profiles/peer'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/profiles'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/wallets/pa'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/wallets/root'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/wallets/prdr'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/wallets/peer'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3/wallets'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507/stg__12cnode3'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp/gpnp_bcp__2016_3_23_18507'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/gpnp'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/tfa'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/racg'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/deinstall'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/cfgtoollogs'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/inventory'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ohasd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/sqlpatch'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/dbs'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/root.sh'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/OPatch'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/clone'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gipc'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/utl'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/admin/crf12cnode3.cfg'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/admin/run/crflogd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/admin/run/crfmond'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/admin/run'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/admin/crf12cnode3.ora'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/admin'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/db/12cnode3/proc/procdump.l01'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/db/12cnode3/proc/procdump.l02'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/db/12cnode3/proc/localdump.hdr'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crf/db/12cnode3/proc/procdump.log'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/db/12cnode3/proc'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/db/12cnode3'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf/db'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crf'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/srvm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/cv'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/owm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/addnode'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/oc4j'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gnsd/init'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/gnsd'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/nls'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/scheduler'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/ldap'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/usm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/jdbc'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/eons'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsrvmcred12.so'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/lib/stubs'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libba12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcorejava.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmddisk.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_afdlib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxn2.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/clntshcore.map'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntshcore.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfstoolsdriver.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/s0main.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libexpat.la'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcore12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnls12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liboramysql12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcommon12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libuini12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libpatchgensh12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfsroot.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libordimt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libgx12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libexpat.so.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/naeet.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/nnfgt.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsrvm12.so0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libosbws12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcell12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfssinglefsmount.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntsh.so.11.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libagtsh.so.1.0'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippdcemerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libgns12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_unix_linux_afdlib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskvol12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpksse4212.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdambr.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxpd.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_afdroot.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/sscoreed.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsnls12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnoname12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdaudit.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libntcp12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libntcpaio12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libplc12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsqlplus.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpkavx12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/librdjni12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libvsn12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libvsn12_std.a.dbl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocijdbc12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnsgr12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libntns12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcommand.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libodm12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpkavx212.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnnetd12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libkubsagt12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxpg.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsvml.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libacfs12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libwwg.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libonsx.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_base_non_fips.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xmlparserv2.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libgnsjni12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_okaroot.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/activation.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libmm.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcryptocme.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnnz12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/afdlib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_asym.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libons.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippdcmerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libipp_bz2.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxpcompat.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxns.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libxml12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/okalib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxp12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntsh.so.12.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocrutl12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_ecc_accel_non_fips.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_base.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libexpat.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/afdtoolsdriver.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocci.so.12.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmddiag.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnid.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcell12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/facility.lis'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libowm2.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/afdload.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcrf12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libpsa12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libheteroxa12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdvol.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnzjs12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liboraz.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfslib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnbeq12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_ecc_accel_fips.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xmlparserv2_sans_jaxp_services.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libodm12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libasmclnt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocci12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libeonsserver.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/oc4jctl_lib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdparser.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntsh.so.10.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdamdu.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libasmperl12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcryptocme.sig'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xmlparserv2_jaxp_services.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdbase.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_afddriverstate.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpkavx12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/okadriverstate.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libplp12_pic.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdanlz.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libordim12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libirc.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libn12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntshcore.so.12.1'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libgeneric12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclsr12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/nigcon.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_okadriverstate.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfsload.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libldapclnt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfsregistrymount.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libpls12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/sysliblist'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xschema.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/okatoolsdriver.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libzx12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/afdroot.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libons.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libvsn12_cee.a.dbl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libexpat.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntst12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/nautab.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/ldflagsO'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsqlplus.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/nigtab.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippsemerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libncrypt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libplp12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocr12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippsmerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsrvm12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liborion12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libzt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskgxpr.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnus12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libasmclntsh12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfsreplcrs.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libntmq12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libntcps12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libeons.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnnz12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libdbcfg12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/ntcontab.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libagfw12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_ecc.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/ldflags'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnro12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdexceptions.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libqsmashr.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_unix_linux_okalib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnque12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcxaguard.so.5'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippcore.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdtmpl.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclib_jiio.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfsregistrymount.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libordim12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsrvmhas12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdglobal.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libimf.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclient12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libztkg12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libpls12_pic.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libcrf_mdb12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/oc4jctl_common.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libvsn12_cse.a.dbl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xmlmesg.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libxdb.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/afddriverstate.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnhost12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libccme_ecc_non_fips.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfsroot.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/okaload.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libmql1.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libldapjclnt12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libhasgen12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/naedhs.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfsdriverstate.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libipp_z.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libldapjclnt12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsql12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libplc12_pic.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnl12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libserver12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libctxc12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osntabst.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/scorept.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/http_client.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippcpemerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libagent12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdattr.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libasmclntsh12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnjni12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libctx12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libpatchgensh12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclntsh.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdpasswd.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libodmd12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liboramysql12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdug.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnnzst12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libskjcx12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xmlcomp2.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/okaroot.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libslax12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocrb12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_afdload.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libintlc.so.5'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclsce12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_okalib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdshare.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xsu12.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnfsodm12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libavstub12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libippcpmerged.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfslib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/clntsh.map'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfssinglefsmount.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liblxled.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpkavx212.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liboevm.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libocci.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libshpksse4212.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnnet12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_acfsdriverstate.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libagtsh.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libipc1.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/acfsload.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liborabz2.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libordsdo12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/naect.o'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libexpat.so.1.5.2'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdsys.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_unix_linux_acfslib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libasmperl12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libafd12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libsrvmocr12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libclsra12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/xml.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libunls12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/mail.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmdxmlexceptions.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libnldap12.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libusmcrs12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libopc12.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libmql1.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/asmcmd_disk_header_format'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/libipc1.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/osds_okaload.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/liblzopro.a'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/lib/s_oc4jctl_lib.pm'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/lib'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/plsql'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/acfs'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/diskmon'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/admin'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/log/12cnode3/alert12cnode3.log'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/ctssd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/client'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/gipcd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/crsd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/racg'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/ohasd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/gpnpd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/evmd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/crflogd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/mdnsd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/srvm'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/gnsd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/cssd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/afd'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/crfmond'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3/xag'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/log/12cnode3'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/log'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/auth'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/demo'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/script'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/config'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/install'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/public'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/utl/12cnode3/crsconfig_fileperms'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/utl/12cnode3/crsconfig_dirs'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/utl/12cnode3'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/utl'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/trace'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/ohasd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/oka'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/init.ohasd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/12cnode3.pid'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/afd.sles'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/ohasd.sles'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/afd'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/init/12cnode3'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/init'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/lib'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/log'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/profile'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/mesg'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/diskmon.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/gpnp.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/cssd.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/asm.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/mdns.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/evm.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/drivers.acfs.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/daemon.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/application.tdf'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/ctss.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/storage.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/ohasdbase.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/generic.tdf'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/appvipx.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/TYPE_application.cap'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/crf.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/TYPE_generic.cap'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/driver.afd.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/oka.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/haip.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/gipc.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/registry.acfs.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/cssdmonitor.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/appvip.type'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/crs/template/crs.type'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/template'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs/sbs'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/crs'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/precomp'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/network'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/hs'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/dmu'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/opmn'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/instantclient'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/css'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/md'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/QOpatch'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/rest'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/evm'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/internal/usableports.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/internal/runstatus.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/internal/.buildid'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/internal'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/jewt4.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/je-4.1.27.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/ojdbc6.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/share.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/jdev-rt.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/Symlink.so'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/jsch.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/RATFA.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/je-5.0.84.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib/commons-io-2.2.jar'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/jlib'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/public.jks'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/getppid.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/uninstalltfa'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/tfasetup.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/Text/ASCIITable/Wrap.pm'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/Text/ASCIITable'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/Text/ASCIITable.pm'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/Text'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/discover_ora_stack.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/profiling.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/uninstalltfa.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/tfactl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/tfactl.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/tfactl_lib.pm'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/patchtfa.sh'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/collectfiles.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/metric_iorm.pl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin/tfactl.tmpl'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/bin'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/inittab.sunos'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/oracle-tfa.conf'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/inittab.aix'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/oracle-tfa.service'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/inittab.hpux'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/init.tfa.tmpl'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/inittab'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install/inittab.linux'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/install'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/tfa.jks'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/tfa_directories.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/bin/tnt'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/bin'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/conf/tnt.prop.tmpl'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/conf'. The directory is not empty. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/commons-cli-1.0.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/commons-lang-2.1.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/xmlparserv2.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/xternal.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/commons-logging-1.1.jar'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib/tnt.jar'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt/lib'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext/tnt'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/ext'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/input'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/tfa_setup.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/tfa.md5'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/problemset.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/file_type_patterns.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/tasks.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/ignore_extensions.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/searchStrings.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/collect_all_directories.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/date_patterns.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/file_type_patterns_internal.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/ignorefiles.txt'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/scanFileList.xml'. The file is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources/mask_strings.xml'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home/resources'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release/tfa_home'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa/release'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/tfa'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools/orachk'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/suptools'. The directory is not empty. Failed to delete the directory '/u01/app/12.1.0.2/grid/xag'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/perl'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/wlm'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/rootupgrade.sh'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/relnotes'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/mdns'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/rdbms'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/slax'. The directory is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid/oui'. The directory is in use. Failed to delete the file '/u01/app/12.1.0.2/grid/oraInst.loc'. The file is in use. Failed to delete the directory '/u01/app/12.1.0.2/grid'. The directory is not empty. Delete directory '/u01/app/12.1.0.2/grid' on the local node : Failed <<<< Delete directory '/u01/app/oraInventory' on the local node : Done The Oracle Base directory '/u01/app/oracle' will not be removed on local node. The directory is not empty. Oracle Universal Installer cleanup was successful. Oracle Universal Installer clean END ## [START] Oracle install clean ## Clean install operation removing temporary directory '/tmp/deinstall2017-09-03_08-13-07PM' on node '12cnode3' ## [END] Oracle install clean ## ######################### DEINSTALL CLEAN OPERATION END ######################### ####################### DEINSTALL CLEAN OPERATION SUMMARY ####################### Successfully detached Oracle home '/u01/app/12.1.0.2/grid' from the central inventory on the local node. Failed to delete directory '/u01/app/12.1.0.2/grid' on the local node. Successfully deleted directory '/u01/app/oraInventory' on the local node. Oracle Universal Installer cleanup was successful. Run 'rm -r /etc/oraInst.loc' as root on node(s) '12cnode3' at the end of the session. Run 'rm -r /opt/ORCLfmap' as root on node(s) '12cnode3' at the end of the session. Run 'rm -r /etc/oratab' as root on node(s) '12cnode3' at the end of the session. Oracle deinstall tool successfully cleaned up temporary directories. ####################################################################### ############# ORACLE DEINSTALL TOOL END ############# [oracle@12cnode3 deinstall]$
The above error message can be ignored to remove the GI home.
Now, update the inventory file on the remaining nodes that the node list does not contain 12cnode3.
[oracle@12cnode1 ~]$ . oraenv ORACLE_SID = [+ASM1] ? The Oracle base remains unchanged with value /u01/app/oracle [oracle@12cnode1 ~]$ [oracle@12cnode1 ~]$ cd $ORACLE_HOME/oui/bin [oracle@12cnode1 bin]$ ./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={12cnode1,12cnode2}" CRS=TRUE -silent Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 8126 MB Passed The inventory pointer is located at /etc/oraInst.loc 'UpdateNodeList' was successful. [oracle@12cnode1 bin]$
Just verifying the inventory file post update.
[oracle@12cnode1 bin]$ cat /u01/app/oraInventory/ContentsXML/inventory.xml <?xml version="1.0" standalone="yes" ?> <!-- Copyright (c) 1999, 2014, Oracle and/or its affiliates. All rights reserved. --> <!-- Do not modify the contents of this file by hand. --> <INVENTORY> <VERSION_INFO> <SAVED_WITH>12.1.0.2.0</SAVED_WITH> <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER> </VERSION_INFO> <HOME_LIST> <HOME NAME="OraGI12Home1" LOC="/u01/app/12.1.0.2/grid" TYPE="O" IDX="1" CRS="true"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> </NODE_LIST> </HOME> <HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.1.0.2/db_1" TYPE="O" IDX="2"> <NODE_LIST> <NODE NAME="12cnode1"/> <NODE NAME="12cnode2"/> </NODE_LIST> </HOME> </HOME_LIST> <COMPOSITEHOME_LIST> </COMPOSITEHOME_LIST> </INVENTORY> [oracle@12cnode1 bin]$
“olsnodes” too now just shows 12cnode1 and 12cnode2 as the nodes of the cluster and not 12cnode3.
[oracle@12cnode1 bin]$ olsnodes -s 12cnode1 Active 12cnode2 Active [oracle@12cnode1 bin]$
Verify the cluster resources status from any one of the existing nodes.
[oracle@12cnode1 bin]$ crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ora.DATA.dg ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ora.FRA.dg ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ora.LISTENER.lsnr ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ora.net1.network ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE ora.ons ONLINE ONLINE 12cnode1 STABLE ONLINE ONLINE 12cnode2 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.12cnode1.vip 1 ONLINE ONLINE 12cnode1 STABLE ora.12cnode2.vip 1 ONLINE ONLINE 12cnode2 STABLE ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE 12cnode1 STABLE ora.MGMTLSNR 1 ONLINE ONLINE 12cnode1 169.254.3.189 192.16 8.1.107,STABLE ora.asm 1 ONLINE ONLINE 12cnode1 Started,STABLE 3 ONLINE ONLINE 12cnode2 Started,STABLE ora.cvu 1 ONLINE ONLINE 12cnode1 STABLE ora.mgmtdb 1 ONLINE ONLINE 12cnode1 Open,STABLE ora.oc4j 1 ONLINE ONLINE 12cnode1 STABLE ora.scan1.vip 1 ONLINE ONLINE 12cnode1 STABLE ora.srprim.db 1 ONLINE ONLINE 12cnode1 Open,STABLE 2 ONLINE ONLINE 12cnode2 Open,STABLE ora.srprim.srprim_i03.svc 1 ONLINE ONLINE 12cnode1 STABLE 2 ONLINE ONLINE 12cnode2 STABLE -------------------------------------------------------------------------------- [oracle@12cnode1 bin]$
We can run the “cluvfy” utility to see if any messages are reported by post node deletion.
[oracle@12cnode1 ~]$ cluvfy stage -post nodedel -n 12cnode3 -verbose Performing post-checks for node removal Checking CRS integrity... The Oracle Clusterware is healthy on node "12cnode1" CRS integrity check passed Clusterware version consistency passed. Result: Node removal check passed Post-check for node removal was successful. [oracle@12cnode1 ~]$
All looks good now and the node is removed from the cluster.
COPYRIGHT
© Shivananda Rao P, 2012 to 2018. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Shivananda Rao and http://www.shivanandarao-oracle.com with appropriate and specific direction to the original content.
DISCLAIMER
The views expressed here are my own and do not necessarily reflect the views of any other individual, business entity, or organisation. The views expressed by visitors on this blog are theirs solely and may not reflect mine
Leave a Reply