R12.2.10 Manual DB Clone between VMDBs

High Level Steps

1. Install OS Pre-requisites on CM, APP nodes(Target) 

2 Install required VMDB patches on Target DB OH 

3 Running Pre-Clone on the Source Apps Tier(s) and DB Tier 

4 Backup Full database and Archivelogs via RMAN on Source 

5 Tar Ball of Application Tier directories on Source and Copy to Target 

6 Pre DB restore steps on target  

7    Configure and start Oracle Instance on Target 

8 RMAN Duplicate & Restore and Recover database and open clone database on Target 

9 Run Post Clone Steps on Target Database Server ( DB Tier ) 

10 Run Post Clone Steps on Target Application Server ( Apps Tier ) 

11 Start EBS Clone Environment and Validate Single Web and DB node 

12 Multi-node Architecture Conversion 

13 Load Balancer Setup and Configuration 

14 Final Steps – Performance Best Practices and Validate the interface mount point and dba_directories 

  1. Install required VMDB patches on Target DB OH

Login to AM501AMERAP0005 and AM501AMERAP0006 as “root” user 

# yum update 

# yum install oracle-ebs-server-R12-preinstall 

Create a softlink as “root” user as below on Target CM/Web nodes – AM501AMERAP0005 and AM501AMERAP0006 

[root@AM501AMERAP0005 lib]#  cd /usr/lib 

[root@AM501AMERAP0005 lib]#  ln -s libXm.so.4.0.4 libXm.so.2 

[root@AM501AMERAP0005 lib]# ls -s libXm.so.4.0.4 libXm.so.2 

   0 libXm.so.2  2728 libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# ls -ltr libXm.so.2 

lrwxrwxrwx 1 root root 14 May 12 16:40 libXm.so.2 -> libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# 

2. Install required VMDB patches on Target DB OH

Downloaded below patches, followed readme to apply using “opatch apply”. NOTE: Apply these patches(matching Source DB Opatch lsinventory) before dropping the Shell DB that comes with OCI VMDB. Bring down the DB before applying these patches.  

22747454 

33568947 

33566041 

19239846 

21322448 

19472320 

27051384 

21321429 

24007737 

22731026 

22828765 

21967332 

21864513 

Startup the DB and Execute datapatch at the end. 

cd $ORACLE_HOME/Opatch 

datapatch 

Ensure Source and Target DB patches are matching.  

  • Error 1

On starting DB using sqlplus, it gave below error 

ORA-00210: cannot open the specified control file 

ORA-00202: control file: ‘+RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381’ 

ORA-17503: ksfdopn:2 Failed to open file +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 

ORA-15001: diskgroup “RECO” does not exist or is not mounted 

ORA-15040: diskgroup is incomplete 

ORA-205 signalled during: ALTER DATABASE   MOUNT… 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

Solution:  

[OCI] Oracle Database Startup Failed with ORA-00210/ ORA-00202 / ORA-15001/ ORA-15040 after Patching DB System OCI (Doc ID 2514240.1) 

[root@am501amerdb0002 bin]# ls -ltr oracle 

-rwsr-s–x 1 oracle oinstall 323514824 May  4 03:56 oracle 

[root@am501amerdb0002 bin]# chgrp asmadmin oracle 

[root@am501amerdb0002 bin]# chmod 6751 oracle 

[root@am501amerdb0002 bin]# ls -ltr oracle 

-rwsr-s–x 1 oracle asmadmin 323514824 May  4 03:56 oracle 

[root@am501amerdb0002 bin]# 

  • Error 2 

Restarted DB using sqlplus but faced below error 

Errors in file /u01/app/oracle/diag/rdbms/orcl_iad12w/ORCL/trace/ORCL_pmon_71054.trc: 

ORA-27140: attach to post/wait facility failed 

ORA-27300: OS system dependent operation:invalid_egid failed with status: 1 

ORA-27301: OS failure message: Operation not permitted 

ORA-27302: failure occurred at: skgpwinit6 

ORA-27303: additional information: startup egid = 1004 (asmadmin), current egid = 1001 (oinstall) 

USER (ospid: 71054): terminating the instance due to error 27140 

Wed May 04 03:56:49 2022 

Instance terminated by USER, pid = 71054 

Wed May 04 03:58:08 2022 

Starting ORACLE instance (normal) (OS id: 71847) 

Wed May 04 03:58:08 2022 

CLI notifier numLatches:13 maxDescs:1346 

Wed May 04 03:58:08 2022 

********************************************************************** 

Wed May 04 03:58:08 2022 

Dump of system resources acquired for SHARED GLOBAL AREA (SGA) 

Solution 

[root@am501amerdb0002 ~]# usermod -a -G asmadmin oracle 

[root@am501amerdb0002 ~]# sudo su – oracle 

Last login: Wed May  4 04:47:15 UTC 2022 

[oracle@am501amerdb0002 ~]$ id 

uid=101(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dbaoper),1003(dba),1004(asmadmin),1006(asmdba) 

[oracle@am501amerdb0002 ~]$ 

Target Shell DB restarted after this 

3. Running Pre-Clone on the Source Apps Tier(s) and DB Tier 

Login to am501amerdb0001 server 

cd $ORACLE_HOME/appsutil/scripts/PROD_am501amerdb0001 

perl adpreclone.pl dbTier 

Login to AM501AMERAP0001 server  

echo $FILE_EDITION 

run 

cd $INST_TOP/admin/scripts 

perl adpreclone.pl appsTier 

4. Backup Full database and Archivelogs via RMAN on Source 

[oracle@am501amerdb0001 OCI]$ pwd 

/home/oracle/OCI 

[oracle@am501amerdb0001 OCI]$ cat rman_r12clone_FULL.sh 

rman target / msglog=RMAN_FULL_rman_r12_uat_22July.log <<EOF 

SET ENCRYPTION ON; 

CONFIGURE BACKUP OPTIMIZATION OFF; 

run 

allocate channel c1 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c2 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c3 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c4 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c5 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c6 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c7 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c8 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

BACKUP AS COMPRESSED BACKUPSET FULL DATABASE FORCE TAG =’RMAN_FULL_UAT_July22′ PLUS ARCHIVELOG TAG =’RMAN_FULL_ARC_UAT_July22′; 

BACKUP CURRENT CONTROLFILE TAG =’RMAN_FULL_CNTRL_UAT_July22′; 

release channel c1; 

release channel c2; 

release channel c3; 

release channel c4; 

release channel c5; 

release channel c6; 

release channel c7; 

release channel c8; 

EOF 

[oracle@am501amerdb0001 OCI]$ 

Execute backup 

============= 

nohup ./rman_r12clone_FULL.sh & 

5. Trigger tar ball of application binaries in the background 

As root user on Source Admin Server(AM501AMERAP0001) 

mkdir /BACKUPS/R12Clone 

chmod 777 /BACKUPS/R12Clone 

cd /BACKUPS/R12Clone 

nohup gtar -czvf runfilesystem_fs1_ebs.gtar.gz /u01/install/APPS/fs1/EBSapps & 

jobs 

[root@AM501AMERAP0001 R12Clone]# jobs 

[1]+  Running                 nohup gtar -czvf runfilesystem_fs1_ebs.gtar.gz /u01/install/APPS/fs1/EBSapps & 

[root@AM501AMERAP0001 R12Clone]# pwd 

/BACKUPS/R12Clone 

[root@AM501AMERAP0001 R12Clone]# 

6. Pre DB restore steps on target  

Delete the Target Database created by Cloud Tooling (I took a cold RMAN backup before deleting it) 

sqlplus / as sysdba 

select * from v$encryption_wallet; — Capture the location of TDE encryption/auto login keys and take a backup of the same.  

set heading off linesize 999 pagesize 0 feedback off trimspool on 

spool /tmp/remove_files.sh 

select ‘asmcmd rm ‘||name from v$datafile 

union all 

select ‘asmcmd rm ‘||name from v$tempfile 

union all 

select ‘asmcmd rm ‘||member from v$logfile; 

spool off 

chmod 777 /tmp/remove_files.sh 

Shutdown the DB using sqlplus  

sudo su – grid 

. oraenv 

+ASM1 

sh /tmp/remove_files.sh (Validate the file for any syntax errors before executing) 

Get the Source Database ID 

SQL> select dbid from v$database; 

        

DBID 

—————— 

1624776053 

7. Configure and start Oracle Instance on Target 

  1. Get the Source Database ID 

SQL> select dbid from v$database; 

        

DBID 

—————— 

1624776053 

  1. Copy the RMAN Configuration File(used in backup script i.e /home/oracle/OCI/config) from Source to Target  /home/oracle/OCI/object_storage/ 

This file contains the information about the Object Storage endpoint, wallet location where the Object Storage credentials are stored, and the Object Storage bucket name: 

  1. Copy the Wallet containing the Object Storage Credentials(location mentioned in above file) from am501amerdb0001 to am501amerdb0002 (/home/oracle/OCI/object_storage/) 
  1. Copy the library for the Backup Module(/home/oracle/OCI/lib/libopc.so) from am501amerdb0001 to am501amerdb0002 
  1. Copy the TDE wallet containing the TDE master encryption keys and autologin keys from am501amerdb0001 to am501amerdb0002(location indicated in target v$encryption_wallet before dropping the DB). Ensure we have original keys backups on target before copying from Source so they don’t get overwritten. 

sqlplus “/as sysdba” 

select * from v$encryption_wallet;  

  1. Check the Copied Files 

The RMAN configuration file, wallet, and library for the Backup Module: 

[oracle@am501amerdb0002 ~]$ cd object_storage 

[oracle@am501amerdb0002 object_storage]$ ls -ltr /home/oracle/object_storage/ 

total 91240 

-rw——- 1 oracle oinstall     1621 May  5 18:57 cwallet.sso 

-rw-r–r– 1 oracle oinstall 93414592 May  5 19:04 libopc.so 

-rw-r–r– 1 oracle oinstall      189 May  5 19:27 configam501amerdb0001 

-rw-r–r– 1 oracle oinstall      193 May  5 20:09 config 

[oracle@am501amerdb0002 object_storage]$ pwd 

/home/oracle/object_storage 

[oracle@am501amerdb0002 object_storage]$ 

The TDE password and auto-login wallets: 

[oracle@am501amerdb0002 ORCL_iad18r]$ ls -ltr /opt/oracle/dcs/commonstore/wallets/tde/ORCL_iad18r/ 

total 24 

-rw——- 1 oracle oinstall 10475 May 10 18:37 ewallet.p12 

-rw——- 1 oracle oinstall 10520 May 10 18:37 cwallet.sso 

[oracle@am501amerdb0002 ORCL_iad18r]$ 

  1. Start the DB in “No Mount” mode 

SQL>startup nomount; 

SQL>select * from v$encryption_wallet;  – should be open and autologin 

8. RMAN Duplicate & Restore and Recover database and open clone database on Target 

  •   Start DB in nomount by commenting recovery_file_dest & recovery_file_size 

Create pfile from spfile; 

Comment recovery_file_dest & recovery_file_size 

start DB with pfile 

  • Restore the Control File 

         rman target / 

         RMAN> set dbid 1624776053; 

         RMAN> 

         run  

        { 

        allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so,SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

        restore controlfile from autobackup; 

        } 

  • Mount the database. 

        SQL> alter database mount; 

  • Create set new name script on Source DB to use in restore script 

set head off pages 0 feed off echo off verify off 

set lines 200 

spool /tmp/rename_datafiles_CB.lst 

select ‘set newname for datafile ‘ || file# || ‘ to ”+DATA”;’ 

from V$datafile ; 

spool off 

exit; 

  • Restore the DB 

[oracle@am501amerdb0002 OCI]$ pwd 

/home/oracle/OCI 

[oracle@am501amerdb0002 OCI]$ cat RMAN_RESTORE_TEST.sh 

rman target / msglog=rman_restore_test.log <<EOF 

CONFIGURE BACKUP OPTIMIZATION OFF;  

run 

allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c2 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c3 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c4 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c5 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c6 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c7 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c8 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

set newname for database to ‘+DATA’;  

restore database; 

EOF 

[oracle@am501amerdb0002 OCI]$ 

  • Switch all datafiles 

[oracle@am501amerdb0002 OCI]$ rman target / 

Recovery Manager: Release 12.1.0.2.0 – Production on Fri Jul 22 12:18:26 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

connected to target database: ORCL (DBID=1624776053, not open) 

RMAN> switch database to copy; 

  • Recover the database 

                 RMAN_TEST_RECOVER.sh 

                 rman target / msglog=rman_TEST_recover_22JULY22.log <<EOF 

                 CONFIGURE BACKUP OPTIMIZATION OFF; 

                 run 

                 { 

                 allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

                 recover database; 

                 } 

EOF 

[oracle@am501amerdb0002 OCI]$ tail -40f rman_TEST_recover_22JULY22.log 

Recovery Manager: Release 12.1.0.2.0 – Production on Fri Jul 22 12:25:55 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

connected to target database: ORCL (DBID=1624776053, not open) 

RMAN> 

using target database control file instead of recovery catalog 

old RMAN configuration parameters: 

CONFIGURE BACKUP OPTIMIZATION OFF; 

new RMAN configuration parameters: 

CONFIGURE BACKUP OPTIMIZATION OFF; 

new RMAN configuration parameters are successfully stored 

RMAN> 2> 3> 4> 5> 

allocated channel: c1 

channel c1: SID=4 device type=SBT_TAPE 

channel c1: Oracle Database Backup Service Library VER=19.0.0.1 

Starting recover at 22-JUL-22 

starting media recovery 

channel c1: starting archived log restore to default destination 

channel c1: restoring archived log 

archived log thread=1 sequence=3106 

channel c1: reading from backup piece Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013 

channel c1: ORA-19870: error while restoring backup piece Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013 

ORA-19507: failed to retrieve sequential file, handle=”Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013″, parms=”” 

ORA-27029: skgfrtrv: sbtrestore returned error 

ORA-19511: non RMAN, but media manager or vendor specific failure, error text: 

   KBHS-07502: File not found 

KBHS-01404: See trace file /u01/app/oracle/produc 

failover to previous backup 

released channel: c1 

RMAN-00571: =========================================================== 

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== 

RMAN-00571: =========================================================== 

RMAN-03002: failure of recover command at 07/22/2022 12:26:52 

RMAN-20506: no backup of archived log found 

RMAN-06053: unable to perform media recovery because of missing log 

RMAN-06025: no backup of archived log for thread 1 with sequence 3106 and starting SCN of 6011136313975 found to restore 

RMAN> 

Incase of above error, perform manual recovery by copying the requested logs from am501amerdb0001 to am501amerdb0002. Please use RMAN to copy archive logs from ASM to local filesystem and scp the archive files to target.  

  • Restart the database 

SQL> select name,open_mode from v$database; 

NAME      OPEN_MODE 

——— ——————– 

ORCL      MOUNTED 

  • Manually copy all required archive log files from source to target 

[oracle@am501amerdb0002 ~]$ sqlplus ‘/as sysdba’ 

SQL*Plus: Release 12.1.0.2.0 Production on Fri Jul 22 13:30:50 2022 

Copyright (c) 1982, 2014, Oracle.  All rights reserved. 

Connected to: 

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 – 64bit Production 

With the Automatic Storage Management and Real Application Testing options 

SQL>  recover database until cancel using backup controlfile; 

ORA-00279: change 6011136314031 generated at 07/22/2022 08:14:02 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-00280: change 6011136314031 for thread 1 is in sequence #3106 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

ORA-00308: cannot open archived log 

‘+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709’ 

ORA-17503: ksfdopn:2 Failed to open file 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-15012: ASM file 

‘+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709’ 

does not exist 

ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below 

ORA-01152: file 1 was not restored from a sufficiently old backup 

ORA-01110: data file 1: ‘+DATA/ORCL_IAD12W/DATAFILE/system.271.1110732659’ 

SQL> recover database until cancel using backup controlfile; 

ORA-00279: change 6011136314031 generated at 07/22/2022 08:14:02 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-00280: change 6011136314031 for thread 1 is in sequence #3106 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3106.1838.1110729709 

ORA-00279: change 6011136438482 generated at 07/22/2022 16:01:48 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3107.1837.1110730643 

ORA-00280: change 6011136438482 for thread 1 is in sequence #3107 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3106.1838.1110729709’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3107.1837.1110730643 

ORA-00279: change 6011136480729 generated at 07/22/2022 16:17:23 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136480729 for thread 1 is in sequence #3108 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3107.1837.1110730643’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3108.1836.1110733273 

ORA-00279: change 6011136769586 generated at 07/22/2022 17:01:12 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136769586 for thread 1 is in sequence #3109 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3108.1836.1110733273’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3109.1835.1110736887 

ORA-00279: change 6011136961194 generated at 07/22/2022 18:01:26 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136961194 for thread 1 is in sequence #3110 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3109.1835.1110736887’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3110.1834.1110740497 

ORA-00279: change 6011137109018 generated at 07/22/2022 19:01:36 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011137109018 for thread 1 is in sequence #3111 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3110.1834.1110740497’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3111.1833.1110744077 

ORA-00279: change 6011137258778 generated at 07/22/2022 20:01:16 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011137258778 for thread 1 is in sequence #3112 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3111.1833.1110744077’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

CANCEL 

Media recovery cancelled. 

SQL> alter database open resetlogs; 

Database altered. 

SQL> show pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           READ WRITE NO 

SQL> exit  

9. Run Post Clone Steps on Target Database Server ( DB Tier ) 

  • Change DBID of the Target DB 

Shut immediate 

Startup mount 

[oracle@am501amerdb0002 ~]$ nid TARGET=SYS 

DBNEWID: Release 12.1.0.2.0 – Production on Fri Jul 22 13:41:30 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

Password: 

Connected to database ORCL (DBID=1624776053) 

Connected to server version 12.1.0 

Control Files in database: 

    +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 

Change database ID of database ORCL? (Y/[N]) => Y 

Proceeding with operation 

Changing database ID from 1624776053 to 1638538147 

    Control File +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 – modified 

    — 

    — 

    Datafile +DATA/ORCL_IAD12W/6570B3097CD64775E053010A5A0AFBA9/TEMPFILE/temp1.580.111074610 – dbid changed 

    Control File +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 – dbid changed 

    Instance shut down 

Database ID for database ORCL changed to 1638538147. 

All previous backups and archived redo logs for this database are unusable. 

Database is not aware of previous backups and archived logs in Recovery Area. 

Database has been shutdown, open database with RESETLOGS option. 

Succesfully changed database ID. 

DBNEWID – Completed succesfully. 

[oracle@am501amerdb0002 ~]$ 

  • Restart the DB and validate the DBID 

sqlplus “/as sysdba” 

SQL> startup mount; 

SQL> alter database open resetlogs; 

SQL> sho pdbs 

SQL> select dbid from v$database; 

  • Rename the pluggable DB name – Rename a Pluggable Database In Oracle 12c (Doc ID 2439885.1) 

sqlplus “/as sysdba” 

Connect to the CDB and check the information about the PDB first: 

SQL>select name, open_mode, restricted from v$pdbs; 

SQL>select name, con_id, dbid,con_uid,guid from v$containers order by con_id; 

SQL>select service_id,name,network_name,creation_date,pdb,con_id from cdb_services; 

SQL> sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           READ WRITE NO 

SQL> 

SQL> alter pluggable database PROD close; 

SQL>  sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           MOUNTED 

SQL> alter pluggable database PROD open restricted; 

SQL> select name, open_mode, restricted from v$pdbs; 

NAME                           OPEN_MODE  RES 

—————————— ———- — 

PDB$SEED                       READ ONLY  NO 

PROD                           READ WRITE YES 

SQL> alter session set container=PROD; 

SQL> alter pluggable database rename global_name to TEST; 

SQL> sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         3 TEST                                          READ WRITE YES 

SQL> alter pluggable database close immediate; 

SQL> alter pluggable database open; 

Connect to the CDB and check the information again: 

SQL> alter session set container=CDB$ROOT; 

SQL> select name, open_mode, restricted from v$pdbs; 

SQL>select name, con_id, dbid,con_uid,guid from v$containers order by con_id; 
SQL>select service_id,name,network_name,creation_date,pdb,con_id from cdb_services; 

  • Copy tnsnames.ora and listener.ora from Source $TNS_ADMIN to Target $TNS_ADMIN. Make changes of “PROD” to “TEST” and “am501amerdb0001” to “am501amerdb0002” 
  • Create a pfile out of spfile from PROD and scp to target. Make appropriate directory changes as per target.  

Ensure below changes are in place to avoid TNS issues 

db_domain as null  

sec_case_sensitive_logon as FALSE  

Start DB with the new pfile from PROD.  

Create spfile 

Restart the DB 

Restart the listener 

Check “test” is mentioned in listener services. 

[oracle@am501amerdb0002 ~]$ lsnrctl status 

LSNRCTL for Linux: Version 12.1.0.2.0 – Production on 17-MAY-2022 22:17:40 

Copyright (c) 1991, 2014, Oracle.  All rights reserved. 

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521)) 

STATUS of the LISTENER 

———————— 

Alias                     LISTENER 

Version                   TNSLSNR for Linux: Version 19.0.0.0.0 – Production 

Start Date                10-MAY-2022 15:44:32 

Uptime                    7 days 6 hr. 33 min. 8 sec 

Trace Level               off 

Security                  ON: Local OS Authentication 

SNMP                      OFF 

Listener Parameter File   /u01/app/19.0.0.0/grid/network/admin/listener.ora 

Listener Log File         /u01/app/grid/diag/tnslsnr/am501amerdb0002/listener/alert/log.xml 

Listening Endpoints Summary… 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.215.254.53)(PORT=1521))) 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=am501amerdb0002.ad.global)(PORT=5500))(Security=(my_wallet_directory=/u01/app/oracle/product/12.1.0.2/dbhome_1/admin/ORCL_iad12w/xdb_wallet))(Presentation=HTTP)(Session=RAW)) 

Services Summary… 

Service “+APX” has 1 instance(s). 

  Instance “+APX1”, status READY, has 1 handler(s) for this service… 

Service “+ASM” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “+ASM_DATA” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “+ASM_RECO” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “ORCLXDB” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

Service “ORCL_iad12w” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

Service “test” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

The command completed successfully 

[oracle@am501amerdb0002 ~]$ 

  • Create a appsutil zip from Source Admin Server(AM501AMERAP0001) under $AD_TOP/bin(run filesystem) 

[applmgr@AM501AMERAP0001 bin]$ perl admkappsutil.pl 

Starting the generation of appsutil.zip 

Log file located at /u01/install/APPS/fs1/inst/apps/PROD_am501amerap0001/admin/log/MakeAppsUtil_05092211.log 

output located at /u01/install/APPS/fs1/inst/apps/PROD_am501amerap0001/admin/out/appsutil.zip 

MakeAppsUtil completed successfully. 

[applmgr@AM501AMERAP0001 bin]$ 

[applmgr@AM501AMERAP0001 bin]$ 

  • Execute below scripts as “APPS” user 

                sqlplus apps@test 

                SQL> exec fnd_conc_clone.setup_clean; 

                SQL> exec ad_zd_fixer.clear_valid_nodes_info; 

  •    Copy the DB context from Source OH/appsutil to Target OH/appsutil and make necessary changes  
  • Create nls/data/9idata directory 

                On the database server node, as the owner of the Oracle RDBMS file system and database instance, run the following command to create the $ORACLE_HOME/nls/data/9idata directory. 

$ perl $ORACLE_HOME/nls/data/old/cr9idata.pl 

After creating the directory, make sure that the ORA_NLS10 environment variable is set to the full path of the 9idata directory whenever you enable the 12c Oracle home. 

[oracle@am501amerdb0002 old]$ ls -ltr cr9idata.pl 

-rw-r–r– 1 oracle oinstall 11785 Oct 12  2021 cr9idata.pl 

[oracle@am501amerdb0002 old]$ perl cr9idata.pl 

Creating directory /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata … 

Copying files to /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata… 

Copy finished. 

Please reset environment variable ORA_NLS10 to /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata! 

[oracle@am501amerdb0002 old]$ 

  • Set the env file and execute autoconfig 

cat ORCL.env 

export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 

export ORACLE_SID=ORCL 

export ORA_NLS10=/u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata 

. ORCL.env 

cd $ORACLE_HOME/appsutil/bin 

sh adconfig.sh 

  • Update the Site Name 

sqlplus apps@test 

update fnd_profile_option_values set profile_option_value = ‘R12 SIT’ where profile_option_id=125; 

commit; 

Decide if you have to cancel or suspend any source concurrent jobs or WF setups. I did not do it as we were cloning CRP for SIT

Apps Cloning Steps continues in the next blog https://orama.blog/2022/09/04/r12210-apps-clone-between-oci-compute-nodes/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s