Blog

Multinode Conversion and LB Configuration on EBS 12.2.10 + 12c DB

  1. Backup the $CONTEXT_FILE & perform following, 

  

The value of the variable s_atName is defaulted to the hostname of the primary application tier node.  

In addition to setting the s_atName to the hostname of the primary application tier, the value of the applications context variable “ s_shared_file_system” also need to be set to true.  

  

      

S_atName= AT_am501amerap0001 

s_shared_file_system=true 

  1. Autoconfig 

Run autoconfig post $CONTEXT_FILE changes 

  1. Source Patch_fs 

./EBSapps.ev patch 

  1. Patch FS 

Update the Patch $CONTEXT_FILE  

  1. Update the DB 

[applmgr@AM501AMERAP0005 ~]$ $ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer action=upload contextfile=/u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml logfile=/tmp/patchctxupload.log 

                ————————————————————————————————————————– 

                [applmgr@AM501AMERAP0005 ~]$ cat /tmp/patchctxupload.log 

                   ADX Database Utility 

getConnectionUsingAppsJDBCConnector() à 

    APPS_JDBC_URL=’jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=am501amerdb0002.ad.global)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=TEST)))’ 

    Trying to connect using APPS_JDBC_URL 

    Connection obtained. 

——————-ADX Database Utility Finished————— 

                   ADX Database Utility 

getConnectionUsingAppsJDBCConnector() à 

    APPS_JDBC_URL=’jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=am501amerdb0002.ad.global)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=TEST)))’ 

    Trying to connect using APPS_JDBC_URL 

    Connection obtained. 

——————-ADX Database Utility Finished————— 

[applmgr@AM501AMERAP0005 ~]$ 

  1. Run fs_clone on “run” file system 

adop phase=fs_clone 

adop -status 

  1. Start admin server on the run_fs 

$INST_TOP/admin/scripts/adadminsrvctl.sh start 

  1. Start the adminserver on the patch_fs 

$INST_TOP/admin/scripts/adadminsrvctl.sh start forcepatchfs 

  1. Preclone on RUN FS 

$INST_TOP/admin/scripts/adpreclone.pl appsTier 

  1. Preclone on PATCH FS 

$INST_TOP/admin/scripts/adpreclone.pl appsTier 

  1. Stop firewalld on AM501AMERAP0005 and AM501AMERAP0006 

sudo systemctl status firewalld 

sudo systemctl stop firewalld 

  1. Adding Connection Filter Rule for a new Application tier node – Run Filesystem 

java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

[applmgr@AM501AMERAP0005 ~]$ . EBSapps.env 

  E-Business Suite Environment Information 

  —————————————- 

  RUN File System           : /u01/install/APPS/fs1/EBSapps/appl 

  PATCH File System         : /u01/install/APPS/fs2/EBSapps/appl 

  Non-Editioned File System : /u01/install/APPS/fs_ne 

  DB Host: am501amerdb0002.ad.global  Service/SID: TEST 

  E-Business Suite Environment Setting 

  ———————————— 

  – Enter [R/r] for sourcing Run File System Environment file, or 

  – Enter [P/p] for sourcing Patch File System Environment file, or 

  – Enter anything else to exit 

  Please choose the environment file you wish to source [R/P]:R 

  Sourcing the RUN File System … 

[applmgr@AM501AMERAP0005 ~]$ pwd 

/home/applmgr 

[applmgr@AM501AMERAP0005 ~]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

For details please refer log file /home/applmgr/RegisterNodePreReq.log 

Enter the WebLogic Server Admin Password: 

add-filter-rule option executed successfully. 

[applmgr@AM501AMERAP0005 ~]$ cat /home/applmgr/RegisterNodePreReq.log 

Validated the passed arguments for the option add-filter-rule 

Validating hostname :AM501AMERAP0006.ad.global 

Validating the AM501AMERAP0006.ad.global :true 

Existing Connection Filter rules: 

[am501amerap0005.ad.global * * allow, 0.0.0.0/0 * * deny] 

Updating Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

[applmgr@AM501AMERAP0005 ~]$ 

java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

[applmgr@AM501AMERAP0005 Heema]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global^C 

[applmgr@AM501AMERAP0005 Heema]$ ls -ltr /u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar 

-rwxr-x— 1 applmgr oinstall 36339849 May 11 18:15 /u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar 

[applmgr@AM501AMERAP0005 Heema]$ ls -ltr /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

-rw-r–r–1 applmgr oinstall 121325 May 12 17:48 /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

[applmgr@AM501AMERAP0005 Heema]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

For details please refer log file /home/applmgr/Heema/RegisterNodePreReq.log 

Enter the WebLogic Server Admin Password: 

add-filter-rule option executed successfully. 

[applmgr@AM501AMERAP0005 Heema]$ cat /home/applmgr/Heema/RegisterNodePreReq.log 

Validated the passed arguments for the option add-filter-rule 

Validating hostname :am501amerap0006.ad.global 

Validating the am501amerap0006.ad.global :true 

Existing Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

Updating Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, am501amerap0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

[applmgr@AM501AMERAP0005 Heema]$ 

  1. Adding Connection Filter Rule for a new Application tier node – Patch Filesystem 

. EBSenv Patch 

java -cp $CLASSPATH:/u01/install/APPS/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

java -cp $CLASSPATH:/u01/install/APPS/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

  1. Enable SSH connectivity 

Enable SSH connectivity between the nodes 

                Login as applmgr 

                 cd $HOME 

cd .ssh 

ssh-keygen -t rsa on each node and take defaults 

Login to AM501AMERAP0005: cp id_rsa.pub id_rsa.pub_cm 

Login to AM501AMERAP0006: cp id_rsa.pub id_rsa.pub_app 

scp id_rsa.pub_cm from AM501AMERAP0005 to AM501AMERAP0006 

scp id_rsa.pub_app from AM501AMERAP006 to AM501AMERAP0005 

mv authorized_keys authorized_keys_bkp 

for the AM501AMERAP0005/ AM501AMERAP0006 

cat id_rsa.pub>authorized_keys 

 

scp id_rsa.pub_cm from AM501AMERAP0005 to AM501AMERAP0006 

scp id_rsa.pub_app from AM501AMERAP006 to AM501AMERAP0005 

On AM501AMERAP0005 

cat id_rsa.pub_app>>authorized_keys 

On AM501AMERAP0006 

cat id_rsa.pub_cm>> authorized_keys 

 

Validate the ssh connectivity 

On AM501AMERAP0005 

ssh AM501AMERAP0005 – Self ssh 

ssh AM501AMERAP0006  

On AM501AMERAP0006 

ssh AM501AMERAP0006  – Self ssh  

ssh AM501AMERAP0005 

  1. Prepare the pair file 

/BACKUPS/TEST/TEST_am501amerap0005_run.txt 

  1. Set the env on the new node 

On the app node, perform the following. 

  

[applmgr@AM501AMERAP0006 ~]$ export PATH=/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin:$PATH 

[applmgr@AM501AMERAP0006 ~]$ cd /u01/install/APPS/fs1/EBSapps/comn/clone/bin 

[oracle@am501amerdb0002 ETCC]$ sh checkDBpatch.sh 

[applmgr@AM501AMERAP0005 ETCC]$ sh checkMTpatch.sh 

  1. Add node 

/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin/perl ./adclonectx.pl addnode contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml pairsfile=/BACKUPS/TEST/TEST_am501amerap0005_run.txt dualfs=yes 

[applmgr@AM501AMERAP0006 ~]$ export PATH=/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin:$PATH 

[applmgr@AM501AMERAP0006 ~]$ cd /u01/install/APPS/fs1/EBSapps/comn/clone/bin 

[applmgr@AM501AMERAP0006 bin]$ /u01/install/APPS/fs1/FMW_Home/webtier/perl/bin/perl ./adclonectx.pl addnode contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml pairsfile=/BACKUPS/TEST/TEST_am501amerap0005_run.txt dualfs=yes 

                     Copyright I 2011, 2015 Oracle Corporation 

                        Redwood Shores, California, USA 

                        Oracle E-Business Suite Rapid Clone 

                                 Version 12.2 

                      adclonectx Version 120.30.12020000.22 

Enter the APPS passwd : 

Enter the Weblogic AdminServer password : 

 Executing command: /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jre/bin/java -Xmx600M -Doracle.jdbc.autoCommitSpecCompliant=false -classpath /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/ojdbc6.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/xmlparserv2.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/java::/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wlfullclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wlclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wljmxclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/ojmisc.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/obfuscatepassword.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/emCfg.jar oracle.apps.ad.clone.RCloneSFSAddNode -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -configfile /BACKUPS/TEST/TEST_am501amerap0005_run.txt -machinelistenaddress am501amerap0006.ad.global -logdir /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog -promptmsg hide 

Verifying: Apps schema password 

Verifying: 

        pairsfile /BACKUPS/TEST/TEST_am501amerap0005_run.txt 

        Run file system AdminServer is running 

        Patch file system AdminServer is running 

        Run file system domain 

        Patch file system domain 

Executing: run clone context… 

Log file located at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog/run/RCloneSFSAddNode_05131337.log 

Target System Base Directory set to /u01/install/APPS 

Target System Current File System Base set to /u01/install/APPS/fs1 

Target System Other File System Base set to /u01/install/APPS/fs2 

Target System Fusion Middleware Home set to /u01/install/APPS/fs1/FMW_Home 

Target System Other File System Fusion Middleware Home set to /u01/install/APPS/fs2/FMW_Home 

Target System Web Oracle Home set to /u01/install/APPS/fs1/FMW_Home/webtier 

Target System Other File System Web Oracle Home set to /u01/install/APPS/fs2/FMW_Home/webtier 

Target System Appl TOP set to /u01/install/APPS/fs1/EBSapps/appl 

Target System Other File System Appl TOP set to /u01/install/APPS/fs2/EBSapps/appl 

Target System COMMON TOP set to /u01/install/APPS/fs1/EBSapps/comn 

Target System Other File System COMMON TOP set to /u01/install/APPS/fs2/EBSapps/comn 

Target System Current File System Instance Top set to /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006 

Report file located at /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/admin/out/portpool.lst 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

Configuring: run fs add node… 

Executing: patch clone context… 

Log file located at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog/patch/RCloneSFSAddNode_05131344.log 

Target System Other File System Instance Top set to /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006 

Report file located at /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/admin/out/portpool.lst 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

contextfile=/u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

Configuring: patch fs add node… 

— Add node operation completed – 

Node am501amerap0006.ad.global has been added successfully 

[applmgr@AM501AMERAP0006 bin]$ 

  1. Run Autoconfig in the following order only  5-6-5 (CM-WEB-CM)  

19. Start EBS Clone Environment and Validate Multi-Node EBS (One Node has “CM” and One Node has “Web”) and DB node. Start the EBS services from “run” filesystem and validate the front-end forms.

http://am501amerap0006.ad.global:8000/OA_HTML/AppsLogin 

 

R12210 Apps Clone between OCI Compute Nodes

  1. Install OS Pre-requisites on CM, APP nodes(Target)

Login to AM501AMERAP0005 and AM501AMERAP0006 as “root” user 

# yum update 

# yum install oracle-ebs-server-R12-preinstall 

Create a softlink as “root” user as below on Target CM/Web nodes – AM501AMERAP0005 and AM501AMERAP0006 

[root@AM501AMERAP0005 lib]#  cd /usr/lib 

[root@AM501AMERAP0005 lib]#  ln -s libXm.so.4.0.4 libXm.so.2 

[root@AM501AMERAP0005 lib]# ls -s libXm.so.4.0.4 libXm.so.2 

   0 libXm.so.2  2728 libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# ls -ltr libXm.so.2 

lrwxrwxrwx 1 root root 14 May 12 16:40 libXm.so.2 -> libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# 

2. Running Pre-Clone on the Source Apps Tier(s) and DB Tier 

This was already done in my DB manual clone blog. Just need to validate if it is done.

3. Target DB must be already cloned along with Post DB refresh steps as in my DB manual clone bloghttps://wordpress.com/post/orama.blog/1766

4. Run Post Clone Steps on Target Application Server (Apps Tier ) https://wordpress.com/post/orama.blog/1766

5. Run Post Clone Steps on Target Application Server (Apps Tier ) 

cd /u01 

tar -xvzf /BACKUPS/R12Clone/runfilesystem_fs1_ebs.gtar.gz 

It should create directory like /u01/install/APPS/fs1/EBSapps 

  • Create below directories(if not in place) on AM501AMERAP0005 and match the ownership/permission from Source Admin Server(AM501AMERAP0001) 

/u01/app/oraInventory 

/u01/install/APPS/fs1 

/u01/install/APPS/fs2 

/u01/install/APPS/fs_ne 

  • Execute adcfgclone.pl appsTier dualfs 

echo $FILE_EDITION 

run  

cd $COMMON_TOP/clone/bin/ 

perl adcfgclone.pl appsTier dualfs 

Provide the values required for creation of the new APPL_TOP Context file. 

Target System Hostname (virtual or normal) [am501amerap0005] : 

Target System Database SID : TEST 

Target System Database Server Node [am501amerap0005] : am501amerdb0002 

Target System Database Domain Name [ad.global] : 

Target System Base Directory : /u01/install/APPS 

Target System Base Directory set to /u01/install/APPS 

Target System Current File System Base set to /u01/install/APPS/fs1 

Target System Other File System Base set to /u01/install/APPS/fs2 

Target System Fusion Middleware Home set to /u01/install/APPS/fs1/FMW_Home 

Target System Other File System Fusion Middleware Home set to /u01/install/APPS/fs2/FMW_Home 

Target System Web Oracle Home set to /u01/install/APPS/fs1/FMW_Home/webtier 

Target System Other File System Web Oracle Home set to /u01/install/APPS/fs2/FMW_Home/webtier 

Target System Appl TOP set to /u01/install/APPS/fs1/EBSapps/appl 

Target System Other File System Appl TOP set to /u01/install/APPS/fs2/EBSapps/appl 

Target System COMMON TOP set to /u01/install/APPS/fs1/EBSapps/comn 

Target System Other File System COMMON TOP set to /u01/install/APPS/fs2/EBSapps/comn 

Target System Instance Home Directory [/u01/install/APPS] : 

Target System Current File System Instance Top set to /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005 

Do you want to preserve the Display [am501amerap0001:0.0] (y/n)  : y 

Target System Root Service [enabled] : 

Target System Web Entry Point Services [disabled] : enabled 

Target System Web Application Services [disabled] : enabled 

Target System Batch Processing Services [enabled] : enabled 

Target System Other Services [disabled] : 

Do you want the target system to have the same port values as the source system (y/n) [y] ? : 

Validating if the source port numbers are available on the target system.. 

Complete port information available at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/out/TEST_am501amerap0005/portpool.lst 

Target System proxy port [80] : 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

Check Clone Context logfile /u01/install/APPS/fs1/EBSapps/comn/clone/bin/CloneContext_0511174230.log for details. 

Creating Patch file system context file….. 

Log file located at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/CloneContextPatch_0511180352.log 

Target System Other File System Instance Top set to /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005 

Validating if the source port numbers are available on the target system.. 

Complete port information available at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/out/TEST_am501amerap0005/portpool.lst 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

Check Clone Context logfile /u01/install/APPS/fs1/EBSapps/comn/clone/bin/CloneContextPatch_0511180352.log for details. 

FMW Pre-requisite check log file location : /u01/install/APPS/fs1/EBSapps/comn/clone/FMW/logs/prereqcheck.log 

Running: FMW pre-req check… 

Configuring: Run file system…. 

LogFile located at /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/admin/log/clone/run/RCloneApplyAppstier_05111804.log 

<May 11, 2022 6:45:47 PM GMT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.> 

<May 11, 2022 6:54:55 PM GMT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.> 

Configuring: Patch file system…. 

LogFile located at /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/admin/log/clone/patch/RCloneApplyAppstier_05111854.log 

<May 11, 2022 8:26:26 PM GMT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread tha                                                                                  n the one in which it was created.> 

Do you want to startup the Application Services for TEST? (y/n) [n] : n  

  • Shutdown EBS services and make below changes highlighted in YELLLOW to “RUN” filesystem context file. Ensure you take a backup before making changes.  Execute autoconfig on run filesystem after the changes.  

[applmgr@AM501AMERAP0005 ~]$. EBSapps.env run 

[applmgr@AM501AMERAP0005 ~]$ diff /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml05122022 

148c148 

<          <EndUserMonitoringURL oa_var=”s_endUserMonitoringURL”>http://am501amerap0005.ad.global:8000/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL> 

>          <EndUserMonitoringURL oa_var=”s_endUserMonitoringURL”>http://am501amerap0005.ad.global:4443/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL> 

199c199 

<          <externURL oa_var=”s_external_url”>http://am501amerap0005.ad.global:8000</externURL> 

>          <externURL oa_var=”s_external_url”>http://am501amerap0005.ad.global:4443</externURL> 

236c236 

<          <login_page oa_var=”s_login_page”>http://am501amerap0005.ad.global:8000/OA_HTML/AppsLogin</login_page> 

>          <login_page oa_var=”s_login_page”>http://am501amerap0005.ad.global:4443/OA_HTML/AppsLogin</login_page> 

815c815 

<       <activewebport oa_var=”s_active_webport” oa_type=”DUP_PORT” base=”8000” step=”1” range=”-1” label=”Active Web Port”>8000</activewebport> 

>       <activewebport oa_var=”s_active_webport” oa_type=”DUP_PORT” base=”8000” step=”1” range=”-1” label=”Active Web Port”>4443</activewebport> 

[applmgr@AM501AMERAP0005 ~]$ 

[applmgr@AM501AMERAP0005 admin]$ cat $CONTEXT_FILE |egrep ‘s_opmnstatus|s_apcstatus|s_oacorestatus|s_formsstatus|s_oafmstatus|s_web_entry_status|s_web_applications_status’ 

<oa_service_status oa_var=”s_opmnstatus”>enabled</oa_service_status> 

<oa_service_status oa_var=”s_apcstatus”>enabled</oa_service_status> 

<oa_service_status oa_var=”s_oacorestatus”>enabled</oa_service_status> 

<oa_service_status oa_var=”s_formsstatus”>enabled</oa_service_status> 

<oa_service_status oa_var=”s_oafmstatus”>enabled</oa_service_status> 

<oa_service_group_status oa_var=”s_web_entry_status”>enabled</oa_service_group_status> 

<oa_service_group_status oa_var=”s_web_applications_status”>enabled</oa_service_group_status> 

[applmgr@AM501AMERAP0005 admin]$ cd $ADMIN_SCRIPTS_HOME 
[applmgr@AM501AMERAP0005 admin]$./adautocfg.sh 

  • Ensure EBS services are down and make above changes highlighted in YELLLOW to “PATCH” filesystem context file. Ensure you take a backup before making changes 

Connect to sqlplus using system user. 

. EBSapps.env run 
sqlplus system/manager 
alter trigger ebs_logon disable; 
 
 

. EBSapps.env patch 

Source Patch file system  
cd $ADMIN_SCRIPTS_HOME 
./adautocfg.sh  

. EBSapps.env run 

sqlplus system/manager 
alter trigger ebs_logon enable; 

6. Start EBS Clone Environment and Validate Single node EBS and DB node 

Start the EBS services from “run” filesystem and validate the front-end forms. “SYSADMIN” password is same as Source 

http://am501amerap0005.ad.global:8000/OA_HTML/AppsLogin 

Shutdown EBS services after the validation 

7. Multi-node Architecture Conversion 

  1. Backup the $CONTEXT_FILE & perform following, 

  

The value of the variable s_atName is defaulted to the hostname of the primary application tier node.  

In addition to setting the s_atName to the hostname of the primary application tier, the value of the applications context variable “ s_shared_file_system” also need to be set to true.  

  

      

S_atName= AT_am501amerap0001 

s_shared_file_system=true 

  1. Autoconfig 

Run autoconfig post $CONTEXT_FILE changes 

  1. Source Patch_fs 

./EBSapps.ev patch 

  1. Patch FS 

Update the Patch $CONTEXT_FILE  

  1. Update the DB 

[applmgr@AM501AMERAP0005 ~]$ $ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer action=upload contextfile=/u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml logfile=/tmp/patchctxupload.log 

                ————————————————————————————————————————– 

                [applmgr@AM501AMERAP0005 ~]$ cat /tmp/patchctxupload.log 

                   ADX Database Utility 

getConnectionUsingAppsJDBCConnector() à 

    APPS_JDBC_URL=’jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=am501amerdb0002.ad.global)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=TEST)))’ 

    Trying to connect using APPS_JDBC_URL 

    Connection obtained. 

——————-ADX Database Utility Finished————— 

                   ADX Database Utility 

getConnectionUsingAppsJDBCConnector() à 

    APPS_JDBC_URL=’jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=am501amerdb0002.ad.global)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=TEST)))’ 

    Trying to connect using APPS_JDBC_URL 

    Connection obtained. 

——————-ADX Database Utility Finished————— 

[applmgr@AM501AMERAP0005 ~]$ 

  1. Run fs_clone on “run” file system 

adop phase=fs_clone 

adop -status 

  1. Start admin server on the run_fs 

$INST_TOP/admin/scripts/adadminsrvctl.sh start 

  1. Start the adminserver on the patch_fs 

$INST_TOP/admin/scripts/adadminsrvctl.sh start forcepatchfs 

  1. Preclone on RUN FS 

$INST_TOP/admin/scripts/adpreclone.pl appsTier 

  1. Preclone on PATCH FS 

$INST_TOP/admin/scripts/adpreclone.pl appsTier 

  1. Stop firewalld on AM501AMERAP0005 and AM501AMERAP0006 

sudo systemctl status firewalld 

sudo systemctl stop firewalld 

  1. Adding Connection Filter Rule for a new Application tier node – Run Filesystem 

java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

[applmgr@AM501AMERAP0005 ~]$ . EBSapps.env 

  E-Business Suite Environment Information 

  —————————————- 

  RUN File System           : /u01/install/APPS/fs1/EBSapps/appl 

  PATCH File System         : /u01/install/APPS/fs2/EBSapps/appl 

  Non-Editioned File System : /u01/install/APPS/fs_ne 

  DB Host: am501amerdb0002.ad.global  Service/SID: TEST 

  E-Business Suite Environment Setting 

  ———————————— 

  – Enter [R/r] for sourcing Run File System Environment file, or 

  – Enter [P/p] for sourcing Patch File System Environment file, or 

  – Enter anything else to exit 

  Please choose the environment file you wish to source [R/P]:R 

  Sourcing the RUN File System … 

[applmgr@AM501AMERAP0005 ~]$ pwd 

/home/applmgr 

[applmgr@AM501AMERAP0005 ~]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

For details please refer log file /home/applmgr/RegisterNodePreReq.log 

Enter the WebLogic Server Admin Password: 

add-filter-rule option executed successfully. 

[applmgr@AM501AMERAP0005 ~]$ cat /home/applmgr/RegisterNodePreReq.log 

Validated the passed arguments for the option add-filter-rule 

Validating hostname :AM501AMERAP0006.ad.global 

Validating the AM501AMERAP0006.ad.global :true 

Existing Connection Filter rules: 

[am501amerap0005.ad.global * * allow, 0.0.0.0/0 * * deny] 

Updating Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

[applmgr@AM501AMERAP0005 ~]$ 

java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

[applmgr@AM501AMERAP0005 Heema]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global^C 

[applmgr@AM501AMERAP0005 Heema]$ ls -ltr /u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar 

-rwxr-x— 1 applmgr oinstall 36339849 May 11 18:15 /u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar 

[applmgr@AM501AMERAP0005 Heema]$ ls -ltr /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

-rw-r–r–1 applmgr oinstall 121325 May 12 17:48 /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml 

[applmgr@AM501AMERAP0005 Heema]$ java -cp $CLASSPATH:/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

For details please refer log file /home/applmgr/Heema/RegisterNodePreReq.log 

Enter the WebLogic Server Admin Password: 

add-filter-rule option executed successfully. 

[applmgr@AM501AMERAP0005 Heema]$ cat /home/applmgr/Heema/RegisterNodePreReq.log 

Validated the passed arguments for the option add-filter-rule 

Validating hostname :am501amerap0006.ad.global 

Validating the am501amerap0006.ad.global :true 

Existing Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

Updating Connection Filter rules: 

[am501amerap0005.ad.global * * allow, AM501AMERAP0006.ad.global * * allow   #MT, am501amerap0006.ad.global * * allow   #MT, 0.0.0.0/0 * * deny] 

[applmgr@AM501AMERAP0005 Heema]$ 

  1. Adding Connection Filter Rule for a new Application tier node – Patch Filesystem 

. EBSenv Patch 

java -cp $CLASSPATH:/u01/install/APPS/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname AM501AMERAP0006.ad.global 

java -cp $CLASSPATH:/u01/install/APPS/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar oracle.apps.ad.tools.configuration.RegisterNodePreReq add-filter-rule -contextfile /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -hostname am501amerap0006.ad.global 

  1. Enable SSH connectivity 

Enable SSH connectivity between the nodes 

                Login as applmgr 

                 cd $HOME 

cd .ssh 

ssh-keygen -t rsa on each node and take defaults 

Login to AM501AMERAP0005: cp id_rsa.pub id_rsa.pub_cm 

Login to AM501AMERAP0006: cp id_rsa.pub id_rsa.pub_app 

scp id_rsa.pub_cm from AM501AMERAP0005 to AM501AMERAP0006 

scp id_rsa.pub_app from AM501AMERAP006 to AM501AMERAP0005 

mv authorized_keys authorized_keys_bkp 

for the AM501AMERAP0005/ AM501AMERAP0006 

cat id_rsa.pub>authorized_keys 

 

scp id_rsa.pub_cm from AM501AMERAP0005 to AM501AMERAP0006 

scp id_rsa.pub_app from AM501AMERAP006 to AM501AMERAP0005 

On AM501AMERAP0005 

cat id_rsa.pub_app>>authorized_keys 

On AM501AMERAP0006 

cat id_rsa.pub_cm>> authorized_keys 

 

Validate the ssh connectivity 

On AM501AMERAP0005 

ssh AM501AMERAP0005 – Self ssh 

ssh AM501AMERAP0006  

On AM501AMERAP0006 

ssh AM501AMERAP0006  – Self ssh  

ssh AM501AMERAP0005 

  1. Prepare the pair file 

/BACKUPS/TEST/TEST_am501amerap0005_run.txt 

  1. Set the env on the new node 

On the app node, perform the following. 

  

[applmgr@AM501AMERAP0006 ~]$ export PATH=/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin:$PATH 

[applmgr@AM501AMERAP0006 ~]$ cd /u01/install/APPS/fs1/EBSapps/comn/clone/bin 

[oracle@am501amerdb0002 ETCC]$ sh checkDBpatch.sh 

[applmgr@AM501AMERAP0005 ETCC]$ sh checkMTpatch.sh 

  1. Add node 

/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin/perl ./adclonectx.pl addnode contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml pairsfile=/BACKUPS/TEST/TEST_am501amerap0005_run.txt dualfs=yes 

[applmgr@AM501AMERAP0006 ~]$ export PATH=/u01/install/APPS/fs1/FMW_Home/webtier/perl/bin:$PATH 

[applmgr@AM501AMERAP0006 ~]$ cd /u01/install/APPS/fs1/EBSapps/comn/clone/bin 

[applmgr@AM501AMERAP0006 bin]$ /u01/install/APPS/fs1/FMW_Home/webtier/perl/bin/perl ./adclonectx.pl addnode contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml pairsfile=/BACKUPS/TEST/TEST_am501amerap0005_run.txt dualfs=yes 

                     Copyright I 2011, 2015 Oracle Corporation 

                        Redwood Shores, California, USA 

                        Oracle E-Business Suite Rapid Clone 

                                 Version 12.2 

                      adclonectx Version 120.30.12020000.22 

Enter the APPS passwd : 

Enter the Weblogic AdminServer password : 

 Executing command: /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jre/bin/java -Xmx600M -Doracle.jdbc.autoCommitSpecCompliant=false -classpath /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/ojdbc6.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/xmlparserv2.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/java::/u01/install/APPS/fs1/FMW_Home/wlserver_10.3/server/lib/weblogic.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wlfullclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wlclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/wljmxclient.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/ojmisc.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/obfuscatepassword.jar:/u01/install/APPS/fs1/EBSapps/comn/clone/bin/../jlib/emCfg.jar oracle.apps.ad.clone.RCloneSFSAddNode -contextfile /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml -configfile /BACKUPS/TEST/TEST_am501amerap0005_run.txt -machinelistenaddress am501amerap0006.ad.global -logdir /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog -promptmsg hide 

Verifying: Apps schema password 

Verifying: 

        pairsfile /BACKUPS/TEST/TEST_am501amerap0005_run.txt 

        Run file system AdminServer is running 

        Patch file system AdminServer is running 

        Run file system domain 

        Patch file system domain 

Executing: run clone context… 

Log file located at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog/run/RCloneSFSAddNode_05131337.log 

Target System Base Directory set to /u01/install/APPS 

Target System Current File System Base set to /u01/install/APPS/fs1 

Target System Other File System Base set to /u01/install/APPS/fs2 

Target System Fusion Middleware Home set to /u01/install/APPS/fs1/FMW_Home 

Target System Other File System Fusion Middleware Home set to /u01/install/APPS/fs2/FMW_Home 

Target System Web Oracle Home set to /u01/install/APPS/fs1/FMW_Home/webtier 

Target System Other File System Web Oracle Home set to /u01/install/APPS/fs2/FMW_Home/webtier 

Target System Appl TOP set to /u01/install/APPS/fs1/EBSapps/appl 

Target System Other File System Appl TOP set to /u01/install/APPS/fs2/EBSapps/appl 

Target System COMMON TOP set to /u01/install/APPS/fs1/EBSapps/comn 

Target System Other File System COMMON TOP set to /u01/install/APPS/fs2/EBSapps/comn 

Target System Current File System Instance Top set to /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006 

Report file located at /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/admin/out/portpool.lst 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

Configuring: run fs add node… 

Executing: patch clone context… 

Log file located at /u01/install/APPS/fs1/EBSapps/comn/clone/bin/../FMW/logs/sfslog/patch/RCloneSFSAddNode_05131344.log 

Target System Other File System Instance Top set to /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006 

Report file located at /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/admin/out/portpool.lst 

The new APPL_TOP context file has been created : 

  /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

contextfile=/u01/install/APPS/fs2/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml 

Configuring: patch fs add node… 

— Add node operation completed – 

Node am501amerap0006.ad.global has been added successfully 

[applmgr@AM501AMERAP0006 bin]$ 

  1. Run Autoconfig in the following order only  5-6-5 (CM-WEB-CM)  

7. Start EBS Clone Environment and Validate Multi-Node EBS and DB node 

  1. Start the EBS services from “run” filesystem and validate the front-end forms. “SYSADMIN” password is same as Source 

http://am501amerap0006.ad.global:8000/OA_HTML/AppsLogin 

Shutdown EBS services after the validation 

8. Load Balancer Setup and Configuration 

  1. Make changes to “run” filesystem context file to incorporate TEST LB URL – https://ebs.test.erp.oci.assaabloy.net:4443/OA_HTML/AppsLogin  

cat $CONTEXT_FILE| egrep ‘s_endUserMonitoringURL|s_external_url|s_webentryhost|s_webentrydomain|s_login_page|s_webentryurlprotocol|s_active_webport’ 

[applmgr@AM501AMERAP0005 .ssh]$ cat $CONTEXT_FILE| egrep ‘s_endUserMonitoringURL|s_external_url|s_webentryhost|s_webentrydomain|s_login_page|s_webentryurlprotocol|s_active_webport’ 

         <EndUserMonitoringURL oa_var=”s_endUserMonitoringURL”>https://ebs.test.erp.oci.assaabloy.net:4443/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL> 

         <externURL oa_var=”s_external_url”>https://ebs.test.erp.oci.assaabloy.net:4443</externURL> 

         <webentryhost oa_var=”s_webentryhost”>ebs.test.erp.oci</webentryhost> 

         <webentrydomain oa_var=”s_webentrydomain”>assaabloy.net</webentrydomain> 

         <login_page oa_var=”s_login_page”>https://ebs.test.erp.oci.assaabloy.net:4443/OA_HTML/AppsLogin</login_page> 

         <webentryurlprotocol oa_var=”s_webentryurlprotocol”>https</webentryurlprotocol> 

      <activewebport oa_var=”s_active_webport” oa_type=”DUP_PORT” base=”8000” step=”1” range=”-1” label=”Active Web Port”>4443</activewebport> 

[applmgr@AM501AMERAP0005 .ssh]$ 

  1. Run Autoconfig in the following order only  5-6-5 (CM-WEB-CM)  
  1. Update patch $CONTEXT_FILE 

Upload the Patch file system context file to the Database 

source patch File system 

$ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer action=upload contextfile= /u01/install/APPS/fs2/inst/apps/TEST_am501amerap0005/appl/admin/TEST_am501amerap0005.xml logfile=/tmp/patchctxupload.log 

9. Final Steps 

  • Create all /interface/TEST directories on AM501AMERAP0005 and sub-directories as AM501AMERAP0001. /interface should be a FSS among all “3” nodes – AM501AMERAP0005, AM501AMERAP0006 and am501amerdb0002. 
  • Connect to PDB and Change all dba_directories location for “/interface/PROD/*” to “/interface/TEST/*” individually. Ensure all those directories are existing at Unix level with same permissions as AM501AMERAP0001. 

CREATE [OR REPLACE] DIRECTORY directory_name AS ‘path_name’; 

  • Request ASSA ABLOY Team to create SAMBA configuration for /interface/TEST mountpoint 
  • Validate oacore_server in the $CONTEXT_FILE 

[applmgr@AM501AMERAP0005 .ssh]$ grep -I oacore_server $CONTEXT_FILE 

      <oacore_server> 

      </oacore_server> 

         <oacore_server_ports oa_var=”s_oacore_server_ports”>oacore_server1:7201</oacore_server_ports> 

         <oacore_server_sslports oa_var=”s_oacore_server_sslports”/> 

            <oa_service_name oa_var=”s_oacorename” type=”managed_server”>oacore_server1</oa_service_name> 

            <oa_managed_server_name oa_var=”s_oacore_managed_servers”>oacore_server1</oa_managed_server_name> 

[applmgr@AM501AMERAP0005 .ssh]$ 

  • Start the admin server 

adadminsrvctl.sh start 

  • Create below directories for run and patch fs 

[applmgr@AM501AMERAP0006 servers]$ pwd 

/u01/install/APPS/fs1/FMW_Home/user_projects/domains/EBS_domain/servers 

[applmgr@AM501AMERAP0006 servers]$ mkdir -p oacore_server3/logs 

[applmgr@AM501AMERAP0006 servers]$ cd /u01/install/APPS/fs2/FMW_Home/user_projects/domains/EBS_domain/servers 

[applmgr@AM501AMERAP0006 servers]$ mkdir -p oacore_server3/logs 

  • Adding Additional Oacore 

Command : 

perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml -managedsrvname=oacore_server3 -servicetype=oacore -managedsrvport=7203 

[applmgr@AM501AMERAP0006 Heema]$ perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/appl/admin/TEST_am501amerap0006.xml -managedsrvname=oacore_server3 -servicetype=oacore -managedsrvport=7203 -logfile=/u01/install/APPS/fs1/FMW_Home/user_projects/domains/EBS_domain/servers/oacore_server3/logs/oacore3.log 

Enter the APPS Schema password: 

Enter the WebLogic AdminServer password: 

The logfile for this session is located at /u01/install/APPS/fs1/FMW_Home/user_projects/domains/EBS_domain/servers/oacore_server3/logs/oacore3.log 

ManagedServer oacore_server3 created successfully. 

[applmgr@AM501AMERAP0006 Heema]$ pwd 

/home/applmgr/Heema 

[applmgr@AM501AMERAP0006 Heema]$ ls -ltr 

total 0 

Command : 

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl -contextfile=$CONTEXT_FILE -configoption=addMS -oacore=am501amerap0006.ad.global:7203 

[applmgr@AM501AMERAP0006 Heema]$ perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl -contextfile=$CONTEXT_FILE -configoption=addMS -oacore=am501amerap0006.ad.global:7203 

Enter apps schema password : 

*** LOG FILE: /u01/install/APPS/fs1/inst/apps/TEST_am501amerap0006/logs/appl/rgf/TXK/txkSetAppsConf_05150052.log *** 

[applmgr@AM501AMERAP0006 Heema]$ 

am501amerap0005.ad.global:17001/console 

  • Add entry in adop_sync.drv 

$ cd $APPL_TOP_NE/ad/custom/ 

$ ls 

adop_sync.drv 

$ cp adop_sync.drv adop_sync.drv_backup 

Add below two lines in adop_sync.drv 

# Custom Application sync # 

rsync -az %s_current_base%/EBSapps/appl/xxcus/ %s_other_base%/EBSapps/appl/xxcus 

  • Update JVM size for OPP 

OPP Manager 

select DEVELOPER_PARAMETERS from apps.FND_CP_SERVICES where SERVICE_ID = (select MANAGER_TYPE from apps.FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = ‘FNDCPOPP’); 

                Output should be as below 

DEVELOPER_PARAMETERS 

J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx2048m 

If note, please perform below 

SQL> update FND_CP_SERVICES set DEVELOPER_PARAMETERS = ‘J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx2048m’ where SERVICE_ID =(select MANAGER_TYPE from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = ‘FNDCPOPP’); 

1 row updated. 

SQL>  select DEVELOPER_PARAMETERS from apps.FND_CP_SERVICES where SERVICE_ID = (select MANAGER_TYPE from apps.FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = ‘FNDCPOPP’); 

DEVELOPER_PARAMETERS 

J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx2048m 

SQL> SQL> commit; 

Commit complete. 

[applmgr@am501amerap0005]echo $AFJSMARG 

                -server -Xmx384m -Doracle.apps.fnd.common.Pool.leak.mode=stderr:off -verbose:gc 

  • Switch to Framework menus 

“Framework” menus are mandatory in R12. 

Check the Profile Option at the Site level: 

Self Service Personal Home Page mode = Framework only 

Check that no other values exist for this profile other than ‘FWK’ or ‘NONE’ : 

SQL> select PROFILE_OPTION_ID from fnd_profile_options where PROFILE_OPTION_NAME = ‘APPLICATIONS_HOME_PAGE’; 

PROFILE_OPTION_ID 

             7496 

SQL>select distinct PROFILE_OPTION_VALUE from fnd_profile_option_values where PROFILE_OPTION_ID =’6963’; 

PROFILE_OPTION_VALUE 

FWK 

NONE 

If any ‘PHP’ values exist, update those rows to ‘FWK’. 

SQL> update fnd_profile_option_values set PROFILE_OPTION_VALUE=’FWK’ where PROFILE_OPTION_ID = ‘1003494’ and PROFILE_OPTION_VALUE=’PHP’; 

SQL> select distinct PROFILE_OPTION_VALUE from fnd_profile_option_values where PROFILE_OPTION_ID = ‘1003494’; 

Emtek 

SQL> select PROFILE_OPTION_ID from fnd_profile_options where PROFILE_OPTION_NAME = ‘APPLICATIONS_HOME_PAGE’; 

PROFILE_OPTION_ID 

             7496 

SQL> select distinct PROFILE_OPTION_VALUE from fnd_profile_option_values where PROFILE_OPTION_ID =’7496’; 

PROFILE_OPTION_VALUE 

FWK_TREE 

NONE 

SQL> create table fnd_profile_option_values_Moh as select * from fnd_profile_option_values; 

Table created. 

SQL> select count(*) from fnd_profile_option_values; 

  COUNT(*) 

      7990 

SQL> select count(*) from fnd_profile_option_values_Moh; 

  COUNT(*) 

      7990 

SQL> update fnd_profile_option_values set PROFILE_OPTION_VALUE=’FWK’ where PROFILE_OPTION_ID = ‘7496’ and PROFILE_OPTION_VALUE=’FWK_TREE’; 

1 row updated. 

SQL> commit; 

Commit complete. 

SQL> pwd 

Ls 

PROFILE_OPTION_VALUE 

FWK 

NONE 

SQL> 

  • Set XML Publisher TEMP directory  

Add, or remove the end-date from XML Publisher Administrator responsibility for the SYSADMIN user if this responsibility is not available. 

Navigate> XML Publisher Administrator > Home > Administration. 

Self-service screen appears. 

Open : Properties > General 

Set Temporary Directory = /u01/install/APPS/fs1/EBSapps/comn/temp 

Save 

Close Window 

  • CARD connector post clone steps 

IBY: ECAPP URL : https://ebs.erp.oci.assaabloy.net:4443/OA_HTML/ibyecapp 
ICX: Oracle Payment Server URL : https://ebs.erp.oci.assaabloy.net:4443/OA_HTML/ibyecapp 
Database Wallet Directory : /home/oracle/dbwallet 

IBY: HTTP Proxy  — blank 

  • Execute grants  

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/ARLockbox’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/PO_PROMSE_DATE_UPD’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/CustShipAddrUpd’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/AutoReceipt’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/InvShortage’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/MassPO’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/SO_DFF_Upd’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/SubInvTransfer’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/UPSTrack’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/ASN_INV’, ‘read’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/PO_PROMSE_DATE_UPD/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/CustShipAddrUpd/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/ARLockbox/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/AutoReceipt/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/InvShortage/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/MassPO/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/SO_DFF_Upd/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/SubInvTransfer/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/UPSTrack/*’, ‘read,write’ ); 

execute dbms_java.grant_permission( ‘APPS’, ‘SYS:java.io.FilePermission’, ‘/interface/TEST/ASN_INV/*’, ‘read,write’ ); 

You can verify by running below SQLs. 

select * from XXCUS_ARLKBX_FNAME_V; 

select * from XXCUS_AR_CUSTADDRUPD_FNAME_V; 

select * from XXCUS_AUTORECEIPT_FNAME_V; 

select * from XXCUS_PO_PRMISE_DATE_FNAME_V; 

select * from XXCUS_SO_DFFUPD_FNAME_V; 

select * from XXCUS_UPS_TRACK_FNAME_V; 

select * from XXCUS_SUBINV_TRANSF_FNAME_V; 

select * from XXCUS_ASN_INV_FNAME_V; 

select * from XXCUS_PO_MASSCRE_FNAME_V; 

select * from XXCUS_INV_SHORTG_FNAME_V; 

Appendix A

Update dba_directories 

SQL> col OWNER for a30 

SQL> col DIRECTORY_NAME for a40 

SQL> col DIRECTORY_PATH for a60 

SQL> set lines 200 pages 200 

SQL> select OWNER,DIRECTORY_NAME,DIRECTORY_PATH from dba_directories; 

OWNER                          DIRECTORY_NAME                           DIRECTORY_PATH 

—————————— —————————————- ———————————————————— 

SYS                            ORACLE_HOME                              / 

SYS                            ORACLE_BASE                              / 

SYS                            OPATCH_LOG_DIR                           /u01/app/oracle/product/12.1.0.2/dbhome_1/QOpatch 

SYS                            OPATCH_SCRIPT_DIR                        /u01/app/oracle/product/12.1.0.2/dbhome_1/QOpatch 

SYS                            DBMS_OPTIM_ADMINDIR                      /u01/app/oracle/product/12.1.0/dbhome_1/rdbms/admin 

SYS                            DBMS_OPTIM_LOGDIR                        /u01/app/oracle/product/12.1.0/dbhome_1/cfgtoollogs 

SYS                            DATA_PUMP_DIR                            /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/log/ 

SYS                            OPATCH_INST_DIR                          /u01/app/oracle/product/12.1.0.2/dbhome_1/OPatch 

SYS                            ECX_UTL_LOG_DIR_OBJ                      /usr/tmp 

SYS                            APPS_DATA_FILE_DIR                       /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil/outbound/ 

                                                                        TEST_am501amerdb0002 

SYS                            ECX_UTL_XSLT_DIR_OBJ                     /usr/tmp 

SYS                            FND_DIAG_DIR                             /u01/TEMP/12.1.0/rdbms/log 

SYS                            WORK_DIR                                 /devl/ohome/10.2.0/work 

SYS                            PREUPGRADE_DIR                           /prod/ohome/10.2.0/cfgtoollogs/PEMTKI/preupgrade/ 

SYS                            AUDIT_DIR                                /tmp/ 

SYS                            AW_FILES                                 /home/oraprdn/migration 

SYS                            ORACLE_OCM_CONFIG_DIR                    /devl/ohome/10.2.0/ccr/state 

SYS                            UTL_FEDEX                                /mnt/fedex 

SYS                            EX_FILES                                 /home/oraprdn/migration 

SYS                            INVOICE_PDF_PATH1                        /interface/PROD/invoice_pdf_path 

SYS                            CUST_CSV                                 /interface/PROD/custcsv 

SYS                            TNWORK                                   /interface/PROD/TNWorking 

SYS                            ESKERWORK                                /interface/PROD/Esker 

SYS                            WORK                                     /interface/PROD/Working 

SYS                            LABEL_PRINT                              /interface/PROD/LABEL_PRINT 

SYS                            ASSAWORK                                 /interface/PROD/ASSA 

SYS                            ESKER_CUST_ADDR                          /interface/PROD/Esker_Cust_Addr 

SYS                            PREUPG_OUTPUT_DIR                        /prod/ohome/12c/cfgtoollogs/dbua/PEMTKI/upgrade1 

SYS                            BACKUP                                   /interface/PROD/Backup 

SYS                            EDI810                                   /interface/PROD/EDI/810 

SYS                            EDI850                                   /interface/PROD/EDI/850 

SYS                            SHBWORK                                  /interface/PROD/SHBWorking 

SYS                            AUTORECEIPT                              /interface/PROD/AutoReceipt 

SYS                            ASSAWORK_EMTEK                           /interface/PROD/ASSA 

SYS                            PO_PROMSE_DATE                           /interface/PROD/PO_PROMSE_DATE_UPD 

SYS                            ASSAWORK_SCHAUB                          /interface/PROD/ASSA/Schaub 

SYS                            TABLEDUMP                                /backup/PEMTKI/TABLEDUMP 

SYS                            SQLT$STAGE                               /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            TRCA$STAGE                               /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            SQLT$UDUMP                               /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            SQLT$BDUMP                               /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            SQLT$DIAG                                /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            TRCA$INPUT1                              /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            TRCA$INPUT2                              /u01/TEMP/CRPTKI/oracle/diag/rdbms/crptki_trgt/PROD/trace 

SYS                            TRACE_DIR_SRC_4_TCB                      /prod/ohome/12c/diag/rdbms/pemtki/PEMTKI/trace 

SYS                            OPATCH_TEMP_DIR                          /u01/app/oracle/product/12.1.0.2/dbhome_1/QOpatch 

SYS                            MASSPO                                   /interface/PROD/MassPO 

SYS                            CSR_XML_TOP                              /u01/install/APPS/fs1/EBSapps/appl/csr/12.0.0/patch/115/xml 

SYS                            ODPDIR                                   /tmp 

SYS                            CUSTSHIPADDRUPD                          /interface/PROD/CustShipAddrUpd 

SYS                            ASN_INV                                  /interface/PROD/ASN_INV 

SYS                            CRC                                      /interface/PROD/CRC 

SYS                            CUSTCSV                                  /interface/PROD/custcsv 

SYS                            INVSHORTAGE                              /interface/PROD/InvShortage 

SYS                            SO_DFF_UPD                               /interface/PROD/SO_DFF_Upd 

SYS                            SUBINV                                   /interface/PROD/SubInvTransfer 

SYS                            UPSTRACK                                 /interface/PROD/UPSTrack 

SYS                            UPSTXT                                   /interface/PROD/upstxt 

58 rows selected. 

SQL> 

SQL> show pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 TEST                           READ WRITE NO 

SQL> alter session set container=TEST; 

Session altered. 

CREATE OR REPLACE DIRECTORY INVOICE_PDF_PATH1  AS’/interface/TEST/invoice_pdf_path’; 

CREATE OR REPLACE DIRECTORY INVOICE_PDF_PATH1  AS ‘/interface/TEST/invoice_pdf_path’; 

CREATE OR REPLACE DIRECTORY CUST_CSV AS ‘/interface/TEST/custcsv’; 

CREATE OR REPLACE DIRECTORY TNWORK AS ‘/interface/TEST/TNWorking’; 

CREATE OR REPLACE DIRECTORY ESKERWORK AS ‘/interface/TEST/Esker’; 

CREATE OR REPLACE DIRECTORY WORK AS ‘/interface/TEST/Working’; 

CREATE OR REPLACE DIRECTORY LABEL_PRINT AS ‘/interface/TEST/LABEL_PRINT’; 

CREATE OR REPLACE DIRECTORY ASSAWORK  AS ‘/interface/TEST/ASSA’; 

CREATE OR REPLACE DIRECTORY ESKER_CUST_ADDR  AS ‘/interface/TEST/Esker_Cust_Addr’; 

CREATE OR REPLACE DIRECTORY BACKUP  AS ‘/interface/TEST/Backup’; 

CREATE OR REPLACE DIRECTORY EDI810  AS ‘/interface/TEST/EDI/810’; 

CREATE OR REPLACE DIRECTORY EDI850  AS ‘/interface/TEST/EDI/850’; 

CREATE OR REPLACE DIRECTORY SHBWORK  AS ‘/interface/TEST/SHBWorking’; 

CREATE OR REPLACE DIRECTORY AUTORECEIPT  AS ‘/interface/TEST/AutoReceipt’; 

CREATE OR REPLACE DIRECTORY ASSAWORK_EMTEK  AS ‘/interface/TEST/ASSA’; 

CREATE OR REPLACE DIRECTORY PO_PROMSE_DATE  AS ‘/interface/TEST/PO_PROMSE_DATE_UPD’; 

CREATE OR REPLACE DIRECTORY ASSAWORK_SCHAUB  AS ‘/interface/TEST/ASSA/Schaub’; 

CREATE OR REPLACE DIRECTORY MASSPO  AS ‘/interface/TEST/MassPO’; 

CREATE OR REPLACE DIRECTORY CUSTSHIPADDRUPD  AS ‘/interface/TEST/CustShipAddrUpd’; 

CREATE OR REPLACE DIRECTORY ASN_INV AS ‘/interface/TEST/ASN_INV’; 

CREATE OR REPLACE DIRECTORY CRC AS ‘/interface/TEST/CRC’; 

CREATE OR REPLACE DIRECTORY CUSTCSV AS ‘/interface/TEST/custcsv’; 

CREATE OR REPLACE DIRECTORY INVSHORTAGE AS ‘/interface/TEST/InvShortage’; 

CREATE OR REPLACE DIRECTORY SO_DFF_UPD AS ‘/interface/TEST/SO_DFF_Upd’; 

CREATE OR REPLACE DIRECTORY SUBINV AS ‘/interface/TEST/SubInvTransfer’; 

CREATE OR REPLACE DIRECTORY UPSTRACK AS ‘/interface/TEST/UPSTrack’; 

CREATE OR REPLACE DIRECTORY UPSTXT AS ‘/interface/TEST/upstxt’; 

Match APPLTMP to /utl/file/ 

Match APPLTMP to /utl/file/ 

Validate APPLTMP values at AM501AMERAP0005 and AM501AMERAP0006 

[applmgr@AM501AMERAP0005 .ssh]$ echo $APPLPTMP 

/usr/tmp 

[applmgr@AM501AMERAP0005 .ssh]$ 

[applmgr@AM501AMERAP0006 ~]$ echo $APPLPTMP 

/usr/tmp 

[applmgr@AM501AMERAP0006 ~]$ 

Issue the following at both cdb & pdb level 

SQL> alter system set utl_file_dir=’/usr/tmp’ scope=spfile; 

System altered. 

Bounce EBS and DB. 

POST Clone steps 

Create softlinks in $OA_HTML as below. 

[applmgr@AM501AMERAP0006 html]$ pwd 

/u01/install/APPS/fs1/FMW_Home/Oracle_EBS-app1/applications/oacore/html 

[applmgr@AM501AMERAP0006 html]$ ls -ltr|grep lrwx 

lrwxrwxrwx  1 applmgr oinstall        22 Aug 12 04:13 Backup -> /interface/TEST/Backup 

lrwxrwxrwx  1 applmgr oinstall        23 Aug 12 04:13 Working -> /interface/TEST/Working 

lrwxrwxrwx  1 applmgr oinstall        25 Aug 12 04:13 SHBBackup -> /interface/TEST/SHBBackup 

lrwxrwxrwx  1 applmgr oinstall        26 Aug 12 04:13 SHBWorking -> /interface/TEST/SHBWorking 

lrwxrwxrwx  1 applmgr oinstall        23 Aug 12 04:13 ASN_INV -> /interface/TEST/ASN_INV 

lrwxrwxrwx  1 applmgr oinstall        27 Aug 12 04:13 SHBOriginal -> /interface/TEST/SHBOriginal 

lrwxrwxrwx  1 applmgr oinstall        24 Aug 12 04:13 Original -> /interface/TEST/Original 

lrwxrwxrwx  1 applmgr oinstall        19 Aug 12 04:13 CRC -> /interface/TEST/CRC 

lrwxrwxrwx  1 applmgr oinstall        22 Aug 12 04:13 MassPO -> /interface/TEST/MassPO 

lrwxrwxrwx  1 applmgr oinstall        31 Aug 12 04:13 CustShipAddrUpd -> /interface/TEST/CustShipAddrUpd 

lrwxrwxrwx  1 applmgr oinstall        32 Aug 12 04:13 emtek_user_files -> /interface/TEST/emtek_user_files 

lrwxrwxrwx  1 applmgr oinstall        24 Aug 12 04:13 UPSTrack -> /interface/TEST/UPSTrack 

lrwxrwxrwx  1 applmgr oinstall        26 Aug 12 04:13 SO_DFF_Upd -> /interface/TEST/SO_DFF_Upd 

lrwxrwxrwx  1 applmgr oinstall        27 Aug 12 04:13 LABEL_PRINT -> /interface/TEST/LABEL_PRINT 

lrwxrwxrwx  1 applmgr oinstall        27 Aug 12 04:13 InvShortage -> /interface/TEST/InvShortage 

lrwxrwxrwx  1 applmgr oinstall        31 Aug 12 04:13 Esker_Cust_Addr -> /interface/TEST/Esker_Cust_Addr 

lrwxrwxrwx  1 applmgr oinstall        21 Aug 12 04:13 Esker -> /interface/TEST/Esker 

lrwxrwxrwx  1 applmgr oinstall        19 Aug 12 04:13 EDI -> /interface/TEST/EDI 

lrwxrwxrwx  1 applmgr oinstall        20 Aug 12 04:13 ASSA -> /interface/TEST/ASSA 

lrwxrwxrwx  1 applmgr oinstall        22 Aug 12 04:13 upstxt -> /interface/TEST/upstxt 

lrwxrwxrwx  1 applmgr oinstall        27 Aug 12 04:13 AutoReceipt -> /interface/TEST/AutoReceipt 

lrwxrwxrwx  1 applmgr oinstall        30 Aug 12 04:13 SubInvTransfer -> /interface/TEST/SubInvTransfer 

lrwxrwxrwx  1 applmgr oinstall        23 Aug 12 04:13 custcsv -> /interface/TEST/custcsv 

lrwxrwxrwx  1 applmgr oinstall        34 Aug 12 04:13 PO_PROMSE_DATE_UPD -> /interface/TEST/PO_PROMSE_DATE_UPD 

lrwxrwxrwx  1 applmgr oinstall        25 Aug 12 04:13 ARLockbox -> /interface/TEST/ARLockbox 

lrwxrwxrwx  1 applmgr oinstall        26 Aug 12 04:13 TNOriginal -> /interface/TEST/TNOriginal 

lrwxrwxrwx  1 applmgr oinstall        25 Aug 12 04:13 TNWorking -> /interface/TEST/TNWorking 

lrwxrwxrwx  1 applmgr oinstall        25 Aug 12 04:13 riverhead -> /interface/TEST/riverhead 

lrwxrwxrwx  1 applmgr oinstall        32 Aug 12 04:13 invoice_pdf_path -> /interface/TEST/invoice_pdf_path 

R12.2.10 Manual DB Clone between VMDBs

High Level Steps

1. Install OS Pre-requisites on CM, APP nodes(Target) 

2 Install required VMDB patches on Target DB OH 

3 Running Pre-Clone on the Source Apps Tier(s) and DB Tier 

4 Backup Full database and Archivelogs via RMAN on Source 

5 Tar Ball of Application Tier directories on Source and Copy to Target 

6 Pre DB restore steps on target  

7    Configure and start Oracle Instance on Target 

8 RMAN Duplicate & Restore and Recover database and open clone database on Target 

9 Run Post Clone Steps on Target Database Server ( DB Tier ) 

10 Run Post Clone Steps on Target Application Server ( Apps Tier ) 

11 Start EBS Clone Environment and Validate Single Web and DB node 

12 Multi-node Architecture Conversion 

13 Load Balancer Setup and Configuration 

14 Final Steps – Performance Best Practices and Validate the interface mount point and dba_directories 

  1. Install required VMDB patches on Target DB OH

Login to AM501AMERAP0005 and AM501AMERAP0006 as “root” user 

# yum update 

# yum install oracle-ebs-server-R12-preinstall 

Create a softlink as “root” user as below on Target CM/Web nodes – AM501AMERAP0005 and AM501AMERAP0006 

[root@AM501AMERAP0005 lib]#  cd /usr/lib 

[root@AM501AMERAP0005 lib]#  ln -s libXm.so.4.0.4 libXm.so.2 

[root@AM501AMERAP0005 lib]# ls -s libXm.so.4.0.4 libXm.so.2 

   0 libXm.so.2  2728 libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# ls -ltr libXm.so.2 

lrwxrwxrwx 1 root root 14 May 12 16:40 libXm.so.2 -> libXm.so.4.0.4 

[root@AM501AMERAP0005 lib]# 

2. Install required VMDB patches on Target DB OH

Downloaded below patches, followed readme to apply using “opatch apply”. NOTE: Apply these patches(matching Source DB Opatch lsinventory) before dropping the Shell DB that comes with OCI VMDB. Bring down the DB before applying these patches.  

22747454 

33568947 

33566041 

19239846 

21322448 

19472320 

27051384 

21321429 

24007737 

22731026 

22828765 

21967332 

21864513 

Startup the DB and Execute datapatch at the end. 

cd $ORACLE_HOME/Opatch 

datapatch 

Ensure Source and Target DB patches are matching.  

  • Error 1

On starting DB using sqlplus, it gave below error 

ORA-00210: cannot open the specified control file 

ORA-00202: control file: ‘+RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381’ 

ORA-17503: ksfdopn:2 Failed to open file +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 

ORA-15001: diskgroup “RECO” does not exist or is not mounted 

ORA-15040: diskgroup is incomplete 

ORA-205 signalled during: ALTER DATABASE   MOUNT… 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

NOTE: ASMB mounting group 2 (RECO) 

NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) 

WARNING: group 2 (RECO) has missing disks 

ORA-15040: diskgroup is incomplete 

WARNING: group 2 is being dismounted. 

WARNING: ASMB force dismounting group 2 (RECO) due to missing disks 

SUCCESS: diskgroup RECO was dismounted 

Solution:  

[OCI] Oracle Database Startup Failed with ORA-00210/ ORA-00202 / ORA-15001/ ORA-15040 after Patching DB System OCI (Doc ID 2514240.1) 

[root@am501amerdb0002 bin]# ls -ltr oracle 

-rwsr-s–x 1 oracle oinstall 323514824 May  4 03:56 oracle 

[root@am501amerdb0002 bin]# chgrp asmadmin oracle 

[root@am501amerdb0002 bin]# chmod 6751 oracle 

[root@am501amerdb0002 bin]# ls -ltr oracle 

-rwsr-s–x 1 oracle asmadmin 323514824 May  4 03:56 oracle 

[root@am501amerdb0002 bin]# 

  • Error 2 

Restarted DB using sqlplus but faced below error 

Errors in file /u01/app/oracle/diag/rdbms/orcl_iad12w/ORCL/trace/ORCL_pmon_71054.trc: 

ORA-27140: attach to post/wait facility failed 

ORA-27300: OS system dependent operation:invalid_egid failed with status: 1 

ORA-27301: OS failure message: Operation not permitted 

ORA-27302: failure occurred at: skgpwinit6 

ORA-27303: additional information: startup egid = 1004 (asmadmin), current egid = 1001 (oinstall) 

USER (ospid: 71054): terminating the instance due to error 27140 

Wed May 04 03:56:49 2022 

Instance terminated by USER, pid = 71054 

Wed May 04 03:58:08 2022 

Starting ORACLE instance (normal) (OS id: 71847) 

Wed May 04 03:58:08 2022 

CLI notifier numLatches:13 maxDescs:1346 

Wed May 04 03:58:08 2022 

********************************************************************** 

Wed May 04 03:58:08 2022 

Dump of system resources acquired for SHARED GLOBAL AREA (SGA) 

Solution 

[root@am501amerdb0002 ~]# usermod -a -G asmadmin oracle 

[root@am501amerdb0002 ~]# sudo su – oracle 

Last login: Wed May  4 04:47:15 UTC 2022 

[oracle@am501amerdb0002 ~]$ id 

uid=101(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dbaoper),1003(dba),1004(asmadmin),1006(asmdba) 

[oracle@am501amerdb0002 ~]$ 

Target Shell DB restarted after this 

3. Running Pre-Clone on the Source Apps Tier(s) and DB Tier 

Login to am501amerdb0001 server 

cd $ORACLE_HOME/appsutil/scripts/PROD_am501amerdb0001 

perl adpreclone.pl dbTier 

Login to AM501AMERAP0001 server  

echo $FILE_EDITION 

run 

cd $INST_TOP/admin/scripts 

perl adpreclone.pl appsTier 

4. Backup Full database and Archivelogs via RMAN on Source 

[oracle@am501amerdb0001 OCI]$ pwd 

/home/oracle/OCI 

[oracle@am501amerdb0001 OCI]$ cat rman_r12clone_FULL.sh 

rman target / msglog=RMAN_FULL_rman_r12_uat_22July.log <<EOF 

SET ENCRYPTION ON; 

CONFIGURE BACKUP OPTIMIZATION OFF; 

run 

allocate channel c1 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c2 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c3 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c4 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c5 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c6 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c7 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

allocate channel c8 device type ‘SBT’ PARMS  ‘SBT_LIBRARY=/home/oracle/OCI/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/OCI/config)’; 

BACKUP AS COMPRESSED BACKUPSET FULL DATABASE FORCE TAG =’RMAN_FULL_UAT_July22′ PLUS ARCHIVELOG TAG =’RMAN_FULL_ARC_UAT_July22′; 

BACKUP CURRENT CONTROLFILE TAG =’RMAN_FULL_CNTRL_UAT_July22′; 

release channel c1; 

release channel c2; 

release channel c3; 

release channel c4; 

release channel c5; 

release channel c6; 

release channel c7; 

release channel c8; 

EOF 

[oracle@am501amerdb0001 OCI]$ 

Execute backup 

============= 

nohup ./rman_r12clone_FULL.sh & 

5. Trigger tar ball of application binaries in the background 

As root user on Source Admin Server(AM501AMERAP0001) 

mkdir /BACKUPS/R12Clone 

chmod 777 /BACKUPS/R12Clone 

cd /BACKUPS/R12Clone 

nohup gtar -czvf runfilesystem_fs1_ebs.gtar.gz /u01/install/APPS/fs1/EBSapps & 

jobs 

[root@AM501AMERAP0001 R12Clone]# jobs 

[1]+  Running                 nohup gtar -czvf runfilesystem_fs1_ebs.gtar.gz /u01/install/APPS/fs1/EBSapps & 

[root@AM501AMERAP0001 R12Clone]# pwd 

/BACKUPS/R12Clone 

[root@AM501AMERAP0001 R12Clone]# 

6. Pre DB restore steps on target  

Delete the Target Database created by Cloud Tooling (I took a cold RMAN backup before deleting it) 

sqlplus / as sysdba 

select * from v$encryption_wallet; — Capture the location of TDE encryption/auto login keys and take a backup of the same.  

set heading off linesize 999 pagesize 0 feedback off trimspool on 

spool /tmp/remove_files.sh 

select ‘asmcmd rm ‘||name from v$datafile 

union all 

select ‘asmcmd rm ‘||name from v$tempfile 

union all 

select ‘asmcmd rm ‘||member from v$logfile; 

spool off 

chmod 777 /tmp/remove_files.sh 

Shutdown the DB using sqlplus  

sudo su – grid 

. oraenv 

+ASM1 

sh /tmp/remove_files.sh (Validate the file for any syntax errors before executing) 

Get the Source Database ID 

SQL> select dbid from v$database; 

        

DBID 

—————— 

1624776053 

7. Configure and start Oracle Instance on Target 

  1. Get the Source Database ID 

SQL> select dbid from v$database; 

        

DBID 

—————— 

1624776053 

  1. Copy the RMAN Configuration File(used in backup script i.e /home/oracle/OCI/config) from Source to Target  /home/oracle/OCI/object_storage/ 

This file contains the information about the Object Storage endpoint, wallet location where the Object Storage credentials are stored, and the Object Storage bucket name: 

  1. Copy the Wallet containing the Object Storage Credentials(location mentioned in above file) from am501amerdb0001 to am501amerdb0002 (/home/oracle/OCI/object_storage/) 
  1. Copy the library for the Backup Module(/home/oracle/OCI/lib/libopc.so) from am501amerdb0001 to am501amerdb0002 
  1. Copy the TDE wallet containing the TDE master encryption keys and autologin keys from am501amerdb0001 to am501amerdb0002(location indicated in target v$encryption_wallet before dropping the DB). Ensure we have original keys backups on target before copying from Source so they don’t get overwritten. 

sqlplus “/as sysdba” 

select * from v$encryption_wallet;  

  1. Check the Copied Files 

The RMAN configuration file, wallet, and library for the Backup Module: 

[oracle@am501amerdb0002 ~]$ cd object_storage 

[oracle@am501amerdb0002 object_storage]$ ls -ltr /home/oracle/object_storage/ 

total 91240 

-rw——- 1 oracle oinstall     1621 May  5 18:57 cwallet.sso 

-rw-r–r– 1 oracle oinstall 93414592 May  5 19:04 libopc.so 

-rw-r–r– 1 oracle oinstall      189 May  5 19:27 configam501amerdb0001 

-rw-r–r– 1 oracle oinstall      193 May  5 20:09 config 

[oracle@am501amerdb0002 object_storage]$ pwd 

/home/oracle/object_storage 

[oracle@am501amerdb0002 object_storage]$ 

The TDE password and auto-login wallets: 

[oracle@am501amerdb0002 ORCL_iad18r]$ ls -ltr /opt/oracle/dcs/commonstore/wallets/tde/ORCL_iad18r/ 

total 24 

-rw——- 1 oracle oinstall 10475 May 10 18:37 ewallet.p12 

-rw——- 1 oracle oinstall 10520 May 10 18:37 cwallet.sso 

[oracle@am501amerdb0002 ORCL_iad18r]$ 

  1. Start the DB in “No Mount” mode 

SQL>startup nomount; 

SQL>select * from v$encryption_wallet;  – should be open and autologin 

8. RMAN Duplicate & Restore and Recover database and open clone database on Target 

  •   Start DB in nomount by commenting recovery_file_dest & recovery_file_size 

Create pfile from spfile; 

Comment recovery_file_dest & recovery_file_size 

start DB with pfile 

  • Restore the Control File 

         rman target / 

         RMAN> set dbid 1624776053; 

         RMAN> 

         run  

        { 

        allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so,SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

        restore controlfile from autobackup; 

        } 

  • Mount the database. 

        SQL> alter database mount; 

  • Create set new name script on Source DB to use in restore script 

set head off pages 0 feed off echo off verify off 

set lines 200 

spool /tmp/rename_datafiles_CB.lst 

select ‘set newname for datafile ‘ || file# || ‘ to ”+DATA”;’ 

from V$datafile ; 

spool off 

exit; 

  • Restore the DB 

[oracle@am501amerdb0002 OCI]$ pwd 

/home/oracle/OCI 

[oracle@am501amerdb0002 OCI]$ cat RMAN_RESTORE_TEST.sh 

rman target / msglog=rman_restore_test.log <<EOF 

CONFIGURE BACKUP OPTIMIZATION OFF;  

run 

allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c2 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c3 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c4 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c5 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c6 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c7 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

allocate channel c8 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

set newname for database to ‘+DATA’;  

restore database; 

EOF 

[oracle@am501amerdb0002 OCI]$ 

  • Switch all datafiles 

[oracle@am501amerdb0002 OCI]$ rman target / 

Recovery Manager: Release 12.1.0.2.0 – Production on Fri Jul 22 12:18:26 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

connected to target database: ORCL (DBID=1624776053, not open) 

RMAN> switch database to copy; 

  • Recover the database 

                 RMAN_TEST_RECOVER.sh 

                 rman target / msglog=rman_TEST_recover_22JULY22.log <<EOF 

                 CONFIGURE BACKUP OPTIMIZATION OFF; 

                 run 

                 { 

                 allocate channel c1 device type sbt PARMS ‘SBT_LIBRARY=/home/oracle/object_storage/libopc.so, SBT_PARMS=(OPC_PFILE=/home/oracle/object_storage/config)’; 

                 recover database; 

                 } 

EOF 

[oracle@am501amerdb0002 OCI]$ tail -40f rman_TEST_recover_22JULY22.log 

Recovery Manager: Release 12.1.0.2.0 – Production on Fri Jul 22 12:25:55 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

connected to target database: ORCL (DBID=1624776053, not open) 

RMAN> 

using target database control file instead of recovery catalog 

old RMAN configuration parameters: 

CONFIGURE BACKUP OPTIMIZATION OFF; 

new RMAN configuration parameters: 

CONFIGURE BACKUP OPTIMIZATION OFF; 

new RMAN configuration parameters are successfully stored 

RMAN> 2> 3> 4> 5> 

allocated channel: c1 

channel c1: SID=4 device type=SBT_TAPE 

channel c1: Oracle Database Backup Service Library VER=19.0.0.1 

Starting recover at 22-JUL-22 

starting media recovery 

channel c1: starting archived log restore to default destination 

channel c1: restoring archived log 

archived log thread=1 sequence=3106 

channel c1: reading from backup piece Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013 

channel c1: ORA-19870: error while restoring backup piece Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013 

ORA-19507: failed to retrieve sequential file, handle=”Auto_Archive_arc_ORCL_1624776053_kd1380c5_1_1_20220722_1110704517_set16013″, parms=”” 

ORA-27029: skgfrtrv: sbtrestore returned error 

ORA-19511: non RMAN, but media manager or vendor specific failure, error text: 

   KBHS-07502: File not found 

KBHS-01404: See trace file /u01/app/oracle/produc 

failover to previous backup 

released channel: c1 

RMAN-00571: =========================================================== 

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== 

RMAN-00571: =========================================================== 

RMAN-03002: failure of recover command at 07/22/2022 12:26:52 

RMAN-20506: no backup of archived log found 

RMAN-06053: unable to perform media recovery because of missing log 

RMAN-06025: no backup of archived log for thread 1 with sequence 3106 and starting SCN of 6011136313975 found to restore 

RMAN> 

Incase of above error, perform manual recovery by copying the requested logs from am501amerdb0001 to am501amerdb0002. Please use RMAN to copy archive logs from ASM to local filesystem and scp the archive files to target.  

  • Restart the database 

SQL> select name,open_mode from v$database; 

NAME      OPEN_MODE 

——— ——————– 

ORCL      MOUNTED 

  • Manually copy all required archive log files from source to target 

[oracle@am501amerdb0002 ~]$ sqlplus ‘/as sysdba’ 

SQL*Plus: Release 12.1.0.2.0 Production on Fri Jul 22 13:30:50 2022 

Copyright (c) 1982, 2014, Oracle.  All rights reserved. 

Connected to: 

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 – 64bit Production 

With the Automatic Storage Management and Real Application Testing options 

SQL>  recover database until cancel using backup controlfile; 

ORA-00279: change 6011136314031 generated at 07/22/2022 08:14:02 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-00280: change 6011136314031 for thread 1 is in sequence #3106 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

ORA-00308: cannot open archived log 

‘+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709’ 

ORA-17503: ksfdopn:2 Failed to open file 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-15012: ASM file 

‘+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709’ 

does not exist 

ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below 

ORA-01152: file 1 was not restored from a sufficiently old backup 

ORA-01110: data file 1: ‘+DATA/ORCL_IAD12W/DATAFILE/system.271.1110732659’ 

SQL> recover database until cancel using backup controlfile; 

ORA-00279: change 6011136314031 generated at 07/22/2022 08:14:02 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3106.1838.1110729709 

ORA-00280: change 6011136314031 for thread 1 is in sequence #3106 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3106.1838.1110729709 

ORA-00279: change 6011136438482 generated at 07/22/2022 16:01:48 needed for 

thread 1 

ORA-00289: suggestion : 

+RECO/ORCL_IAD18R/ARCHIVELOG/2022_07_22/thread_1_seq_3107.1837.1110730643 

ORA-00280: change 6011136438482 for thread 1 is in sequence #3107 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3106.1838.1110729709’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3107.1837.1110730643 

ORA-00279: change 6011136480729 generated at 07/22/2022 16:17:23 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136480729 for thread 1 is in sequence #3108 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3107.1837.1110730643’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3108.1836.1110733273 

ORA-00279: change 6011136769586 generated at 07/22/2022 17:01:12 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136769586 for thread 1 is in sequence #3109 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3108.1836.1110733273’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3109.1835.1110736887 

ORA-00279: change 6011136961194 generated at 07/22/2022 18:01:26 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011136961194 for thread 1 is in sequence #3110 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3109.1835.1110736887’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3110.1834.1110740497 

ORA-00279: change 6011137109018 generated at 07/22/2022 19:01:36 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011137109018 for thread 1 is in sequence #3111 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3110.1834.1110740497’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

/BACKUPS/thread_1_seq_3111.1833.1110744077 

ORA-00279: change 6011137258778 generated at 07/22/2022 20:01:16 needed for 

thread 1 

ORA-00289: suggestion : +RECO 

ORA-00280: change 6011137258778 for thread 1 is in sequence #3112 

ORA-00278: log file ‘/BACKUPS/thread_1_seq_3111.1833.1110744077’ no longer 

needed for this recovery 

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} 

CANCEL 

Media recovery cancelled. 

SQL> alter database open resetlogs; 

Database altered. 

SQL> show pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           READ WRITE NO 

SQL> exit  

9. Run Post Clone Steps on Target Database Server ( DB Tier ) 

  • Change DBID of the Target DB 

Shut immediate 

Startup mount 

[oracle@am501amerdb0002 ~]$ nid TARGET=SYS 

DBNEWID: Release 12.1.0.2.0 – Production on Fri Jul 22 13:41:30 2022 

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved. 

Password: 

Connected to database ORCL (DBID=1624776053) 

Connected to server version 12.1.0 

Control Files in database: 

    +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 

Change database ID of database ORCL? (Y/[N]) => Y 

Proceeding with operation 

Changing database ID from 1624776053 to 1638538147 

    Control File +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 – modified 

    — 

    — 

    Datafile +DATA/ORCL_IAD12W/6570B3097CD64775E053010A5A0AFBA9/TEMPFILE/temp1.580.111074610 – dbid changed 

    Control File +RECO/ORCL_IAD12W/CONTROLFILE/current.256.1096996381 – dbid changed 

    Instance shut down 

Database ID for database ORCL changed to 1638538147. 

All previous backups and archived redo logs for this database are unusable. 

Database is not aware of previous backups and archived logs in Recovery Area. 

Database has been shutdown, open database with RESETLOGS option. 

Succesfully changed database ID. 

DBNEWID – Completed succesfully. 

[oracle@am501amerdb0002 ~]$ 

  • Restart the DB and validate the DBID 

sqlplus “/as sysdba” 

SQL> startup mount; 

SQL> alter database open resetlogs; 

SQL> sho pdbs 

SQL> select dbid from v$database; 

  • Rename the pluggable DB name – Rename a Pluggable Database In Oracle 12c (Doc ID 2439885.1) 

sqlplus “/as sysdba” 

Connect to the CDB and check the information about the PDB first: 

SQL>select name, open_mode, restricted from v$pdbs; 

SQL>select name, con_id, dbid,con_uid,guid from v$containers order by con_id; 

SQL>select service_id,name,network_name,creation_date,pdb,con_id from cdb_services; 

SQL> sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           READ WRITE NO 

SQL> 

SQL> alter pluggable database PROD close; 

SQL>  sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         2 PDB$SEED                       READ ONLY  NO 

         3 PROD                           MOUNTED 

SQL> alter pluggable database PROD open restricted; 

SQL> select name, open_mode, restricted from v$pdbs; 

NAME                           OPEN_MODE  RES 

—————————— ———- — 

PDB$SEED                       READ ONLY  NO 

PROD                           READ WRITE YES 

SQL> alter session set container=PROD; 

SQL> alter pluggable database rename global_name to TEST; 

SQL> sho pdbs 

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED 

———- —————————— ———- ———- 

         3 TEST                                          READ WRITE YES 

SQL> alter pluggable database close immediate; 

SQL> alter pluggable database open; 

Connect to the CDB and check the information again: 

SQL> alter session set container=CDB$ROOT; 

SQL> select name, open_mode, restricted from v$pdbs; 

SQL>select name, con_id, dbid,con_uid,guid from v$containers order by con_id; 
SQL>select service_id,name,network_name,creation_date,pdb,con_id from cdb_services; 

  • Copy tnsnames.ora and listener.ora from Source $TNS_ADMIN to Target $TNS_ADMIN. Make changes of “PROD” to “TEST” and “am501amerdb0001” to “am501amerdb0002” 
  • Create a pfile out of spfile from PROD and scp to target. Make appropriate directory changes as per target.  

Ensure below changes are in place to avoid TNS issues 

db_domain as null  

sec_case_sensitive_logon as FALSE  

Start DB with the new pfile from PROD.  

Create spfile 

Restart the DB 

Restart the listener 

Check “test” is mentioned in listener services. 

[oracle@am501amerdb0002 ~]$ lsnrctl status 

LSNRCTL for Linux: Version 12.1.0.2.0 – Production on 17-MAY-2022 22:17:40 

Copyright (c) 1991, 2014, Oracle.  All rights reserved. 

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521)) 

STATUS of the LISTENER 

———————— 

Alias                     LISTENER 

Version                   TNSLSNR for Linux: Version 19.0.0.0.0 – Production 

Start Date                10-MAY-2022 15:44:32 

Uptime                    7 days 6 hr. 33 min. 8 sec 

Trace Level               off 

Security                  ON: Local OS Authentication 

SNMP                      OFF 

Listener Parameter File   /u01/app/19.0.0.0/grid/network/admin/listener.ora 

Listener Log File         /u01/app/grid/diag/tnslsnr/am501amerdb0002/listener/alert/log.xml 

Listening Endpoints Summary… 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.215.254.53)(PORT=1521))) 

  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=am501amerdb0002.ad.global)(PORT=5500))(Security=(my_wallet_directory=/u01/app/oracle/product/12.1.0.2/dbhome_1/admin/ORCL_iad12w/xdb_wallet))(Presentation=HTTP)(Session=RAW)) 

Services Summary… 

Service “+APX” has 1 instance(s). 

  Instance “+APX1”, status READY, has 1 handler(s) for this service… 

Service “+ASM” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “+ASM_DATA” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “+ASM_RECO” has 1 instance(s). 

  Instance “+ASM1”, status READY, has 1 handler(s) for this service… 

Service “ORCLXDB” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

Service “ORCL_iad12w” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

Service “test” has 1 instance(s). 

  Instance “ORCL”, status READY, has 2 handler(s) for this service… 

The command completed successfully 

[oracle@am501amerdb0002 ~]$ 

  • Create a appsutil zip from Source Admin Server(AM501AMERAP0001) under $AD_TOP/bin(run filesystem) 

[applmgr@AM501AMERAP0001 bin]$ perl admkappsutil.pl 

Starting the generation of appsutil.zip 

Log file located at /u01/install/APPS/fs1/inst/apps/PROD_am501amerap0001/admin/log/MakeAppsUtil_05092211.log 

output located at /u01/install/APPS/fs1/inst/apps/PROD_am501amerap0001/admin/out/appsutil.zip 

MakeAppsUtil completed successfully. 

[applmgr@AM501AMERAP0001 bin]$ 

[applmgr@AM501AMERAP0001 bin]$ 

  • Execute below scripts as “APPS” user 

                sqlplus apps@test 

                SQL> exec fnd_conc_clone.setup_clean; 

                SQL> exec ad_zd_fixer.clear_valid_nodes_info; 

  •    Copy the DB context from Source OH/appsutil to Target OH/appsutil and make necessary changes  
  • Create nls/data/9idata directory 

                On the database server node, as the owner of the Oracle RDBMS file system and database instance, run the following command to create the $ORACLE_HOME/nls/data/9idata directory. 

$ perl $ORACLE_HOME/nls/data/old/cr9idata.pl 

After creating the directory, make sure that the ORA_NLS10 environment variable is set to the full path of the 9idata directory whenever you enable the 12c Oracle home. 

[oracle@am501amerdb0002 old]$ ls -ltr cr9idata.pl 

-rw-r–r– 1 oracle oinstall 11785 Oct 12  2021 cr9idata.pl 

[oracle@am501amerdb0002 old]$ perl cr9idata.pl 

Creating directory /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata … 

Copying files to /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata… 

Copy finished. 

Please reset environment variable ORA_NLS10 to /u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata! 

[oracle@am501amerdb0002 old]$ 

  • Set the env file and execute autoconfig 

cat ORCL.env 

export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 

export ORACLE_SID=ORCL 

export ORA_NLS10=/u01/app/oracle/product/12.1.0.2/dbhome_1/nls/data/9idata 

. ORCL.env 

cd $ORACLE_HOME/appsutil/bin 

sh adconfig.sh 

  • Update the Site Name 

sqlplus apps@test 

update fnd_profile_option_values set profile_option_value = ‘R12 SIT’ where profile_option_id=125; 

commit; 

Decide if you have to cancel or suspend any source concurrent jobs or WF setups. I did not do it as we were cloning CRP for SIT

Apps Cloning Steps continues in the next blog https://orama.blog/2022/09/04/r12210-apps-clone-between-oci-compute-nodes/

How to Enable Database Management for Oracle Cloud Databases

Introduction

Database Management is a cloud service that provides broad capabilities for managing and monitoring your Oracle Databases on-premises and in the cloud. The new release of Database Management service now supports Oracle Databases on Virtual Machine, Bare Metal, and Exadata Cloud Service in OCI without installing the Management Agent.

For databases running on OCI IaaS, ExaCC, on-premises, or other clouds, you need to install a Management Agent and register the database as External Database resource in the Database Management service.

Database Management service provides the following features:

Fleet Monitoring and Management

  • Unified view for monitoring and managing Oracle Databases across on-premises and cloud

Performance Diagnostics

  • Integrated view of database activity for easy performance diagnostics
  • Automatic Database Diagnostic Monitor (ADDM), Active Session History (ASH) Analytics, SQL monitoring, blocking sessions, and metrics
  • Advanced SQL execution plan analysis

Database Administration

  • Tablespace management
  • Database parameter configuration
  • Scheduled jobs, and more…

This blog post provides you a step-by-step guide to enable Database Management service for Oracle Cloud databases.

The Environment

  • Oracle Database on VM DB Systems in OCI
  • OCI Vault for secrets management

Preparation

Step 1: Set the monitoring user password and privileges in the database

Grant a database user, e.g. SYSTEM or DBSNMP, the required privileges needed for monitoring and managing the database. I used DBSNMP user.

SQL> GRANT CREATE PROCEDURE TO dbsnmp;

Grant succeeded.

SQL> GRANT SELECT ANY DICTIONARY, SELECT_CATALOG_ROLE TO dbsnmp;

Grant succeeded.

SQL> GRANT ALTER SYSTEM TO dbsnmp;

Grant succeeded.

SQL> GRANT ADVISOR TO dbsnmp;

Grant succeeded.

SQL> GRANT EXECUTE ON DBMS_WORKLOAD_REPOSITORY TOdbsnmp;

Grantsucceeded.

The level of password checks in the Database Management service is different from the one in the database. For Database Management service, the user password must be FIPS-compliant following these rules:

  • Password length must be between 14 to 127 characters.
  • Password must have at least one digitone lowercaseone uppercase, and one special character.

The database verification might have stronger rules at some points, e.g. having at least two digitstwo lowercasetwo uppercase, and two special character.

SQL> alter user dbsnmp account unlock;
User altered.

SQL> alter user dbsnmp identified by”X1N4_$T@Xp<>0#7n”;
User altered.

Step 2: Save the monitoring user password as a Secret in OCI Vault

Search for the Vault service in Identity & Security.

Create a new Vault if needed. Within your Vault, create a key that will be used to encrypt the user password.

Create a secret providing the database user password as Secret Content.

Step 3: Assign IAM permissions to Database Management service

Grant the Database Management service the permission to:

  • Read the database monitoring user password stored in OCI Vault secret
  • Write results of the scheduled jobs for Oracle Cloud Databases to the Object Storage service

Create a new IAM Policy with the following statements:

Allow service dpd to read secret-family in compartment ABC

Allow service dpd to manage objects in compartment ABC

You can restrict these permissions further to specific resources by using the IAM WHERE clause.

Step 4: Assign IAM permissions to your OCI user

To enable and use Database Management, you must belong to your tenancy’s Administrators group or to a group (here named DB-MGMT-ADMIN) that have the following permissions:

Allow group DB-MGMT-ADMIN to manage dbmgmt-family in compartment ABC
Allow group DB-MGMT-ADMIN to read database-family in compartment ABC
Allow group DB-MGMT-ADMIN to manage vnics in compartment ABC
Allow group DB-MGMT-ADMIN to use subnets in compartment ABC
Allow group DB-MGMT-ADMIN to use network-security-groups in compartment ABC
Allow group DB-MGMT-ADMIN to use security-lists in compartment ABC
Allow group DB-MGMT-ADMIN to manage secret-family in compartment ABC
Allow group DB-MGMT-ADMIN to read buckets in compartment ABC

You can restrict these permissions further to specific resources by using the IAM WHERE clause.

Network Configuration

Step 5: Create a Private Endpoint in your VCN

Database Management service resides in an Oracle-managed tenancy and interacts with your Cloud databases via a Private Endpoint that is created within your VCN. The Private Endpoint does not need to be in the same subnet as your Cloud databases but must be on a subnet that allows communication with your Cloud databases.

Search for Administration service in Database Management,

Click on the Private Endpoints link, then Create Private Endpoint. Name your Private Endpoint, choose a compartment, select and VCN, a subnet, and a Network Security Group. Finally, click Create Private Endpoint.

A private IP address will be assigned to your Private Endpoint.

Step 6: Allow network traffic between the Private Endpoint and your Cloud databases

Add the appropriate ingress and egress security rules to your Security Lists or Network Security Groups to enable Database Management service to communicate with your Cloud databases via the Private Endpoint. For RAC databases, use the SCAN IPs for communicating with the Database Management service.

Add stateful security rules as follows:

  • In the Network Security Group of your VM DB System, add the private IP of the Private Endpoint into the Ingress Rules, so your database can receive traffic from the Database Management service.
  • In the Network Security Group of the Private Endpoint, add the private IP of the VM DB System into the Egress Rules, so the Database Management service can send requests to the database.

Enable Database Management

Step 7: Enable Database Management for your Cloud database

When you enable Database Management, you can select the full or basic management option:

  • Full Management: includes all database management features for EE. The Performance Hub features are not available for SE.
  • Basic Management: includes 14 basic monitoring metrics such as CpuUtilization and StorageAllocated. It also includes ASH Analytics and SQL Monitoring features in Performance Hub for CDBs, but not for PDBs. I selected Basic Management as it is FREE Option

From the Database Details page, click on the Enable link for Database Management in the Associated Services section.

Enter the database user name DBSNMP, choose the Secret created in step 2, choose the Private Endpoint created in step 5, and click the Enable Database Management button.

If the user password was not FIPS-compliant, enabling will fail. Make sure you create a FIPS-compliant password for your DBSNMP user and save it in the OCI Vault Secret!

Once the enabling process successfully completes, you’ll see the Metrics on the Metric page.

If the metric page is enabled, but not being populated with any data, check the database user privileges needed as described in step 1.

For ASH Analytics, SQL Monitoring, and Blocking Session, click on Performance Hub on the Database Details page.

If your database user is missing any needed privileges, Performance Hub will indicate this in an error message on the upper right side of the page.

References

How to Enable Database Management for Oracle Cloud Databases – Database Heartbeat (database-heartbeat.com)