[ERROR] ETCC not run in the database node <node name>
Validating system setup. Node registry is valid. [WARNING] SYS.DUAL table has incorrect privileges Shut down the application tier services and then run the following commands in the order shown sqlplus / as sysdba @ $AD_TOP/patch/115/sql/ADFIXUSER.sql sqlplus apps/ @ $AD_TOP/sql/adutlrcmp.sql [ERROR] ETCC not run in the database node calrdbuat03 The EBS Technology Codelevel Checker needs to be run on the database node. It is available as Patch 17537119. [WARNING] MTCC has not been run. Apply Patch 17537119 and run checkMTpatch.sh.
The node name in table APPS.TXK_TCC_RESULTS and /etc/hosts can be different, causing an adop failure to identify the database host name correctly.
Solution:
Validate /etc/hosts file has entries with a correct format. IP Address hostname.domain hostname.
Validate entries in APPS.TXK_TCC_RESULTS and make updates if needed
update APPLSYS.TXK_TCC_RESULTS set node_name=’uatfin’ where node_name =’uatfin.node.com’;
Validate the s_dbhost value in the database context file ${CONTEXT_FILE} and ensured it has only NODE_NAME and not the domain. Validate the same in applications context file ${CONTEXT_FILE} as well.
Downloaded the latest version of ETCC and ran on DB node.
(I had to do this as my servers were not connecting via putty and original ppk keys. Hence I wanted to check if it is a server issue or network issue apart from the fact tracert to the servers were timing out all throughout)
First create a OCI serial console connection
Open your OCI Dashboard Using your Web Browser.
Open the Cloud Menu (Hamburger Icon – ) and select “Compute”.
On the “Instances” list Select the instance you would like to connect o using “Serial connection”.
Scroll down and Select the “Console Connections” option.
Press the “Create local connection” button.
On the Popup window, Click on “Generate a keypair for me” and save public and private keys on your local PC.
In few mins, you will see an active console connection.
Now connect using SSH
8. Copy the Console connection string (according to your desired method – SSH / VNC).
9. Open your Terminal / SSH Client and Paste the copied command. I used windows command prompt
Important
Don’t forget the -i switch. it is very important, especially if your SSH keys is stored on a different location or if it is a specific key you generated for this scenario.
The correct syntax will be to add the -i after the ssh word and add the path to your key file, afterwords the rest of the command will remain on it’s original form.
Note: When using single quotes with ProxyCommand=, I got Bad stdio forwarding specification ‘%h:%p’
Now we are connected to the server and it will ask for username/password. Since we dont have ‘opc’ password, we will follow below steps to create a new password
With your open console connection, you want to go back to the webpage where you built the console connection and reboot your instance, by clicking on the Reboot button. Once you click Reboot go back to the terminal window and click on it to make it the active application. You’ll see the Linux stop and reboot routine scroll past the console eventually you’ll see the GRUB menu – From the GRUB menu take the following steps:
As soon as you see the GRUB boot menu, use the up/down arrows to stop the process, enabling you to use the boot menu.
In the boot menu, highlight the top item in the menu, and press e to edit the boot entry.
In edit mode, scroll down until you reach the line that starts with either linuxefi and add the following to the line: init=/bin/bash
4. You don’t need to save the file. Reboot the instance using keyboard shortcut CTRL+X
5. After the instance has rebooted, you’ll see the Bash prompt – Now you have cosmic powers to the instance, play safe and only do what you understand. Maybe even take a snapshot before if you care about the instance.
6. Run the following command to load the Linux policies to preserve the files you’re about to modify: /usr/sbin/load_policy -i //NOTE// If you skip this step you will not able to login to your instance via SSH or console and you’ll be sad.
7. The following command will remount the root partition into read and write mode so you can modify the password files : /bin/mount -o remount, rw /
8. The following command will reset the password of the OPC default user account: sudo passwd opc
9. After the password has been reset you’re done. Reboot the machine with the following command : sudo reboot -f
Now you’ve done all of the needful and you should be to login to your Linux instance from the console after the reboot.
After creating a new EBS instance using Rapidclone, managed servers such as oacore_server2/oacore_server3/oacore_server4 and forms_server1 fail to start and their status on WLS console are not “RUNNING” but “ADMIN” while oafm_server1 is “RUNNING”.
ORA-01017 is shown in the log files of oacore_server1 and forms_server1 as follows.
###<Nov 26, 2021 11:45:44 AM JST> <Error> <Deployer> <hostname.domain> <oacore_server1> <[ACTIVE] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<WLS Kernel>> <> <> <1637894744672> <BEA-149205> <Failed to initialize the application ‘EBSDataSource’ due to error weblogic.application.ModuleException: at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:327) at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:199) at weblogic.application.internal.flow.DeploymentCallbackFlow$1.next(DeploymentCallbackFlow.java:517) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:159) at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:45) at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:648) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:191) at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:44) at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:154) at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:61) at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:55) at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39) at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191) at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21) at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240) at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165) at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122) at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:180) at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:96) at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64) at weblogic.work.ExecuteThread.execute(ExecuteThread.java:263) at weblogic.work.ExecuteThread.run(ExecuteThread.java:221) Caused By: weblogic.common.resourcepool.ResourceSystemException: Could not connect to ‘oracle.jdbc.OracleDriver’.
The returned message is: ORA-01017: invalid username/password; logon denied
It is likely that the login or password is not valid. It is also possible that something else is invalid in the configuration or that the database is not available. at weblogic.jdbc.common.internal.JDBCUtil.parseException(JDBCUtil.java:301) at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:383) at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:241) at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1322) at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1239) at weblogic.common.resourcepool.ResourcePoolImpl.start(ResourcePoolImpl.java:250)The
The customer confirmed followings.
– The initial parameter SEC_CASE_SENSITIVE_LOGON is set to FALSE. – They can connect to Database using sqlplus by password both in upper and lower case.
Solution
Reset the password of EBSDataSource as follows.
(1) Login to WLS Console and proceed as follows.
Domain Structure > Services > Data Sources > EBSDataSource > Configuration TAB > Connection Pool Sub-TAB
(2) Push “Lock & Edit” button in Change Center. Then set APPS schema’s password for “Password” & “Confirm Password” fields and push “Save” button.
(3) Push “Activate Chang” button in Change Center.
(4) Stop oacore_server1 and forms_server1 as follows.
[ERROR] txkADOPValidation failed on Node: “am501amerap0005”
[ERROR] txkADOPValidation failed or is incomplete on Admin node: am501amerap0005
[ERROR] Unable to continue processing on other available nodes: am501amerap0006
[UNEXPECTED]Error running “txkADOPValidation” on node(s): am501amerap0005.
Summary report for current adop session:
Node am501amerap0005:
– Fs_clone status: Failed
Node am501amerap0006:
– Fs_clone status: Not Started
For more details, run the command: adop -status -detail
[STATEMENT] Please run adopscanlog utility, using the command
“adopscanlog -latest=yes”
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 2 (Fail)
[applmgr@AM501AMERAP0005 ~]$
Cause
The global inventory on secondary node was corrupt or missing on secondary node(s) where the txkADOPValidation was failing. ADOPValidation Log files showed below error for secondary node
ERROR: Either /u01/app/oraInventory/ContentsXML/inventory.xml file doesn’t exist or doesn’t have READ permission.
Solution
Copy the contents of /etc/oraInst.loc from Primary to Secondary. Take a tar ball of global inventory from Primary to Secondary nodes. Ensure permissions are same as primary node.
[STATEMENT] ICM status cannot be determined. [STATEMENT] [END 2022/11/01 10:48:57] Check if ICM is running [UNEXPECTED]Error determining status of Internal Concurrent Manager [UNEXPECTED]Error calling runConcurrentProgram subroutine.
I was running “adop phase=prepare” from a directory that contained a “*.dbc” file
Solution
Move the dbc file to a backup directory(if possible) or run adop prepare command from a different directory where “*.dbc” file is not presetn
) and select “Compute”.
Orama.blog 