Problem Description:
We observed below ORA-00600 in alert.log of 11.2.0.4 database, but fortunately DB was up and running
Dump file /u01/app/oracle/diag/rdbms/nascxthzprod/NASCXTHZPROD/incident/incdir_1517411/NASCXTHZPROD_ora_94908_i1517411.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0.4/dbhome_1
System name: Linux
Node name: nasodaprd051
Release: 2.6.39-400.290.1.el6uek.x86_64
Version: #1 SMP Mon Nov 7 12:59:33 PST 2016
Machine: x86_64
Instance name: NASCXTHZPROD
Redo thread mounted by this instance: 1
Oracle process number: 669
Unix process pid: 94908, image: oracle@nasodaprd051
…
*** 2017-07-29 12:05:19.586
Dump continued from file: /u01/app/oracle/diag/rdbms/nascxthzprod/NASCXTHZPROD/trace/NASCXTHZPROD_ora_94908.trc
ORA-00600: internal error code, arguments: [ktbdchk1: bad dscn], [], [], [], [], [], [], [], [], [], [], []
========= Dump for incident 1517411 (ORA 600 [ktbdchk1: bad dscn]) ========
—– Beginning of Customized Incident Dump(s) —–
…
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error/Stack/SQL/Version from file “NASCXTHZPROD_ora_94908_i1517411.trc”
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: ORA-00600 [ktbdchk1: bad dscn]
Error Stack: ORA-600[ktbdchk1: bad dscn]
Main Stack:
ktbValidateDependentScn <- ktbgcl1 <- ktbgtl0 <- kdiins0 <- kdiinsp <- kauxsin
<- qesltcLoadIndexList <- qesltcLoadIndexes <- qerltcNoKdtBufferedInsRowCBK
<- qerltcSingleRowLoad <- qerltcFetch <- insexe <- opiexe <- kpoal8 <- opiodr <- ttcpip
<- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real <- ssthrdmain <- main
Current SQL: INSERT INTO CASES (IC_MEMBER_ID, DATE_CREATED, IC_RULE_ID, ACTION, REPORTING_CATEGORY, CERTAINTY, BRIEF_JUSTIFICATION_ID, COMPR_JUSTIFICATION_ID, JUSTIFICATION_ID, MEMBER_EMPLOYEE_FLAG, STATE, DENY_ADJUSTMENT_CODE, ADD_ADJUSTMENT_CODE, SUPPORT_ADJUSTMENT_CODE, SAVINGS_PERCENTAGE, MODULE_NAME, MSG, H…
Cause:
Bug 22241601 : ORA-600 [KDSGRP1] IN ADG AFTER FAILOVER
Solution:
Install Patch for Bug 22241601 from note 22241601.8
Workaround
Install the fix to prevent this issue from being introduced; installing the fix
also by default tries to repair existent invalid ITL commit scn (healing).
There is not need to set any parameter for it.
Databases already having init.ora parameter _ktb_debug_flags=8 can
remove the parameter after the fix is installed as _ktb_debug_flags=8 is now
the default so the healing is enabled by default. Note that this fix is not disabled
if _ktb_debug_flags is set to 0; a value not including 8 will only disable the healing
for already affected ITL SCN but the fix still solves the problem of the invalid ITL SCN
in the INDEX to be introduced as it does not depend of any parameter.
Sometimes the fix may not repair the block for an already existent invalid SCN on disk;
then repair this issue with:
1. Recreate the affected Index in the Primary Database
Create Index may also fail with the same ORA-600 [ktbdchk1: bad dscn] if Oracle
builds the index based on another index with the same corruption for which the
solution is to drop all the affected indexes for the table and recreate them.
(OR)
2. If the issue is in the standby database: refresh the affected file from primary to standby.
If the error is in the primary database, refresh the file from the standby to the primary database.