hannel ORA_DISK_1: finished piece 1 at 16-NOV-15

piece handle=+DFILE/ORCL/BACKUPSET/2015_11_16/nnndf0_tag20151116t060442_0.263.89
5903483 tag=TAG20151116T060442 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:25
Finished backup at 16-NOV-15
Starting backup at 16-NOV-15
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=11 RECID=3 STAMP=895903509
channel ORA_DISK_1: starting piece 1 at 16-NOV-15
channel ORA_DISK_1: finished piece 1 at 16-NOV-15
piece handle=+DFILE/ORCL/BACKUPSET/2015_11_16/annnf0_tag20151116t060509_0.265.89
5903509 tag=TAG20151116T060509 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
Finished backup at 16-NOV-15
Starting Control File and SPFILE Autobackup at 16-NOV-15
piece handle=+DFILE/ORCL/AUTOBACKUP/2015_11_16/s_895903516.266.895903517 comment
=NONE
Finished Control File and SPFILE Autobackup at 16-NOV-15
SQL> column path format a40
SQL> /
NAME
-----------------------------DATA011
DATA012
DATA013

PATH
---------------------------------------ORCL:DATA011
ORCL:DATA012
ORCL:DATA013

select label,path,name from v$asm_disk

LABEL
----------------------------------------------------DATA011
DATA012
DATA013

PATH
NAME
---------------------------------------- ------ORCL:DATA011
ORCL:DATA012
ORCL:DATA013

DATA011
DATA012
DATA013

asmcmd md_backup /dbms/oracle/+ASM/adm/back/asm_meta_backup_$(date +%Y%m%d_%H%M)

.bkp
md_backup /u01/app/oracle/product/12.1.0/DATA_DISKGROUP_backup_$(date +%Y%m%d_%H
%M).bkp -G DATA
ASMCMD> md_backup /u01/app/oracle/product/12.1.0/DATA_DISKGROUP_backup.bkp -G DA
TA
Disk group metadata to be backed up: DATA
Current alias directory path: ORCL/DATAFILE
Current alias directory path: ORCL/TEMPFILE
Current alias directory path: ORCL
Current alias directory path: ORCL/PARAMETERFILE
Current alias directory path: ASM/PASSWORD
Current alias directory path: ASM/ASMPARAMETERFILE

Current alias directory path: ORCL/ONLINELOG

Current alias directory path: ASM
Current alias directory path: ORCL/CONTROLFILE
********************************************************************************
***************************************************
ASMCMD> md_backup /u01/app/oracle/product/12.1.0/ALL_DISKGROUP.bkp
Disk group metadata to be backed up: DFILE
Disk group metadata to be backed up: DATA
Current alias directory path: ORCL/ONLINELOG
Current alias directory path: ORCL/BACKUPSET
Current alias directory path: ORCL/CONTROLFILE
Current alias directory path: ORCL/BACKUPSET/2015_11_16
Current alias directory path: ORCL/ARCHIVELOG/2015_11_16
Current alias directory path: ORCL/AUTOBACKUP/2015_11_16
Current alias directory path: ORCL/AUTOBACKUP
Current alias directory path: ORCL
Current alias directory path: ORCL/ARCHIVELOG
Current alias directory path: ORCL/DATAFILE
Current alias directory path: ORCL/TEMPFILE
Current alias directory path: ORCL
Current alias directory path: ORCL/PARAMETERFILE
Current alias directory path: ASM/PASSWORD
Current alias directory path: ORCL/ONLINELOG
Current alias directory path: ASM/ASMPARAMETERFILE
Current alias directory path: ASM
Current alias directory path: ORCL/CONTROLFILE
ASMCMD> cp s_895903516.266.895903517 /u01/app/oracle
copying +dfile/orcl/AUTOBACKUP/2015_11_16/s_895903516.266.895903517 -> /u01/app/
oracle/s_895903516.266.895903517

Version: 12.1.0.2.0
kfed read /dev/asm_h002 | egrep 'ausize|dsknum|dskname|grpname|fgname'
kfed read /dev/oracleasm/disks/DATA011 | egrep 'ausize|dsknum|dskname|grpname|fg
name'
[oracle@collabn2 disks]$ kfed read /dev/oracleasm/disks/DATA011 | egrep 'ausize|
dsknum|dskname|grpname|fgname'
kfdhdb.dsknum:
0 ; 0x024: 0x0000
kfdhdb.dskname:
DATA011 ; 0x028: length=7
kfdhdb.grpname:
DATA ; 0x048: length=4
kfdhdb.fgname:
DATA011 ; 0x068: length=7
kfdhdb.ausize:
1048576 ; 0x0bc: 0x00100000
pvcreate /dev/oracleasm/disks/DATA011
[root@collabn2 ~]# pvcreate /dev/oracleasm/disks/DATA011
Writing physical volume data to disk "/dev/oracleasm/disks/DATA011"
Physical volume "/dev/oracleasm/disks/DATA011" successfully created
[oracle@collabn2 disks]$ dd if=/dev/zero of=/dev/oracleasm/disks/DATA011 bs=512
count=1
1+0 records in
1+0 records out

512 bytes (512 B) copied, 1.8231e-05 s, 28.1 MB/s

ASM instance started
Total System Global Area 1140850688 bytes
Fixed Size
2933400 bytes
Variable Size
1112751464 bytes
ASM Cache
25165824 bytes
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DFILE" cannot be mounted
ORA-15013: diskgroup "DFILE" is already mounted
dd if=/dev/zero of=/dev/oracleasm/disks/DATA012 bs=512 count=1
[root@collabn2 ~]# dd if=/dev/zero of=/dev/oracleasm/disks/DATA012 bs=512 count=
1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000167976 s, 3.0 MB/s
[root@collabn2
DATA011
DATA012
DATA013
[root@collabn2
Disk "DATA011"
Dropping disk:
[root@collabn2
Disk "DATA012"
Dropping disk:

~]# oracleasm listdisks

~]# oracleasm deletedisk DATA011

defines an unmarked device
done
~]# oracleasm deletedisk DATA012
defines an unmarked device
done

[root@collabn2 ~]# oracleasm deletedisk DATA011

Disk "DATA011" defines an unmarked device
Dropping disk: done
[root@collabn2 ~]# oracleasm deletedisk DATA012
Disk "DATA012" defines an unmarked device
Dropping disk: done
[root@collabn2 ~]# oracleasm createdisk DATA011 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@collabn2 ~]# oracleasm createdisk DATA012 /dev/sdb2
Writing disk header: done
Instantiating disk: done
Disk "DATA012" defines an unmarked device
Dropping disk: done
[root@collabn2 ~]# oracleasm createdisk DATA011 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@collabn2 ~]# oracleasm createdisk DATA012 /dev/sdb2
Writing disk header: done
Instantiating disk: done
[root@collabn2 ~]# oracleasm listdisks
DATA011
DATA012
DATA013
[root@collabn2 ~]# oracleasm scandisks
Reloading disk partitions: done

Cleaning any stale ASM disks...

Scanning system for ASM disks...
md_restore -S /u01/app/oracle/restore.sql --silent /tmp/backup_11.bkp --full -G
IKP_TEST
md_restore /u01/app/oracle/product/12.1.0/DATA_DISKGROUP_backup.bkp -G DATA
ASMCMD> md_restore /u01/app/oracle/product/12.1.0/DATA_DISKGROUP_backup.bkp -G D
ATA
Current Diskgroup metadata being restored: DATA
Diskgroup DATA created!
System template INCR XTRANSPORT BACKUPSET modified!
System template VOTINGFILE modified!
System template FLASHBACK modified!
System template ARCHIVELOG modified!
System template OCRFILE modified!
System template ASMPARAMETERFILE modified!
System template FLASHFILE modified!
System template PARAMETERFILE modified!
System template ONLINELOG modified!
System template BACKUPSET modified!
System template KEY_STORE modified!
System template DATAGUARDCONFIG modified!
System template XTRANSPORT modified!
System template XTRANSPORT BACKUPSET modified!
System template TEMPFILE modified!
System template DATAFILE modified!
System template DUMPSET modified!
System template CHANGETRACKING modified!
System template AUDIT_SPILLFILES modified!
System template CONTROLFILE modified!
System template AUTOBACKUP modified!
System template AUTOLOGIN_KEY_STORE modified!
Directory +DATA/ASM re-created!
Directory +DATA/ORCL re-created!
SQL> select status from v$instance;
STATUS
-----------STARTED

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64
bit Production
With the Automatic Storage Management option
[oracle@collabn2 grid]$ asmcmd
ASMCMD> cd data
ASMCMD> ls -ltr
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type Redund Striped Time
ASMCMD> cd orcl

Sys Name
N
ASM/
N
ORCL/

ASMCMD> ls
ASMCMD> cd ..
ASMCMD> cd asm
ASMCMD> ls -ltr
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
ASMCMD> ls
SQL> startup nomount;
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/orcl/spfileorcl.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora
ORA-15056: additional error message
ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora
ORA-15173: entry 'spfileorcl.ora' does not exist in directory 'orcl'
ORA-06512: at line 4
RMAN> restore controlfile from '/u01/app/oracle/s_895903516.266.895903517';
Starting restore at 16-NOV-15
using channel ORA_DISK_1
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/cntrlorcl.dbf
Finished restore at 16-NOV-15
DBID=1423683324
CREATE DISKGROUP DATA EXTERNAL REDUNDANCY DISK'/dev/oracleasm/disks/DATA011' NAM
E DATA011,'/dev/oracleasm/disks/DATA012' NAME DATA012;
'/dev/asm_h002' NAME DATA013
ATTRIBUTE 'au_size'='4M',

srvctl modify database -d orcl -p +DATA/orcl/parameterfilespfile.259.895909591

[oracle@collabn2 dbs]$ srvctl modify database -d orcl -p +DATA/ORCL/PARAMETERFIL
E/spfile.259.895909591
[oracle@collabn2 dbs]$
[oracle@collabn2 dbs]$ . oraenv
ORACLE_SID = [+ASM] ? orcl
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@collabn2 dbs]$ sqlplus / as sysdba
SQL*Plus: Release 12.1.0.2.0 Production on Mon Nov 16 07:49:37 2015
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

RMAN> list backup of database;

List of Backup Sets
===================
BS Key Type LV Size
Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ --------------2
Full
1.27G
DISK
00:00:23
16-NOV-15
BP Key: 2 Status: AVAILABLE Compressed: NO Tag: TAG20151116T060442
Piece Name: +DFILE/ORCL/BACKUPSET/2015_11_16/nnndf0_tag20151116t060442_0
.263.895903483
List of Datafiles in backup set 2
File LV Type Ckp SCN
Ckp Time Name
---- -- ---- ---------- --------- ---1
Full 1681551
16-NOV-15 +DATA/ORCL/DATAFILE/system.258.895902483
3
Full 1681551
16-NOV-15 +DATA/ORCL/DATAFILE/sysaux.257.895902427
4
Full 1681551
16-NOV-15 +DATA/ORCL/DATAFILE/undotbs1.260.895902541
6
Full 1681551
16-NOV-15 +DATA/ORCL/DATAFILE/users.259.895902537
RMAN> restore database;
Starting restore at 16-NOV-15
Starting implicit crosscheck backup at 16-NOV-15
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=29 device type=DISK
Crosschecked 3 objects
Finished implicit crosscheck backup at 16-NOV-15
Starting implicit crosscheck copy at 16-NOV-15
using channel ORA_DISK_1
Finished implicit crosscheck copy at 16-NOV-15
searching for all files in the recovery area
cataloging files...
cataloging done
List of Cataloged Files
=======================
File Name: +DFILE/ORCL/AUTOBACKUP/2015_11_16/s_895903516.266.895903517
File Name: +DFILE/ORCL/ARCHIVELOG/2015_11_16/thread_1_seq_12.267.895904615
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to +DATA/ORCL/DATAFILE/system.258.8
95902483
channel ORA_DISK_1: restoring datafile 00003 to +DATA/ORCL/DATAFILE/sysaux.257.8
95902427
channel ORA_DISK_1: restoring datafile 00004 to +DATA/ORCL/DATAFILE/undotbs1.260
.895902541
channel ORA_DISK_1: restoring datafile 00006 to +DATA/ORCL/DATAFILE/users.259.89
5902537
channel ORA_DISK_1: reading from backup piece +DFILE/ORCL/BACKUPSET/2015_11_16/n
nndf0_tag20151116t060442_0.263.895903483

ORA-01589: must use RESETLOGS or NORESETLOGS option for database open

RMAN> sql 'alter database open resetlogs';
sql statement: alter database open resetlogs
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of sql command on default channel at 11/16/2015 07:56:49
RMAN-11003: failure during parse/execution of SQL statement: alter database open
resetlogs
ORA-01152: file 1 was not restored from a sufficiently old backup
ORA-01110: data file 1: '+DATA/ORCL/DATAFILE/system.261.895910159'
RMAN> recover database;
Starting recover at 16-NOV-15
using channel ORA_DISK_1
starting media recovery
archived log for thread 1 with sequence 11 is already on disk as file +DFILE/ORC
L/ARCHIVELOG/2015_11_16/thread_1_seq_11.264.895903507
archived log for thread 1 with sequence 12 is already on disk as file +DFILE/ORC
L/ARCHIVELOG/2015_11_16/thread_1_seq_12.267.895904615
archived log for thread 1 with sequence 13 is already on disk as file +DFILE/ORC
L/ONLINELOG/group_1.257.895902593
archived log file name=+DFILE/ORCL/ARCHIVELOG/2015_11_16/thread_1_seq_11.264.895
903507 thread=1 sequence=11
archived log file name=+DFILE/ORCL/ARCHIVELOG/2015_11_16/thread_1_seq_12.267.895
904615 thread=1 sequence=12
archived log file name=+DFILE/ORCL/ONLINELOG/group_1.257.895902593 thread=1 sequ
ence=13
media recovery complete, elapsed time: 00:00:01
Finished recover at 16-NOV-15
RMAN> sql 'alter database open resetlogs';
sql statement: alter database open resetlog
********************************************************************************
********************************************************************************
*********
KFED Command
********************************************************************************
********************************************************************************
**********

Hi Friends,
I want to share a situation that happened to me recently.

I closed my database running on ASM and also ASM instance successfully. I moved
my ASM disks to a new server. After the installation Oracle software (Grid Infra
structure and RDBMS) on the new server, I got the following error when I tried
to mount the DATA diskgroup.
ovmdbtest1.asyalocal.com.tr@09:52:05> asmcmd
ASMCMD> mount DATA
ORA-15032: not all alterations performed
ORA-15017: diskgroup DATA cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup
ERROR: OCIStmtExecute)
ASMCMD>

DATA (DBD

Same as;
SQL> alter diskgroup DATA mount;
alter diskgroup DATA mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15017: diskgroup DATA cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup

DATA

I think, one or more of ASM disk s header was corrupted. I checked my disks usi
ng ASMCA utility and it shows the status of 2 disks as PROVISIONED . Similarly, k
fod utility also shows the status of 2 disks as PROVISIONED .
# $GRID_HOME/bin/kfod status=TRUE asm_diskstring= /dev/raw/raw*
TRUE

disk=all dscvgroup=

Disk Size Header Path Disk Group User Group

================================================================================
1: 512000 Mb MEMBER /dev/raw/raw1 DATA ora11g dba
2: 512000 Mb PROVISIONED /dev/raw/raw2 DATA ora11g dba
3: 512000 Mb MEMBER /dev/raw/raw3 DATA ora11g dba
4: 512000 Mb PROVISIONED /dev/raw/raw4 DATA ora11g dba
5: 512000 Mb MEMBER /dev/raw/raw5 DATA ora11g dba
6: 512000 Mb MEMBER /dev/raw/raw6 DATA ora11g dba
ORACLE_SID ORACLE_HOME
================================================================================
+ASM /oracle/grid11g
I closed ASM instance and checked the header information of damaged disk using k
fed utility.
# $GRID_HOME/bin/kfed read /dev/raw/raw2
kfbh.endian:
1
kfbh.hard:
130
kfbh.type:
1
kfbh.datfmt:
1
kfbh.block.blk:
0
kfbh.block.obj:
2147483648
kfbh.check:
945480537
kfbh.fcn.base:
0
kfbh.fcn.wrap:
0
kfbh.spare1:
0
kfbh.spare2:
0
kfdhdb.driver.provstr:ORCLDISK ; 0x000:

; 0x000: 0x01
; 0x001: 0x82
; 0x002: KFBTYP_DISKHEAD
; 0x003: 0x01
; 0x004: blk=0
; 0x008: disk=0
; 0x00c: 0x385ae359
; 0x010: 0x00000000
; 0x014: 0x00000000
; 0x018: 0x00000000
; 0x01c: 0x00000000
length=32

kfdhdb.driver.reserved[0]:
kfdhdb.driver.reserved[1]:
kfdhdb.driver.reserved[2]:
kfdhdb.driver.reserved[3]:
kfdhdb.driver.reserved[4]:
kfdhdb.driver.reserved[5]:
kfdhdb.compat:
kfdhdb.dsknum:
kfdhdb.grptyp:
kfdhdb.hdrsts:
kfdhdb.dskname:
kfdhdb.grpname:
kfdhdb.fgname:
kfdhdb.capname:
kfdhdb.crestmp.hi:
AR=0x7dd
kfdhdb.crestmp.lo:
INS=0x3a
kfdhdb.mntstmp.hi:
R=0x7dd
kfdhdb.mntstmp.lo:
NS=0x3b
kfdhdb.secsize:
kfdhdb.blksize:
kfdhdb.ausize:
kfdhdb.mfact:
kfdhdb.dsksize:
kfdhdb.pmcnt:
kfdhdb.fstlocn:
kfdhdb.altlocn:
kfdhdb.f1b1locn:
kfdhdb.redomirrors[0]:
kfdhdb.redomirrors[1]:
kfdhdb.redomirrors[2]:
kfdhdb.redomirrors[3]:
kfdhdb.dbcompat:
kfdhdb.grpstmp.hi:
AR=0x7dd
kfdhdb.grpstmp.lo:
INS=0x3a
kfdhdb.vfstart:
kfdhdb.vfend:
kfdhdb.spfile:
kfdhdb.spfflg:
kfdhdb.ub4spare[0]:
kfdhdb.ub4spare[1]:
kfdhdb.ub4spare[2]:
kfdhdb.ub4spare[3]:
kfdhdb.ub4spare[4]:
kfdhdb.ub4spare[5]:
kfdhdb.ub4spare[6]:
kfdhdb.ub4spare[7]:
kfdhdb.ub4spare[8]:
kfdhdb.ub4spare[9]:
kfdhdb.ub4spare[10]:
kfdhdb.ub4spare[11]:
kfdhdb.ub4spare[12]:
kfdhdb.ub4spare[13]:
kfdhdb.ub4spare[14]:
kfdhdb.ub4spare[15]:

33686018
33686018
33686018
33686018
33686018
33686018
186646528
0
1
3
DATA_0000
DATA
DATA_0000

;
;
;
;
;
;
;
;
;
;
;
;
;
;
32983952 ;

0x008:
0x00c:
0x010:
0x014:
0x018:
0x01c:
0x020:
0x024:
0x026:
0x027:
0x028:
0x048:
0x068:
0x088:
0x0a8:

0x02020202
0x02020202
0x02020202
0x02020202
0x02020202
0x02020202
0x0b200000
0x0000
KFDGTP_EXTERNAL
KFDHDR_MEMBER
length=9
length=4
length=9
length=0
HOUR=0x10 DAYS=0x1c MNTH=0x2 YE

3911025664 ; 0x0ac: USEC=0x0 MSEC=0x361 SECS=0x11 M

32984464 ; 0x0b0: HOUR=0x10 DAYS=0xc MNTH=0x3 YEA
3973637120 ; 0x0b4: USEC=0x0 MSEC=0x239 SECS=0xd MI
512
4096
1048576
113792
512000
6
1
2
2
0
0
0
0
168820736
32983952

;
;
;
;
;
;
;
;
;
;
;
;
;
;
;

0x0b8:
0x0ba:
0x0bc:
0x0c0:
0x0c4:
0x0c8:
0x0cc:
0x0d0:
0x0d4:
0x0d8:
0x0da:
0x0dc:
0x0de:
0x0e0:
0x0e4:

0x0200
0x1000
0x00100000
0x0001bc80
0x0007d000
0x00000006
0x00000001
0x00000002
0x00000002
0x0000
0x0000
0x0000
0x0000
0x0a100000
HOUR=0x10 DAYS=0x1c MNTH=0x2 YE

3909777408 ; 0x0e8: USEC=0x0 MSEC=0x29e SECS=0x10 M

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;

0x0ec:
0x0f0:
0x0f4:
0x0f8:
0x0fc:
0x100:
0x104:
0x108:
0x10c:
0x110:
0x114:
0x118:
0x11c:
0x120:
0x124:
0x128:
0x12c:
0x130:
0x134:
0x138:

0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000

kfdhdb.ub4spare[16]:
kfdhdb.ub4spare[17]:
kfdhdb.ub4spare[18]:
kfdhdb.ub4spare[19]:
kfdhdb.ub4spare[20]:
kfdhdb.ub4spare[21]:
kfdhdb.ub4spare[22]:
kfdhdb.ub4spare[23]:
kfdhdb.ub4spare[24]:
kfdhdb.ub4spare[25]:
kfdhdb.ub4spare[26]:
kfdhdb.ub4spare[27]:
kfdhdb.ub4spare[28]:
kfdhdb.ub4spare[29]:
kfdhdb.ub4spare[30]:
kfdhdb.ub4spare[31]:
kfdhdb.ub4spare[32]:
kfdhdb.ub4spare[33]:
kfdhdb.ub4spare[34]:
kfdhdb.ub4spare[35]:
kfdhdb.ub4spare[36]:
kfdhdb.ub4spare[37]:
kfdhdb.ub4spare[38]:
kfdhdb.ub4spare[39]:
kfdhdb.ub4spare[40]:
kfdhdb.ub4spare[41]:
kfdhdb.ub4spare[42]:
kfdhdb.ub4spare[43]:
kfdhdb.ub4spare[44]:
kfdhdb.ub4spare[45]:
kfdhdb.ub4spare[46]:
kfdhdb.ub4spare[47]:
kfdhdb.ub4spare[48]:
kfdhdb.ub4spare[49]:
kfdhdb.ub4spare[50]:
kfdhdb.ub4spare[51]:
kfdhdb.ub4spare[52]:
kfdhdb.ub4spare[53]:
kfdhdb.acdb.aba.seq:
kfdhdb.acdb.aba.blk:
kfdhdb.acdb.ents:
kfdhdb.acdb.ub2spare:

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
254613
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
43605

;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;

0x13c:
0x140:
0x144:
0x148:
0x14c:
0x150:
0x154:
0x158:
0x15c:
0x160:
0x164:
0x168:
0x16c:
0x170:
0x174:
0x178:
0x17c:
0x180:
0x184:
0x188:
0x18c:
0x190:
0x194:
0x198:
0x19c:
0x1a0:
0x1a4:
0x1a8:
0x1ac:
0x1b0:
0x1b4:
0x1b8:
0x1bc:
0x1c0:
0x1c4:
0x1c8:
0x1cc:
0x1d0:
0x1d4:
0x1d8:
0x1dc:
0x1de:

0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x0003e295
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x0000
0xaa55

The header information was corrupted. kfed output for the disk affected shows 0x
aa55 on
kfdhdb.acdb.ub2spare field. I backed up the header information using dd command.
# dd if=/dev/raw/raw2 of=/tmp/DATA.dd bs=1M count=10
I got the allocation unit size of damaged disk.
# $GRID_HOME/bin/kfed read /dev/raw/raw2 | grep ausize
Now we can repair damaged header information.
# $GRID_HOME/bin/kfed repair /dev/raw/raw2 aus=1048576
I rechecked the header information of damaged disk using kfed utility. Now,
kfed output for the disk affected shows 0x0000 on kfdhdb.acdb.ub2spare field

# $GRID_HOME/bin/kfed read /dev/raw/raw2

kfbh.endian:
1
kfbh.hard:
130
kfbh.type:
1
kfbh.datfmt:
1
kfbh.block.blk:
0
kfbh.block.obj:
2147483648
kfbh.check:
945480537
kfbh.fcn.base:
0
kfbh.fcn.wrap:
0
kfbh.spare1:
0
kfbh.spare2:
0
kfdhdb.driver.provstr:ORCLDISK ; 0x000:
kfdhdb.driver.reserved[0]:
33686018
kfdhdb.driver.reserved[1]:
33686018
kfdhdb.driver.reserved[2]:
33686018
kfdhdb.driver.reserved[3]:
33686018
kfdhdb.driver.reserved[4]:
33686018
kfdhdb.driver.reserved[5]:
33686018
kfdhdb.compat:
186646528
kfdhdb.dsknum:
0
kfdhdb.grptyp:
1
kfdhdb.hdrsts:
3
kfdhdb.dskname:
DATA_0000
kfdhdb.grpname:
DATA
kfdhdb.fgname:
DATA_0000
kfdhdb.capname:
kfdhdb.crestmp.hi:
32983952
AR=0x7dd
kfdhdb.crestmp.lo:
3911025664
INS=0x3a
kfdhdb.mntstmp.hi:
32984464
R=0x7dd
kfdhdb.mntstmp.lo:
3973637120
NS=0x3b
kfdhdb.secsize:
512
kfdhdb.blksize:
4096
kfdhdb.ausize:
1048576
kfdhdb.mfact:
113792
kfdhdb.dsksize:
512000
kfdhdb.pmcnt:
6
kfdhdb.fstlocn:
1
kfdhdb.altlocn:
2
kfdhdb.f1b1locn:
2
kfdhdb.redomirrors[0]:
0
kfdhdb.redomirrors[1]:
0
kfdhdb.redomirrors[2]:
0
kfdhdb.redomirrors[3]:
0
kfdhdb.dbcompat:
168820736
kfdhdb.grpstmp.hi:
32983952
AR=0x7dd
kfdhdb.grpstmp.lo:
3909777408
INS=0x3a
kfdhdb.vfstart:
0
kfdhdb.vfend:
0
kfdhdb.spfile:
0
kfdhdb.spfflg:
0
kfdhdb.ub4spare[0]:
0
kfdhdb.ub4spare[1]:
0

; 0x000: 0x01
; 0x001: 0x82
; 0x002: KFBTYP_DISKHEAD
; 0x003: 0x01
; 0x004: blk=0
; 0x008: disk=0
; 0x00c: 0x385ae359
; 0x010: 0x00000000
; 0x014: 0x00000000
; 0x018: 0x00000000
; 0x01c: 0x00000000
length=32
; 0x008: 0x02020202
; 0x00c: 0x02020202
; 0x010: 0x02020202
; 0x014: 0x02020202
; 0x018: 0x02020202
; 0x01c: 0x02020202
; 0x020: 0x0b200000
; 0x024: 0x0000
; 0x026: KFDGTP_EXTERNAL
; 0x027: KFDHDR_MEMBER
; 0x028: length=9
; 0x048: length=4
; 0x068: length=9
; 0x088: length=0
; 0x0a8: HOUR=0x10 DAYS=0x1c MNTH=0x2 YE
; 0x0ac: USEC=0x0 MSEC=0x361 SECS=0x11 M
; 0x0b0: HOUR=0x10 DAYS=0xc MNTH=0x3 YEA
; 0x0b4: USEC=0x0 MSEC=0x239 SECS=0xd MI
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;

0x0b8:
0x0ba:
0x0bc:
0x0c0:
0x0c4:
0x0c8:
0x0cc:
0x0d0:
0x0d4:
0x0d8:
0x0da:
0x0dc:
0x0de:
0x0e0:
0x0e4:

0x0200
0x1000
0x00100000
0x0001bc80
0x0007d000
0x00000006
0x00000001
0x00000002
0x00000002
0x0000
0x0000
0x0000
0x0000
0x0a100000
HOUR=0x10 DAYS=0x1c MNTH=0x2 YE

; 0x0e8: USEC=0x0 MSEC=0x29e SECS=0x10 M

;
;
;
;
;
;

0x0ec:
0x0f0:
0x0f4:
0x0f8:
0x0fc:
0x100:

0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000

kfdhdb.ub4spare[2]:
kfdhdb.ub4spare[3]:
kfdhdb.ub4spare[4]:
kfdhdb.ub4spare[5]:
kfdhdb.ub4spare[6]:
kfdhdb.ub4spare[7]:
kfdhdb.ub4spare[8]:
kfdhdb.ub4spare[9]:
kfdhdb.ub4spare[10]:
kfdhdb.ub4spare[11]:
kfdhdb.ub4spare[12]:
kfdhdb.ub4spare[13]:
kfdhdb.ub4spare[14]:
kfdhdb.ub4spare[15]:
kfdhdb.ub4spare[16]:
kfdhdb.ub4spare[17]:
kfdhdb.ub4spare[18]:
kfdhdb.ub4spare[19]:
kfdhdb.ub4spare[20]:
kfdhdb.ub4spare[21]:
kfdhdb.ub4spare[22]:
kfdhdb.ub4spare[23]:
kfdhdb.ub4spare[24]:
kfdhdb.ub4spare[25]:
kfdhdb.ub4spare[26]:
kfdhdb.ub4spare[27]:
kfdhdb.ub4spare[28]:
kfdhdb.ub4spare[29]:
kfdhdb.ub4spare[30]:
kfdhdb.ub4spare[31]:
kfdhdb.ub4spare[32]:
kfdhdb.ub4spare[33]:
kfdhdb.ub4spare[34]:
kfdhdb.ub4spare[35]:
kfdhdb.ub4spare[36]:
kfdhdb.ub4spare[37]:
kfdhdb.ub4spare[38]:
kfdhdb.ub4spare[39]:
kfdhdb.ub4spare[40]:
kfdhdb.ub4spare[41]:
kfdhdb.ub4spare[42]:
kfdhdb.ub4spare[43]:
kfdhdb.ub4spare[44]:
kfdhdb.ub4spare[45]:
kfdhdb.ub4spare[46]:
kfdhdb.ub4spare[47]:
kfdhdb.ub4spare[48]:
kfdhdb.ub4spare[49]:
kfdhdb.ub4spare[50]:
kfdhdb.ub4spare[51]:
kfdhdb.ub4spare[52]:
kfdhdb.ub4spare[53]:
kfdhdb.acdb.aba.seq:
kfdhdb.acdb.aba.blk:
kfdhdb.acdb.ents:
kfdhdb.acdb.ub2spare:

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;
;

0x104:
0x108:
0x10c:
0x110:
0x114:
0x118:
0x11c:
0x120:
0x124:
0x128:
0x12c:
0x130:
0x134:
0x138:
0x13c:
0x140:
0x144:
0x148:
0x14c:
0x150:
0x154:
0x158:
0x15c:
0x160:
0x164:
0x168:
0x16c:
0x170:
0x174:
0x178:
0x17c:
0x180:
0x184:
0x188:
0x18c:
0x190:
0x194:
0x198:
0x19c:
0x1a0:
0x1a4:
0x1a8:
0x1ac:
0x1b0:
0x1b4:
0x1b8:
0x1bc:
0x1c0:
0x1c4:
0x1c8:
0x1cc:
0x1d0:
0x1d4:
0x1d8:
0x1dc:
0x1de:

0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x00000000
0x0000
0x0000

Now, we can mount DATA diskgroup successfully.

I recommend that you should add ASM disk header information backup to your curre

nt backup plan.
I did not take a backup of damaged ASM disk header information before. So how wa
s the recovery
process above? There is a second copy of ASM disk header information. kfed repai
rs the corrupted block from second copy.
If our second copy of the header information was corrupted also then we could r
epair the header information using backup