Tuesday, October 15, 2013

How to Resume an RMAN Duplicate on Failure - Just Re-Run it!

If you find yourself at the with an error message like below during an RMAN Duplicate, don’t worry about it. As I found out from dumb luck, it’s quite easy!
RMAN-03002: failure of recover command at 10/07/2013 15:11:47
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 53302 and starting SCN of 143193979270
The requirement was to clone production, without causing additional load on it, to a new environment that I would use for Golden Gate replication.
 
Database: 11.1.0.7.0 64bit
Server: Solaris 5.10 x86 64bit
Backup Location: Data Domain
 
Let me show the duplicate script first:
RUN
{
        ALLOCATE auxiliary CHANNEL CH01 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";
        ALLOCATE auxiliary CHANNEL CH02 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";
        ALLOCATE auxiliary CHANNEL CH03 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";
        ALLOCATE auxiliary CHANNEL CH04 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";
        ALLOCATE auxiliary CHANNEL CH05 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";
        ALLOCATE auxiliary CHANNEL CH06 TYPE 'SBT_TAPE' parms "ENV=(NB_ORA_POLICY=ORACLE_RAC01_PRD1_FULL,NB_ORA_CLIENT=RAC01,NB_ORA_SCHED=Default-Application-Backup)";

        SET NEWNAME FOR DATAFILE 54 TO '/u03/oradata/RPTPRD/data_01.365.668441285';
        SET NEWNAME FOR DATAFILE 54 TO '/u03/oradata/RPTPRD/data_01.365.668441285';
        . . .
        . . .
        SET NEWNAME FOR DATAFILE 54 TO '/u03/oradata/RPTPRD/data_01.365.668441285';

        DUPLICATE TARGET DATABASE TO RPTPRD
                UNTIL SCN 143197743846
                LOGFILE GROUP 1 ('/r01/oradata/RPTPRD/redo01a.log',
                                '/r02/oradata/RPTPRD/redo01b.log') SIZE 500M REUSE,
                        GROUP 2 ('/r01/oradata/RPTPRD/redo02a.log',
                                '/r02/oradata/RPTPRD/redo02b.log') SIZE 500M REUSE,
                        GROUP 3 ('/r01/oradata/RPTPRD/redo03a.log',
                                '/r02/oradata/RPTPRD/redo03b.log') SIZE 500M REUSE
                NOFILENAMECHECK
                SPFILE
                PARAMETER_VALUE_CONVERT 'PRD','RPTPRD'
                SET CONTROL_FILES='/u01/oradata/RPTPRD/control01.ctl','/u02/oradata/RPTPRD/control02.ctl','/u03/oradata/RPTPRD/control03.ctl'
                SET DB_CREATE_FILE_DEST='/u03/oradata/RPTPRD'
                SET AUDIT_FILE_DEST='/u01/app/oracle/admin/RPTPRD/adump'
                SET DB_RECOVERY_FILE_DEST='/a01/flash_recovery_area'
                set DIAGNOSTIC_DEST='/u01/app/oracle/'
                SET DB_RECOVERY_FILE_DEST_SIZE='20G'
                SET local_listener='(address=(protocol=TCP)(host=apvmrdb01)(port=1529))'
                SET FAL_CLIENT=''
                SET FAL_SERVER=''
                SET log_archive_dest_2=''
                SET dg_broker_start='FALSE'
                SET CLUSTER_DATABASE='FALSE'
                SET log_archive_dest_1='LOCATION=/a01/flash_recovery_area'
                SET db_create_online_log_dest_2='/r02/oradata/RPTPRD'
                SET db_create_online_log_dest_1='/r01/oradata/RPTPRD'
                SET standby_archive_dest='';
}
The actual run error is below.
Starting recover at 07-OCT-13

starting media recovery

unable to find archived log
archived log thread=1 sequence=53302
Oracle Error:
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '/u03/oradata/RPTPRD/system.326.665261191'

released channel: CH01
released channel: CH02
released channel: CH03
released channel: CH04
released channel: CH05
released channel: CH06
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 10/07/2013 15:11:47
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 53302 and starting SCN of 143193979270
I knew that Data Domain had the backups, so it couldn’t be a case of missing pieces. By the way, the failure actually happened because my log_archive_dest_1 parameter was set to a location that didn’t exist on my auxiliary server.
 
So what now? I thought maybe I should drop the database and start from scratch. But, curiosity sometimes works in my favor. I simply restarted the script and bingo! It worked!!
 
Relevant excerpt from duplicate log:
contents of Memory Script:
{
   set until scn  143197743846;
   recover
   clone database
    delete archivelog
   ;
}
executing Memory Script

executing command: SET until clause

Starting recover at 07-OCT-13

starting media recovery

channel CH01: starting archived log restore to default destination
channel CH01: restoring archived log
archived log thread=1 sequence=53302
channel CH01: reading from backup piece al_180608_1_828139086
channel CH02: starting archived log restore to default destination
channel CH02: restoring archived log
archived log thread=2 sequence=46813
channel CH02: restoring archived log
archived log thread=1 sequence=53303
channel CH02: reading from backup piece al_180607_1_828139086
channel CH03: starting archived log restore to default destination
channel CH03: restoring archived log
archived log thread=2 sequence=46814
channel CH03: restoring archived log
archived log thread=2 sequence=46815
channel CH03: reading from backup piece al_180609_1_828139086
channel CH04: starting archived log restore to default destination
channel CH04: restoring archived log
archived log thread=1 sequence=53304
channel CH04: reading from backup piece al_180612_1_828145845
channel CH05: starting archived log restore to default destination
channel CH05: restoring archived log
archived log thread=2 sequence=46816
channel CH05: reading from backup piece al_180611_1_828145845
channel CH06: starting archived log restore to default destination
channel CH06: restoring archived log
archived log thread=1 sequence=53305
channel CH06: reading from backup piece al_180614_1_828153049
channel CH01: piece handle=al_180608_1_828139086 tag=TAG20131006T223805
channel CH01: restored backup piece 1
channel CH01: restore complete, elapsed time: 00:00:25
archived log file name=/a01/flash_recovery_area/1_53302_663959504.dbf thread=1 sequence=53302
channel CH01: starting archived log restore to default destination
channel CH01: restoring archived log
archived log thread=2 sequence=46817
channel CH01: reading from backup piece al_180616_1_828153049
channel CH03: piece handle=al_180609_1_828139086 tag=TAG20131006T223805
channel CH03: restored backup piece 1
channel CH03: restore complete, elapsed time: 00:00:25
channel CH02: piece handle=al_180607_1_828139086 tag=TAG20131006T223805
channel CH02: restored backup piece 1
channel CH02: restore complete, elapsed time: 00:00:40
archived log file name=/a01/flash_recovery_area/2_46813_663959504.dbf thread=2 sequence=0
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/2_46813_663959504.dbf RECID=7 STAMP=828199660
archived log file name=/a01/flash_recovery_area/2_46814_663959504.dbf thread=2 sequence=46814
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/2_46814_663959504.dbf RECID=4 STAMP=828199644
archived log file name=/a01/flash_recovery_area/2_46815_663959504.dbf thread=2 sequence=46815
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/1_53302_663959504.dbf RECID=3 STAMP=828199644
archived log file name=/a01/flash_recovery_area/1_53303_663959504.dbf thread=1 sequence=53303
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/1_53303_663959504.dbf RECID=2 STAMP=828199643
channel CH04: piece handle=al_180612_1_828145845 tag=TAG20131007T003044
channel CH04: restored backup piece 1
channel CH04: restore complete, elapsed time: 00:01:00
archived log file name=/a01/flash_recovery_area/1_53304_663959504.dbf thread=1 sequence=53304
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/2_46815_663959504.dbf RECID=1 STAMP=828199642
channel CH05: piece handle=al_180611_1_828145845 tag=TAG20131007T003044
channel CH05: restored backup piece 1
channel CH05: restore complete, elapsed time: 00:01:00
archived log file name=/a01/flash_recovery_area/2_46816_663959504.dbf thread=2 sequence=46816
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/1_53304_663959504.dbf RECID=5 STAMP=828199657
channel CH01: piece handle=al_180616_1_828153049 tag=TAG20131007T023048
channel CH01: restored backup piece 1
channel CH01: restore complete, elapsed time: 00:01:03
channel CH06: piece handle=al_180614_1_828153049 tag=TAG20131007T023048
channel CH06: restored backup piece 1
channel CH06: restore complete, elapsed time: 00:01:28
archived log file name=/a01/flash_recovery_area/1_53305_663959504.dbf thread=1 sequence=53305
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/2_46816_663959504.dbf RECID=6 STAMP=828199659
archived log file name=/a01/flash_recovery_area/2_46817_663959504.dbf thread=2 sequence=46817
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/1_53305_663959504.dbf RECID=8 STAMP=828199666
channel clone_default: deleting archived log(s)
archived log file name=/a01/flash_recovery_area/2_46817_663959504.dbf RECID=9 STAMP=828199670
media recovery complete, elapsed time: 00:00:08
Finished recover at 07-OCT-13
I’m a big fan of crediting others work, after I a re-ran the duplicate command, I  google’d for “resume rman duplicate” and found April C. Sim’s blog on it!
 
Hope this helps.
 

Cheers!

at
Labels: , , ,

No comments :

Post a Comment

Subscribe to: Post Comments ( Atom )