RMAN : ORA-01861: literal does not match format string

The issue occurred for some of the databases around the month end.
The backup started failing with same message and even connecting to database or catalog, no rman command was working.

$rman target / catalog=<catalog details>

Recovery Manager: Release 10.2.0.4.0 - Production on Tue Dec 1 07:40:36 2015 

Copyright (c) 1982, 2007, Oracle. All rights reserved. 

connected to target database: ********
connected to recovery catalog database 

RMAN> resync catalog; 

starting full resync of recovery catalog 
RMAN-00571: =========================================================== 
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== 
RMAN-00571: =========================================================== 
RMAN-03009: failure of resync command on default channel at 12/01/2015 07:40:55 
ORA-01861: literal does not match format string 

RMAN> 

Issue:

Taken the dump of controlfile with following commands.

SQL> Alter session set tracefile_identifier='dump'; 

SQL> alter session set events 'immediate trace name controlf level 9'; 

The dump showed something like following.

RECID #49661 Recno 1277 Record timestamp 11/30/15 12:45:55 piece #1 copy #1 pool 0 
Backup set key: stamp=897133568, count=46318 
V$RMAN_STATUS: recid=140735545929200, stamp=140735545929192 
Flags: 
Device: DISK 
Handle: *******
Media-Handle: <Handle Name>
Comment: 
Tag: ********
Completion time 11/31/15 01:37:53 


The point to notice here was the date i.e. 31 Nov. This is a bug for database versions 10.2.0.4 and 11.0.2.1 and oracle has patches for them.
But the problem here is existing malicious backup which are not allowing to proceed.

To resolve those following were the step taken.

Check the target database for following.

SQL> select recid, stamp, handle, set_stamp, set_count, piece# 
from v$backup_piece 
where recid in (<recId>);

This can be recid for handle name or anything  from above list to identify the backups taken on a malicious date.

Once we have the details. Take backup of controlfile before proceding futher.

SQL> alter database backup controlfile to '/tmp/backupcontrol.ct' 

SQL> exec sys.dbms_backup_restore.deleteBackupPiece(recid, stamp, 'handle',set_stamp, set_count, piece#); 

Note:- Need to run above for all backup pieces, and it will delete the entries from v$rman_status as well as backup piece from disk.

Now the rman can work without connecting catalog, but the catalog still throws the same error. Even resync catalog too fails with same error.

Later following commands were  run on target database.

NOTE:- you loose all earlier backup information after following activities.

SQL> EXECUTE DBMS_BACKUP_RESTORE.RESETCFILESECTION(19); #delete piece

PL/SQL procedure successfully completed.

SQL> EXECUTE DBMS_BACKUP_RESTORE.RESETCFILESECTION(13); #active piece

PL/SQL procedure successfully completed.

SQL>

Post the activity, there are two options.(you may try both in some cases).

1. Unregister database and register database in catalog database.

2. Connect catalog schema using sqlplus and run following.

SQL> update dbinc set high_do_recid =0 where db_name = '<your target's DB_NAME here>'; 
SQL> commit; 


Now connect to rman using target and catalog, and do a resync catalog.

$rman target / catalog=<catalog details>

RMAN> resync catalog;

Finally this came back to a state where backups can work with catalog using existing scripts.

"ORA-00245: control file backup failed; target is likely on a local file system" Backup fails on oracle 11.2 RAC database

On a multi node RAC database we were occasionally facing issue with RMAN backup.

RMAN> 2> 3> 4> 5>
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of allocate command at 04/25/2013 04:00:30
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of full resync command on default channel at 04/25/2013 04:00:30
ORA-00245: control file backup failed; target is likely on a local file system


The issue is intermittent and does not appear every time, and so we are not sure if this is actually some issue with configuration.

Reason:-

Oracle [Support Note ID 1472171.1] says that snapshot control file should be on shared location.

Solution:-

I logged into RMAN from each instance and checked the configuration

RMAN> show all;

RMAN> .
.
.
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '<oracle_home_dbs>/snapcf_DB01.f'; # default

Thus in different instances I found the value is different as snapcf_DB01.f/snapcf_DB02.f etc, also noticed "#default" at the end.

Though we did not have a shared location for oracle home, the path is similar on our servers, thus making rest of the path same except the DB01.f part.

So at the place of making it on shared location we just gave it a common name, by just giving following command from any one node.

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '<oracle_home_dbs>/snapcf_<SID>.f';

And running at one place makes the change working at all the instances. (its only default that can come different or else it should be same if you have common control files for all the instances).
Now not having the shared snapshot controlfile, but a same name is making it work and issue resolved.

ORA-17627: ORA-01017: during standby creation with RMAN

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 11/17/2011 07:38:34
RMAN-03015: error occurred in stored script Memory Script
RMAN-03009: failure of backup command on p1 channel at 11/17/2011 07:38:34
ORA-17629: Cannot connect to the remote database server
ORA-17627: ORA-01017: invalid username/password; logon denied
ORA-17629: Cannot connect to the remote database server

We need to see first the script we execute to create the standby.

As this is 11g, we have a feature that we can make a standby database using rman duplicate db, even when the backup of primary database is not available.

So to use that feature executed below RMAN commands on target server.

rman target / auxiliary sys/pass@STANDB

run
{
allocate channel p1 type disk;
allocate auxiliary channel sb1 type disk;
duplicate target database for standby from active database;
}

And this ended up with above error.

Two things to be checked in this case:

1. Simply try logging in to each of the database using service name from the other server.(remote login)
from standby - sqlplus sys/passs@PRIME
from primary - sqlplus sys/passs@STANDB

2. For executing this we have to connect both the databases using service name and if the target is connected as / then above error occurs.

Another error message will appear if we connect standby as /, but that states clearly to connect auxiliary database using service name.

So the right way of connecting using service name is

rman target sys/passs@PRIME auxiliary sys/pass@STANDB

RMAN-06091: no channel allocated for maintenance (of an appropriate type)

RMAN-06091: no channel allocated for maintenance (of an appropriate type)
The error message appears when we try to delete some backups. Set of commands could be like below.

RMAN> show retention policy;
RMAN configuration parameters are:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
RMAN> delete obsolete;