Multiple databases were upgraded to a new version in our project and our catalog version was older ( RCAT.DBMS_RCVCAT version 11.02.00.04 ) than database version. So we tried upgrade the catalog database
[oracle@orcl19x apps01]$ rman target / catalog rcat/password@orcl11x
Recovery Manager: Release 19.0.0.0.0 - Production on Thu Jul 29 20:45:26 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: ORCL19X1 (DBID=2012336792)
connected to recovery catalog database
PL/SQL package RCAT.DBMS_RCVCAT version 11.02.00.04 in RCVCAT database is too old
RMAN> upgrade catalog;
recovery catalog owner is RCAT
enter UPGRADE CATALOG command again to confirm catalog upgrade
RMAN> upgrade catalog;
error creating modify_grsp_pdb_key_not_null
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-06004: Oracle error from recovery catalog database: ORA-02296: cannot enable (RCAT.) - null values found
RMAN> exit
And this weird error was thrown
recovery catalog is partially upgraded to 19.03.00.00
error creating modify_grsp_pdb_key_not_null
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-06004: Oracle error from recovery catalog database: ORA-02296: cannot enable (RCAT.) - null values found
After deeper analysis for long time, we come to know that we enabled guarantee restore point on the catalog database itself for rollback purpose which was causing this issue. So we dropped the existing GRP
SQL> drop restore point GUA;
Restore point dropped.
We created a normal restore point
SQL> create restore point A;
Restore point created.
After that, catalog was upgraded successfully
[oracle@orcl19x apps01]$ rman target / catalog rcat/password@orcl11x
Recovery Manager: Release 19.0.0.0.0 - Production on Thu Jul 29 21:17:30 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: ORCL19X1 (DBID=2012336792, not open)
connected to recovery catalog database
PL/SQL package RCAT.DBMS_RCVCAT version 11.02.00.04 in RCVCAT database is too old
RMAN> upgrade catalog;
recovery catalog owner is RCAT
enter UPGRADE CATALOG command again to confirm catalog upgrade
RMAN> upgrade catalog;
recovery catalog upgraded to version 19.03.00.00.00
DBMS_RCVMAN package upgraded to version 19.03.00.00
DBMS_RCVCAT package upgraded to version 19.03.00.00.
This also seems to be bug in 12c versions with different errors. But the above error is different w.r.t GRP
Bug 20861957
Bug 19677999 - related to container databases(CDB/PDB)
Solution:
Connect to the catalog database. In my case, orcl11x is the catalog database which throws error during upgrade. If you create a guaranteed restore point in your catalog database, then you may get this bug. Check the table related to problem. In my case the table name is grsp which got created as a result of GRP
SQL> conn rcat/password@orcl11x
Connected.
SQL> select count(*) from grsp;
COUNT(*)
----------
1
SQL> select * from grsp;
DBINC_KEY SITE_KEY
---------- ----------
RSPNAME
--------------------------------------------------------------------------------
CREATION_ RSPTIME FROM_SCN TO_SCN GUA PDB_KEY
--------- --------- ---------- ---------- --- ----------
2 3
GUA
02-AUG-21 964316 964360 YES
Delete the table itself directory because this table is related to guranateed restore point which is not going to affect database. For extra caution, you can delete the null values of the table grsp to avoid further conflicts with respect to this table
SQL> delete from grsp;
1 row deleted.
SQL> commit;
Commit complete.
SQL> select * from grsp;
no rows selected
After delete of the table, i was able to upgrade the catalog to higher version!
[oracle@orcl19x ~]$ rman target / catalog rcat/password@orcl11x
Recovery Manager: Release 19.0.0.0.0 - Production on Tue Aug 10 19:45:41 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: ORCL19X1 (DBID=2012336792)
connected to recovery catalog database
recovery catalog is partially upgraded to 19.03.00.00
RMAN> upgrade catalog;
Oracle error from recovery catalog database: ORA-06550: line 1, column 41:
PLS-00201: identifier 'DBMS_RCVMAN.GETPACKAGEVERSION' must be declared
ORA-06550: line 1, column 34:
PL/SQL: Statement ignored
recovery catalog owner is RCAT
enter UPGRADE CATALOG command again to confirm catalog upgrade
RMAN> upgrade catalog;
recovery catalog is partially upgraded to 19.03.00.00
recovery catalog upgraded to version 19.03.00.00.00
DBMS_RCVMAN package upgraded to version 19.03.00.00
DBMS_RCVCAT package upgraded to version 19.03.00.00.