Category - RMAN

STEP BY STEP RMAN ACTIVE DUPLICATION OF ORACLE Database - RAC TO RAC 19c

STEP BY STEP RMAN ACTIVE DUPLICATE Database From RAC TO RAC 19c

In this article we will be  using ACTIVE DUPLICATION method to prepare Auxiliary Instance on RAC 19c.

1=> Environment.

We are having below envionment.
Currently only Source side is having completely ready database.

Target Side only have Cluster_Home and Oracle_Home installed, NO DB installed yet



On Source Side

[oracle@racsetn1 dbs]$ /grid_home/app/19c/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       racsetn1                 STABLE
               ONLINE  ONLINE       racsetn2                 STABLE
ora.chad
               ONLINE  ONLINE       racsetn1                 STABLE
               ONLINE  ONLINE       racsetn2                 STABLE
ora.net1.network
               ONLINE  ONLINE       racsetn1                 STABLE
               ONLINE  ONLINE       racsetn2                 STABLE
ora.ons
               ONLINE  ONLINE       racsetn1                 STABLE
               ONLINE  ONLINE       racsetn2                 STABLE
ora.proxy_advm
               OFFLINE OFFLINE      racsetn1                 STABLE
               OFFLINE OFFLINE      racsetn2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 STABLE
      2        ONLINE  ONLINE       racsetn2                 STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 STABLE
      2        ONLINE  ONLINE       racsetn2                 STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 STABLE
      2        ONLINE  ONLINE       racsetn2                 STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racsetn1                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 STABLE
      2        ONLINE  ONLINE       racsetn2                 STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 Started,STABLE
      2        ONLINE  ONLINE       racsetn2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       racsetn1                 STABLE
      2        ONLINE  ONLINE       racsetn2                 STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.ntnt.db
      1        ONLINE  ONLINE       racsetn1                 Open,HOME=/ora_home/
                                                             app/19c/db,STABLE
      2        ONLINE  ONLINE       racsetn2                 Open,HOME=/ora_home/
                                                             app/19c/db,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.racsetn1.vip
      1        ONLINE  ONLINE       racsetn1                 STABLE
ora.racsetn2.vip
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       racsetn1                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       racsetn2                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       racsetn2                 STABLE
--------------------------------------------------------------------------------
[oracle@racsetn1 dbs]$
[oracle@racsetn1 dbs]$ /grid_home/app/19c/grid/bin/crsctl check cluster -all
**************************************************************
racsetn1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racsetn2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

On Target Side (Where DB will be Duplicated).

[oracle@racnoden1 dbs]$
[oracle@racnoden1 dbs]$ /grid_home/app/19c/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.chad
               ONLINE  OFFLINE      racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.net1.network
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.ons
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.proxy_advm
               OFFLINE OFFLINE      racnoden1                STABLE
               OFFLINE OFFLINE      racnoden2                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA_REP.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA_RECO.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                Started,STABLE
      2        ONLINE  ONLINE       racnoden2                Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.qosmserver
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.racnoden1.vip
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.racnoden2.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
--------------------------------------------------------------------------------
[oracle@racnoden1 dbs]$ /grid_home/app/19c/grid/bin/crsctl check cluster -all
**************************************************************
racnoden1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racnoden2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[oracle@racnoden1 dbs]$


2=> Setting Up tnsnames.ora file on Source & Target Side.

We need to create tnsnames.ora file on Source and Target side database hosts as below.

-> Source Side - All Nodes

[oracle@racsetn1 tmp]$ cat $ORACLE_HOME/network/admin/tnsnames.ora
# tnsnames.ora Network Configuration File: /ora_home/app/19c/db/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.

NTNT =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = NTNT.ace2oracle.lab)
    )
  )

TART =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.103)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.104)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TART.ace2oracle.lab)
    )
  )
[oracle@racsetn1 tmp]$

[oracle@racsetn2 ~]$ cat $ORACLE_HOME/network/admin/tnsnames.ora
# tnsnames.ora.racsetn2 Network Configuration File: /ora_home/app/19c/db/network/admin/tnsnames.ora.racsetn2
# Generated by Oracle configuration tools.

NTNT =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = NTNT.ace2oracle.lab)
    )
  )

TART =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.103)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.104)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TART.ace2oracle.lab)
    )
  )

[oracle@racsetn2 ~]$

-> Target Side - All Nodes

[oracle@racnoden1 dbs]$ cat $ORACLE_HOME/network/admin/tnsnames.ora
# tnsnames.ora Network Configuration File: /ora_home/app/19c/db/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.

TART =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racnode-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TART.ace2oracle.lab)(UR=A)
    )
  )

NTNT =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = NTNT.ace2oracle.lab)
    )
  )
[oracle@racnoden1 dbs]$

[oracle@racnoden2 ~]$ cat $ORACLE_HOME/network/admin/tnsnames.ora
# tnsnames.ora Network Configuration File: /ora_home/app/19c/db/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.

TART =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racnode-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TART.ace2oracle.lab)(UR=A)
    )
  )

NTNT =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = NTNT.ace2oracle.lab)
    )
  )
[oracle@racnoden2 ~]$


3=> Setting Up Static Listener on Target Side Cluster_Home on NODE-1 only as below.

[oracle@racnoden1 dbs]$ su - grid
Password:
[grid@racnoden1 ~]$ cat $ORACLE_HOME/network/admin/listener.ora
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))))        # line added by Agent
LISTENER_SCAN3=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3))))        # line added by Agent
LISTENER_SCAN2=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2))))        # line added by Agent
LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))))        # line added by Agent
ASMNET1LSNR_ASM=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))))        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_ASMNET1LSNR_ASM=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_ASMNET1LSNR_ASM=SUBNET        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN1=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN2=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN2=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN3=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN3=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER=SUBNET        # line added by Agent

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = TART.ace2oracle.lab)
      (ORACLE_HOME = /ora_home/app/19c/db)
      (SID_NAME = TART1)
    )
  )

[grid@racnoden1 ~]$ lsnrctl status

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))

STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                25-JAN-2023 20:23:30
Uptime                    0 days 5 hr. 32 min. 56 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /grid_home/app/19c/grid/network/admin/listener.ora
Listener Log File         /grid_home/app/oracle/diag/tnslsnr/racnoden1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.101)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.103)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_DATA_REP" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_FRA_RECO" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_OCR" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "TART.ace2oracle.lab" has 1 instance(s).
  Instance "TART1", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
[grid@racnoden1 ~]$
[grid@racnoden1 ~]$ lsnrctl reload

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 01:56:25

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
The command completed successfully
[grid@racnoden1 ~]$ lsnrctl service

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 01:56:30

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
Services Summary...
Service "TART.ace2oracle.lab" has 1 instance(s).
  Instance "TART1", status UNKNOWN, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0
         LOCAL SERVER
The command completed successfully
[grid@racnoden1 ~]$ lsnrctl status

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 01:56:34

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                25-JAN-2023 20:23:30
Uptime                    0 days 5 hr. 33 min. 9 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /grid_home/app/19c/grid/network/admin/listener.ora
Listener Log File         /grid_home/app/oracle/diag/tnslsnr/racnoden1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
Services Summary...
Service "TART.ace2oracle.lab" has 1 instance(s).
  Instance "TART1", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

[grid@racnoden1 ~]$ ssh racnoden2

[grid@racnoden2 ~]$
[grid@racnoden2 ~]$ cat $ORACLE_HOME/network/admin/listener.ora
LISTENER_SCAN3=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3))))        # line added by Agent
LISTENER_SCAN2=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2))))        # line added by Agent
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))))        # line added by Agent
LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))))        # line added by Agent
ASMNET1LSNR_ASM=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))))        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_ASMNET1LSNR_ASM=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_ASMNET1LSNR_ASM=SUBNET        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN1=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER=SUBNET        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN2=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN2=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN3=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN3=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set

[grid@racnoden2 ~]$
[grid@racnoden2 ~]$ lsnrctl status

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                25-JAN-2023 20:22:34
Uptime                    0 days 5 hr. 34 min. 21 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /grid_home/app/19c/grid/network/admin/listener.ora
Listener Log File         /grid_home/app/oracle/diag/tnslsnr/racnoden2/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.102)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.104)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "+ASM_DATA_REP" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "+ASM_FRA_RECO" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "+ASM_OCR" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@racnoden2 ~]$ lsnrctl services

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 01:57:00

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0 state:ready
         LOCAL SERVER
Service "+ASM_DATA_REP" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0 state:ready
         LOCAL SERVER
Service "+ASM_FRA_RECO" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0 state:ready
         LOCAL SERVER
Service "+ASM_OCR" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0 state:ready
         LOCAL SERVER
The command completed successfully


4=> Copying Password file from Source to Destination on ALL NODES.
-> Get password file from Source as below.

[oracle@racsetn1 tmp]$ su - grid
Password:

[grid@racsetn1 ~]$ asmcmd
ASMCMD>  pwget --dbuniquename NTNT
+DATA/NTNT/PASSWORD/pwdntnt.365.1123698383
ASMCMD>
ASMCMD> pwcopy +DATA/NTNT/PASSWORD/pwdntnt.365.1123698383 /tmp/pwfile
copying +DATA/NTNT/PASSWORD/pwdntnt.365.1123698383 -> /tmp/pwfile

ASMCMD> exit

[grid@racsetn1 ~]$ ls -lrt /tmp/pwfile
-rw-r-----. 1 grid oinstall 2048 Jan 26 02:01 /tmp/pwfile

[grid@racsetn1 ~]$ chmod 777 /tmp/pwfile
[grid@racsetn1 ~]$ logout
[oracle@racsetn1 tmp]$

-> Send password file from Source to TARGET ALL NODES in $ORACLE_HOME/dbs as below , per instancwise.

[oracle@racsetn1 tmp]$ scp /tmp/pwfile racnoden1:/ora_home/app/19c/db/dbs/orapwTART1

oracle@racnoden1's password:
pwfile                                                                                    100% 2048     3.6MB/s   00:00    
[oracle@racsetn1 tmp]$

[oracle@racsetn1 tmp]$ scp /tmp/pwfile racnoden2:/ora_home/app/19c/db/dbs/orapwTART2

oracle@racnoden2's password:
pwfile                                                                                    100% 2048     4.8MB/s   00:00    
[oracle@racsetn1 tmp]$

5=> Test TNSPING and connections as below.

-> TNSPING from Source - ALL Nodes

[oracle@racsetn1 tmp]$
[oracle@racsetn1 tmp]$ tnsping TART

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:05:59

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.103)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.104)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TART.ace2oracle.lab)))
OK (0 msec)
[oracle@racsetn1 tmp]$
[oracle@racsetn1 tmp]$ tnsping NTNT

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:02

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NTNT.ace2oracle.lab)))
OK (10 msec)

[oracle@racsetn1 tmp]$ ssh racsetn2
Last login: Thu Jan 26 01:53:45 2023 from racsetn1.ace2oracle.lab

[oracle@racsetn2 ~]$ tnsping TART

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:20

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racnode-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TART.ace2oracle.lab)))
OK (0 msec)
[oracle@racsetn2 ~]$
[oracle@racsetn2 ~]$
[oracle@racsetn2 ~]$ tnsping NTNT

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:23

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NTNT.ace2oracle.lab)))
OK (0 msec)
[oracle@racsetn2 ~]$ logout
Connection to racsetn2 closed.
[oracle@racsetn1 tmp]$

-> TNSPING from Target - ALL Nodes

[oracle@racnoden1 dbs]$ tnsping TART

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:35

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racnode-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TART.ace2oracle.lab)(UR=A)))
OK (0 msec)
[oracle@racnoden1 dbs]$ tnsping NTNT

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:37

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NTNT.ace2oracle.lab)))
OK (0 msec)

[oracle@racnoden1 dbs]$ ssh racnoden2
[oracle@racnoden2 ~]$
[oracle@racnoden2 ~]$ tnsping NTNT

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:50

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racset-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NTNT.ace2oracle.lab)))
OK (0 msec)
[oracle@racnoden2 ~]$ tnsping TART

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 26-JAN-2023 02:06:53

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/ora_home/app/19c/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racnode-scan)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TART.ace2oracle.lab)(UR=A)))
OK (0 msec)
[oracle@racnoden2 ~]$

6=> Check connection as Auxiliary and Target as below.

-> From Source DB

[oracle@racsetn1 dbs]$ rman target sys/Oracle_4U@NTNT

Recovery Manager: Release 19.0.0.0.0 - Production on Thu Jan 26 02:09:51 2023
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: NTNT (DBID=3125572952)

RMAN>

Recovery Manager complete.


[oracle@racsetn1 dbs]$ rman auxiliary sys/Oracle_4U@TART

Recovery Manager: Release 19.0.0.0.0 - Production on Thu Jan 26 02:10:04 2023
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to auxiliary database (not started)

RMAN>


7=> Prepare Parameter file copy to Target side as below.

[oracle@racsetn1 tmp]$ sqlplus "/as sysdba"

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> create pfile='/tmp/pfile_s.ora' from spfile;

created.

-> Get DB_FILE and LOG_FIEL path from Source DB as below.

[oracle@racsetn1 dbs]$ sqlplus "/as sysdba"

SQL> select member from v$logfile;

MEMBER
--------------------------------------------------------------------------------
+DATA/NTNT/ONLINELOG/group_2.272.1123698529
+FRA/NTNT/ONLINELOG/group_2.789.1123698531
+DATA/NTNT/ONLINELOG/group_1.280.1123698529
+FRA/NTNT/ONLINELOG/group_1.783.1123698531
+DATA/NTNT/ONLINELOG/group_3.279.1123699801
+FRA/NTNT/ONLINELOG/group_3.821.1123699803
+DATA/NTNT/ONLINELOG/group_4.278.1123699813
+FRA/NTNT/ONLINELOG/group_4.851.1123699813

8 rows selected.

SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
+DATA/NTNT/DATAFILE/system.366.1123698405
+DATA/NTNT/DATAFILE/sysaux.358.1123698431
+DATA/NTNT/DATAFILE/undotbs1.363.1123698445
+DATA/NTNT/DATAFILE/users.347.1123698447
+DATA/NTNT/DATAFILE/undotbs2.262.1123699645
+DATA/NTNT/DATAFILE/rman_tspitr.257.1126727007

6 rows selected.
[oracle@racsetn1 dbs]$


8=> Edit parameters as per TARGET DB as below.

[oracle@racnoden1 dbs]$ cat pfile_tart.ora
*.audit_file_dest='/ora_home/app/oracle/admin/TART/adump'
*.audit_trail='db'
*.cluster_database=FALSE
*.compatible='19.0.0'
*.control_files='+DATA_REP/TART/CONTROLFILE/cont01.ctl','+DATA_REP/TART/CONTROLFILE/cont02.ctl'
*.db_block_size=8192
*.db_domain='ace2oracle.lab'
*.db_name='TART'
*.db_recovery_file_dest='+FRA_RECO'
*.db_recovery_file_dest_size=8256m
*.diagnostic_dest='/ora_home/app/oracle'
*.log_archive_format='%t_%s_%r.dbf'
*.nls_language='AMERICAN'
*.nls_territory='AMERICA'
*.open_cursors=300
*.pga_aggregate_target=871m
*.processes=320
*.remote_login_passwordfile='exclusive'
*.sga_target=2612m
TART1.thread=1
TART1.undo_tablespace='UNDOTBS1'
*.log_file_name_convert='+DATA/NTNT','+DATA_REP/TART','+FRA/NTNT','+FRA_RECO/TART'
*.db_file_name_convert='+DATA/NTNT','+DATA_REP/TART'
[oracle@racnoden1 dbs]$


9=>Create Required Directories on Target Side DB as below.

-> Create File system on both nodes

[oracle@racnoden1 dbs]$ mkdir -p /ora_home/app/oracle/admin/TART/adump
[oracle@racnoden1 dbs]$ ssh racnoden2
Last login: Thu Jan 26 02:06:43 2023 from racnoden1.ace2oracle.lab
[oracle@racnoden2 ~]$  mkdir -p /ora_home/app/oracle/admin/TART/adump

-> Create ASM directories as below

[oracle@racnoden1 dbs]$ su - grid
[grid@racnoden1 ~]$ asmcmd
ASMCMD> ls
DATA_REP/
FRA_RECO/
OCR/
ASMCMD> cd DATA_REP
ASMCMD> mkdir TART
ASMCMD> cd TART
ASMCMD> pwd
+DATA_REP/TART
ASMCMD> cd ../..
ASMCMD> ls
DATA_REP/
FRA_RECO/
OCR/
ASMCMD> cd FRA_RECO
ASMCMD> mkdir TART
ASMCMD> cd TART
ASMCMD> pwd
+FRA_RECO/TART
ASMCMD>

10=> Export ORACLE_SID and start database on TARGET Side as NOMOUNT.

[oracle@racnoden1 dbs]$ echo $ORACLE_SID
TART1
[oracle@racnoden1 dbs]$ sqlplus "/as sysdba"

SQL> startup nomount pfile='pfile_tart.ora';
ORACLE instance started.

Total System Global Area 2751463160 bytes
Fixed Size            8900344 bytes
Variable Size          570425344 bytes
Database Buffers     2164260864 bytes
Redo Buffers            7876608 bytes
SQL>

11=> Prepare rman script as below on Source DB

[oracle@racsetn1 dbs]$ cat rman_clone.cmd
run
{
allocate channel src1 type disk;
allocate channel src2 type disk;
allocate channel src3 type disk;
allocate channel src4 type disk;
allocate auxiliary channel aux1 type disk;
allocate auxiliary channel aux2 type disk;
allocate auxiliary channel aux3 type disk;
allocate auxiliary channel aux4 type disk;
allocate auxiliary channel aux5 type disk;
allocate auxiliary channel aux6 type disk;
duplicate target database to TART from active database USING  BACKUPSET ;
}
[oracle@racsetn1 dbs]$

12=> Start DUPLICATING Source database as below .

nohup rman target sys/Oracle_4U@NTNT auxiliary sys/Oracle_4U@TART msglog /ora_home/app/19c/db/dbs/rman_clone2_log cmdfile=/ora_home/app/19c/db/dbs/rman_clone.cmd &

[oracle@racsetn1 dbs]$
[oracle@racsetn1 dbs]$ nohup rman target sys/Oracle_4U@NTNT auxiliary sys/Oracle_4U@TART msglog /ora_home/app/19c/db/dbs/rman_clone2_log cmdfile=/ora_home/app/19c/db/dbs/rman_clone.cmd &
[1] 24234
[oracle@racsetn1 dbs]$ nohup: ignoring input and appending output to ‘nohup.out’

[oracle@racsetn1 dbs]$ tail -100f /ora_home/app/19c/db/dbs/rman_clone2_log

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: NTNT (DBID=3125572952)
connected to auxiliary database: TART (not mounted)

RMAN> run
2> {
3> allocate channel src1 type disk;
4> allocate channel src2 type disk;
5> allocate channel src3 type disk;
6> allocate channel src4 type disk;
7> allocate auxiliary channel aux1 type disk;
8> allocate auxiliary channel aux2 type disk;
9> allocate auxiliary channel aux3 type disk;
10> allocate auxiliary channel aux4 type disk;
11> allocate auxiliary channel aux5 type disk;
12> allocate auxiliary channel aux6 type disk;
13> duplicate target database to TART from active database USING  BACKUPSET ;
14> }
15>
using target database control file instead of recovery catalog
allocated channel: src1
channel src1: SID=27 instance=NTNT1 device type=DISK

allocated channel: src2
channel src2: SID=28 instance=NTNT2 device type=DISK

allocated channel: src3
channel src3: SID=409 instance=NTNT1 device type=DISK

allocated channel: src4
channel src4: SID=160 instance=NTNT2 device type=DISK

allocated channel: aux1
channel aux1: SID=15 device type=DISK

allocated channel: aux2
channel aux2: SID=140 device type=DISK

allocated channel: aux3
channel aux3: SID=268 device type=DISK

allocated channel: aux4
channel aux4: SID=393 device type=DISK

allocated channel: aux5
channel aux5: SID=16 device type=DISK

allocated channel: aux6
channel aux6: SID=141 device type=DISK

Starting Duplicate Db at 26-JAN-23
current log archived

contents of Memory Script:
{
   sql clone "create spfile from memory";
}
executing Memory Script

sql statement: create spfile from memory

contents of Memory Script:
{
   shutdown clone immediate;
   startup clone nomount;
}
executing Memory Script

...
...

Oracle instance shut down

connected to auxiliary database (not started)
Oracle instance started

Total System Global Area    2751463160 bytes

Fixed Size                     8900344 bytes
Variable Size                570425344 bytes
Database Buffers            2164260864 bytes
Redo Buffers                   7876608 bytes
allocated channel: aux1
channel aux1: SID=379 device type=DISK
allocated channel: aux2
channel aux2: SID=13 device type=DISK
allocated channel: aux3
channel aux3: SID=138 device type=DISK
allocated channel: aux4
channel aux4: SID=266 device type=DISK
allocated channel: aux5
channel aux5: SID=267 device type=DISK
allocated channel: aux6
channel aux6: SID=394 device type=DISK

contents of Memory Script:
{
   sql clone "alter system set  db_name =
 ''NTNT'' comment=
 ''Modified by RMAN duplicate'' scope=spfile";
   sql clone "alter system set  db_unique_name =
 ''TART'' comment=
 ''Modified by RMAN duplicate'' scope=spfile";
   shutdown clone immediate;
   startup clone force nomount
   restore clone from service  'NTNT' primary controlfile;
   alter clone database mount;
}
executing Memory Script

sql statement: alter system set  db_name =  ''NTNT'' comment= ''Modified by RMAN duplicate'' scope=spfile

sql statement: alter system set  db_unique_name =  ''TART'' comment= ''Modified by RMAN duplicate'' scope=spfile

Oracle instance shut down

Oracle instance started

Total System Global Area    2751463160 bytes

Fixed Size                     8900344 bytes
Variable Size                570425344 bytes
Database Buffers            2164260864 bytes
Redo Buffers                   7876608 bytes
allocated channel: aux1
channel aux1: SID=379 device type=DISK
allocated channel: aux2
channel aux2: SID=13 device type=DISK
allocated channel: aux3
channel aux3: SID=138 device type=DISK
allocated channel: aux4
channel aux4: SID=266 device type=DISK
allocated channel: aux5
channel aux5: SID=140 device type=DISK
allocated channel: aux6
channel aux6: SID=15 device type=DISK

Starting restore at 26-JAN-23

channel aux1: starting datafile backup set restore
channel aux1: using network backup set from service NTNT
channel aux1: restoring control file
channel aux1: restore complete, elapsed time: 00:00:05
output file name=+DATA_REP/TART/CONTROLFILE/cont01.ctl
output file name=+DATA_REP/TART/CONTROLFILE/cont02.ctl
Finished restore at 26-JAN-23

database mounted
RMAN-05529: warning: DB_FILE_NAME_CONVERT resulted in invalid ASM names; names changed to disk group only.

contents of Memory Script:
{
   sql clone 'alter database flashback off';
   set newname for datafile  1 to
 "+DATA_REP";
   set newname for datafile  2 to
 "+DATA_REP";
   set newname for datafile  3 to
 "+DATA_REP";
   set newname for datafile  4 to
 "+DATA_REP";
   set newname for datafile  5 to
 "+DATA_REP";
   set newname for datafile  7 to
 "+DATA_REP";
   restore
   from  nonsparse   from service
 'NTNT'   clone database
   ;
   sql 'alter system archive log current';
}
executing Memory Script

sql statement: alter database flashback off

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting restore at 26-JAN-23

channel aux1: starting datafile backup set restore
channel aux1: using network backup set from service NTNT
channel aux1: specifying datafile(s) to restore from backup set
channel aux1: restoring datafile 00001 to +DATA_REP
channel aux2: starting datafile backup set restore
channel aux2: using network backup set from service NTNT
channel aux2: specifying datafile(s) to restore from backup set
channel aux2: restoring datafile 00002 to +DATA_REP
channel aux3: starting datafile backup set restore
channel aux3: using network backup set from service NTNT
channel aux3: specifying datafile(s) to restore from backup set
channel aux3: restoring datafile 00003 to +DATA_REP
channel aux4: starting datafile backup set restore
channel aux4: using network backup set from service NTNT
channel aux4: specifying datafile(s) to restore from backup set
channel aux4: restoring datafile 00004 to +DATA_REP
channel aux5: starting datafile backup set restore
channel aux5: using network backup set from service NTNT
channel aux5: specifying datafile(s) to restore from backup set
channel aux5: restoring datafile 00005 to +DATA_REP
channel aux6: starting datafile backup set restore
channel aux6: using network backup set from service NTNT
channel aux6: specifying datafile(s) to restore from backup set
channel aux6: restoring datafile 00007 to +DATA_REP
channel aux2: restore complete, elapsed time: 00:00:06
channel aux4: restore complete, elapsed time: 00:00:04
channel aux5: restore complete, elapsed time: 00:00:05
channel aux6: restore complete, elapsed time: 00:00:03
channel aux1: restore complete, elapsed time: 00:00:10
channel aux3: restore complete, elapsed time: 00:00:09
Finished restore at 26-JAN-23

sql statement: alter system archive log current
current log archived

contents of Memory Script:
{
   restore clone force from service  'NTNT'
           archivelog from scn  2987129;
   switch clone datafile all;
}
executing Memory Script

Starting restore at 26-JAN-23

channel aux1: starting archived log restore to default destination
channel aux1: using network backup set from service NTNT
channel aux1: restoring archived log
archived log thread=1 sequence=61
channel aux2: starting archived log restore to default destination
channel aux2: using network backup set from service NTNT
channel aux2: restoring archived log
archived log thread=1 sequence=62
channel aux3: starting archived log restore to default destination
channel aux3: using network backup set from service NTNT
channel aux3: restoring archived log
archived log thread=1 sequence=63
channel aux4: starting archived log restore to default destination
channel aux4: using network backup set from service NTNT
channel aux4: restoring archived log
archived log thread=2 sequence=65
channel aux5: starting archived log restore to default destination
channel aux5: using network backup set from service NTNT
channel aux5: restoring archived log
archived log thread=2 sequence=66
channel aux1: restore complete, elapsed time: 00:00:06
channel aux2: restore complete, elapsed time: 00:00:06
channel aux3: restore complete, elapsed time: 00:00:07
channel aux4: restore complete, elapsed time: 00:00:06
channel aux5: restore complete, elapsed time: 00:00:04
Finished restore at 26-JAN-23

datafile 1 switched to datafile copy
input datafile copy RECID=13 STAMP=1127096670 file name=+DATA_REP/TART/DATAFILE/system.278.1127096633
datafile 2 switched to datafile copy
input datafile copy RECID=14 STAMP=1127096670 file name=+DATA_REP/TART/DATAFILE/rman_tspitr.280.1127096635
datafile 3 switched to datafile copy
input datafile copy RECID=15 STAMP=1127096670 file name=+DATA_REP/TART/DATAFILE/sysaux.261.1127096635
datafile 4 switched to datafile copy
input datafile copy RECID=16 STAMP=1127096670 file name=+DATA_REP/TART/DATAFILE/undotbs1.257.1127096637
datafile 5 switched to datafile copy
input datafile copy RECID=17 STAMP=1127096670 file name=+DATA_REP/TART/DATAFILE/undotbs2.258.1127096639
datafile 7 switched to datafile copy
input datafile copy RECID=18 STAMP=1127096671 file name=+DATA_REP/TART/DATAFILE/users.259.1127096639

contents of Memory Script:
{
   set until scn  2987359;
   recover
   clone database
    delete archivelog
   ;
}
executing Memory Script

executing command: SET until clause

Starting recover at 26-JAN-23

starting media recovery

archived log for thread 1 with sequence 62 is already on disk as file +FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_62.258.1127096663
archived log for thread 1 with sequence 63 is already on disk as file +FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_63.257.1127096667
archived log for thread 2 with sequence 65 is already on disk as file +FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_65.690.1127096669
archived log for thread 2 with sequence 66 is already on disk as file +FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_66.654.1127096669
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_62.258.1127096663 thread=1 sequence=62
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_65.690.1127096669 thread=2 sequence=65
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_63.257.1127096667 thread=1 sequence=63
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_66.654.1127096669 thread=2 sequence=66
media recovery complete, elapsed time: 00:00:04
Finished recover at 26-JAN-23

contents of Memory Script:
{
   delete clone force archivelog all;
}
executing Memory Script

deleted archived log
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_62.258.1127096663 RECID=2 STAMP=1127096668
Deleted 1 objects

deleted archived log
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_61.259.1127096663 RECID=1 STAMP=1127096667
Deleted 1 objects

deleted archived log
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_1_seq_63.257.1127096667 RECID=3 STAMP=1127096669
Deleted 1 objects

deleted archived log
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_66.654.1127096669 RECID=5 STAMP=1127096670
Deleted 1 objects

deleted archived log
archived log file name=+FRA_RECO/TART/ARCHIVELOG/2023_01_26/thread_2_seq_65.690.1127096669 RECID=4 STAMP=1127096669
Deleted 1 objects

released channel: src1
released channel: src2
released channel: src3
released channel: src4
released channel: aux1
released channel: aux2
released channel: aux3
released channel: aux4
released channel: aux5
released channel: aux6
Oracle instance started

Total System Global Area    2751463160 bytes

Fixed Size                     8900344 bytes
Variable Size                570425344 bytes
Database Buffers            2164260864 bytes
Redo Buffers                   7876608 bytes

contents of Memory Script:
{
   sql clone "alter system set  db_name =
 ''TART'' comment=
 ''Reset to original value by RMAN'' scope=spfile";
   sql clone "alter system reset  db_unique_name scope=spfile";
}
executing Memory Script

sql statement: alter system set  db_name =  ''TART'' comment= ''Reset to original value by RMAN'' scope=spfile

sql statement: alter system reset  db_unique_name scope=spfile
Oracle instance started

Total System Global Area    2751463160 bytes

Fixed Size                     8900344 bytes
Variable Size                570425344 bytes
Database Buffers            2164260864 bytes
Redo Buffers                   7876608 bytes
sql statement: CREATE CONTROLFILE REUSE SET DATABASE "TART" RESETLOGS ARCHIVELOG
  MAXLOGFILES    192
  MAXLOGMEMBERS      3
  MAXDATAFILES     1024
  MAXINSTANCES    32
  MAXLOGHISTORY      292
 LOGFILE
  GROUP     1 ( '+FRA_RECO', '+DATA_REP' ) SIZE 200 M  REUSE,
  GROUP     2 ( '+FRA_RECO', '+DATA_REP' ) SIZE 200 M  REUSE
 DATAFILE
  '+DATA_REP/TART/DATAFILE/system.278.1127096633'
 CHARACTER SET AL32UTF8

sql statement: ALTER DATABASE ADD LOGFILE
 
  INSTANCE 'i2'
  GROUP     3 ( '+DATA_REP', '+FRA_RECO' ) SIZE 200 M  REUSE,
  GROUP     4 ( '+DATA_REP', '+FRA_RECO' ) SIZE 200 M  REUSE

contents of Memory Script:
{
   set newname for tempfile  1 to
 "+DATA_REP";
   switch clone tempfile all;
   catalog clone datafilecopy  "+DATA_REP/TART/DATAFILE/rman_tspitr.280.1127096635",
 "+DATA_REP/TART/DATAFILE/sysaux.261.1127096635",
 "+DATA_REP/TART/DATAFILE/undotbs1.257.1127096637",
 "+DATA_REP/TART/DATAFILE/undotbs2.258.1127096639",
 "+DATA_REP/TART/DATAFILE/users.259.1127096639";
   switch clone datafile all;
}
executing Memory Script

executing command: SET NEWNAME

renamed tempfile 1 to +DATA_REP in control file

cataloged datafile copy
datafile copy file name=+DATA_REP/TART/DATAFILE/rman_tspitr.280.1127096635 RECID=1 STAMP=1127096783
cataloged datafile copy
datafile copy file name=+DATA_REP/TART/DATAFILE/sysaux.261.1127096635 RECID=2 STAMP=1127096783
cataloged datafile copy
datafile copy file name=+DATA_REP/TART/DATAFILE/undotbs1.257.1127096637 RECID=3 STAMP=1127096783
cataloged datafile copy
datafile copy file name=+DATA_REP/TART/DATAFILE/undotbs2.258.1127096639 RECID=4 STAMP=1127096783
cataloged datafile copy
datafile copy file name=+DATA_REP/TART/DATAFILE/users.259.1127096639 RECID=5 STAMP=1127096783

datafile 2 switched to datafile copy
input datafile copy RECID=1 STAMP=1127096783 file name=+DATA_REP/TART/DATAFILE/rman_tspitr.280.1127096635
datafile 3 switched to datafile copy
input datafile copy RECID=2 STAMP=1127096783 file name=+DATA_REP/TART/DATAFILE/sysaux.261.1127096635
datafile 4 switched to datafile copy
input datafile copy RECID=3 STAMP=1127096783 file name=+DATA_REP/TART/DATAFILE/undotbs1.257.1127096637
datafile 5 switched to datafile copy
input datafile copy RECID=4 STAMP=1127096783 file name=+DATA_REP/TART/DATAFILE/undotbs2.258.1127096639
datafile 7 switched to datafile copy
input datafile copy RECID=5 STAMP=1127096783 file name=+DATA_REP/TART/DATAFILE/users.259.1127096639

contents of Memory Script:
{
   Alter clone database open resetlogs;
}
executing Memory Script

database opened
Reenabling controlfile options for auxiliary database
Executing: alter database flashback on
Cannot remove created server parameter file
Finished Duplicate Db at 26-JAN-23

Recovery Manager complete.
 Done                    nohup rman target sys/Oracle_4U@NTNT auxiliary sys/Oracle_4U@TART msglog /ora_home/app/19c/db/dbs/rman_clone2_log cmdfile=/ora_home/app/19c/db/dbs/rman_clone.cmd

13=>As OUR DB is cloned now. Let we create  spfile to ASM DG as below.

SQL> create spfile='+DATA_REP/TART/PARAMETERFILE/spfiletnt.ora' from memory;

File created.

SQL> exit

-> EDIT Below entry on both nodes as below.

[oracle@racnoden1 dbs]$
[oracle@racnoden1 dbs]$ cat init
init.ora        initTATRT1.ora  
[oracle@racnoden1 dbs]$ cat initTATRT1.ora
spfile='+DATA_REP/TART/PARAMETERFILE/spfiletnt.ora'
[oracle@racnoden1 dbs]$
[oracle@racnoden1 dbs]$ ssh racnoden2
Last login: Fri Jan 27 02:24:43 2023 from racnoden1.ace2oracle.lab
[oracle@racnoden2 ~]$
[oracle@racnoden2 ~]$ cd $ORACLE_HOME/dbs
[oracle@racnoden2 dbs]$ cat initTATRT2.ora
spfile='+DATA_REP/TART/PARAMETERFILE/spfiletnt.ora'
[oracle@racnoden2 dbs]$

-> Recycle database as below post removing spfileTART1.ora from NODE-1 $ORACLE_HOME/dbs location.

[oracle@racnoden1 dbs]$ cd $ORACLE_HOME/dbs
[oracle@racnoden1 dbs]$ rm spfileTART1.ora

-> Recycle as below.

[oracle@racnoden1 dbs]$ sqlplus "/as sysdba"

SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.

[oracle@racnoden1 dbs]$ sqlplus "/as sysdba"
SQL> startup
ORACLE instance started.

Total System Global Area 2751463160 bytes
Fixed Size            8900344 bytes
Variable Size          570425344 bytes
Database Buffers     2164260864 bytes
Redo Buffers            7876608 bytes
Database mounted.
Database opened.

14=> Set RAC parameters to ENABLE NODE-2 as below and start DATABASE as Clustered on ALL NODES

[oracle@racnoden1 dbs]$ sqlplus "/as sysdba"

SQL> show parameter pfile

NAME            TYPE     VALUE
------------------------------------ ----------- ------------------------------
spfile             string     +DATA_REP/TART/PARAMETERFILE/spfiletnt.ora

SQL> alter system set undo_tablespace=UNDOTBS2 sid='TART2' scope=spfile;

SQL> alter system set instance_number=1 sid='TART1' scope=spfile;
SQL> alter system set instance_number=2 sid='TART2' scope=spfile;

SQL> alter system set instance_name='TART1' sid='TART1' scope=spfile;
SQL> alter system set instance_name='TART2' sid='TART2' scope=spfile;

SQL> alter system set thread=1 sid='TART1' scope=spfile;
SQL> alter system set thread=2 sid='TART2' scope=spfile;

SQL> alter system set cluster_database=TRUE scope=spfile;
SQL> alter system set remote_listener='racnode-scan:1521' scope=spfile;
System altered.

SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 2751463160 bytes
Fixed Size            8900344 bytes
Variable Size          570425344 bytes
Database Buffers     2164260864 bytes
Redo Buffers            7876608 bytes
Database mounted.
Database opened.

-> Start database on ALL NOES as .

[oracle@racnoden1 dbs]$ srvctl start database -d TART

[oracle@racnoden1 dbs]$ /grid_home/app/19c/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.chad
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.net1.network
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.ons
               ONLINE  ONLINE       racnoden1                STABLE
               ONLINE  ONLINE       racnoden2                STABLE
ora.proxy_advm
               OFFLINE OFFLINE      racnoden1                STABLE
               OFFLINE OFFLINE      racnoden2                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA_REP.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA_RECO.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                Started,STABLE
      2        ONLINE  ONLINE       racnoden2                Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       racnoden1                STABLE
      2        ONLINE  ONLINE       racnoden2                STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.qosmserver
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.racnoden1.vip
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.racnoden2.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       racnoden1                STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       racnoden2                STABLE
ora.tart.db
      1        ONLINE  ONLINE       racnoden1                Open,HOME=/ora_home/app/19c/db,STABLE
      2        ONLINE  ONLINE       racnoden2                Open,HOME=/ora_home/app/19c/db,STABLE
--------------------------------------------------------------------------------

15=> Change passwordfile location at OCR level as below.

-> Get current OCR level password info

[oracle@racnoden1 dbs]$ srvctl config database -d TART
Database unique name: TART
Database name:
Oracle home: /ora_home/app/19c/db
Oracle user: oracle
Spfile: +DATA_REP/TART/PARAMETERFILE/spfiletnt.ora
Password file:                                                                             <<<< NOT AVAILABLE
Domain: ace2oracle.lab
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA_REP
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: oinstall
OSOPER group: oper
Database instances: TART1,TART2
Configured nodes: racnoden1,racnoden2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed

-> As we copied password file on dbs locations during setup as below.

[oracle@racnoden1 dbs]$ ls -lrt orapwTART1
-rw-r-----. 1 oracle oinstall 2048 Jan 26 02:03 orapwTART1

[oracle@racnoden1 dbs]$ ls -lrt orapwTART1
-rw-r-----. 1 oracle oinstall 2048 Jan 26 02:03 orapwTART1

->Copy to temp location to access for other user
[oracle@racnoden1 dbs]$ cp orapwTART1 /tmp/orapwTART1

-> Swith to grid and move to ASM as below

[oracle@racnoden1 dbs]$ su - grid
[grid@racnoden1 ~]$ asmcmd
ASMCMD> ls
DATA_REP/
FRA_RECO/
OCR/
ASMCMD> cd DATA_REP
ASMCMD> cd TART
ASMCMD> ls
CONTROLFILE/
DATAFILE/
ONLINELOG/
PARAMETERFILE/
TEMPFILE/
ASMCMD> mkdir PASSWORD
ASMCMD> cd PASSWORD
ASMCMD> pwd
+DATA_REP/TART/PASSWORD
ASMCMD> pwcopy --dbuniquename TART /tmp/orapwTART1 +DATA_REP/TART/PASSWORD/pwdtart
copying /tmp/orapwTART1 -> +DATA_REP/TART/PASSWORD/pwdtart
ASMCMD-9453: failed to register password file as a CRS resource
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type      Redund  Striped  Time             Sys  Name
PASSWORD  UNPROT  COARSE   JAN 27 02:00:00  N    pwdtart => +DATA_REP/TART/PASSWORD/pwdtart.267.1127182975
PASSWORD  UNPROT  COARSE   JAN 27 02:00:00  Y    pwdtart.267.1127182975
ASMCMD> pwd
+DATA_REP/TART/PASSWORD
ASMCMD> exit
[grid@racnoden1 ~]$ logout

-> Login to oracle user and register with CRS as below
[oracle@racnoden1 dbs]$
[oracle@racnoden1 dbs]$ srvctl modify database -d TART -pwfile +DATA_REP/TART/PASSWORD/pwdtart
[oracle@racnoden1 dbs]$ srvctl config database -d TART
Database unique name: TART
Database name:
Oracle home: /ora_home/app/19c/db
Oracle user: oracle
Spfile: +DATA_REP/TART/PARAMETERFILE/spfiletnt.ora
Password file: +DATA_REP/TART/PASSWORD/pwdtart                             <<<< INFORMATION available now.
Domain: ace2oracle.lab
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA_REP
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: oinstall
OSOPER group: oper
Database instances: TART1,TART2
Configured nodes: racnoden1,racnoden2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed

-> Move/remove password file from Both nodes from DBS location

[oracle@racnoden1 dbs]$ ls -lrt ora*
-rw-r-----. 1 oracle oinstall 2048 Jan 26 02:03 orapwTART1
[oracle@racnoden1 dbs]$ mv orapwTART1 orapwTART1_OLD
[oracle@racnoden1 dbs]$ ssh racnoden2
Last login: Fri Jan 27 02:06:51 2023 from racnoden1.ace2oracle.lab
[oracle@racnoden2 ~]$ cd $ORACLE_HOME/dbs
[oracle@racnoden2 dbs]$ mv orapwTART2 orapwTART2_OLD

-> Test password as below.

[oracle@racnoden2 dbs]$ sqlplus sys/Oracle_4U@TART as sysdba
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
[oracle@racnoden2 dbs]$


16=> Check database Health as below.

[oracle@racnoden2 dbs]$ sqlplus "/as sysdba"

SQL> select name,open_mode,database_role,cdb from gv$database;

NAME      OPEN_MODE           DATABASE_ROLE    CDB
--------- -------------------- ---------------- ---
TART      READ WRITE           PRIMARY        NO
TART      READ WRITE           PRIMARY        NO


SQL> select * from v$logfile;


    GROUP# STATUS     TYPE    MEMBER                         IS_     CON_ID
---------- ---------- ------- -------------------------------------------------- --- ----------
     2          ONLINE  +FRA_RECO/TART/ONLINELOG/group_2.259.1127096787     YES          0
     1          ONLINE  +FRA_RECO/TART/ONLINELOG/group_1.257.1127096787     YES          0
     3          ONLINE  +DATA_REP/TART/ONLINELOG/group_3.260.1127096749     NO          0
     3          ONLINE  +FRA_RECO/TART/ONLINELOG/group_3.690.1127096749     NO          0
     4          ONLINE  +DATA_REP/TART/ONLINELOG/group_4.276.1127096769     NO          0
     4          ONLINE  +FRA_RECO/TART/ONLINELOG/group_4.654.1127096769     NO          0

6 rows selected.

SQL> SELECT FILE_NAME FROM DBA_DATA_FILES;

FILE_NAME
----------------------------------------------------------------------------------------
+DATA_REP/TART/DATAFILE/system.278.1127096633
+DATA_REP/TART/DATAFILE/sysaux.261.1127096635
+DATA_REP/TART/DATAFILE/undotbs1.257.1127096637
+DATA_REP/TART/DATAFILE/users.259.1127096639
+DATA_REP/TART/DATAFILE/undotbs2.258.1127096639
+DATA_REP/TART/DATAFILE/rman_tspitr.280.1127096635

6 rows selected.

SQL> SELECT FILE_NAME FROM DBA_TEMP_FILES;

FILE_NAME
----------------------------------------------------------------------------------------
+DATA_REP/TART/TEMPFILE/temp.277.1127096815

SQL> SHOW PARAMETER CONTROL

NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
control_files                 string     +DATA_REP/TART/CONTROLFILE/cont01.ctl,
                                                  +DATA_REP/TART/CONTROLFILE/cont02.ctl
control_management_pack_access         string     DIAGNOSTIC+TUNING

SQL> select DBID,name,DBID,CONTROLFILE_TYPE from gv$database;

      DBID NAME        DBID CONTROL
---------- --------- ---------- -------
3144695979 TART      3144695979 CURRENT
3144695979 TART      3144695979 CURRENT

SQL> exit

17=>Remove SID_LIST_LISTENER as we set in above steps while creation.

[grid@racnoden1 ~]$ lsnrctl status

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 27-JAN-2023 02:31:55

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                27-JAN-2023 01:35:53
Uptime                    0 days 0 hr. 56 min. 1 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /grid_home/app/19c/grid/network/admin/listener.ora
Listener Log File         /grid_home/app/oracle/diag/tnslsnr/racnoden1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.101)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.103)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_DATA_REP" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_FRA_RECO" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_OCR" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "TART.ace2oracle.lab" has 2 instance(s).
  Instance "TART1", status UNKNOWN, has 1 handler(s) for this service...
  Instance "TART1", status READY, has 1 handler(s) for this service...
The command completed successfully

[grid@racnoden1 ~]$ vi /grid_home/app/19c/grid/network/admin/listener.ora

[grid@racnoden1 ~]$ cat  /grid_home/app/19c/grid/network/admin/listener.ora
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))))        # line added by Agent
LISTENER_SCAN3=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3))))        # line added by Agent
LISTENER_SCAN2=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2))))        # line added by Agent
LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))))        # line added by Agent
ASMNET1LSNR_ASM=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))))        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_ASMNET1LSNR_ASM=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_ASMNET1LSNR_ASM=SUBNET        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN1=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN2=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN2=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN3=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN3=OFF        # line added by Agent - Disabled by Agent because REMOTE_REGISTRATION_ADDRESS is set
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON        # line added by Agent
VALID_NODE_CHECKING_REGISTRATION_LISTENER=SUBNET        # line added by Agent

#SID_LIST_LISTENER =
#  (SID_LIST =
#    (SID_DESC =
#      (GLOBAL_DBNAME = TART.ace2oracle.lab)
#      (ORACLE_HOME = /ora_home/app/19c/db)
#      (SID_NAME = TART1)
#    )
#  )
[grid@racnoden1 ~]$

[grid@racnoden1 ~]$ lsnrctl reload

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 27-JAN-2023 02:32:32

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
The command completed successfully

[grid@racnoden1 ~]$ lsnrctl status

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 27-JAN-2023 02:33:52

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                27-JAN-2023 01:35:53
Uptime                    0 days 0 hr. 57 min. 58 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /grid_home/app/19c/grid/network/admin/listener.ora
Listener Log File         /grid_home/app/oracle/diag/tnslsnr/racnoden1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.101)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.103)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_DATA_REP" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_FRA_RECO" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_OCR" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "TART.ace2oracle.lab" has 1 instance(s).
  Instance "TART1", status READY, has 1 handler(s) for this service...
The command completed successfully