ORA-12640, ORA-21561: OID generation failed

Problem Description
In the sqlnet.ora file there is an entry SQLNET.AUTHENTICATION_SERVICES=NONE and now whenever you connect to database using RMAN it fails with following error.
ORA-21561: OID generation failed

Whenever you change the sqlnet.ora entry to SQLNET.AUTHENTICATION_SERVICES=NTS, it fails with error ORA-12640: Authentication adapter initialization failed.

Cause of the Problem
The problem happened due to lower value of Windows SharedSection heap size.

Solution of the Problem
1. Click on start menu on your windows machine and then go to Run.

2. Type regedit in the Run box and press Enter.

3. In the Registry Editor window navigate to
\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\
You will see key name Windows.

4. Right click on windows name and select modify. You should see value data field containing an entry like below.

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,3072,512 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off MaxRequestThreads=16

From the field value data the parameter SharedSection's third value (512) is the size of the desktop heap for each desktop that is associated with a "noninteractive" window station.

Increase the desktop heap size that is modify the third value to 1024, so the values are now listed as 1024, 3072, 1024.

After you modify the entry try to connect to database using RMAN and it should resolve the problem.