Saturday, July 17, 2010

How to manually install Oracle Data Mining in 11g

There are some substantial changes with the Data Mining option in 11g than the previous versions. In 11g, no DMSYS schema exists. Also it has a tight integration with Oracle Database. Data Mining metadata and PL/SQL packages have been migrated from DMSYS to SYS schema. Moreover there is no longer an 'odm' directory in the ORACLE_HOME.

If you look into further details, you will notice that there is no entry for Data Mining in the DBA_REGISTRY but there is a value in V$OPTION:
SQL> col value format a30
SQL> col parameter format a30
SQL> select * from v$option where PARAMETER = 'Data Mining';

PARAMETER VALUE
------------------------------ ------------------------------
Data Mining TRUE
In general, the option is installed with the use of the Database Configuration Assistant (dbca).

However if there is a need to install Data Mining manually, then you need to perform following steps.

1) Go to the $ORACLE_HOME/admin directory of the RDBMS instance where you want to install Data Mining.

2) Ensure that your ORACLE_HOME and ORACLE_SID environmental variable setup is correct.
On unix/linux issue,
$ echo $ORACLE_HOME
$ echo $ORACLE_SID


3) Login as SYS and run catodm.sql, dbmsodm.sql, prvtodm.plb scripts from SQL*Plus:
SQL> conn / as sysdba
SQL> @catodm.sql
SQL> @dbmsodm.sql
SQL> @prvtodm.plb

How to manually install Data Mining in Oracle 10g

Manually Install Oracle Data Mining for RDBMS 10.1.x
Step 01: Ensure that your ORACLE_HOME and ORACLE_SID environmental variable setup is correct.
On unix/linux issue,
$ echo $ORACLE_HOME
$ echo $ORACLE_SID


Step 02: Start Sql*plus and connect with sys user as sysdba privilege.
$ sqlplus /nolog
SQL> conn / as sysdba

Step 03: Execute dminst.sql, odmpatch.sql and utlrp.sql scripts like below.

On Unix - Linux,
SQL> run $ORACLE_HOME/dm/admin/dminst.sql SYSAUX TEMP
SQL> run $ORACLE_HOME/dm/admin/odmpatch.sql
SQL> run $ORACLE_HOME/dm/admin/utlrp.sql
On Windows,
SQL> start %ORACLE_HOME%\dm\admin\dminst.sql SYSAUX TEMP 
SQL> start %ORACLE_HOME%\dm\admin\odmpatch.sql
SQL> start %ORACLE_HOME%\dm\admin\utlrp.sql
Step 04: Ensure that 'Oracle Data Mining' is at Valid status in dba_registry by executing the following query,

SQL> select COMP_NAME,VERSION,STATUS from dba_registry where COMP_NAME='Oracle Data Mining';

Manually Install Oracle Data Mining for RDBMS 10.2.x
Step 01: Ensure that your ORACLE_HOME and ORACLE_SID environmental variable setup is correct.
On unix/linux issue,
$ echo $ORACLE_HOME
$ echo $ORACLE_SID


Step 02: Start Sql*plus and connect with sys user as sysdba privilege.
$ sqlplus /nolog
SQL> conn / as sysdba

Step 03: Execute dminst.sql, odmpatch.sql and utlrp.sql scripts like below.

On Unix/Linux issue,
SQL> run $ORACLE_HOME/rdbms/admin/dminst.sql SYSAUX TEMP
SQL> run $ORACLE_HOME/rdbms/admin/odmpatch.sql
SQL> run $ORACLE_HOME/rdbms/admin/utlrp.sql

On Windows,
SQL> start %ORACLE_HOME%\rdbms\admin\dminst.sql SYSAUX TEMP 
SQL> start %ORACLE_HOME%\rdbms\admin\odmpatch.sql
SQL> start %ORACLE_HOME%\rdbms\admin\utlrp.sql

Step 04: Ensure that 'Oracle Data Mining' is at Valid status in dba_registry by executing the following query,

SQL> select COMP_NAME,VERSION,STATUS from dba_registry where COMP_NAME='Oracle Data Mining';

Friday, July 16, 2010

How to remove data mining option from database

Oracle data mining is a licensed database option that is generally installed by default. However if you want to uninstall it the following procedures will help you.

Uninstall for database versions 9iR2 (9.2.x) and 10gR1 (10.1.x)
From the Oracle Universal Installer (OUI) page, you should be able to deinstall Data Mining from the 'Deinstall' icon. This is a supported way.

If OUI does not work for some reasons, you may do the following to deinstall ODM.

1. Start SQLPLUS and connect with user sys as sysdba
$ sqlplus / as sysdba

2. Remove ODM repositories from the database.

2.1 For Database release 9iR2 (9.2.x).

- Drop ODM, ODM_MTR schemas.

SQL> DROP USER ODM_MTR CASCADE;
SQL> DROP USER ODM CASCADE;


- Drop ODM public synonyms.
SQL> set heading off
SQL> spool /home/oracle/drop_odm_synonyms.sql
SQL> select 'Drop public synonym ' || SYNONYM_NAME || ' ;' from DBA_SYNONYMS where TABLE_OWNER = 'ODM';
SQL> spool off
SQL> @/home/oracle/drop_odm_synonyms.sql
Note that 264 synonyms should be selected and dropped.

2.2 For Database release 10gR1 (10.1.x)

- Drop DMSYS schema.

SQL> DROP USER DMSYS CASCADE;

- Drop DMSYS public synonyms.
SQL> set heading off 
SQL> spool /home/oracle/drop_dmsys_synonyms.sql
SQL> select 'Drop public synonym '" || SYNONYM_NAME || '" ;' from DBA_SYNONYMS where TABLE_OWNER = 'DMSYS';
SQL> spool off
SQL> @/home/oracle/drop_dmsys_synonyms.sql
Note that 568 synonyms should be selected and dropped.

Issue the following DELETE statement.

SQL> DELETE FROM exppkgact$ WHERE SCHEMA='DMSYS';

3. Update the DBA registry :

Before updating the DBA registry, first make sure Data Mining is registered in your database by running the following:
SQL> connect / as sysdba; 
SQL> select comp_id, version, status from dba_registry;
If ODM (Data Mining) is returned by the above query than execute the following to remove Data Mining from the DBA registry:

SQL> exec dbms_registry.removed('ODM');

For UNIX Platform the following steps allow to remake Oracle executable without DM libraries.
1. Shutdown database
2. cd $ORACLE_HOME/rdbms/lib
3. make -f ins_rdbms.mk dm_off
4. make -f ins_rdbms.mk ioracle
5. startup the database

The sqlplus banner should no longer display the Data Mining option and v$option should show 'Oracle Data Mining' as false on a Unix system based install.

Please note that it is possible synonyms may exist for Data Mining objects. If any exist, remove them to complete the removal.

Uninstall for Database version 10gR2 (10.2.x)

With Database Release 10gR2, the only supported way to remove the Data Mining option is to use the Oracle Universal Installer.

The steps would be,

- Start The Oracle Universal Installer.

-In Installer , click on the "Deinstall" Button.

- Find you RDBMS home and expand it.

- Under it you will find 'Oracle Database 10g 10.2.0.x' . Expand it.

- Under it you will find another folder also called Oracle Database 10g 10.2.0.1'. Expand it.

- In the bottom of this tree you will find 'Oracle Data Mining RDBMS Files 10.2.0.1'. This is what you need to remove.

- This Data Mining uninstall process should relink oracle executable with dm_off target, which should be reflected in sys.v$option view and SQLPlus banner.

Note that on some platform (e.g: HP UX) you may notice that after selecting the 'Oracle Data Mining RDBMS Files 10.2.0.1' and pressing OK you will be asked if you're sure you want to remove basically everything, including "Oracle Database 10g" itself. If this is the case stop deinstall process and do the following:

On unix and linux platform,
1. Shutdown database
2. cd $ORACLE_HOME/rdbms/lib
3. make -f ins_rdbms.mk dm_off
4. make -f ins_rdbms.mk ioracle
5. startup database.
On all platforms,
Update the database registry to remove the Data Mining component from it by performing the following :

Start SQL*Plus and log in as SYSDBA and execute dbms_registry.removed('ODM')

SQL> conn / as sysdba
SQL> exec dbms_registry.removed('ODM');

The sqlplus banner should no longer display the Data Mining option and v$option should show 'Oracle Data Mining' as false on a Unix system based install.

Uninstall for Database version 11gR1 (11.1.x)

With 11g, Oracle Data Mining is installed as part of the database installation. Data Mining is now part of Oracle binary and SYS metadata.

The Data Mining option cannot be removed but it can be disabled:

On Linux/Unix platform as below:
1. Shutdown database 
2. cd $ORACLE_HOME/rdbms/lib
3. make -f ins_rdbms.mk dm_off
4. make -f ins_rdbms.mk ioracle
5. startup database

On Windows platform as below:

Use the remove option from the Universal Installer and select the Data Mining under Enterprise Edition.

To verify the option have been disabled:

- Start sqlplus
$ sqlplus /nolog

- Connect as DBA user
SQL> conn / as sysdba

- Query the sys.v$option using :
SQL> Select parameter, value from v$option where parameter ='Data Mining';

If the returned value for column "value" is FALSE, then the option have been disabled.

Expdp fails with PLS-00201: identifier DMSYS.DBMS_DM_MODEL_EXP must be declared

Problem Description
Oracle data pump export fails with error "PLS-00201: identifier 'DMSYS.DBMS_DM_MODEL_EXP' must be declared ORA-06550:" like below.
ORA-39125: Worker unexpected fatal error in KUPW$WORKER.
GET_TABLE_DATA_OBJECTS while calling DBMS_METADATA.FETCH_XML_CLOB []
ORA-31642: the following SQL statement fails:
BEGIN
DMSYS.DBMS_DM_MODEL_EXP.SCHEMA_CALLOUT('CLUS1PROD1',0,1,'10.01.00.03.00');
END;
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 86
ORA-06512: at "SYS.DBMS_METADATA", line 872
ORA-06550: line 1, column 7:
PLS-00201: identifier 'DMSYS.DBMS_DM_MODEL_EXP' must be declared ORA-06550: line 1, column 7:
PL/SQL: Statement ignored

Cause of the Problem
The problem arises because DMSYS schema objects have been accidentally removed or DMSYS schema objects have Invalid status or DMSYS schema has been dropped.

Solution of the Problem
Case 01: DMSYS has been dropped
1. Start SQLPlus.
$ sqlplus /nolog

2. Connect with user SYS as SYSDBA and issue the following commands:
SQL> conn / as sysdba
SQL> DELETE FROM exppkgact$ WHERE SCHEMA='DMSYS';
SQL> exit;
3. Run export jobs.

Case 02: DMSYS schema objects have been accidentally removed / have invalid objects
1. Start SQLPlus and connect with user SYS as SYSDBA
$ sqlplus / as sysdba

2.
i) If Database is version 10.1.0.x do the following steps:
 SQL> run $ORACLE_HOME/dm/admin/dminst.sql SYSAUX TEMP $ORACLE_HOME/dm/admin/ 
SQL> run $ORACLE_HOME/dm/admin/odmpatch.sql (if the database is at a patch level)
SQL> run $ORACLE_HOME/rdbms/admin/utlrp.sql
ii) If Database is version 10.2.0.x do the following steps:
 SQL> run $ORACLE_HOME/rdbms/admin/dminst.sql SYSAUX TEMP $ORACLE_HOME/rdbms/admin/ 
SQL> run $ORACLE_HOME/rdbms/admin/odmpatch.sql
SQL> run $ORACLE_HOME/rdbms/admin/utlrp.sql
3. Ensure that 'Oracle Data Mining' is at valid status in dba_registry using,
SQL> select COMP_NAME,VERSION,STATUS from dba_registry where COMP_NAME='Oracle Data Mining';

4. Run the export jobs.

Export DataPump fails with ORA-39125 while Calling DMSYS.DBMS_DM_MODEL_EXP.SCHEMA_CALLOUT

Problem Description
While invoking oracle datapump schema level export it fails with error
"ORA-39125: Worker unexpected fatal error in KUPW$WORKER.GET_TABLE_DATA_OBJECTS
while calling DBMS_METADATA.FETCH_XML_CLOB []
ORA-31642: the following SQL statement fails:
BEGIN DMSYS.DBMS_DM_MODEL_EXP.SCHEMA_CALLOUT" like below.
#> expdp userid=system/password DIRECTORY=mydir DUMPFILE=expdp_scott.dmp LOGFILE=expdp_scott.log SCHEMAS=scott

Export: Release 10.2.0.1.0 - 64bit Production on Sunday, 27 July, 2008 10:00:38

Copyright (c) 2003, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
Starting "TEST"."JOB1": userid=system/***** DIRECTORY=mydir DUMPFILE=expdp_scott.dmp LOGFILE=expdp_scott.log SCHEMAS=scott
Estimate in progress using BLOCKS method...
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
ORA-39125: Worker unexpected fatal error in KUPW$WORKER.GET_TABLE_DATA_OBJECTS
while calling DBMS_METADATA.FETCH_XML_CLOB []
ORA-31642: the following SQL statement fails:
BEGIN DMSYS.DBMS_DM_MODEL_EXP.SCHEMA_CALLOUT('SCOTT',0,1,'10.01.00.02.00'); END;
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 86
ORA-06512: at "SYS.DBMS_METADATA", line 872
ORA-04063: package body "DMSYS.DBMS_DM_UTIL" has errors
ORA-06508: PL/SQL: could not find program unit being called

Cause of the Problem
As the line "ORA-04063: package body "DMSYS.DBMS_DM_UTIL" has errors" appears the problem happened due to invalid state of the package DMSYS.DBMS_DM_UTIL. Package DMSYS.DBMS_DM_UTIL is used by the Oracle Data Mining option. You can verify the invalid objects in oracle by following query,
SQL> set lines 200
SQL> select status,
object_id,
object_type,
owner||'.'||object_name "OWNER.OBJECT"
from dba_objects
where status != 'VALID'
order by 4,2;

Solution of the Problem
Step 01: Log in to database as dmsys user and run the script dmutil.plb to re-create the invalid package.
SQL> CONNECT dmsys/dmsys
Connected.

SQL> @$ORACLE_HOME/dm/admin/dmutil.plb
Package created.
Package created.

Step 02: Run the script $ORACLE_HOME/rdbms/admin/utlrp.sql to recompile the invalid objects.
SQL> @$ORACLE_HOME/rdbms/admin/utlrp.sql

Step 03: Delete the Export DataPump log file and dump file of the previous failed attempt. Then re-run the export DataPump operation.

Tuesday, July 13, 2010

New features in Oracle 11g Data Guard

Oracle 11g already have two releases. Oracle 11g Release 1 (11.1g) and Oracle 11g Release 2(11.2g). In this post I will specify new features available to Oracle data guard 11g separately.

New Features in Oracle Data Guard 11.1 (Applicable to both Redo Apply and SQL Apply)
- The COMPRESSION attribute is used to specify whether redo data is compressed before transmission to a redo transport destination.

- With NET_TIMEOUT attribute it is specified the number of seconds that the LGWR background process will block waiting for a redo transport destination to acknowledge redo data sent to it. If an acknowledgement is not received within NET_TIMEOUT seconds, an error is logged and the redo transport session to that destination is terminated.

- Role transitions happened faster than previous versions.

- Strong authentication for redo transport network sessions.

- Simplified Data Guard management interface by deprecation of redundant SQL clauses and initialization parameters.

- A physical standby database can now take advantage of the rolling upgrade feature provided by a logical standby. Through the use of the new KEEP IDENTITY clause option to the SQL ALTER DATABASE RECOVER TO LOGICAL STANDBY statement, a physical standby database can be temporarily converted into a logical standby database for the rolling upgrade, and then reverted back to the original configuration of a primary database and a physical standby database when the upgrade is done.

- In the same Data Guard configuration now it allows a mix of Linux and Windows primary and standby databases.

New Features in Oracle Data Guard 11.1 (Applicable to only Redo Apply)
- You can open ans query physical standby database while continuing to receive and apply redo data from a primary database.

- There comes snapshot standby database which is new type of updatable standby database.

- You can use the RMAN DUPLICATE command to create a physical standby database over the network without a need for pre-existing database backups.

- Lost-write detection feature using a physical standby.

New Features in Oracle Data Guard 11.1 (Applicable to SQL Apply only)
- DBMS_RLS, DBMS_FGA PL/SQL Packages are supported and also XML stored as CLOB.

- Transparent Data Encryption (TDE) is supported in SQL apply. This allows a logical standby database to provide data protection for applications with advanced security requirements.

- Data Guard SQL Apply parameters can be set dynamically.

- When switching over to a logical standby database where either the primary database or the standby database is using Oracle RAC, the SWITCHOVER command can be used without having to shut down any instance, either at the primary or at the logical standby database.

- Scheduler Jobs can be created on a standby database using the PL/SQL DBMS_SCHEDULER package and can be associated with an appropriate database role so that they run when intended (for example, when the database is the primary, standby, or both).


New Features in Oracle Data Guard 11.2 (Applicable to both Redo Apply and SQL Apply)

- Data Guard configuration now support up to 30 standby databases.

- In Oracle 10g there was introduced flash recovery area. Starting from Oracle 11g the place is known as fast recovery area. In 11gR2 the fast recovery area location changed from LOG_ARCHIVE_DEST_10 to LOG_ARCHIVE_DEST_1.

- The initialization parameter FAL_CLIENT is deprecated.

- Redo transport compression is no longer limited to compressing redo data only when a redo gap is being resolved. When compression is enabled, all redo data sent is compressed.

- The ALTER SYSTEM FLUSH REDO SQL statement can be used at failover time to flush unsent redo from a mounted primary database to a standby database, thereby allowing a zero data loss failover even if the primary database is not running in a zero data loss data protection mode.


New Features in Oracle Data Guard 11.2 (Applicable to both Redo Apply only)

- Apply lag tolerance can be configured by using the new STANDBY_MAX_DATA_DELAY parameter.

- The "ALTER SESSION SYNC WITH PRIMARY" SQL statement can be used to ensure that a physical standby database is synchronized with the primary database as of the time the statement is issued.

- The V$DATAGUARD_STATS view now has apply lag and transport lag columns. Data inside the view columns also contain more accurate information.

- The new V$STANDBY_EVENT_HISTOGRAM view has been introduced where we can see the histogram of apply lag values on the physical standby database.

- A corrupted data block in a primary database can be automatically replaced with an uncorrupted copy of that block from a physical standby database and vice versa.

New Features in Oracle Data Guard 11.2 (Applicable to only SQL Apply)
- Logical standby databases and the LogMiner utility support tables with basic table compression and OLTP table compression.

- Logical standby and the LogMiner utility support tables with SecureFile LOB columns. Compression and encryption operations on SecureFile LOB columns are also supported.

- Online redefinition performed at the primary database using the DBMS_REDEFINITION PL/SQL package is transparently replicated on a logical standby database.

- Logical Standby supports the use of editions at the primary database, including the use of edition-based redefinition to upgrade applications with minimal downtime.

- Logical standby databases support Streams Capture. This allows you to offload processing from the primary database in one-way information propagation configurations and make the logical standby the hub that propagates information to multiple databases. Streams Capture can also propagate changes that are local to the logical standby database.

Saturday, July 10, 2010

While startup listener it fails with HPUX Error: 29: Illegal seek

Problem Description
While issuing "lsnrctl start" command on HP-UX it fails with HPUX Error: 29: Illegal seek error like below.
ORACLE:/home>lsnrctl start 

LSNRCTL for HPUX: Version 10.2.0.4.0 - Production on 09-JUL-2010 22:19:41

Copyright (c) 1991, 2007, Oracle. All rights reserved.

Starting /SIDS/app/oracle/product/10.2.0/db/bin/tnslsnr: please wait...

TNS-12537: TNS:connection closed
TNS-12560: TNS:protocol adapter error
TNS-00507: Connection closed
HPUX Error: 29: Illegal seek
"lsnrctl status" command also with error HPUX Error: 239: Connection refused
ORACLE:/home>lsnrctl status 

LSNRCTL for HPUX: Version 10.2.0.4.0 - Production on 09-JUL-2010 22:33:21

Copyright (c) 1991, 2007, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1621)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
HPUX Error: 239: Connection refused
On Linux, the error stack may look like this:
TNS-12547: TNS:lost contact 
TNS-12560: TNS:proto adapter error
TNS-00517: Lost contact
Linux IA64 Error: 104: Connection reset by peer

Cause of the Problem
The problem happened due to incorrect settings of /etc/hosts file or host user is unable to read the /etc/hosts file. TNS Listener fails while attempting to initiate subscription to ONS node down event.

Solution of the Problem
1) Verify that OS user has permission on /etc/hosts file.
$ls -l /etc/hosts

2) Verify that the /etc/hosts file contains the localhost entry in the following form:

127.0.0.1 localhost.localdomain localhost
10.1.10.2 myhost.arjudba.blogspot.com myhost

3) If you are not in RAC environment explicitly disable the 10g TNS Listener's subscription to ONS by editing the listener.ora file and add the parameter below in the listener.ora file.

SUBSCRIBE_FOR_NODE_DOWN_EVENT_{listener_name}=OFF

where {listener_name} would be replaced with the actual listener name configured in the listener.ora file.

For example, if the listener name is LISTENER (which is default listener), the parameter would be:

SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF

4) Start the listener.
$ lsnrctl start