Showing posts with label Internals. Show all posts
Showing posts with label Internals. Show all posts

Wednesday, November 11, 2009

When datafile bytes size greater than maxbytes

From dba_data_files we can check the size of the datafile, the maximum size of the datafile it can grows if AUTOEXTENSIBLE option is set and their status with many other fields.

Describing the dba_data_files yield following result.
SQL> desc dba_data_files;
Name Null? Type
----------------------------------------- -------- ----------------------------
FILE_NAME VARCHAR2(513)
FILE_ID NUMBER
TABLESPACE_NAME VARCHAR2(30)
BYTES NUMBER
BLOCKS NUMBER
STATUS VARCHAR2(9)
RELATIVE_FNO NUMBER
AUTOEXTENSIBLE VARCHAR2(3)
MAXBYTES NUMBER
MAXBLOCKS NUMBER
INCREMENT_BY NUMBER
USER_BYTES NUMBER
USER_BLOCKS NUMBER
ONLINE_STATUS VARCHAR2(7)

Where,
Bytes represent the size of the data file in bytes.

Status can be AVAILABLE or INVALID (INVALID means that the file number is not in use, for example, a file in a tablespace was dropped)

Maxbytes represent maximum file size in bytes.

User_Bytes represent the actual size of the datafile that can be usable to store user data. In fact,
USER_BYTES= Actual size of the datafile - Size used to store file related metadata.

Online_status can have following values,
Online status of the file:
- SYSOFF
- SYSTEM
- OFFLINE
- ONLINE
- RECOVER

From the column description we see MAXBYTES should be greater than BYTES.

But let's have a look at the following query.
SQL> select bytes/1024/1024, maxbytes/1024/1024, user_bytes/1024/1024 from dba_data_files where tablespace_name='DATA';

BYTES/1024/1024 MAXBYTES/1024/1024 USER_BYTES/1024/1024
--------------- ------------------ --------------------
400 300 399.9375

We see in the query it shows BYTES size is greater then USER_BYTES which is acceptable. Because 400-399.9375 = 0.0625 Megabytes size is used to store metadata information. But why the MAXBYTES is greater than the size BYTES?

Well, this happens if after creating tablespace and datafile later we resize the datafile to a bigger size than the maxsize.
Following example will make you clear.

1)Create a tablespace whose datafile size is less than maxsize
SQL> create tablespace data datafile 'data01.dbf' size 40M autoextend on maxsize 300M;
Tablespace created.

2)Let's have a query to see bytes and maxbytes size from dba_data_files.
SQL> select bytes/1024/1024, maxbytes/1024/1024 from dba_data_files where tablespace_name='DATA';
BYTES/1024/1024 MAXBYTES/1024/1024
--------------- ------------------
40 300

3)Resize the datafile to a bigger size than MAXBYTES.
SQL> alter database datafile 'data01.dbf' resize 400M;

Database altered.

4)Let's have a query to see BYTES and MAXBYTES information.
SQL> select bytes/1024/1024, maxbytes/1024/1024 from dba_data_files where tablespace_name='DATA';

BYTES/1024/1024 MAXBYTES/1024/1024
--------------- ------------------
400 300


Related Documents
http://arjudba.blogspot.com/2008/04/maximum-oracle-database-size.html
http://arjudba.blogspot.com/2008/07/how-to-determine-os-block-size-for.html
http://arjudba.blogspot.com/2009/02/estimate-dumpfile-size-before-taking.html
http://arjudba.blogspot.com/2008/05/what-is-overall-database-size.html
http://arjudba.blogspot.com/2008/06/choose-optimal-data-block-size-in.html
http://arjudba.blogspot.com/2008/05/settting-maximum-size-of-backup-set-or.html
http://arjudba.blogspot.com/2008/06/playing-with-oracle-data-block-size.html

Monday, August 31, 2009

What happens during oracle database hot backup

According to oracle documentation you already heard/aware that during an Oracle tablespace hot backup, a script or program or command puts a tablespace into backup mode, then copies the datafiles to disk or tape, then takes the tablespace out of backup mode. We can check the backup mode from V$BACKUP view. In case of user managed backup, backup process starts after issuing ALTER TABLESPACE tbs_name BEGIN BACKUP; or ALTER DATABASE BEGIN BACKUP; command and backup process ends by ALTER TABLESPACE tbs_name END BACKUP; or ALTER DATABASE END BACKUP; command.

Hot backup is demonstrated inside the topic http://arjudba.blogspot.com/2008/05/user-managed-hot-backup-of-oracle.html.

Although the process is very clear and well understood but there are many misconception around hot backup. The misconception start what is actually done during hot backup, is data file opens writeable during backup process? or changes are stored somewhere in the SGA, the redologs, the rollback/undo segments or some combination thereof, and then written back into the datafile when the tablespace is taken out of backup mode?

Well, around the writeable issue inside datafile there is other misconception like "During hot backup process there is generated huge amount of redo data which in fact slows down the database dramatically if the database is in archivelog mode."

Now let's know what actually happens during hot backup. The hot backup steps are,

1)The corresponding tablespace is checkpointed.

2)The checkpoint SCN marker in the datafile headers cease to increment with checkpoints.

3)Full images of changed DB blocks are written to the redologs.

Whenever you issue,

ALTER TABLESPACE tbs_name BEGIN BACKUP;

command, at that point a checkpoint is performed against the target tablespace and the datafile header is frozen, so no more updates are allowed on it (the datafile header), this is for the database to know which was the last time the tablespace had a consistent image of the data.

But during backup process, the corresponding datafiles in the tablespace allow just normal read/write operations, that is I/O activity is not frozen.

In case of redo log generation, each block will be recorded into the redo log files, the first time it the block is changed. So if a row is modified for the first time inside date block since hot backup started the complete block image is recorded in the redo log files but subsequent transactions on the block will only record the transaction just as normal.

Above three steps are required to guarantee consistency during the file is restored and recovery. By freezing the checkpoint SCN in the file headers, any subsequent recovery on that backup copy of the file will know that it must commence at that SCN. Having an old SCN in the file header tells recovery that the file is an old one, and that it should look for the redolog file containing that SCN, and apply recovery starting there. Note that checkpoints to datafiles in hot backup mode are not suppressed during the backup, only the incrementing of the main checkpoint SCN flag. A "hot backup checkpoint" SCN marker in the file header continues to increment as periodic or incremental checkpoints progress normally.

By initially checkpointing the datafiles that comprise the tablespace and logging full block images to redo, Oracle guarantees that any blocks changed in the datafile while in hot backup mode will also be available in the redologs in case they are ever used for a recovery.

Now many one claims that during hot backup process there is excessive redo log generation than in normal mode. It actually depends on the amount of blocks changes during hot backup process. Because the first time a block is changed logging of full images of changed blocks in these tablespaces are recorded to the redo logs. Normally, Oracle logs an entry in the redologs for every change in the database, but it does not log the whole image of the database block. But during the hot backup process by logging full images of changed DB blocks to the redologs, Oracle eliminates the possibility of the backup containing irresolvable split blocks. To understand this reasoning, you must first understand what a split block is.

Typically, Oracle database blocks are a multiple of O/S blocks. For instance, most windows filesystems have a default block size of 512 bytes and unix filesystems have a default blocksize 2k, while Oracle’s default block size is 8k. This means that the filesystem stores data in 512 byte chunks, while Oracle performs reads and writes in 2k chunks, or multiples thereof. While backing up a datafile, your backup script makes a copy of the datafile from the filesystem, using O/S utilities such as copy, dd, cpio, or OCOPY. As it is making this copy, it is reading in O/S-block sized increments. If the database writer happens to be writing a DB block into the datafile at the same time that your script is reading that block’s constituent O/S blocks, your backup copy of the DB block could contain some O/S blocks from before the database performed the write, and some from after. This would be a split block.

By logging the full block image of the changed block to the redologs, it guarantees that in the event of a recovery, any split blocks that might be in the backup copy of the datafile will be resolved by overlaying them with the full legitimate image of the block from the redologs. Upon completion of a recovery, any blocks that got copied in a split state into the backup will have been resolved through application of full block images from the redologs.

Related Documents

Thursday, August 27, 2009

Does oversize of datatype VARCHAR2 causes performance problem

From the beginning of learning Oracle SQL you have possibly heard that in case of VARCHAR2 datatype it allocates space exactly what it needs. So if you allocates 4000 bytes of VARCHAR2 data type and database needs 10 bytes only then exactly 10 bytes are allocated.

That is, in case of VARCHAR2(4000) and VARCHAR2(16) columns, if we store less then 16 bytes data in these two columns then same amount of space will be allocated, and performance should be the same. But, have you ever tested it? I got a funny example http://hrivera99.blogspot.com/2008/05/why-is-varchar2-oversizing-bad.html here. There it is said performance problem but in reality there is not. In the example it is shown problem in physical reads but I don't agree with the example. In fact in the first example it is cached data and hence physical reads is reduced.

In the following section I simulate same example and see no performance differences. However there may rise, http://arjudba.blogspot.com/2008/09/ora-01450-maximum-key-length-3215.html while creating index in case of bigger VARCHAR2 length.

The most misleading example can be created by omitting
"
ALTER TABLESPACE EXAMPLE OFFLINE;

ALTER TABLESPACE EXAMPLE ONLINE;"

If you omit this step you may get different result as data become cached. And you need to take tablespace offline in order to get most accurate result as offlining a tablespace uncache of corresponding tablespace data.

Step 1)Create varchar2_length_test table with VARCHAR2(4000) and insert data into it.
SQL> create table varchar2_length_test(
2 ID NUMBER,
3 COL2 VARCHAR2(4000),
4 COL3 VARCHAR2(4000),
5 COL4 VARCHAR2(4000),
6 COL5 VARCHAR2(4000),
7 COL6 VARCHAR2(4000),
8 COL7 VARCHAR2(4000),
9 COL8 VARCHAR2(4000),
10 COL9 VARCHAR2(4000),
11 COL10 VARCHAR2(4000),
12 COL11 VARCHAR2(4000),
13 COL12 VARCHAR2(4000),
14 COL13 VARCHAR2(4000)) TABLESPACE EXAMPLE;

Table created.

SQL>
SQL> begin
2 for i in 1 .. 100000
3 LOOP
4 INSERT into varchar2_length_test VALUES(
5 i, i||'Col2',i||'Col3',i||'Col4',i||'Col5',i||'Col6',i||'Col7',i||'Col8',i||'Col9',i||'Col10',i||'Col11',i||'Col12',
6 i||'Col13');
7 END LOOP;
8 END;
9 /

PL/SQL procedure successfully completed.

Step 2)Create varchar2_length_test_short table with VARCHAR2(16) and insert data into it.
SQL> create table varchar2_length_test_short(
2 ID NUMBER,
3 COL2 VARCHAR2(16),
4 COL3 VARCHAR2(16),
5 COL4 VARCHAR2(16),
6 COL5 VARCHAR2(16),
7 COL6 VARCHAR2(16),
8 COL7 VARCHAR2(16),
9 COL8 VARCHAR2(16),
10 COL9 VARCHAR2(16),
11 COL10 VARCHAR2(16),
12 COL11 VARCHAR2(16),
13 COL12 VARCHAR2(16),
14 COL13 VARCHAR2(16)) TABLESPACE EXAMPLE;

Table created.

SQL> begin
2 for i in 1 .. 100000
3 LOOP
4 INSERT into varchar2_length_test_short VALUES(
5 i, i||'Col2',i||'Col3',i||'Col4',i||'Col5',i||'Col6',i||'Col7',i||'Col8',i||'Col9',i||'Col10',i||'Col11',i||'Col12',
6 i||'Col13');
7 END LOOP;
8 END;
9 /

PL/SQL procedure successfully completed.

Step 3)Clear caching in the tablespace.
SQL> ALTER TABLESPACE EXAMPLE OFFLINE;
Tablespace altered.

SQL> ALTER TABLESPACE EXAMPLE ONLINE;
Tablespace altered.


Step 4)Enable tracing and look at statistics
SQL> SET AUTOT TRACE
SQL> select count(*) from varchar2_length_test;

1 row selected.


Execution Plan
----------------------------------------------------------
Plan hash value: 1500664439

-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 418 (2)| 00:00:06 |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS FULL| VARCHAR2_LENGTH_TEST | 88364 | 418 (2)| 00:00:06 |
-----------------------------------------------------------------------------------

Note
-----
- dynamic sampling used for this statement


Statistics
----------------------------------------------------------
29 recursive calls
1 db block gets
1980 consistent gets
1912 physical reads
176 redo size
411 bytes sent via SQL*Net to client
396 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

SQL> ALTER TABLESPACE EXAMPLE OFFLINE;

Tablespace altered.

SQL>
SQL> ALTER TABLESPACE EXAMPLE ONLINE;

Tablespace altered.

SQL> select count(*) from varchar2_length_test_short;

1 row selected.


Execution Plan
----------------------------------------------------------
Plan hash value: 161270611

-----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 418 (2)| 00:00:06 |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS FULL| VARCHAR2_LENGTH_TEST_SHORT | 109K| 418 (2)| 00:00:06 |
-----------------------------------------------------------------------------------------

Note
-----
- dynamic sampling used for this statement


Statistics
----------------------------------------------------------
29 recursive calls
1 db block gets
1993 consistent gets
1912 physical reads
176 redo size
411 bytes sent via SQL*Net to client
396 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed



So we see in both VARCHAR2(4000) and VARCHAR2(16) almost same consistent gets and physical reads. So oversize of varchar2 does not cause performance problem issue but lead to other problems.

Related Documents

ORA-01450: maximum key length (3215) exceeded

Saturday, May 31, 2008

Hidden Parameters in Oracle. How to Change Hidden Parameter

The hidden parameters start with an "_".They can not be viewed from the output of show parameter
or querying v$parameter unless and untill they are set explicitly in init.ora.
However if you want to view all the hidden parameters and their default values the following query
could be of help,


SELECT
a.ksppinm "Parameter", b.ksppstvl "Session Value", c.ksppstvl "Instance Value"
FROM x$ksppi a, x$ksppcv b, x$ksppsv c
WHERE
a.indx = b.indx AND a.indx = c.indx AND a.ksppinm LIKE '/_%' escape '/'
/


In order to see the listing of all hidden parameters query,

select *
from SYS.X$KSPPI
where substr(KSPPINM,1,1) = '_';


Change Hidden Parameters in Oracle
---------------------------------------

It is never recommended to modify these hidden parameters without the assistance of Oracle Support.Changing these parameters may lead to high performance degradation and other problems in the database.
In order to change hidden parameter,
1)If you use pfile then in your initSID.ora you can entry of the hidden parameter and start the database.

2)If you want to use for the current session you can use ALTER SESSION SET ....

3)To set it permanently if you use spfile then use, ALTER SYSTEM SET ...... SCOPE=SPFILE. Since hidden parameter starts with underscore(_) to access it you have to specify within double quotes. If you use SCOPE=SPFILE then in order to take effect you need to restart database. You can use SCOPE=BOTH if parameter can be set in the session also.

Monday, May 5, 2008

How to Discover find DBID

We need DBID whenever we lost our all database files, or whenever we need to restore spfile or control file.

After some analysis on discovering DBID I got several ways to find DBID. I will try to demonstrate the procedure.

A)If the database is up: You can query V$database and get the DBID and record it in somewhere.

or,if the database is down and you have control file then you can mount the database and query from V$SATABASE.

SQL> SELECT DBID FROM V$DATABASE;
DBID
----------
2869417476


B) If you log the RMAN backup or if you preserve output of RMAN session then you can get DBID from that output.

$rman TARGET /
Recovery Manager: Release 10.2.0.1.0 - Production on Tue May 6 01:25:48 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
connected to target database: ARJU (DBID=2869417476)


C)If you have configured AUTOBACKUP ON,

RMAN> SHOW CONTROLFILE AUTOBACKUP;
RMAN configuration parameters are:
CONFIGURE CONTROLFILE AUTOBACKUP ON;


Now we take BACKUP DATAFILE 4,
RMAN>BACKUP DATAFILE 4;
.
.
Starting Control File and SPFILE Autobackup at 06-MAY-08
piece handle=/oracle/app/oracle/product/10.2.0/db_1/dbs/c-2869417476-20080506-04


The autobackup format is by default c-IIIIIIIIII-YYYYMMDD-QQ,(when it is %F) where:

IIIIIIIIII stands for the DBID.
YYYYMMDD is a time stamp of the day the backup is generated
QQ is the hex sequence that starts with 00 and has a maximum of FF

I have seen that this format works when we set specifically/explicitly configure as
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F';

Now in the file from c-2869417476-20080506-04 we get DBID is 2869417476.


D)If you did not set Autobackup on which is by default. After many analysis I found that DBID can be seen from backup piece or any image copy that holds either SYSTEM or SYSAUX or UNDO datafiles.
Though if you backup your database as backup as compressed then with this method you will not be able to discover DBID.
If you have SYSTEM datafile or UNDO datafile either as image copy or as backup piece then you can use,
strings file_name |grep MAXVALUE, (In case of SYSTEM datafile)
strings file_name |grep MAXVALUE (In case of UNDO datafile)

to find DBID.

If you have SYSAUX datafile either as image copy or as backup piece then you can use,
strings file_name |grep DBID= to find DBID.

Examples:
-----------------

Both of these example is based on UNIX scenario.

1)
RMAN> backup datafile 1;


Starting backup at 06-MAY-08
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=153 devtype=DISK
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
input datafile fno=00001 name=/oradata1/arju/datafiles/ARJU/system01.dbf
channel ORA_DISK_1: starting piece 1 at 06-MAY-08
channel ORA_DISK_1: finished piece 1 at 06-MAY-08
piece handle=/oradata2/arju/flash_recovery_area/ARJU/backupset/
2008_05_06/o1_mf_nnndf_TAG20080506T014702_41zw6p8j_.bkp
.
.

RMAN> exit;
Recovery Manager complete.

bash-3.00$ strings /oradata2/arju/flash_recovery_area/ARJU/backupset/
2008_05_06/o1_mf_nnndf_TAG20080506T014702_41zw6p8j_.bkp |grep MAXVALUE,

.
.

2869417476, MAXVALUE,

So here 2869417476 is the DBID.

2)
RMAN> backup as copy datafile 2;


Starting backup at 06-MAY-08
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=134 devtype=DISK
channel ORA_DISK_1: starting datafile copy
input datafile fno=00002 name=/oradata1/arju/datafiles/ARJU/undotbs01.dbf
output filename=/oradata2/arju/flash_recovery_area/ARJU/datafile/o1_mf_undotbs1_41zwjtx6_.dbf

bash-3.00$ strings /oradata2/arju/flash_recovery_area/ARJU/datafile/o1_mf_undotbs1_41zwjtx6_.dbf |grep MAXVALUE

2869417476, MAXVALUE


Here 2869417476 is the DBID

3)From physical data file you can also follow the same method.

From sysaux datafile,

bash-3.00$ strings /oradata1/arju/datafiles/ARJU/sysaux01.dbf |grep DBID=
connected to target database: ARJU (DBID=2869417476, not open)


Here 2869417476 is the DBID.

4)From whole database backup you can also follow same method.
RMAN> BACKUP DATABASE;

Starting backup at 06-MAY-08
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=145 devtype=DISK
.
.
piece handle=/oradata2/arju/flash_recovery_area/ARJU/backupset/
2008_05_06/o1_mf_nnndf_TAG20080506T015705_41zwskgv_.bkp

bash-3.00$ strings /oradata2/arju/flash_recovery_area/ARJU/backupset/
2008_05_06/o1_mf_nnndf_TAG20080506T015705_41zwskgv_.bkp |grep MAXVALUE,

.
.
2869417476, MAXVALUE,


Here 2869417476 is the DBID.

Enjoy!!!

Related Documents
How to Change Database Name and DBID?

Sunday, April 20, 2008

How undo Space Allocation Works

In two different scenarios two different mechanisms is used to allocate undo space.

A)Whenever a new transaction needs undo space:
-------------------------------------------------

1)First , oracle tries to find a undo segment which has no active transaction and allocate the extent within the undo segment.

2)If no such undo segment found then oracle tries to make online of and off-line undo segment and use it.

3)If no such off-line undo segment found then create a new undo segment and use it.

4)If there is no sufficient space to create a new undo segment then it try to reuse an expired extent from the existing undo segment.

B)Whenever a running transaction needs more undo space:
------------------------------------------------------------

1)If current extent has more free blocks? If it has then allocate the next free block within the extent.

2)If current extent does not have free blocks then check the next extent within the segment. If the next extent is expired then wrap the next extent and allocate the first block of the next extent.

3)If the next extent is not expired then get space from UNDO tablespace.If a free extent is available then allocate it to the undo segment and return the first block in the new extent.

4)If there is no free space available then take space from offline undo segment. Deallocate the extent from the offline undo segment and add it to the current undo segment. Return the first free block of the extent.

5)If there is no offline undo segment then deallocate the extent from the online undo segment and add it to the current undo segment. Return the first free block of the extent.

6)If it fails then extend the datafile of the undo tablespace.If the file can be extended then add an extent to the current undo segment then return the block.

7)If the extend datafile fails then try to reuse extents from its own undo segments. If it is not busy then warp into the next extent.

8)If reuse extents from its own undo segment fails i.e they contain uncommitted transaction then take unexpired extents from it's own offline undo segments.

9)If it fails then take unexpired extents from it's own online undo segments.

10)If all above fails then ORA-30036 (unable to extend segment by %s in undo tablespace '%s' ) arises.

Related Documents:
-------------------------
Why so much undo in my Database

Wednesday, April 16, 2008

What is Alert Log?

Each database has a special file named alert_sid.log. The alert log of a database is a chronological log of messages and errors, and includes the following items:


1)All internal errors (ORA-600), block corruption errors (ORA-1578), and deadlock errors (ORA-60) that occur.

2)Administrative operations, such as CREATE, ALTER, and DROP statements and STARTUP, SHUTDOWN, and ARCHIVELOG statements.

3)Messages and errors relating to the functions of shared server and dispatcher processes.

4)Errors occurring during the automatic refresh of a materialized view.

5)The values of all initialization parameters that had nondefault values at the time the database and instance start.


Oracle Database uses the alert log to record these operations as an alternative to displaying the information on an operator's console.

The alert log file destination is specified by BACKGROUND_DUMP_DEST.