Monday, August 31, 2009

What happens during oracle database hot backup

According to oracle documentation you already heard/aware that during an Oracle tablespace hot backup, a script or program or command puts a tablespace into backup mode, then copies the datafiles to disk or tape, then takes the tablespace out of backup mode. We can check the backup mode from V$BACKUP view. In case of user managed backup, backup process starts after issuing ALTER TABLESPACE tbs_name BEGIN BACKUP; or ALTER DATABASE BEGIN BACKUP; command and backup process ends by ALTER TABLESPACE tbs_name END BACKUP; or ALTER DATABASE END BACKUP; command.

Hot backup is demonstrated inside the topic http://arjudba.blogspot.com/2008/05/user-managed-hot-backup-of-oracle.html.

Although the process is very clear and well understood but there are many misconception around hot backup. The misconception start what is actually done during hot backup, is data file opens writeable during backup process? or changes are stored somewhere in the SGA, the redologs, the rollback/undo segments or some combination thereof, and then written back into the datafile when the tablespace is taken out of backup mode?

Well, around the writeable issue inside datafile there is other misconception like "During hot backup process there is generated huge amount of redo data which in fact slows down the database dramatically if the database is in archivelog mode."

Now let's know what actually happens during hot backup. The hot backup steps are,

1)The corresponding tablespace is checkpointed.

2)The checkpoint SCN marker in the datafile headers cease to increment with checkpoints.

3)Full images of changed DB blocks are written to the redologs.

Whenever you issue,

ALTER TABLESPACE tbs_name BEGIN BACKUP;

command, at that point a checkpoint is performed against the target tablespace and the datafile header is frozen, so no more updates are allowed on it (the datafile header), this is for the database to know which was the last time the tablespace had a consistent image of the data.

But during backup process, the corresponding datafiles in the tablespace allow just normal read/write operations, that is I/O activity is not frozen.

In case of redo log generation, each block will be recorded into the redo log files, the first time it the block is changed. So if a row is modified for the first time inside date block since hot backup started the complete block image is recorded in the redo log files but subsequent transactions on the block will only record the transaction just as normal.

Above three steps are required to guarantee consistency during the file is restored and recovery. By freezing the checkpoint SCN in the file headers, any subsequent recovery on that backup copy of the file will know that it must commence at that SCN. Having an old SCN in the file header tells recovery that the file is an old one, and that it should look for the redolog file containing that SCN, and apply recovery starting there. Note that checkpoints to datafiles in hot backup mode are not suppressed during the backup, only the incrementing of the main checkpoint SCN flag. A "hot backup checkpoint" SCN marker in the file header continues to increment as periodic or incremental checkpoints progress normally.

By initially checkpointing the datafiles that comprise the tablespace and logging full block images to redo, Oracle guarantees that any blocks changed in the datafile while in hot backup mode will also be available in the redologs in case they are ever used for a recovery.

Now many one claims that during hot backup process there is excessive redo log generation than in normal mode. It actually depends on the amount of blocks changes during hot backup process. Because the first time a block is changed logging of full images of changed blocks in these tablespaces are recorded to the redo logs. Normally, Oracle logs an entry in the redologs for every change in the database, but it does not log the whole image of the database block. But during the hot backup process by logging full images of changed DB blocks to the redologs, Oracle eliminates the possibility of the backup containing irresolvable split blocks. To understand this reasoning, you must first understand what a split block is.

Typically, Oracle database blocks are a multiple of O/S blocks. For instance, most windows filesystems have a default block size of 512 bytes and unix filesystems have a default blocksize 2k, while Oracle’s default block size is 8k. This means that the filesystem stores data in 512 byte chunks, while Oracle performs reads and writes in 2k chunks, or multiples thereof. While backing up a datafile, your backup script makes a copy of the datafile from the filesystem, using O/S utilities such as copy, dd, cpio, or OCOPY. As it is making this copy, it is reading in O/S-block sized increments. If the database writer happens to be writing a DB block into the datafile at the same time that your script is reading that block’s constituent O/S blocks, your backup copy of the DB block could contain some O/S blocks from before the database performed the write, and some from after. This would be a split block.

By logging the full block image of the changed block to the redologs, it guarantees that in the event of a recovery, any split blocks that might be in the backup copy of the datafile will be resolved by overlaying them with the full legitimate image of the block from the redologs. Upon completion of a recovery, any blocks that got copied in a split state into the backup will have been resolved through application of full block images from the redologs.

Related Documents

Thursday, August 27, 2009

Does oversize of datatype VARCHAR2 causes performance problem

From the beginning of learning Oracle SQL you have possibly heard that in case of VARCHAR2 datatype it allocates space exactly what it needs. So if you allocates 4000 bytes of VARCHAR2 data type and database needs 10 bytes only then exactly 10 bytes are allocated.

That is, in case of VARCHAR2(4000) and VARCHAR2(16) columns, if we store less then 16 bytes data in these two columns then same amount of space will be allocated, and performance should be the same. But, have you ever tested it? I got a funny example http://hrivera99.blogspot.com/2008/05/why-is-varchar2-oversizing-bad.html here. There it is said performance problem but in reality there is not. In the example it is shown problem in physical reads but I don't agree with the example. In fact in the first example it is cached data and hence physical reads is reduced.

In the following section I simulate same example and see no performance differences. However there may rise, http://arjudba.blogspot.com/2008/09/ora-01450-maximum-key-length-3215.html while creating index in case of bigger VARCHAR2 length.

The most misleading example can be created by omitting
"
ALTER TABLESPACE EXAMPLE OFFLINE;

ALTER TABLESPACE EXAMPLE ONLINE;"

If you omit this step you may get different result as data become cached. And you need to take tablespace offline in order to get most accurate result as offlining a tablespace uncache of corresponding tablespace data.

Step 1)Create varchar2_length_test table with VARCHAR2(4000) and insert data into it.
SQL> create table varchar2_length_test(
2 ID NUMBER,
3 COL2 VARCHAR2(4000),
4 COL3 VARCHAR2(4000),
5 COL4 VARCHAR2(4000),
6 COL5 VARCHAR2(4000),
7 COL6 VARCHAR2(4000),
8 COL7 VARCHAR2(4000),
9 COL8 VARCHAR2(4000),
10 COL9 VARCHAR2(4000),
11 COL10 VARCHAR2(4000),
12 COL11 VARCHAR2(4000),
13 COL12 VARCHAR2(4000),
14 COL13 VARCHAR2(4000)) TABLESPACE EXAMPLE;

Table created.

SQL>
SQL> begin
2 for i in 1 .. 100000
3 LOOP
4 INSERT into varchar2_length_test VALUES(
5 i, i||'Col2',i||'Col3',i||'Col4',i||'Col5',i||'Col6',i||'Col7',i||'Col8',i||'Col9',i||'Col10',i||'Col11',i||'Col12',
6 i||'Col13');
7 END LOOP;
8 END;
9 /

PL/SQL procedure successfully completed.

Step 2)Create varchar2_length_test_short table with VARCHAR2(16) and insert data into it.
SQL> create table varchar2_length_test_short(
2 ID NUMBER,
3 COL2 VARCHAR2(16),
4 COL3 VARCHAR2(16),
5 COL4 VARCHAR2(16),
6 COL5 VARCHAR2(16),
7 COL6 VARCHAR2(16),
8 COL7 VARCHAR2(16),
9 COL8 VARCHAR2(16),
10 COL9 VARCHAR2(16),
11 COL10 VARCHAR2(16),
12 COL11 VARCHAR2(16),
13 COL12 VARCHAR2(16),
14 COL13 VARCHAR2(16)) TABLESPACE EXAMPLE;

Table created.

SQL> begin
2 for i in 1 .. 100000
3 LOOP
4 INSERT into varchar2_length_test_short VALUES(
5 i, i||'Col2',i||'Col3',i||'Col4',i||'Col5',i||'Col6',i||'Col7',i||'Col8',i||'Col9',i||'Col10',i||'Col11',i||'Col12',
6 i||'Col13');
7 END LOOP;
8 END;
9 /

PL/SQL procedure successfully completed.

Step 3)Clear caching in the tablespace.
SQL> ALTER TABLESPACE EXAMPLE OFFLINE;
Tablespace altered.

SQL> ALTER TABLESPACE EXAMPLE ONLINE;
Tablespace altered.


Step 4)Enable tracing and look at statistics
SQL> SET AUTOT TRACE
SQL> select count(*) from varchar2_length_test;

1 row selected.


Execution Plan
----------------------------------------------------------
Plan hash value: 1500664439

-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 418 (2)| 00:00:06 |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS FULL| VARCHAR2_LENGTH_TEST | 88364 | 418 (2)| 00:00:06 |
-----------------------------------------------------------------------------------

Note
-----
- dynamic sampling used for this statement


Statistics
----------------------------------------------------------
29 recursive calls
1 db block gets
1980 consistent gets
1912 physical reads
176 redo size
411 bytes sent via SQL*Net to client
396 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

SQL> ALTER TABLESPACE EXAMPLE OFFLINE;

Tablespace altered.

SQL>
SQL> ALTER TABLESPACE EXAMPLE ONLINE;

Tablespace altered.

SQL> select count(*) from varchar2_length_test_short;

1 row selected.


Execution Plan
----------------------------------------------------------
Plan hash value: 161270611

-----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 418 (2)| 00:00:06 |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS FULL| VARCHAR2_LENGTH_TEST_SHORT | 109K| 418 (2)| 00:00:06 |
-----------------------------------------------------------------------------------------

Note
-----
- dynamic sampling used for this statement


Statistics
----------------------------------------------------------
29 recursive calls
1 db block gets
1993 consistent gets
1912 physical reads
176 redo size
411 bytes sent via SQL*Net to client
396 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed



So we see in both VARCHAR2(4000) and VARCHAR2(16) almost same consistent gets and physical reads. So oversize of varchar2 does not cause performance problem issue but lead to other problems.

Related Documents

ORA-01450: maximum key length (3215) exceeded

Shell script book for the newbie

I have written a book of shell script specially for the newbie who want to learn shell script. The book starts from a very basic idea of shell script programming. Each sections are well demonstrated by examples in order to make clear idea about shell script programming. The book is written in such a way so that you can work with unix and practice shell script even you don't have idea about unix operation systems.

Below is some of the topics that is discussed in the book. All topics are discussed with examples.


- Introduction -What is kernel, shell, shell script.
- Basic Steps to write a shell script
- Variables in shell script
- Output text with echo command
- Arithmetic operation and expression in shell script with expr
- Basic shell programming commands quote, exit and read
- Wildcard characters in linux
- Assignment and comparison variable in shell script
- Command writing, command line arguments in shell script
- Input-Output rediection on linux
- Pipe with example in linux


- If construct in shell script
- Checking condition in shell scipt by test or [expr]
- If else construct in shell script
- For and while loop in shell script
- Case statement in shell script


- What is /dev/null on linux
- Conditional execution && and || in shell script
- Functions in shell script
- Trap command on linux shell script
- The shift command and uses of it in shell script
- getopts command in linux shell script


- Retrieve column from file using cut utility in linux
- Merge lines using paste utility on linux
- Join utility in linux
- Translate or replace characters using tr utility
- Data manipulation using awk utility in linux
- Edit file on linux using sed utility
- Remove duplicate successive line using uniq utility
- Find pattern from a file using grep, egrep or fgrep utility

You can collect this shell script book for $15.

And to collect the book please contact to my email address a_arju@yahoo.co.uk
Related Documents
Oracle DBA scripts ordered by Daily, Weekly, Monthly

Oracle DBA scripts ordered by Daily, Weekly, Monthly

I have written a list of oracle DBA scripts that is essential for the DBAs. I have divided the scripts into three categories as daily, weekly and monthly basis. The scripts included almost every types of tasks that are needed for a DBA.

Scripts included various types of error checking, database health monitoring, automatic backup, recovery process, automatic performance problems findings.

Scripts are written for automatic database start up and shut down, check for backup has been carried out successfully, clear archive logs, automatic checking invalid objects, repair them, database growth checking, oracle database security settings, identify top sqls and provide necessary recommendations, tablespace growth management, various advisor to recommend necessary database changes, parameter suggestion/modifications etc.

The scripts not only included oracle scripts, there is also included shell scripts where it is necessary, so you can schedule your scripts through crontab.

All these scripts you can collect for $50.

And to collect the scripts please contact to my email address a_arju@yahoo.co.uk

Guidelines for choosing the best web hosting service.

In order to make a website operational to Internet users, you at first need to build a web site and then you search for a web hosting service provider in order to host your website. There is over tons of websites that offer web hosting service. So it is your choice to pick the best web hosting service.

Before go through this topic have a look at What is web hosting service? in order to have an idea about web hosting. Also go through different types of web hosting based on application which is discussed in http://arjudba.blogspot.com/2009/06/different-types-of-web-hosting-services.html

In this topic, I am trying to present the guidelines for the best web hosting which will help you to pick a good web hosting provider.

1)Speed and reliability of the servers and hardware:
Be sure about the hosting server's actual hardware configuration as well as the speed of the hosting provider they offers.
- A good hosting service provider must have a high speed Internet connection (T3 or higher).

- It should opt for unlimited bandwidth for your website.

- Good hosting servers must support redundant servers. So that if one server fails another can operate.

- There must be an host administrator to do backup jobs of your web hosting data.

2)Amount of space needed for hosting and flexibility:
Your website contents must need space in the hosting server where it would be reside and if your site is dynamic(in case of blogs. forums, social marketing site), day by day your volume of space needed must increase. So before hosting your web site be sure the amount of space needed for you. Ask them up to how much they can support.

In the market you can get lower cost of 500MB hosting space in one hosting company than in 50MB space of other hosting company. In case of web hosting, this is much true that "The less money you spend in hosting, the poor service you receive".

Also besides choosing space ask the web hosting provider whether they can accommodate your changes in future. Like, in the near future you may need oracle database support or mysql database support which are not belong to your current package.

3)Customer Services:
This is one of the most criteria to choose a web hosting service. Make sure your web hosting service offers 24x7x365 support. That is you can reach them 24 hours/day, seven days a week and 365 days per year with real people on the phone or by email to help you without cost. Before doing hosting, you can easily test this by simply emailing them and check how much time they are taking to response you as well as measure how much effective their mail is.

Also,
- Look for the big customers they are offering service currently.

- Take the existing customers feedback.

- How many years your hosting company have been in the business.

- Are their hardware new and ask for RAM, processor configuration etc for their servers.

4)The features provided by hosting company:
Check for the features that are provided by the company that you are going to host your web site. Some basic and common features are,

- DNS registration for your web site.

- At least 20MB of space to host your files.

- Unlimited bandwidth - to upload files, to support traffic to your site.

- Free Technical Support.

- POP3 E-Mail Accounts - Based on the service you can get many email POP accounts, but they must offer at least 3 email POP accounts.

- Email Forwarding - So that you can forward email to your yahoo/gmail that you check often.

- Email Auto-responders - Some programs that will respond clients answer automatically. May have intelligence into the program or may be generic answer. Vacation response might be a generic auto response.

- Email Aliases - Allows you to create groups and you can assign several emails into the groups.

- FTP Access (File Transfer Protocol) - This is the protocol used on the Internet for sending files.

- FrontPage Support - If your hosting company support FrontPage extensions then you can edit the site online using the program.

- Own CGI-Bin Access (Common Gateway Interface)

- Server Side Includes (SSI) - can be used to dynamically insert information such as local time, date or file information.

- Password Protection - To access your web site contents there must be a authentication.

- Database support - MySQL, Oracle etc. database as you need them.

- Web Statistics - Though you can use various tools to see your web statistics but ask hosting provider whether they support any.

- Secure Server - You should process online credit card transactions through a secure server so the credit card numbers are encrypted.

- Shopping Cart Software - To enable eCommerce on your site you will need some type of method for processing orders online.

- Online Web Site Manager - There should be panel by which you can make changes to your site, maintain your site, account information and emails.

- Search engine submissions - Many hosting service provider offers for extra services like Search engine submissions.

- Virus Scanning - On windows hosting web hosting servers are responsible to do scanning and ensure your files to keep virus free.

- Reseller/referral program - Common program that exist almost in every web hosting company. If someone register to the web hosting service using your referral links then your referral program can pay 20% of the money that someone spend using your referral.

5)The amount you like to spend for hosting:
I already said, the more you spend the better service you get. There may be a little exception into it. However, for a small web hosting package of 20-50MB of allocated space for a site your cost may range from $5 to $100 per month. Anyway, before pick any package look for technical support they will provide.

If you follow the above basic guidelines, prior to choose a hosting company I hope, you will have a reliable, efficient and satisfying hosting service that will contribute to the success of your business for many years to come.

Related Documents

What is web hosting service?

Monday, August 24, 2009

Format Model Modifiers - FX and FM

The FM and FX modifiers are used in the TO_CHAR function in oracle sql in the format model. These two modifiers control blank padding and exact format checking.

FM:
FM is fill mode. If we simply use TO_CHAR function that is without FM then Oracle uses blank characters to fill format elements to a constant width equal to the largest element for the relevant format model in the current session language. For example if NLS_LANGUAGE is set to AMERICAN then, in case of month the highest length month is SEPTEMBER. So if we don't use FM modifier then every month takes the length of 9 characters. (that is month SEPTEMBER). But if we use FM format model modifiers then the blank characters are suppressed.

Below example will clear you.

SQL> select to_char(SYSDATE,'fmDDTH MONTH YYYY') from dual;

TO_CHAR(SYSDATE,'FM
-------------------
24TH AUGUST 2009

SQL> select to_char(SYSDATE,'DDTH MONTH YYYY') from dual;

TO_CHAR(SYSDATE,'DD
-------------------
24TH AUGUST 2009

SQL> show parameter nls_language

NAME TYPE VALUE
------------------------------------ ----------- ---------------
nls_language string AMERICAN

SQL> select to_char(SYSDATE,'Day')||' Time' from dual;

TO_CHAR(SYSDAT
--------------
Monday Time
SQL> select to_char(SYSDATE,'fmDay')||' Time' from dual;

TO_CHAR(SYSDAT
--------------
Monday Time


FX:
FX is format exact.
FX modifier specifies exact matching for the character argument and datetime format model of a TO_DATE function.

Let's have a look at FX format model specification.
- Punctuation and quoted text in the character argument must exactly match the corresponding parts of the format model.

- The character argument cannot have extra blanks. Without FX, Oracle ignores
extra blanks.

- Numeric data in the character argument must have the same number of digits as
the corresponding element in the format model. Without FX, numbers in the
character argument can omit leading zeroes.

When FX is enabled, you can disable this check for leading zeroes by using the FM
modifier as well.

If any portion of the character argument violates any of these conditions, then Oracle returns an error message.

Peter Davies' Internet Marketing Blog, a student of Alex Jeffreys

This review is a summary of what Interactive Blogger, an internet marketing blog owned by Peter Davies. Peter has started and an Internet Marketing Business and is being mentored by a rising star in the Online world, Alex Jeffreys who has a great record of coaching newbies in internet marketing or those who have been struggling and don’t know where to go next. Alex in turn has been mentored by Rich Schefren whom Peter has also had the benefit of.

Alex communicates to his student the ethos of providing large amounts of value up front, so that anyone who has an online business or is thinking of creating an online business can receive great tangible benefits.

He sees internet marketing as a few simple steps;
  • First you build a blog with which you create great content to market with your readership, never try to direct sell to them

  • Use the blog to drive traffic to your site, add valuable information to other blogs, market yourself using several techniques, articles, video marketing, forums

  • Use to your traffic to build a list but don’t always try to sell to that list, give them large amounts of useful content first the sales will come later.


  • Peters’ approach has embraced Alex’s ethos, indeed he has decided as his first real venture to create a free product explaining through the eyes of someone who has recently gone through the process himself he has started a new online business from scratch. This is not a product that will get you rich overnight, he states quite clearly that there is no such thing, this covers the basics of planning, building the blog then blogging, generating traffic and building a subscriber list.

    This is a reflection of a genuine approach to getting started, watching him learn new things then apply them and watching him develop into an authority in his chosen field without the hype. Best of all its free! http://interactive-blogger.com

    Related Documents
    http://arjudba.blogspot.com/2008/08/list-of-available-advertising-network.html
    http://arjudba.blogspot.com/2009/06/how-to-get-targeted-backlinks-to-your.html
    http://arjudba.blogspot.com/2009/08/how-to-increase-your-technorati.html