Problem Description
Dynamic change of CPU_COUNT parameter causes oracle bug ORA-600 [ksrpubwait:deadlock]. For example, after issuing command "alter system set cpu_count=5;" it fails with following errors.
ORA-600 [ksrpubwait:deadlock], [parameters to cluster db instances - broadcast channel]
ORA-2097: parameter cannot be modified because specified value is invalid
And then instance crashes.
If you look at stack trace you will see,
ksrpubwait ksrpublish kspbcast kspset0 kxfpCPUCountUpdate ksbcpcb
Cause of the Problem
This errors are caused by Oracle Bug 7535429
Solution of the Problem
Solution 01: Bug 7535429 is fixed in version 11.2. So upgrade your oracle database to 11gR2 will solve the problem.
There is one-off patch which also help you to solve the problem. Check for the availability of one-off Patch 7535429 for your platform on MetaLink.
Solution 02: Explicitely setting CPU_COUNT to new value.
Solution 03: If you start the instance from pfile instead of using spfile then it will also solve the problem.
Solution 04: Set explicitly PARALLEL_MAX_SERVERS to the current value instead of relying default value.
Saturday, June 26, 2010
While startup, instance terminated after ORA-600 [6006] ORA-600 [6856]
Problem Description
While starting up the instance it fails with following errors:
ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [6856], [0], [60], [], [], []
and then SMON abnormally crashes the database instance.
Cause of the Problem
These are oracle bugs and these ORA-600 [6006] and ORA-600 [6856] bug occurred whenever undo segments are trying to rollback a failed transaction and cannot.
For error "ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], []" ,
Oracle is undoing an index leaf key operation. If the key is not found, ORA-00600 [6006] is logged.
For error "ORA-00600: internal error code, arguments: [6856], [0], [60], [], [], []" , SMON is trying to recover a dead transaction. But the undo application runs into an internal error (trying to delete a row that is already deleted).
Solution of the Problem
1) Review the trace files and look for the object(s) involved. If the trace file does not have a SQL statement, search on the following: "block dump header"
2) In the block header there will be a seg/obj = hex value. Convert the hex to dec and this will give you the data_object_id.
3) The alert.log may also show the affected object, for example:
ORACLE Instance ORCL (pid = 8) - Error 600 encountered while recovering transaction (9999, 36) on object 45879.
SQL>select owner, object_name from dba_objects where data_object_id = object#;
This will be the object you need to work with.
To implement the solution:
1. shutdown the oracle instance.
SQL> shut immediate
2. set event - event="10513 trace name context forever, level 2" (this event and setting disables transaction recovery which was initiated by SMON). Open the pfile and add the following line inside pfile.
event="10513 trace name context forever, level 2"
3. startup instance
SQL> startup
4.
- If the object is an index - drop and recreate.
- If the object is a table - drop / export / Create Table as Select (CTAS) to change the object ID
5. shutdown the instance
SQL> shut immediate
6. remove the event
Open the pfile and remove the event line that was added.
7. startup the instance
SQL> startup
8. recreate the affected object(s)
While starting up the instance it fails with following errors:
ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [6856], [0], [60], [], [], []
and then SMON abnormally crashes the database instance.
Cause of the Problem
These are oracle bugs and these ORA-600 [6006] and ORA-600 [6856] bug occurred whenever undo segments are trying to rollback a failed transaction and cannot.
For error "ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], []" ,
Oracle is undoing an index leaf key operation. If the key is not found, ORA-00600 [6006] is logged.
For error "ORA-00600: internal error code, arguments: [6856], [0], [60], [], [], []" , SMON is trying to recover a dead transaction. But the undo application runs into an internal error (trying to delete a row that is already deleted).
Solution of the Problem
1) Review the trace files and look for the object(s) involved. If the trace file does not have a SQL statement, search on the following: "block dump header"
2) In the block header there will be a seg/obj = hex value. Convert the hex to dec and this will give you the data_object_id.
3) The alert.log may also show the affected object, for example:
ORACLE Instance ORCL (pid = 8) - Error 600 encountered while recovering transaction (9999, 36) on object 45879.
SQL>select owner, object_name from dba_objects where data_object_id = object#;
This will be the object you need to work with.
To implement the solution:
1. shutdown the oracle instance.
SQL> shut immediate
2. set event - event="10513 trace name context forever, level 2" (this event and setting disables transaction recovery which was initiated by SMON). Open the pfile and add the following line inside pfile.
event="10513 trace name context forever, level 2"
3. startup instance
SQL> startup
4.
- If the object is an index - drop and recreate.
- If the object is a table - drop / export / Create Table as Select (CTAS) to change the object ID
5. shutdown the instance
SQL> shut immediate
6. remove the event
Open the pfile and remove the event line that was added.
7. startup the instance
SQL> startup
8. recreate the affected object(s)
ORA-600 [17147] ORA-600 [Kghlkrem1] ORA-7445 kghlkremf() database crash
During normal database activity, in alert log the following ORA-00600 error occurred and it causes database crashed.
The call stack is as follows:
kghfrmrg kghfre qsmqktcc ktcdso ktcrcm ktdcmt k2lcom k2send xctctl xctcom_with_options opicom
or
k2send xctctl xctcom_with_options opicom opiodr
or
kghalo ktcccadd kwqidracbk kwqidcpmc kwqidafm0 kwqididqx kpoaqdq
or
kghadd_reserved_extent kghget_reserved_extent kghgex kghfnd
Cause of the Problem
The problem occurred due to oracle bug. The bug involves abnormal termination of queries involving bloom filters. This manifests itself as memory corruptions. The memory was freed, reallocated to some other client, and corrupted but slaves were still writing into it.
Solution of the Problem
Solution 01:
This bug is fixed in oracle database version 10.2.0.4. So apply 10.2.0.4 patchset to solve this problem.
Solution 02:
Workaround, you can set the oracle hidden parameter _bloom_filter_enabled to false.
This parameter can be set dynamically:
If you are using spfile then just set as,
SQL> connect / as sysdba
SQL> alter system set "_bloom_filter_enabled"=false scope=both;
If you use pfile to startup your database remove the scope parameter that is use,
SQL> alter system set "_bloom_filter_enabled"=false;
ORA-00600: internal error code, arguments: [17147], [0xC0000003E4CE10A5], [], [], [], [], [], []If you look at the trace file you no longer will see any sql that causes this bug.
ORA-00600: internal error code, arguments: [KGHLKREM1], [0xC0000003E4CE10B0], [], [], [], []
ORA-7445: exception encountered: core dump [kghlkremf()+33] [SIGSEGV]
The call stack is as follows:
kghfrmrg kghfre qsmqktcc ktcdso ktcrcm ktdcmt k2lcom k2send xctctl xctcom_with_options opicom
or
k2send xctctl xctcom_with_options opicom opiodr
or
kghalo ktcccadd kwqidracbk kwqidcpmc kwqidafm0 kwqididqx kpoaqdq
or
kghadd_reserved_extent kghget_reserved_extent kghgex kghfnd
Cause of the Problem
The problem occurred due to oracle bug. The bug involves abnormal termination of queries involving bloom filters. This manifests itself as memory corruptions. The memory was freed, reallocated to some other client, and corrupted but slaves were still writing into it.
Solution of the Problem
Solution 01:
This bug is fixed in oracle database version 10.2.0.4. So apply 10.2.0.4 patchset to solve this problem.
Solution 02:
Workaround, you can set the oracle hidden parameter _bloom_filter_enabled to false.
This parameter can be set dynamically:
If you are using spfile then just set as,
SQL> connect / as sysdba
SQL> alter system set "_bloom_filter_enabled"=false scope=both;
If you use pfile to startup your database remove the scope parameter that is use,
SQL> alter system set "_bloom_filter_enabled"=false;
ORA-00600 [730] [SPACE LEAK] OR ORA-600 [LIBRARYCACHENOTEMPTYONCLOSE] during shutdown
Problem Description
While shutting down oracle database the following error occurred.
In oracle 10g and 11g,
ORA-00600: internal error code, arguments: [730], [4100736], [space leak], [], [], [], [], [], [], [], [], []
If you observe stack trace it will look like,
ksesic2 <- ksmshu <- opistp_real <- opistp <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- main <- start
In oracle 10g the error looks like,
ORA-00600: internal error code, arguments: [LibraryCacheNotEmptyOnClose], [], [] ,[], [], [], [], []
If you observe stack trace it will look like, kglshu <- kqlnfy <- kscnfy <- ksmshu <- opistp_real <- opistp <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real
Cause of the Problem
The error occurred due to oracle bug. Oracle named this bug as BUG 7572335. This happens during database shutdown when child cursors had been previously marked as kept in the shared pool. Note that this is a non-corruptive error.
Solution of the Problem
Solution 01: This bug is solved in Oracle database 11gR2. So upgrade to Oracle database 11.2 or higher is one solution.
Solution 02: If you are using Oracle database 10gR2 then up to patchset version 10.2.0.4 this bug remained and it is solved in version 10.2.0.5. You can upgrade to oracle version 10.2.0.5 by applying the Patchset, Patch 8202632.
Solution 03: If available for your platform and version, you can apply one off patch 7572335. Download and apply the fix, Patch 7572335
While shutting down oracle database the following error occurred.
In oracle 10g and 11g,
ORA-00600: internal error code, arguments: [730], [4100736], [space leak], [], [], [], [], [], [], [], [], []
If you observe stack trace it will look like,
ksesic2 <- ksmshu <- opistp_real <- opistp <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- main <- start
In oracle 10g the error looks like,
ORA-00600: internal error code, arguments: [LibraryCacheNotEmptyOnClose], [], [] ,[], [], [], [], []
If you observe stack trace it will look like, kglshu <- kqlnfy <- kscnfy <- ksmshu <- opistp_real <- opistp <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real
Cause of the Problem
The error occurred due to oracle bug. Oracle named this bug as BUG 7572335. This happens during database shutdown when child cursors had been previously marked as kept in the shared pool. Note that this is a non-corruptive error.
Solution of the Problem
Solution 01: This bug is solved in Oracle database 11gR2. So upgrade to Oracle database 11.2 or higher is one solution.
Solution 02: If you are using Oracle database 10gR2 then up to patchset version 10.2.0.4 this bug remained and it is solved in version 10.2.0.5. You can upgrade to oracle version 10.2.0.5 by applying the Patchset, Patch 8202632.
Solution 03: If available for your platform and version, you can apply one off patch 7572335. Download and apply the fix, Patch 7572335
Friday, June 25, 2010
In RAC flashback transaction query fails with ORA-600[kcbgtcr_13]
Problem Description
In Oracle database 11gR1 RAC environment flashback transaction queries fails with Oracle bug ORA-600[kcbgtcr_13].
For example,
SQL> SELECT UNDO_SQL FROM FLASHBACK_TRANSACTION_QUERY WHERE XID = '08001C00ACD20901';
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [], [], [], [], [], [], []
If you investigate into trace file you will see error stack as,
kcbgtcr <- ktuq_get_urec <- ktuqup_get_startslot <- ktuqup_init <- ktuqqp_fetch_next_rec <- ktuqtpgv <- krvxInvokeCallbacks <- krvxrfro_FlashReadOne <- krvxrporr_ProcessOneRedoRecord <- krvxread <- krvxgtsp_GetTxnSingleProcess <- krvxgt <- ktuqfcbk <-qerfxFetch <- opifch2 <- kpoal8 <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real <- main
Cause of the Problem
This is oracle bug and oracle named this bug as bug 7484261.
Solution of the Problem
This issue is fixed in Oracle database 11gR2. So upgrade your database to 11.2g will solve the problem.
If you are still using 11gR1 then download and apply Patch 7484261 from Metalink. Based on your platform and oracle version download from link https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=7484261
In Oracle database 11gR1 RAC environment flashback transaction queries fails with Oracle bug ORA-600[kcbgtcr_13].
For example,
SQL> SELECT UNDO_SQL FROM FLASHBACK_TRANSACTION_QUERY WHERE XID = '08001C00ACD20901';
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [], [], [], [], [], [], []
If you investigate into trace file you will see error stack as,
kcbgtcr <- ktuq_get_urec <- ktuqup_get_startslot <- ktuqup_init <- ktuqqp_fetch_next_rec <- ktuqtpgv <- krvxInvokeCallbacks <- krvxrfro_FlashReadOne <- krvxrporr_ProcessOneRedoRecord <- krvxread <- krvxgtsp_GetTxnSingleProcess <- krvxgt <- ktuqfcbk <-qerfxFetch <- opifch2 <- kpoal8 <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real <- main
Cause of the Problem
This is oracle bug and oracle named this bug as bug 7484261.
Solution of the Problem
This issue is fixed in Oracle database 11gR2. So upgrade your database to 11.2g will solve the problem.
If you are still using 11gR1 then download and apply Patch 7484261 from Metalink. Based on your platform and oracle version download from link https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=7484261
ORA-00600: internal error code, arguments: [4819]
Problem Description
While reviewing database alert log files it was report an ORA-600 [4819] error followed by ORA-00308 and ORA-27037 errors like below.
ORA-00600: internal error code, arguments: [4819], [], [], [], [], [], [], []
ORA-00308: cannot open archived log '/home/oracle/archive/arc_ARJU_001.arc'
ORA-27037: unable to obtain file status
This problem started to happen because recently database has been switched into NOARCHIVELOG mode from ARCHIVELOG mode.
Cause of the Problem
The problem arises because the LOG_ARCHIVE_DEST_n parameter is still set to point to an archive log destination and the default value for the LOG_ARCHIVE_DEST_STATE_n parameter is 'ENABLE'. Based on these two settings database still attempts to open the required archive log file during transaction recovery.
Solution of the Problem
To resolve this problem, mark the destination as deferred and null out the archive destination, That is
1) Login as sysdba.
SQL> connect / as sysdba
2) Set the archive_dest_state_1 to defer.
SQL> alter system set log_archive_dest_state_1 = defer scope=both;
3) Set log_archive_dest_1 to null.
SQL> alter system set log_archive_dest_1 = '' scope=both;
You are done!
While reviewing database alert log files it was report an ORA-600 [4819] error followed by ORA-00308 and ORA-27037 errors like below.
ORA-00600: internal error code, arguments: [4819], [], [], [], [], [], [], []
ORA-00308: cannot open archived log '/home/oracle/archive/arc_ARJU_001.arc'
ORA-27037: unable to obtain file status
This problem started to happen because recently database has been switched into NOARCHIVELOG mode from ARCHIVELOG mode.
Cause of the Problem
The problem arises because the LOG_ARCHIVE_DEST_n parameter is still set to point to an archive log destination and the default value for the LOG_ARCHIVE_DEST_STATE_n parameter is 'ENABLE'. Based on these two settings database still attempts to open the required archive log file during transaction recovery.
Solution of the Problem
To resolve this problem, mark the destination as deferred and null out the archive destination, That is
1) Login as sysdba.
SQL> connect / as sysdba
2) Set the archive_dest_state_1 to defer.
SQL> alter system set log_archive_dest_state_1 = defer scope=both;
3) Set log_archive_dest_1 to null.
SQL> alter system set log_archive_dest_1 = '' scope=both;
You are done!
Thursday, June 10, 2010
crosscheck archivelog fails with validation failed for archived log
Problem Description
Whenever I run "crosscheck archivelog all" it failed with message "validation failed for archived log" like below.
The problem happened because archive log file destination was changed or someone deleted the archive log files manually using OS commands.
Solution of the Problem
The solution is to delete the expired archive log files. So do following,
$ rman
RMAN> connect target /
RMAN> crosscheck archivelog all;
RMAN> delete expired archivelog all;
Whenever I run "crosscheck archivelog all" it failed with message "validation failed for archived log" like below.
RMAN> crosscheck archivelog all;Cause of the Problem
released channel: ORA_DISK_1
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=323 devtype=DISK
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_3_5bkc1o5q_.arc recid=6 stamp=697208725
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_4_5bkd60x4_.arc recid=7 stamp=697209883
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_5_5bkd78q9_.arc recid=8 stamp=697209922
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_6_5bkf2rwx_.arc recid=9 stamp=697210802
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_7_5blbdsqs_.arc recid=10 stamp=697240820
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_10/o1_mf_1_8_5blc2tvx_.arc recid=11 stamp=697241524
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_11/o1_mf_1_9_5bm1t7hk_.arc recid=12 stamp=697264802
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_11/o1_mf_1_10_5bmqwzr6_.arc recid=13 stamp=697287417
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_11/o1_mf_1_11_5bnjqg4n_.arc recid=14 stamp=697312840
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_11/o1_mf_1_12_5bnytoy5_.arc recid=15 stamp=697327280
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_12/o1_mf_1_13_5boplo7l_.arc recid=16 stamp=697351600
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_12/o1_mf_1_14_5bp6vv6w_.arc recid=17 stamp=697368277
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_12/o1_mf_1_15_5bpdtlfn_.arc recid=18 stamp=697374380
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_13/o1_mf_1_16_5br3nh19_.arc recid=19 stamp=697430505
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_13/o1_mf_1_17_5brvlghg_.arc recid=20 stamp=697455015
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_14/o1_mf_1_18_5btzdcv5_.arc recid=21 stamp=697524454
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_14/o1_mf_1_19_5bw0drf0_.arc recid=22 stamp=697558259
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_14/o1_mf_1_20_5bwvyksr_.arc recid=23 stamp=697586476
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_15/o1_mf_1_21_5bxmsqql_.arc recid=24 stamp=697610898
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_15/o1_mf_1_22_5bz0h5on_.arc recid=25 stamp=697656639
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_15/o1_mf_1_23_5bzjbvg2_.arc recid=26 stamp=697672884
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_16/o1_mf_1_24_5c0lgk9r_.arc recid=27 stamp=697707819
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_16/o1_mf_1_25_5c24o21l_.arc recid=28 stamp=697759227
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_17/o1_mf_1_26_5c30s9py_.arc recid=29 stamp=697788035
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_17/o1_mf_1_27_5c4s0wyy_.arc recid=30 stamp=697845623
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_18/o1_mf_1_28_5c5k0jrj_.arc recid=31 stamp=697870186
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_18/o1_mf_1_29_5c6z1g0p_.arc recid=32 stamp=697917319
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_18/o1_mf_1_30_5c7mqfl8_.arc recid=33 stamp=697938502
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_19/o1_mf_1_31_5c89jkhw_.arc recid=34 stamp=697960811
validation failed for archived log
archive log filename=/u01/app/oracle/archivelog/2009_09_19/o1_mf_1_32_5c9ggk5p_.arc recid=35 stamp=697998634
validation succeeded for archived log
archive log filename=/u01/app/oracle/archivelog/2009_12_02/o1_mf_1_343_5kdfc1sv_.arc recid=36 stamp=704560058
validation succeeded for archived log
archive log filename=/u01/app/oracle/archivelog/2009_12_02/o1_mf_1_344_5kdfy3pd_.arc recid=37 stamp=704560667
validation succeeded for archived log
archive log filename=/u05/archive/%t_%s_%r.arc1_345_697204983.dbf recid=38 stamp=704560901
validation succeeded for archived log
archive log filename=/u05/archive/1_346_697204983.dbf recid=39 stamp=704561258
validation succeeded for archived log
archive log filename=/u05/archive/1_347_697204983.dbf recid=40 stamp=704561835
validation succeeded for archived log
archive log filename=/u05/archive/1_348_697204983.dbf recid=41 stamp=704561992
validation succeeded for archived log
archive log filename=/u05/archive/1_349_697204983.dbf recid=42 stamp=704562987
validation succeeded for archived log
archive log filename=/u05/archive/1_350_697204983.dbf recid=43 stamp=704562987
validation succeeded for archived log
archive log filename=/u05/archive/1_351_697204983.dbf recid=44 stamp=704562992
validation succeeded for archived log
archive log filename=/u05/archive/1_352_697204983.dbf recid=45 stamp=704563257
validation succeeded for archived log
archive log filename=/u05/archive/1_353_697204983.dbf recid=46 stamp=704563321
validation succeeded for archived log
archive log filename=/u05/archive/1_354_697204983.dbf recid=47 stamp=704563408
validation succeeded for archived log
archive log filename=/u05/archive/1_355_697204983.dbf recid=48 stamp=704563434
validation succeeded for archived log
archive log filename=/u05/archive/1_356_697204983.dbf recid=49 stamp=704584836
validation succeeded for archived log
archive log filename=/u05/archive/1_357_697204983.dbf recid=50 stamp=704586204
validation succeeded for archived log
archive log filename=/u05/archive/1_358_697204983.dbf recid=51 stamp=704611511
validation succeeded for archived log
archive log filename=/u05/archive/1_359_697204983.dbf recid=52 stamp=704648675
validation succeeded for archived log
archive log filename=/u05/archive/1_360_697204983.dbf recid=53 stamp=704650176
validation succeeded for archived log
archive log filename=/u05/archive/1_361_697204983.dbf recid=54 stamp=704650675
validation succeeded for archived log
archive log filename=/u05/archive/1_362_697204983.dbf recid=55 stamp=704671244
validation succeeded for archived log
archive log filename=/u05/archive/1_363_697204983.dbf recid=56 stamp=704692487
validation succeeded for archived log
archive log filename=/u05/archive/1_364_697204983.dbf recid=57 stamp=704708839
validation succeeded for archived log
archive log filename=/u05/archive/1_365_697204983.dbf recid=59 stamp=704736263
validation succeeded for archived log
archive log filename=/u05/archive/1_366_697204983.dbf recid=58 stamp=704736262
validation succeeded for archived log
archive log filename=/u05/archive/1_367_697204983.dbf recid=60 stamp=704736266
validation succeeded for archived log
archive log filename=/u05/archive/1_368_697204983.dbf recid=61 stamp=704737398
Crosschecked 56 objects
The problem happened because archive log file destination was changed or someone deleted the archive log files manually using OS commands.
Solution of the Problem
The solution is to delete the expired archive log files. So do following,
$ rman
RMAN> connect target /
RMAN> crosscheck archivelog all;
RMAN> delete expired archivelog all;
Wednesday, June 9, 2010
Catchable fatal error: Object of class WP_Error could not be converted to string
Problem Description
In the environment of
WPMU 2.9.2
BP 1.2.3
PHP 5.2.13
BuddyPress Groupblog 1.4.4
the buddypress groupblog accessing blog tab fails with error below.
Cause of the problem
The problem happened due to bug of BuddyPress Groupblog version 1.4.4.
Solution of the problem
Since this is bug of this BuddyPress Groupblog 1.4.4 version so it is expected bug will be resolved in the next version of this plugin. However you can follow following workaround to make the plugin or blog section work.
The idea is don't create the group and groupblog at the same time.
Step 01: Visit the Site Admin > Blogs. Create a new Blog. Ensure that blog is created by visiting wp-admin/wpmu-blogs.php
Step 02: From the frontend after login as admin Create a group as usual but do not create a new groupblog. Instead at 3. Group Blog click the checkbox "Enable group blog" and then Use one of your own available blogs and from dropdown box select your blog.
Step 03: Go through next steps and you will see your blog is working perfect.
In the environment of
WPMU 2.9.2
BP 1.2.3
PHP 5.2.13
BuddyPress Groupblog 1.4.4
the buddypress groupblog accessing blog tab fails with error below.
Catchable fatal error: Object of class WP_Error could not be converted to string in /home/kunstfac/public_html/wp-includes/wp-db.php on line 488The url will be like, http://{website_name}/groups/{group_name}/blog/
Cause of the problem
The problem happened due to bug of BuddyPress Groupblog version 1.4.4.
Solution of the problem
Since this is bug of this BuddyPress Groupblog 1.4.4 version so it is expected bug will be resolved in the next version of this plugin. However you can follow following workaround to make the plugin or blog section work.
The idea is don't create the group and groupblog at the same time.
Step 01: Visit the Site Admin > Blogs. Create a new Blog. Ensure that blog is created by visiting wp-admin/wpmu-blogs.php
Step 02: From the frontend after login as admin Create a group as usual but do not create a new groupblog. Instead at 3. Group Blog click the checkbox "Enable group blog" and then Use one of your own available blogs and from dropdown box select your blog.
Step 03: Go through next steps and you will see your blog is working perfect.
Friday, June 4, 2010
How to change compatible parameter in Oracle
Overview of Compatible parameter
Oracle compatible parameter is a string data type parameter and value of this parameter is oracle database version. It can take value as 11.1.0.7, 11.1.0.6, 10.2.0.4, 10.2.0.3, 10.1.0.5, 10.1.0.4, 9.2.0.8, 9.0.1.4, 8.1.7.4 etc. This parameter controls the database behavior for example whether a feature will work for a database. For example if database version is 10.2.0.4 but compatible parameter is set to 9.2 then certain feature like RMAN compression will not work.
After the database upgrade, if the compatible parameter is set to database version then new feature stores any data on disk (including data dictionary changes) that cannot be processed with your previous release. However after upgrade if compatible parameter is not changed then new feature of the upgraded version will not be available.
Default, minimum and maximum value of compatible parameter
Here goes the compatible parameter default, minimum and maximum values based on oracle database version.
In order to check your compatible parameter issue,
SQL> SELECT name, value, description FROM v$parameter WHERE name = 'compatible';
or if you use sql*plus issue,
SQL> show parameter compatible;
Steps to change compatible parameter
1) Perform full backup of your database (optional).
Before changing compatible parameter, you should take a full backup of your database. Because raising the COMPATIBLE initialization parameter may cause your database to become incompatible with earlier releases of the Oracle Database, and a backup ensures that you can return to the earlier release whenever you want.
2) If you are using spfile parameter to start up your database then complete the following things.
a. Update the server parameter file to set or change the value of the COMPATIBLE initialization parameter.
For example, to set the COMPATIBLE initialization parameter to 10.2.0, issue the following statement:
SQL> ALTER SYSTEM SET COMPATIBLE = '10.2.0' SCOPE=SPFILE;
b. Shut down and restart the instance.
SQL> SHUTDOWN IMMEDIATE
SQL> STARTUP
3) If you are using pfile parameter, then complete the following steps.
a. Shut down the instance if it is running:
SQL> SHUTDOWN IMMEDIATE
b. Edit the initialization parameter file to set or change the value of the COMPATIBLE initialization parameter.
For example, to set the COMPATIBLE initialization parameter to 10.2.0, enter the following in the initialization parameter file:
COMPATIBLE = 10.2.0
c. Start the instance using STARTUP.
SQL> STARTUP
Related Documents
http://arjudba.blogspot.com/2008/04/list-of-oracle-database-version-release.html
http://arjudba.blogspot.com/2008/04/oracle-database-editions.html
http://arjudba.blogspot.com/2008/04/oracle-products.html
http://arjudba.blogspot.com/2008/04/timeline-of-oracle-rdbms-major-release.html
http://arjudba.blogspot.com/2008/04/history-of-oracle-corporation.html
http://arjudba.blogspot.com/2008/10/list-of-patchset-number-in-metalink.html
http://arjudba.blogspot.com/2010/01/how-to-know-whether-patches-applied-to.html
Oracle compatible parameter is a string data type parameter and value of this parameter is oracle database version. It can take value as 11.1.0.7, 11.1.0.6, 10.2.0.4, 10.2.0.3, 10.1.0.5, 10.1.0.4, 9.2.0.8, 9.0.1.4, 8.1.7.4 etc. This parameter controls the database behavior for example whether a feature will work for a database. For example if database version is 10.2.0.4 but compatible parameter is set to 9.2 then certain feature like RMAN compression will not work.
After the database upgrade, if the compatible parameter is set to database version then new feature stores any data on disk (including data dictionary changes) that cannot be processed with your previous release. However after upgrade if compatible parameter is not changed then new feature of the upgraded version will not be available.
Default, minimum and maximum value of compatible parameter
Here goes the compatible parameter default, minimum and maximum values based on oracle database version.
In order to check your compatible parameter issue,
SQL> SELECT name, value, description FROM v$parameter WHERE name = 'compatible';
or if you use sql*plus issue,
SQL> show parameter compatible;
Steps to change compatible parameter
1) Perform full backup of your database (optional).
Before changing compatible parameter, you should take a full backup of your database. Because raising the COMPATIBLE initialization parameter may cause your database to become incompatible with earlier releases of the Oracle Database, and a backup ensures that you can return to the earlier release whenever you want.
2) If you are using spfile parameter to start up your database then complete the following things.
a. Update the server parameter file to set or change the value of the COMPATIBLE initialization parameter.
For example, to set the COMPATIBLE initialization parameter to 10.2.0, issue the following statement:
SQL> ALTER SYSTEM SET COMPATIBLE = '10.2.0' SCOPE=SPFILE;
b. Shut down and restart the instance.
SQL> SHUTDOWN IMMEDIATE
SQL> STARTUP
3) If you are using pfile parameter, then complete the following steps.
a. Shut down the instance if it is running:
SQL> SHUTDOWN IMMEDIATE
b. Edit the initialization parameter file to set or change the value of the COMPATIBLE initialization parameter.
For example, to set the COMPATIBLE initialization parameter to 10.2.0, enter the following in the initialization parameter file:
COMPATIBLE = 10.2.0
c. Start the instance using STARTUP.
SQL> STARTUP
Related Documents
http://arjudba.blogspot.com/2008/04/list-of-oracle-database-version-release.html
http://arjudba.blogspot.com/2008/04/oracle-database-editions.html
http://arjudba.blogspot.com/2008/04/oracle-products.html
http://arjudba.blogspot.com/2008/04/timeline-of-oracle-rdbms-major-release.html
http://arjudba.blogspot.com/2008/04/history-of-oracle-corporation.html
http://arjudba.blogspot.com/2008/10/list-of-patchset-number-in-metalink.html
http://arjudba.blogspot.com/2010/01/how-to-know-whether-patches-applied-to.html
Thursday, June 3, 2010
java.util.zip.ZipException: reading zip file central directory failed
Problem Description
While applying opatch, from the front end I see following message.
From the logfile,
Cause of the Problem
The file "$ORACLE_HOME/rdbms/jlib/qsma.jar" itself is corrupted
The command jar tvf $ORACLE_HOME/rdbms/jlib/qsma.jar generates "java.util.zip.ZipException: unexpected EOF"
Solution of the Problem
For 10.2.0.4, we can consider extracting this file from 10.2.0.4 patchset staging kit.
- cd 10.2.0.4 Patchset Staging kit>/Disk1/stage/Patches/oracle.rdbms.rsf/10.2.0.4.0/1/DataFiles
- mkdir /tmp/qsma_jar
- cp filegroup4.1.1.jar /tmp/qsma_jar
- cd /tmp/qsma_jar
- jar xvf filegroup4.1.1.jar
-- This will give many files , including "qsma.jar" .
Rename the existing qsma.jar in the 10.2.0.4 Oracle Home and replace it with the file present in
-- /tmp/qsma_jar
Now use the ORACLE_HOME/jdk/bin/jar to check if we can list contents of this qsma.jar
If successful, proceed to apply the remaining CPU patch.
While applying opatch, from the front end I see following message.
Do you want to proceed? [y|n]
y
User Responded with: Y
Running make for target iextjob
Running make for target iextjobo
Running make for target client_sharedlib
Running make for target idgmgrl
Running make for target ioracle
Running make for target client_sharedlib
Running make for target itnslsnr
Running make for target iwrap
Running make for target genplusso
ApplySession adding interim patch '9119284' to inventory
Verifying the update...
Inventory check OK: Patch ID 9119284 is registered in Oracle Home inventory with proper meta-data.
java.util.zip.ZipException: unexpected EOF
at java.util.zip.ZipInputStream.read(ZipInputStream.java:159)
at java.io.FilterInputStream.read(FilterInputStream.java:90)
at oracle.opatch.MultiJarUtil.writeOneZipEntryToFile(MultiJarUtil.java:422)
at oracle.opatch.MultiJarUtil.doVerifyLeafLevel(MultiJarUtil.java:259)
at oracle.opatch.JarActionHelper.verify(JarActionHelper.java:291)
at oracle.opatch.JarAction.verifyMultiLevel(JarAction.java:2101)
at oracle.opatch.JarAction.verify(JarAction.java:2022)
at oracle.opatch.OPatchSessionHelper.verifyPatchAction(OPatchSessionHelper.java:2928)
at oracle.opatch.ApplySession.verify(ApplySession.java:2781)
at oracle.opatch.ApplySession.wasThisPatchApplied(ApplySession.java:2391)
at oracle.opatch.ApplySession.processLocal(ApplySession.java:4463)
at oracle.opatch.ApplySession.process(ApplySession.java:5584)
at oracle.opatch.OPatchSession.main(OPatchSession.java:1718)
at oracle.opatch.OPatch.main(OPatch.java:630)
There are 3 jar files under ORACLE_HOME that are not patched.
Files check failed: Some files under ORACLE_HOME are not patched. Please see log file for details.
ApplySession failed: ApplySession failed in system modification phase... 'Verification of patch failed: Files are not updated completely.'
OPatch will attempt to restore the system...
Restoring the Oracle Home...
Checking if OPatch needs to invoke 'make' to restore some binaries...
OPatch was able to restore your system. Look at log file and timestamp of each file to make sure your system is in the state prior to appl ying the patch.
--------------------------------------------------------------------------------
The following warnings have occurred during OPatch execution:
1) OUI-67160:ApplySession for patch ID '9119284': Could not back up following actions for patch rollback:
1) null
2) null
3) null
2) OUI-67124:java.lang.NullPointerException
3) OUI-67124:Files check failed: Some files under ORACLE_HOME are not patched. Please see log file for details.
--------------------------------------------------------------------------------
OPatch failed with error code 73
From the logfile,
INFO:Is the local system ready for patching? [y|n]
INFO:Start to wait for user-input at Fri May 28 00:29:59 EDT 2010
INFO:Finish waiting for user-input at Fri May 28 00:30:05 EDT 2010
INFO:User Responded with: Y
INFO:Start saving patch at Fri May 28 00:30:05 EDT 2010
INFO:Finish saving patch at Fri May 28 00:30:15 EDT 2010
INFO:Start backing up system for restore at Fri May 28 00:30:15 EDT 2010
INFO:Backing up files and inventory (not for auto-rollback) for the Oracle Home
INFO:Backing up files affected by the patch '9119284' for restore. This might take a while...
INFO:Finish backing up system for restore at Fri May 28 00:30:32 EDT 2010
INFO:******* create file /SIDS/app/oracle/product/10.2.0/db/.patch_storage/interim_inventory.txt
INFO:Start backing up system for rollback at Fri May 28 00:30:32 EDT 2010
INFO:Backing up files affected by the patch '9119284' for rollback. This might take a while...
INFO:Going to backup for rollback Jar Action(s) in a consolidated mode.
INFO:Jar Path -> /SIDS/app/oracle/product/10.2.0/db/rdbms/jlib/qsma.jar
INFO:--- Clubbed Actions for this Jar Path to backup for rollback NOW ->
INFO:Backup For Rollback starts for the clubbed jars at --> Fri May 28 00:30:37 EDT 2010
INFO:Stack Description: java.util.zip.ZipException: reading zip file central directory failed
INFO:StackTrace: java.util.zip.ZipFile.open(Native Method)
INFO:StackTrace: java.util.zip.ZipFile.(ZipFile.java:123)
INFO:StackTrace: java.util.jar.JarFile.(JarFile.java:127)
INFO:StackTrace: java.util.jar.JarFile.(JarFile.java:92)
INFO:StackTrace: oracle.opatch.JarActionHelper.backupForRollback(JarActionHelper.java:1032)
INFO:StackTrace: oracle.opatch.JarAction.backupForRollback(JarAction.java:1575)
INFO:StackTrace: oracle.opatch.PatchObject.backupForRollback(PatchObject.java:4339)
INFO:StackTrace: oracle.opatch.ApplySession.processLocal(ApplySession.java:4388)
INFO:StackTrace: oracle.opatch.ApplySession.process(ApplySession.java:5584)
INFO:StackTrace: oracle.opatch.OPatchSession.main(OPatchSession.java:1718)
INFO:StackTrace: oracle.opatch.OPatch.main(OPatch.java:630)
INFO:Stack Description: java.lang.NullPointerException
INFO:StackTrace: oracle.opatch.JarActionHelper.backupForRollback(JarActionHelper.java:1204)
INFO:StackTrace: oracle.opatch.JarAction.backupForRollback(JarAction.java:1575)
INFO:StackTrace: oracle.opatch.PatchObject.backupForRollback(PatchObject.java:4339)
INFO:StackTrace: oracle.opatch.ApplySession.processLocal(ApplySession.java:4388)
INFO:StackTrace: oracle.opatch.ApplySession.process(ApplySession.java:5584)
INFO:StackTrace: oracle.opatch.OPatchSession.main(OPatchSession.java:1718)
INFO:StackTrace: oracle.opatch.OPatch.main(OPatch.java:630)
INFO:Going to backup for rollback Jar Action(s) in a consolidated mode.
INFO:Jar Path -> /SIDS/app/oracle/product/10.2.0/db/rdbms/jlib/qsma.jar
INFO:--- Clubbed Actions for this Jar Path to backup for rollback NOW ->
INFO:Backup For Rollback starts for the clubbed jars at --> Fri May 28 00:30:37 EDT 2010
INFO:Stack Description: java.util.zip.ZipException: reading zip file central directory failed
The following actions have failed:
WARNING:OUI-67124:java.lang.NullPointerException
INFO:
Do you want to proceed? [y|n]
INFO:Start to wait for user-input at Fri May 28 01:43:18 EDT 2010
INFO:Finish waiting for user-input at Fri May 28 01:45:24 EDT 2010
INFO:User Responded with: Y
INFO:Running make for target iextjob
INFO:Start invoking 'make' at Fri May 28 01:45:25 EDT 2010Fri May 28 01:45:25 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:45:28 EDT 2010
INFO:Running make for target iextjobo
INFO:Start invoking 'make' at Fri May 28 01:45:28 EDT 2010Fri May 28 01:45:28 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:45:29 EDT 2010
INFO:Running make for target client_sharedlib
INFO:Start invoking 'make' at Fri May 28 01:45:29 EDT 2010Fri May 28 01:45:29 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:47:12 EDT 2010
INFO:Running make for target idgmgrl
INFO:Start invoking 'make' at Fri May 28 01:47:12 EDT 2010Fri May 28 01:47:12 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:47:14 EDT 2010
INFO:Running make for target ioracle
INFO:Start invoking 'make' at Fri May 28 01:47:14 EDT 2010Fri May 28 01:47:14 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:48:18 EDT 2010
INFO:Running make for target client_sharedlib
INFO:Start invoking 'make' at Fri May 28 01:48:18 EDT 2010Fri May 28 01:48:18 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:49:42 EDT 2010
INFO:Running make for target itnslsnr
INFO:Start invoking 'make' at Fri May 28 01:49:42 EDT 2010Fri May 28 01:49:42 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:49:44 EDT 2010
INFO:Running make for target iwrap
INFO:Start invoking 'make' at Fri May 28 01:49:44 EDT 2010Fri May 28 01:49:44 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:49:54 EDT 2010
INFO:Running make for target genplusso
INFO:Start invoking 'make' at Fri May 28 01:49:54 EDT 2010Fri May 28 01:49:54 EDT 2010
INFO:Finish invoking 'make' at Fri May 28 01:49:56 EDT 2010
INFO:Finish modifying the system at Fri May 28 01:49:57 EDT 2010
INFO:ApplySession adding interim patch '9119284' to inventory
INFO:Start saving patch to inventory at Fri May 28 01:49:57 EDT 2010
INFO:Registered Encoding value is : 8859_1 for the file /SIDS/app/oracle/product/10.2.0/db/inventory/oneoffs/9119284/etc/config/actions.xml
INFO:Finish saving patch to inventory at Fri May 28 01:50:03 EDT 2010
INFO:PatchObject::createPatchObject() Patch location is /SIDS/app/oracle/product/10.2.0/db/inventory/oneoffs/9119284
INFO:PatchObject::createPatchObject() patch location is /SIDS/app/oracle/product/10.2.0/db/inventory/oneoffs/9119284
INFO:
Verifying the update...
INFO:PatchObject::PatchObject() Patch location is /nas/sysadm/oracle/psu/HP/9119284
INFO:PatchObject::createPatchObject() Patch location is /nas/sysadm/oracle/psu/HP/9119284
INFO:PatchObject::createPatchObject() patch location is /nas/sysadm/oracle/psu/HP/9119284
INFO:PatchObject::createPatchObject() Patch location is /SIDS/app/oracle/product/10.2.0/db/inventory/oneoffs/9119284
INFO:PatchObject::createPatchObject() patch location is /SIDS/app/oracle/product/10.2.0/db/inventory/oneoffs/9119284
INFO:Inventory check OK: Patch ID 9119284 is registered in Oracle Home inventory with proper meta-data.
INFO: verifying 38 copy files.
INFO: verifying 4 jar files.
INFO:Verify top-level entry oracle/CDC/Purge.class using file /SIDS/app/oracle/product/10.2.0/db/.patch_storage/9119284_Dec_28_2009_03_45_13/scratch/work/Purge.class and /nas/sysadm/oracle/psu/HP/9119284/files//rdbms/jlib/CDC.jar/oracle/CDC/Purge.class
INFO:Verify top-level entry oracle/qsma/QsmaDataManager.class using file /SIDS/app/oracle/product/10.2.0/db/.patch_storage/9119284_Dec_28_2009_03_45_13/scratch/work/QsmaDataManager.class and /nas/sysadm/oracle/psu/HP/9119284/files//rdbms/jlib/qsma.jar/oracle/qsma/QsmaDataManager.class
INFO:Verify top-level entry oracle/qsma/QsmaDataReports.class using file /SIDS/app/oracle/product/10.2.0/db/.patch_storage/9119284_Dec_28_2009_03_45_13/scratch/work/QsmaDataReports.class and /nas/sysadm/oracle/psu/HP/9119284/files//rdbms/jlib/qsma.jar/oracle/qsma/QsmaDataReports.class
INFO:java.util.zip.ZipException: unexpected EOF
INFO: at java.util.zip.ZipInputStream.read(ZipInputStream.java:159)
INFO: at java.io.FilterInputStream.read(FilterInputStream.java:90)
INFO: at oracle.opatch.MultiJarUtil.writeOneZipEntryToFile(MultiJarUtil.java:422)
INFO: at oracle.opatch.MultiJarUtil.doVerifyLeafLevel(MultiJarUtil.java:259)
INFO: at oracle.opatch.JarActionHelper.verify(JarActionHelper.java:291)
INFO: at oracle.opatch.JarAction.verifyMultiLevel(JarAction.java:2101)
INFO: at oracle.opatch.JarAction.verify(JarAction.java:2022)
INFO: at oracle.opatch.OPatchSessionHelper.verifyPatchAction(OPatchSessionHelper.java:2928)
INFO: at oracle.opatch.ApplySession.verify(ApplySession.java:2781)
INFO: at oracle.opatch.ApplySession.wasThisPatchApplied(ApplySession.java:2391)
INFO: at oracle.opatch.ApplySession.processLocal(ApplySession.java:4463)
INFO: at oracle.opatch.ApplySession.process(ApplySession.java:5584)
INFO: at oracle.opatch.OPatchSession.main(OPatchSession.java:1718)
INFO: at oracle.opatch.OPatch.main(OPatch.java:630)
INFO:Verify top-level entry oracle/qsma/QsmaFileManager.class using file /SIDS/app/oracle/product/10.2.0/db/.patch_storage/9119284_Dec_28_2009_03_45_13/scratch/work/QsmaFileManager.class and /nas/sysadm/oracle/psu/HP/9119284/files//rdbms/jlib/qsma.jar/oracle/qsma/QsmaFileManager.class
INFO: verifying 300 archive files.
SEVERE:There are 3 jar files under ORACLE_HOME that are not patched.
WARNING:OUI-67124:Files check failed: Some files under ORACLE_HOME are not patched. Please see log file for details.
SEVERE:OUI-67073:ApplySession failed: ApplySession failed in system modification phase... 'Verification of patch failed: Files are not updated completely.'
Cause of the Problem
The file "$ORACLE_HOME/rdbms/jlib/qsma.jar" itself is corrupted
The command jar tvf $ORACLE_HOME/rdbms/jlib/qsma.jar generates "java.util.zip.ZipException: unexpected EOF"
Solution of the Problem
For 10.2.0.4, we can consider extracting this file from 10.2.0.4 patchset staging kit.
- cd 10.2.0.4 Patchset Staging kit>/Disk1/stage/Patches/oracle.rdbms.rsf/10.2.0.4.0/1/DataFiles
- mkdir /tmp/qsma_jar
- cp filegroup4.1.1.jar /tmp/qsma_jar
- cd /tmp/qsma_jar
- jar xvf filegroup4.1.1.jar
-- This will give many files , including "qsma.jar" .
Rename the existing qsma.jar in the 10.2.0.4 Oracle Home and replace it with the file present in
-- /tmp/qsma_jar
Now use the ORACLE_HOME/jdk/bin/jar to check if we can list contents of this qsma.jar
If successful, proceed to apply the remaining CPU patch.
Tuesday, June 1, 2010
Abasa Arju born in this earth on 1st June, 2010
Today my first child (boy) came in this earth. He is in a hospital. I would want to keep his name as "Abasa Arju". Here goes some of his cute photos. I want dua from everyone for all of us.
This photo is taken almost just after his birth whenever I took him on my hand.
This photo is taken within 30 minutes of his birth.
Abasa Arju, whenever he was in hospital after his birth.
This photo is taken after 8 hours of his birth. Doctor gave him saline in this photo.
I wish, pray and hope that Allah will reward him a happy life in the earth and hereafter.
This photo is taken almost just after his birth whenever I took him on my hand.
This photo is taken within 30 minutes of his birth.
Abasa Arju, whenever he was in hospital after his birth.
This photo is taken after 8 hours of his birth. Doctor gave him saline in this photo.
I wish, pray and hope that Allah will reward him a happy life in the earth and hereafter.
Subscribe to:
Posts (Atom)