Flex Corporate IT Operations Page 1/2

Handling Oracle Job failures

EDITING

In this document we will cover the Oracle job failures and action to be taken for the same.

Once you receive alert (INC) for Oracle job failure, Please login to TWS console and try to look for the job log.

If you could not get it anything from TWS log, Please look for oracle log/tmp file for more details.

Log or tmp file in LOGS directory will let you know the progress of the oracle job.

You should have basic understanding of Archive/Purge logic of the schema's(tables) if you need to ding more into the issue.

Let us see different reasons for Job failures.

1) LOCK file in script directory.

LOCK will be created in script directory when the job is still executing with its job ID. (This will be deleted automatically if script completes)

When job got failed with LOCK file exists as reason,(it will be specified in the mail you will receive for failure)

Please check whether LOCK file still exists or not. If LOCK file still exists, please check whether related process is still exists or not.

If process was not existing. Please verify the log. If log shows script is completed. You can verify same in DB tables.(whether data got
archive/purge)

If everything completed successfully. Please delete the LOCK file and re execute the job.

2) Process was Still running

If process mentioned in LOCK file exists, means script was still running. You can verify same, by checking the related tmp file (with previous
day date) in LOG directory.

If any script is INPROCESS, the file in LOG location will be *.tmp only. Once sript completes, tmp will be renamed as *.log.

So please verify related tmp file, whether job was started properly or not.

If job was not started properly (means not connected to DB), Please delete the LOCK file and re execute the job and verify the tmp file.

If started properly, you need to verify related tables. (if process is in progress, data in table will get reduce in regular intervals)

If count is decreasing, Wait till process completes. or if takes more time discuss with DB team.

3) NO Process is running but Lock file exists and Data in tables not reduced

In this case, Please check with DB team, if we have any long running queries.

Please check ACTIVE sessions query. If the query is matching our Purge/Archive logic and not changing for long time.Reason could be the
process at DB side would have stuck.

Please kill that session on DB with help of DB team.

Delete the LOCK in our system and re execute the job.

4) Schema Procedure failed

In this case, script execution will be completed and LOCK will be cleared. You will be having a error message in the log file or in the mail
generated.

Need to act based on the error. (ex: unique constraints)

Please note that, while re executing the failed job, you should be careful in checking the Timings. If you are executing the job on next day, you
need to remember that you are archiving/purging two days of data and you need to be prepare that it takes more time than usual. SO it is
advisable to take action on same day(PST hours). OR if you are forced to run job to Archive/purge more no.of days data, it is always suggestible
to run the same manually by alerting the config file for no.of days. For this you need to understand/read script/config file and schema procedure.

Please do not try to modify any scripts/config file with out proper testing and approvals.

Please note that same config file used for all scripts. So be careful in changing the same.

DB user name and password details are stored in a encrypted files.

Printed copies of this Document are Uncontrolled and are considered valid only for the day it is printed -
Flex Corporate IT Operations Page 2/2

Oracle TWS JOB Details :

http://esowiki.flextronics.com:8090/pages/viewpage.action?pageId=15500015
http://esowiki.flextronics.com:8090/display/WBA/Oracle+Jobs+-+PROD

Schema Archive/Purge Details :

http://esowiki.flextronics.com:8090/pages/viewpage.action?pageId=24676668

http://esowiki.flextronics.com:8090/display/WBA/Trading+Network+%28WMTNS%29+Data+Archiving+Policy+and+Retention
http://esowiki.flextronics.com:8090/display/WBA/wMaudit+copying+in+wm7A1

Scripts,Logs and Configuration Files :

On SACVL093 :

Scripts Directory: /wmhome/scripts/oracleJobs/scripts

LOCK File Directory : /wmhome/scripts/oracleJobs/scripts

LOG and TMP File Directory: /wmhome/scripts/oracleJobs/logs/

Configuration Files Directory: /wmhome/scripts/oracleJobs/oracleConfig/

Naming Convention : Naming of Lock file,Log or tmp file, will be based on these values: Environment,SID,Schema,Activity,date.....

Ex : WM7P1_PRD_B2B_WMCUST_PURGE_112916.log PRD_B2B_TN_DATA_ARCHIVE_MDP.lock
WM7A1_PRD_B2B_WMTNS_ARCHIVE_112416.tmp

Printed copies of this Document are Uncontrolled and are considered valid only for the day it is printed -