1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com Iss search MainframeTips.com Tips, tricks and clever workarounds for mainframe systems, especialy IBM's Z/OS. * Device management » © General Frequently troubles during Mainframe ZOS support (Don't trouble trouble until trouble troubles you!) Contents, Bottom 1. Duplicated SYSID in 2 LPARs during SMF processing of SCRT Report for Sub-capacity charges of Mainframe 2. TSO Logon time delay and system wait MORE INFO 3. SMF summary file is_full WTO/SYSLOG buffers are full and SYSLOG file need to be saved on tape or transfered to PC L.CICS online application pending in “system” — stop, cancel, restart LL. Stop CICS online application 1.2. Restart CICS online application 1.3, VTAM generating messages stop 8. TSO user locked ~ cancel (forced) 9, Forgotten TSO user password 10. Forgotten CICS/KBI user password 10.2. KBI password htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 1s.1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com 13, No space in a Storage group to restore archive VSAM DB files — add, delete,move VSAM files and disk volum: 14, The problem “No connection” Edit/View of TSO/ISPF library 15, Logree data set is full 1, Duplicated SYSID in 2 LPARs during SMF processing of SC charges of Mainframe Report for Sub-capacity Ifyou have a trouble related to SCRT report not accepted from IBM/LMS because one of the LPAR’s in the CPU of Mainframe has duplicated SYSID with another LPAR on the same CPU, go to article ” Troubles with duplicated SYSID in 2 LPARs”. 2. TSO Logon time delay and system wait Ifyou or other customers are waiting too much when logging on TSO of z/OS in only one particular LPAR of the CPU and no TSO logon time delay exists on other LPAR’s, go to item 1 of the Contents in article “Troubles with TSO Logon time delay and system wait”. If TSO is in_ system wait troubles during normal operations, go to item 2 of the Contents in article 3. SMF summary file is full ‘When a warning message is received that SMF summary file is full and the job DUMPXY of z/OS (writing regular SMF dumps on SMF summary file) ended with cc=0008 (due to 100% full of SMF summary file) and a new active work SMF file is switched with dump required for the previous active work SMF file, go to article “Troubles when SMF summary file is_full” to solve the problem and to avoid this situation in the future. 4, Bad SMF records when DUMPXY is restarted after co-0008 When the failed job DUMPXY with RC=0008 is resubmitted (as SMF dump was required) and some SMF records are dumped (written ) in a wrong way (bad SMF records) on the summary SMF file resulted in ABENDO02 with re=0004 during the extracting step of SMF records (in preparing the final SCRT Report) , go to article “Troubles with removing bad SMF records” to remove the bad SMF records. 5, Jes2 queue is full - offload on tape and delete If a message appears that JES2 queue is full and nobody can logon in TSO, then some Emergency actions should be performed to reanimate the system. Ifa warming message appears that JES2 limit is reached, some actions are necessary to avoid the status of JES2 full. In the all cases above go to item I (Emergency actions if warning limit reached or full JES2) of the Contents in article “Troubles when Jes? queue is full” If you havea ZIOS functioning), go to item 2 (JES2 offload on tape and delete) of the es2_queue is full” to prevent the possible troubles. about the JES2 queue properly processing and maintenance (not to be 100% full when stops ‘ontents in article “Troubles when htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 261082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com ‘When the message area is full and the 2/OS system stopped working due to message buffer pool limit reached, go to item 1 of the article ‘Troubles when SYSLOG or WTO buffers are full’ to solve the urgent problem and to learn how to avoid it in the future. To save the current SYSLOG data on tape a procedure is used for daily/monthly processing of SYSLOG on a production system , minimizing the size of the SYSLOG. Go to item 2 of the article ‘Troubles when SYSLOG or WTO buffers are full” ‘To save the current SYSLOG STC**** to PC for later investigating and analysis, 2/OS file is created that contains all SYSLOG current data and sent to PC with MF-PC text file transfer. Go to item 3 of the article “Troubles when SYSLOG or WTO buffers are full" 7. CICS online application pending — stop, cancel, restart If the CICS online application is pending and all CICS users are waiting in “system”, then go to next steps explaining how to stop, cancel and restart the CICS application in correct way. 7.1. Stop CICS online application First of all the KBI online application should be stopped if possible as follows: ~ start a new terminal session from the pending one or from TSO/ISPF session with - File— Run the same — and the following VIAM menu appears appears: B BC2ACI0A. Production Kirchman Bankway CICS ~ enter letter “B’ — and sign on in the CICS with ID=xxxxxx PW =yyyyyy. = start KBI online— enter ‘s001” (KBI operation transaction name) with KBI_ ID=zzzzzz. PW = pppppp. — close KBI operation and all KBI files from KBI operation menu. — close CICS system with the command ‘CEMT P SHUT,D’ If this is not possible (pending system wait during KBI stop) the command ‘CEMT P SHUT.’ should be ed from any CICS terminal to try to stop the online system in a regular way. If the above doesn’t work (KBI and CICS can’t be stopped) then go to TSO/ISPF ~ sd.st — to eancel the CICS application BC2ACIOA with: ‘e BC2AC10A STC03521 BC2ACI0A 15 EXECUTION — BC2A BC2A‘ The cancel command could be issued from TSO/ISPF — sd.log— enter ‘fe BC2ACI0A’ or from the system main or alternate console with -*c ~BC2ACIOA’ to stop forced the CICS application job in 2/OS system. 7.2. Restart CICS online application = restart CICS job in z/OS ‘To restart the CICS job again, from TSO/ISPF — sd.log— enter ‘/s_ BC2ACIOA’, The same could be done from the system main or altemate console with —‘s BC2ACIOA\. ~ restart CICS system in z/OS htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ans1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com B BC2ACIOA Production Kirchman Bankway CICS ~ enter letter “B’ — and sign on in the CICS with ID=xxxxxx PW =yyyyyy. ~ restart KBI online application After CICS is started ~ enter ‘so01° (KBI operation transaction name) with KBI ID=zzzzzz PW = pppppp and KBI operation menu appears. — open KBI operation and all KBI files from KBI operation menu, — restart KBI online operations from TSO/ISPF — sd.log — start a new terminal session with — File— Run the same — and the following appears: B BC2ACI0A Production Kirchman Bankway CICS ~ enter letter “B’ — and sign on in the CICS with ID=xxxxxx PW =yyyyyy. Enter ‘sy01’ (KBI processing transaction name) with KBI ID=zzzzzz PW = pppppp and KBI processing ‘menu appears — KBI is restarted. 7.3. VIAM generating messages stop, If the following VTAM messages appear on syslog (sd.log) — stop them: IST5301 AM GBIND PENDING FROM BCIBVTAM TO BC4AVTAM FOR BC4AVTAM 365 IST10S1I EVENT CODE = 0202 1ST10621 EVENT ID = 00000002000100000004000102020F3100000005 IST3141 END ISTS301 AM GBIND PENDING FROM BCIBVTAM TO BC2AVTAM FOR BC2AVTAM 366 IST10511 EVENT CODE = 0202 IST10621 EVENT ID = 00000002000100000001000102020F3100000004 IST3141 END Jv net,id=be4avtam,inact,f — stop communication with BC4AVTAM application ISTO97I VARY ACCEPTED IST6171 DEACTIVATION IN PROGRESS FOR BC4AVTAM IST10SI BC4AVTAM NODE NOW INACTIVE, Ww net,id=belbytam,inact,f — stop communication with BCIBAVTAM application IST097I VARY ACCEPTED IST6171 DEACTIVATION htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ans PROGRESS FOR BCIBVTAM1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com 8. TSO user locked — eancel (forced) If TSO user is locked (pending) and can’t sign on (due to not logged off properly with— PF3 - PF3— PF3— ‘Logoff” command or due to some operator or system error), first of all it should be canceled (to unlock) from system operator in TSO/ISPF — sd.log— with: —/e u-ibmuser— cancel tso ibmuser if pending -if cal mgp,arm — cancel tso user forced — /d ts,a— display active tso users to see if successfuly canceled ed TSO user can logon in TSO session again (with the same VTAM menu session started or from another TSO/ISPF session with — File ~ Run the same) — and when the following VTAM menu appears: ‘T BC2ATSOA Biochim TSO Production * Enter the letter *T” and follow with the TSO user own ID=zzzzzz and PW = pppppp. ‘The TSO user can now operate OK with the TSO/ISPF menu. 9, Forgotten TSO user password To change or reactivate the forgotten TSO user password do the following: From TSOMSPF - p.6: Enter: alu beplfts password(passfts) resume TSO user ID = bepIfts TSO user PW = passfts The new password of TSO UID=bepIfts is now passfts. 10. Forgotten CICS/KBI user password 10.1. CICS password To change or reassign the forgotten CICS user password do the following: From TSO/ISPF — p.6 — issue the command: alu cu330099 password(cu099) resume — Enter From VTAM menu session choose CICS/KBI application *B’ in LPAR BC2A/90: B BC2ACI0A Production Kirchman Bankway CICS — after ‘B’ entered the following appears: htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 5161082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com Password... eu099 Language New Password ‘Type your userid and password, then press ENTER: — Enter Userid....CU330099 Groupid ... Password ... Language New Password DFHCE3525 Your password has expired. Please type your new password, New Password stefl23. —Enter DFHCE3S39 Please reenter the new password for verification. New Password stef123_ — Enter ‘The new password of CICS user CU330099 is stef123 for BC2AC10A/CICS10 in LPAR BC2A/90. 10.2. KBI password To change or reassign the forgotten KBI password , from VTAM menu session choose CICS/KBI application “B’ in LPAR BC2A/90: B BC2ACI0A Production Kirchman Bankway CICS — after ‘B’ entered the following appears: ‘Type your CICS userid and password, then press ENTER: Userid....€330405. Grow ~ authorized CICS user ID Password... XXXXXXXX— — — authorized CICS user PW Language New Password htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ens1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com bs01 ~ special KBI transaction and security code: XXXXXX entered, For your KBI USER IDENTIFIER ( cu330099 ) with CURRENT STATUS INACTIVE enter command bsrest from main BSMENU of bs01 to change STATUS as ACTIVE: PLEASE CONFIRM YOUR REQUEST (1) 1-YES 2-NO Enter command bsaspw from BSMENU to ASSIGN the new USER PASSWORD: cuxxx ASSIGN PASSWORD (Couxxx ) VERIFY ASSIG! ED PASSWORD —(cuxxx) Enter Exit the bs01 transaction. To start KBI application on the same CICS session enter sy0I transaction — the following appears: PLEASE ENTER YOUR KBI USER-ID * 0330099 * = write KBI ID =eu330099 ENTER YOUR PASSWORD * cuxxx * — write KBI new PW=cuxxx ~ Enter A new PW is required htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 761082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com * stefl23 0 * ~ Enter A new PW is required — please enter it now Repeat your new PW again * stefl23 0 * ~ Enter ‘The main KBI menu is displayed and you can work with the CICS online application. 11. Reactivate or add/delete TCPIP routes required for Mainframe If TCPIP configuration is changed it is necessary to display, update and reactivate the TCPIP routes to Mainframe as follows: ~ display Routes From TSO/ISPF ~ sd.log~ enter the following TCPIP command to display active/connected Routes to Mainframe: 1d tepip,t,conn — display tep - 23 routes active, - Update Routes To edit the PROFILE.TCPIP, enter the member tepprof in Library be2a.zos140.parmlib and find (F) ‘HOME addresses’: ; HOME: Provides the list of home IP addres: HOME 10.212.4.90 BC2AJES — the current LPAR BC2A IP address htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ans1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com ; Static routing BEGINROUTES —_- Here it is necessary to update — add/delete Route!!! Route 10.212.4.0/24 = BC2AJES MTU 1500 Route 10.150.48.0/16 10,212.4.3 BCZAJES MTU 1500 [DROUTES ~ activate the new Routes /TSO user should be with AUTHORITY writes — IBMUSER/ From TSO/ISPF — sd.log — enter: /v tepip,,O,be2a.z0s140.parmlib(tepprof) — activate top after update in be2a.zos140parmlib(tepprof): 12, Backup/Restore troubles with saved SMF file on tape — DCB parameters If in the restore job (for saved SMF file on tape) the tape DCB is of not existing file or of existing file (or existing library) without parameters, and the job execution is not successful when the tape DCB in the restore job is of an existing library (DCB=BC1B.Z0S140.0PER.LIB) without params (for saved file SMFDUMPW.D040213 on tape CB0394 with backup using DCB= (IPOSAV.SMFDUMPW), the restore from SMF save tape is not successful - ce=0008 / ADR374E (001)- OPNCL(08), UNABLE TO OPEN DDNAME TAPE, 107: DFDSS_ EXEC PGM=ADRDSSU,REGION=0M,TIME=1440 PARM="TYPRUD =-NORUN’ '/SYSPRINT DD SYSOUT-* JTAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG), 1! VOL=SER=CB0394,UNIT=(300,,DEFER), i! DCB=(BC1B.Z0S140.0PE} 1B), or DCB=BC1B.Z0S140.0PER.LIB i! LABEL=(6,SL) JISYSIN DD * RESTORE - htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ons1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com y= INDDNAME(TAPE) — RENAMEU(KC108.SMF.D040213,IPOSAV.D0402R) — SPHERE —If the tape DCB in the restore job is not of an existing file (KC101,DMNSNTAP.MODEL.DCB), the restore from SMF save tape is not successful — JCL ERROR — /IEF2121 RESMF02 DEDSS TAPE - DATA SET NOT FOUND/: DEDSS EXEC PGM=ADRDSSU,REGION=0M,TIME=1440 PARM="TYPRUN=NORUN’ JISYSPRINT DD SYSOUT=* JITAPE DD DSN=BC2A.BKPKCI08.SMF.D180612.D020712,DISP=(NEW,CATLG), {! VOL=SER=CB0344,UNIT=(300,,DEFER), i! DCB =(KCI01.DMNSNTAP.MODEL.DCB - (| | RECFM=U,LRECL=0,BLKSIZE=32760, i! DSORG=PS), ‘The solution is the tape DCB to have the same DCB used during backup (DCB-IPOSAV.SMFDUMPW) — then the restore from tape is successful — e=0000: JITAPE DD DSN=BCIB.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG), J} VOL=SER=CB0394,UNIT=(300,,DEFER), ji DCB=(IPOSAV.SMFDUMPW), - without params or to have the same DCB-IPOSAV.SMFDUMPW and params as was in backup job: TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG), /) —- VOL=SER=CB0394,UNIT=(300,,DE! R), i! DCB=(IPOSAV.SMFDUMPW, i! RECFM=U,LRECL=0,BLKSIZE=32760, i! DSORG=PS), or to have the DCB of another existing file IPOSAV.LOGREC) and params as was in backup job: TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG), htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 10151082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com j) RECFM=U,LRECL=0,BLKSIZE=32760, i DSORG=PS), or to have the DCB of existing library (BC1B.ZOS140.0PER.LIB) and the same params as was in backup job: TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG), | VOL=SER=CB0394,UNIT=(300,,DEFER), ji DCB=(BC1B.Z0S140.0PER.LIB, i! RECFM=U,LRECL: |, BLKSIZE=32760, 1! DSORG=PS), ‘The conclusion: for SMF restore tape DCB ~ to be of a sequental file or to has params with (DSORG=PS) - doesn’t matter how it was backed up — wth DCB of SMF file or with params (DSORG=PS). = If the tape is from another LPAR do the following to avoid some possible troubles (tape volume not found etc.) 2 Verify if the tape volune I exists (is—r—5 tape volume to RMM (is—r- 5-1-2). — Display volume information) and if not — add the Ifthe tape volume exists but has other params — Delete or release a volume (is -r- 5— 1-4) —add the tape to RMM (is —r—5— 1 -2) with the necessary params 2 Ifnot possible to delete a tape volume label go to “Case 2. The problem: RMM - error re=12 when try to release, force any tape” of article “Mainframe Tape utilities” where this problem and the decision are described, 13. No space in a Storage group to restore archive VSAM DB files — add, deletezmove VSAM files and disk volumes If you need a space to restore archive VSAM DB files on existing storage group without adding new disk volumes, the following must be done: VSAM. ~ delete unused VSAM files and garbage as shown in article ilities or initialize volumes with the obsolete data — move/condense VSAM files to free disk space on volumes as follows: If you have not enough free disk volumes but you have some free space on volumes in a storage group that should be gathered on total free volumes in the same or another storage group (condense VSAM files space), go to“ 1.1, Move(condense) VSAM files to free disk volume/s in a storage group” of “Casel. Copy/move VSAM files and full disk volumes to disk” in article ZOS VSAM utilities. All VSAM files from one or more volumes are copied to volumes in the same storage group with enough available fiee space and the freed volumes are set offline to avoid using them if second such copy is necessary. The freed volumes must be excluded from the old storage group (disable) and can be set as spare volumes ot can be added to another storage group in the same or in another LPAR. htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 ws1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com — add new disk volumes to a storage group as described in article Mainframe_Disk_utilities —copy a big file on chosen disk volumes with free available space for any storage group Use the following job COPYBO1 to copy file IPOSAV.SMFDUMPW on cligible disk volumes DEV302,DEV281,DEV272,DEV269,DEV262 with free space on them: /COPYBO1 EXEC PGM-IEBGENER HSYSPRINT DD SYSOUT=* JISYSIN| DD DUMMY HISYSUT1 DD DISP=OLD,DSN=IPOSAV.SMFDUMPW HSYSUT2_ DD DSN=KCI07.SMF.D050313,U NIT=SYSDA, /!- VOL=SER=(DEV302,DEV281,DEV272,DEV269,DEV262), i), SPACE=(CYL,(1000,110)), DISP=(NEW,CATLG,DELETE) Space information for the copied file: Used cylinders . . : 3,262 Used extents... : 22 Tracks % Used XT Device 49650 98 22 3390 14. The problem “No connection” Edit/View of TSO/ISPF library If you receive a message ‘No Connection’ when you want to edit'view any member of TSO/ISPF library, look at the following explanation If you are working in ISPF C/S WSA for file transfers between PC and Mainframe and when finally you exit from this connection with WSDISCON command in ISPF or stop wsa.exe application on PC, you may receive ‘a message in TSO ‘No Connection’ that appears when you want to edit/view any member in edit /view mode of ISPF library. In this ease go to article “No connection” Edit problem in TSO/ISPF C/S WSA environment to solve the problem and rid of it. 15, Logree data set is full If the Logrec data set is full, look at the following explanation. If the error recording data set (ERDS also known as LOGREC — logree data set) is full, then error recording stops and /OS continues with no error log recording. A daily or monthly procedure is used to copy logrec data set (SYSPLEX.LOGREC.ALLRECS or BC2A.LOGREC) into a daily OUTPUT ACCUMULATION DATA SET (IPOSAV.LOGREC) and to clear logrec data set (SYSPLEX.LOGREC.ALLRECS or BC2A.LOGREC) to be ready for new error recording followed by printing a ‘SYSTEM SUMMARY” REPORT for analyzes. htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 1251012812017 This website regular daily/monthly procedure for Logrec processing is described. End ‘You may also like: ZOS Logree (Error log) processing ‘Troubles with duplicated SYSID in 2 LPARs Transfer z/OS VSAM file via Internet August 2012 Oct 17, 2013 htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 Frequently roubles during Mainttame Z0S support - MainrameTins.com ‘Troubles when SMF summary file is full April 2012 October 2013 ZOS Hardware Management Console VO configuration operations ‘ss by Shareahotc sas1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com + bad SMF records, CICS, CICS user password, CICS/KBI user password, DUMPXY, duplicated SYSID, full, Jes? queue, LOGREC, LPAR, SCRT Report, SMF processing, SMF summary file, TCPIP routes, TSO library, TSO Logon time delay, TSO/ISPF library, VSAM DB files, VTAM, WTO SYSLOG buffers, WTO/SYSLOG buffers Anonymous BSc Leave A Reply Name Email (will not be published) Website Comment A ‘Submit Comment Comments No comments yet, be the first to add one! Tags yad SME records CPU sis disk conSzurtion Dito dourlond DUMPXY file transfer FTP uep HMC wo confswation INDSEILE IOCDS IODF IPL. sue isp¥ isp uco ict ttmaynentes LPAR Mainframe PC rc iter Pama Supan tens restore SMF su sssosés spool SYSLOG tape ‘asi lst TSO TS0 C'S WSA 180 ISPF ypleed Utilities VSAM vroc wwps wro wrog Places © Ab © Structure of data htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 sans1082017 Frequenty troubles during Mainframe 20S suppor - MainrameTips.com BS «es Archives March 2016 March 2014 October 2013 July 2013 March 2013 january 2013 December 2012 November 2012 August 2012 June 2012 May 2012 April 2012 March 2012 January 2012 December 2011 November 2011 October 2011 August 2011 July 2011 June 2011 May 2011 March 2011 Copyright © Stefan Krustey 2011 htpstiwww:maintrametips.convirequenth-roubles-during-meinfame-z05-suppor_ 8 1915