Preflow is usually started around 10:00. Some time between 10:00pm and 11:00pm, the following jobs
should be manually submitted:|
has some date-dependencies. Before running
Dayend, you should always check the dates in the files and correct if necessary. The procedure
for doing this is
found here. (A2K and A2KTEST dates
should be not require correction, as we run daily Dayend cycles for these, but we seldom do
Dayend on the Test LPAR, so usually have to correct the dates there).|
In addition to the Invision jobstreams, we have a jobstream consisting of 'home-grown' jobs.
Invision Dayend runs jobstreams under control of job (GSPJ1$01) called the Runner. Each job has steps which update a file to tell the Runner when a job starts and ends successfully or abends. The Runner has a TSO interface so the operator can control the jobflow and react to problems. When a job bombs, the error is usually intercepted and recovery JCL is dynamically built. This documents how the process works.
The jobflow depends on what parts of Invision are used by a particular installation, what options the installation has set and what day is being processed.
For example: We do not run the EAD/LCR application any more, so those jobs are not part of our jobflow. We have set up certain options in Invision which causes some jobs to run, not run, run with certain options or at certain times. We schedule PCPurge on one specific day, CDReOrg on another specific day, some jobs on a Saturday or Sunday, some MonthEnd, Quarterly or Yearly, etc..
Operator does 'S PSYSJOBS,J=GSPP1$01'. This job examines data in Invision and prepares the jobflow for our environment in its broadest possible content. Operator verifies Sysout from this job contains 'SCHEDULE WILL COMPLETE' - if not, calls Support.
Operator does 'S PSYSJOBS,J=GSPJ1$01'. This is the Runner job which controls Dayend. Job PAP38A7$ will trigger immediately. All other work requires the operator to initiate a jobstream.
Operator does ISPF 6 and enters:
EX 'PLSMS.SMSSM00L.P0000.CLIST(SMJSCNTL)' 'PROD'
(Should chose options "BROWSE Y" and "PRINT 0")
Operator gets following screen and should monitor via O command, which shows what is running and any Abends:
OPTION==> -------- Current Schedule File ------------- Userid - RLS0 PDSMS.SVGS.CP01.SCHDRUN Time - 12:40 Terminal - 3278 Enter: PF Keys - 24 C = Enter Control Transactions R = Refresh Run Tables B = Show all of schedule M = Show Stream and its Jobs T = Show Stream J = Show Job I = Show In-Progress Streams/Jobs A = Show Abended Jobs S = Show Scheduled Streams/Jobs U = Show Unscheduled Streams/Jobs F = Show Finished Streams/Jobs X = Show Suspended Streams/Jobs P = Show Flow G = Show Flow by Stream D = Reset Defaults E = Show all Streams Z = Show Control Transactions Y = Show Missing Jobs O = Show Current Jobs (I,A,M) Q = Exit this Program Stream ID ===> Job ID ===>
E command will show following jobstreams:
CISCORP - status S
This gathers info for Siemens (which we ignore).
DAYEND - Status S
This is the jobflow except Patient Accounting.
DESCHED - status S
Based on the results from SCDAYEND, this customizes the jobstream for this particular day: Purge, Reorg, WeekEnd, MonthEnd, etc. PADAYEND - status I
This is the Patient Accounting jobflow.
SCDAYEND - status S
This sets up jobstreams for our installation, based on what Invision components we run, customizations,etc.
PMHH - status S
This jobstream is our own and consists of several daily in-house jobs.
Operator enters C (Enter Control Transactions) and sees:
OPTION==> -------- Current Schedule File ------------- Userid - RLS0 PDSMS.SVGS.CP01.SCHDRUN Time - 12:42 Terminal - 3278 Enter: PF Keys - 24 SS = Start A Stream JS = Schedule a Job SH = Suspend Stream (Successors ) SC = Suspend Stream (Successors ) JH = Suspend Job ( HOLD ) JC = Suspend Job ( CONTINUE ) US = Unschedule Stream CS = Complete Stream !! Caution !! UJ = Unschedule Job CA = Complete Stream & all Jobs !! XX = Stop the Controller CJ = Complete Job !! Q = Exit Control Transactions RS = Reschedule Stream !! QQ = Exit the Program RJ = Reschedule Jobs !! DR = Demand Recovery !! Stream ID ===> Job ID ===>
Operator enters SS with SCDAYEND in the 'Stream ID' field.
F3 to go back and monitor jobsteam SCDAYEND
When SCDAYEND finishes completely (E command shows SCDAYEND in F status), operator does C command and SS for DESCHED jobstream, then waits for that jobsteam to complete.
SCDAYEND and DESCHED (for A2K) are run before midnight. Operator must then wait until after midnight before continuing. Since we cannot synchronize time between all the systems, we normally wait until 00:05 so that data moving between the Mainframe and other systems is guaranteed to be time-stamped in the new day. After midnight, operator does C, then SS for DAYEND jobstream. Dayend and PADAYEND will then process.
When DAYEND and PADAYEND complete, run CISCORP. After CISCORP, run PMHH.
When PMHH finishes, Dayend cycle is done and operator does F3 to exit. GSPJ1$01 will end in a minute or two.
A2KTEST Dayend differs in that it is all run during the same calendar day, both the preflow jobstreams (SCDAYEND/DESCHED) and Dayend (PADAYEND/DAYEND/CISCORP/TMHH). This results in some of the jobs issuing WTOR "REPLY CURR OR CONT" messages. Operator should reply CONT.
Each job has steps to tell Runner that the job is starting. It also has a step to tell Runner the job has ended successfully. Most jobs have a step to tell Runner the job has abended and to build Recovery jobs.
Telling Runner the job status is done by giving the jobname and ACT=x:
The value of 'JOB' is naturally unique to each job.
ACT=S tells Runner the job is Starting
ACT=R tells Runner to Reset the job Status
ACT=U tells Runner the job completed successfully
ACT=A tells Runner the job Abended
Recovery JCL is via Proc:
RECOVBLD ..... JOB=jobname
In addition to building Recovery JCL, this proc tells Runner 'ACT=A' to flag as Abended.
A typical job would have the following:
//START EXEC DECHCKPT ... JOB=GLP09A7$,ACT=S
(Tell Runner GLP09A7$ is stating)
Steps that actually do the work
//UPDATE EXEC DECHCKPT ... JOB=GLP09A7$,ACT=U
(Tell Runner GLP09A7$ completed successfully)
//ABEND EXEC RECOVBLD ... JOB=GLP09A7$
(Build Recovery JCL for GLP09A7$ and tell Runner it Abended)
The RECOVBLD proc builds JCL to do any cleanup neccessary, restart the job at the appropriate step and to update Runner.
For example, if GLP09A7$ abended in the GLDLY3A proc, RECOVBLD would construct the following in PLSMS.SMSSM00L.P0000.RECOVJCL:
//GLT09A7$ JOB (111111,11),'SMSTEST-INVISION',CLASS=A, // MSGCLASS=H, // RESTART=GLDLY3A.GL2
//RESET EXEC DECHCKPT ... JOB=GLP09A7$,ACT=R (tell Runner RESET) //START EXEC DECHCKPT ... ,JOB=GLP09A7$,ACT=S (tell Runner STARTED) (specical 'cleanup' steps based on where it bombed) //PURGE EXEC PURGE,DG=A7,HL1='PDSMS.' //EXTRACEX.SYSIN DD * SELECT(ALL); PURGE (ORDEST=ALL,RPT=DAILY-GL); PURGE (ORDEST=ALL,RPT=G/L-ERR); //DELETE EXEC RECOVDEL //SCRATCH.SYSIN DD * DELETE PDSMS.SMGL.DGA7.JOURNAL DELETE PDSMS.SMGL.DGA7.JOURNAL NOSCRATCH DELETE PDSMS.SMGL.DGA7.EXPLODJL DELETE PDSMS.SMGL.DGA7.EXPLODJL NOSCRATCH //SUBMIT EXEC RECOVSUB,JOB=GLP09A7$ (Submit PLSMS.SMSSM00L.P0000.RECOVJCL(GLP0917$)
After the error condition is fixed, operator would submit GLR09A7$ from SYSJOBS, which would:
//SUBMIT EXEC RECOVSUB,HL2='PLSMS.',JOB=GLR09A7$
This would submit RECOVJVL(GLU09A7$). RECOVJC(GLU0917$ would first do cleanup, then submit RECOVJCL(GLP09A7$) which would restart GLP09A7$ at the proper step.
Data to recover each job is in PLSMS.SMSSM00L.P0000.RECOVERY
Some jobs do not have Recovery JCL and some jobs do ACT=S and ACT=U but do not run RECOVBLD step.
The recovery mechanism does not work in the case of JCL failures (dataset not found, duplicate dateset, JCL error, etc.)
For example, if GLP09A7$ got a JCL error, the error would be manually corrected and the operator would do a C Runner command, then DR (Demand Recovery) specifying GLP09A7$ as jobname. On the next screen, enter the Proc name and optionally the step name within the Proc, and a reason code (JCLE, etc).
This would build the Recovery JCL and operator would then submit SYSJOBS(GLR090A7$) as above.