Understanding Job Control & Scheduling

In this section:

Introduction to Job Control & Scheduling

Users interact with Callista in two main ways. They can access data and initiate processes via forms, or by running jobs recorded in this subsystem.

The Job Control and Scheduling (JBS) subsystem controls scheduled report production, large volume database updates and system housekeeping tasks. The programs and commands to perform these activities are initiated from executable files or procedures accessed via 'jobs' which are run by users. These are generally listed on menus. The jobs available to specific users depends on their security access. Jobs may produce output in various forms, such as printed and on-screen reports, email or files.

Some terms and distinctions are especially important for an initial understanding of the subsystem:

Jobs, Parameters and Requests

The term 'executable job' is used in this documentation to describe a set of stored programming instructions. Many executable jobs have an associated parameter file. An existing executable job can be defined in Callista under one or more job names, with different output options and security conditions attached to each job name. Use of the term 'job' in this manual refers to a uniquely named job known to Callista.

When a user wants to run a job in batch mode, it is submitted as part of a request that the user sets to run at, or as soon as possible after, a nominated date and time. Often a request will contain just one job, but it is possible for several different jobs to be included in one request, or for one job to be included several times with different parameters.

Immediate or Batch mode

Users can run jobs and reports in different modes. The ability to run an individual report or job in a particular mode depends on the definition recorded for it (in JBSF4110), the security profile of the user and whether or not the user is defined as a Novice Interface user (in SECF0066). The modes are:

·         Immediate

·         As Soon As Possible

·         Batch Mode

Novice Interface users do not have the option of running 'their' reports/jobs in Batch mode. When reports/jobs are run in Immediate mode it is not necessary for the user to access Job Scheduler. When reports/jobs are run in As Soon As Possible mode, access to Job Scheduler is optional and is required only to monitor progress in JBSF5300. Refer to All About Standard Reports and Jobs for further information about the Novice Interface.

The remainder of this section refers to reports/jobs run in Batch mode.  They are handled entirely by the Job Scheduler subsystem (JBS).

Users

Various people will have different responsibilities in maintaining and using the subsystem.

These two groups are likely to act in consultation.

This introductory section is primarily about creating and submitting job requests. Additional information on setting up the subsystem is available within this online manual in the Overview section of Specialist Functions and in technical documentation provided with the relevant release of Callista.

Factors Influencing the Running of Jobs

Run windows

'Run window' is the term used for a standard set of times in a week. In its simplest form, there might be one run window for business hours, and a second for after hours. Each request must stipulate a run window, and the job(s) in the request can only start in that window. Some jobs may only be allowed to run in certain windows. Batch queues are also set up to operate in particular run windows.

Dependencies

Some jobs need to run in sequence. For instance, obtaining an up-to-date report might depend on an update job being run first. For any job, it is possible to stipulate the jobs that should run before it (prerequisites) and those that depend on its running (dependents). If a job is set up with mandatory prerequisite and dependent jobs, users need to ensure that all the jobs in the relationship are included in a request.

Conflicts

It is inadvisable to run certain jobs concurrently, especially where they are updating/extracting from the same set of data. A job that conflicts with another in this way can be defined as a 'conflict job'. Callista prevents the two jobs from running at the same time. System jobs for which conflict resolution is critical will already have conflict records created for them.

Priority

All jobs are given a priority number from one to nine. Unless additional constraints (such as dependencies) dictate otherwise, the system will run all scheduled jobs in priority order, irrespective of which requests they come from, starting with priority one jobs. Some users may be given the ability to override, at request time, the designated priorities for particular jobs.

Batch Queues

Before batch jobs can be run, they must be allocated to queues. This is done automatically by the system. How quickly jobs begin once they reach their scheduled time will depend in part on the number of active batch queues, and any restrictions on the running of jobs in particular queues. Jobs in one request may run in different batch queues, but will not run concurrently.

Running a Report Immediately

Refer to All About Standard Reports and Jobs.

Steps in a Job Request

Jobs run in batch mode must be part of a job request. The following paragraphs outline the steps required to set up a job or jobs in a request, enter a schedule and submit the request to run as scheduled. The principal form used in creating job requests is Maintain Request Details (JBSF5210). All other request forms can be accessed from this form either directly or through intermediary forms, and full documentation is given for each form.

Select a Job. Jobs will normally be run from menus. However, if when selected from the menu, a job is able to be run outside Job Scheduler (Immediate or ASAP mode), its parameter form will be displayed. Select the Schedule button on the parameter form toolbar to invoke the Maintain Request Details form (JBSF5210).  If the job can only be run in batch mode, on selecting it from a menu the Maintain Request Details form (JBSF5210) is opened automatically. The job name selected from the menu is entered into the form by the JBS subsystem.

Alternatively, all batch jobs can be run by accessing this central form directly, in which case the user must enter the names of the required jobs.

Step 1. The Maintain Request Details form (JBSF5210) opens.

Step 2. In the Request block, select or enter a Run Window.

Step 3. Click in the Job Name field. The name of the job will be inserted, unless the form was entered directly, in which case the field will be blank.

Step 4. Enter the names of any other jobs you wish to run in this request. If required, use the arrow buttons to alter the sequencing of these jobs. If required and permitted, alter the priorities of the jobs using the Priority Override field.

Step 5. At this stage, the basic details of a request have been entered. Further steps require the use of JBSF5210 and its associated forms. Refer to JBSF5210's documentation for further information on its use and for links to associated forms.

 

Associated forms are documented under:

 

Example scenario - creating and submitting a request

In the morning, a staff member wants to set up a request to run a hypothetical job producing class lists. On weekdays, the run windows operating are a business hours and an after hours window. The list is not required until the next day, so she elects to schedule the request for that evening in the after hours run window, to run after 6 p.m. (18:00).

  1. She selects the job name, ENRR1234, from the Reports menu. The job can only be run in batch mode, so the parameter form opens automatically with the Immediate Mode icon disabled.
  2. She enters the required class as a prameter and clicks on the Schedule icon.
  3. In the central request form, she enters the AFT_HOURS run window, and then clicks in the Request Job block. Job ENRR1234 is displayed there. Although the job has a default of priority 1, there are other more urgent jobs that she has submitted to run that evening, so she sets the priority down to 4, and saves the job.
  4. The job has the faculty printer already set up as a default destination, but she chooses to override this and specify the printer in her office, using the Job Output button to navigate to form JBSF5230 where destinations are maintained.
  5. Back in the central form, she selects the Job Parameters button, which display the parameter interface. She checks the parameters again, realising that she only requires on-campus students. She changes the attendance mode to ON and saves. She exits the parameter form to return to the central form.
  6. Next, she clicks on the Request Schedule button, navigating to form JBSF5212 to enter the date and time when she wants the request to run. She saves the information and exits the form.
  7. In the central form, she clicks the Submit button, and then the Save button.

The request is submitted, and now waits for the system to schedule it to run at the correct time.

Duplicate requests

Any request in this subsystem can be used as a template for a new job request, with altered request details if required. This gives a quick method to create requests. Full details are given in Maintain Request Details (JBSF5210) form documentation.

Standing Requests

A schedule for a request can be set up in two ways. The more common way is outlined in Steps in a Job Request. An alternative is to set it up as a standing request.

With a standing request, the system will run the request automatically at specified intervals within a given period, rather than on precise dates designated in advance. For example, a standing request can be set up to run every Friday in March, April and May, or on each census date in the academic year. The use of standing requests is confined to jobs where the original parameters remain valid, whenever the job is run. Security provisions allow for the setting up of standing requests to be limited to specific people.

A checkbox in the central form, Maintain Request Details, is used to indicate a standing request. If this checkbox is selected, a button to navigate to the Maintain Standing Request Frequency form (JBSF5211) takes the place of the button to set up the basic request schedule. If a user does not have the security privilege to set up standing requests, this checkbox's label is 'greyed out'.

In all other ways, the creation of a standing request follows the steps to set up a basic request.

Output from Jobs

A job may produce one or more output files as a source of information to be printed, faxed or sent as email. These outputs are specified with the job, but there are different possibilities in handling output destinations for each output file attached to a job. Destinations may or may not be specified when the job is created, and if created, may or may not be mandatory. Theoretically, therefore, the following situations could be encountered for outputs of jobs in just one request:

Provision has been made in Callista for destinations of these types:

 

Depending on how the job output is specified, one output file may be directed to different output destination types, or different output files may be created for each type, even where the content is the same.

Further details about adding output destinations at request time are given in the documentation for Maintain Request Job Output Destinations (JBSF5230).

Once a request has run and produced output, this output can be 're-sent' to its original destinations or to new destinations, as long as the person who set up the request has not specified output deletion after sending. See details in the documentation for the form, Inquire on Job Run Outputs (JBSF5410).

Checking on the Progress of a Request

Once requests have been submitted, two background processes keep track of their progress and schedule and run the jobs they contain. These processes run at predetermined intervals, so that there may be a slight time delay between submitting a request and subsequent activity. The time intervals for the two background processes are set by information technology staff responsible for the JBS subsystem.

Users can follow the course of their job requests

Request statuses

A request's status is maintained by the system, and can be seen in the Request Status field in the central form, Maintain Request Details (JBSF5210). The possible request status values are: PLANNED, WAIT, SCHEDULED, COMPLETE, CANCELLED.

A request is set from PLANNED to WAIT status as soon as it is submitted, and to SCHEDULED when it reaches its scheduled time to run. It can be CANCELLED at any time before completion, and during the WAIT status it can be re-planned. Re-planning can include altering its schedule.

Because a request can be scheduled to run on several occasions, either as a standing request or as a normal request with several schedule dates/times, it will revert to WAIT status between runs, and only be set to COMPLETE at the end of the last cycle.

Request Statuses

Status

Progress of request

User intervention possible

PLANNED

Not yet submitted

Most details of request can be changed. A run window can be changed as long as no other request details have been created.
Request can be CANCELLED.

WAIT

Request submitted. It may not yet have been picked up by the relevant background process, or has not reached date/time given in its schedule. Requests with more than one schedule date/time revert to WAIT status between runs.
The (next) run date/time is decided at the point when the WAIT status is set or re-set.

Request can be:

  • returned to PLANNED status by using Re-plan Request button.
  • CANCELLED.

SCHEDULED

The relevant background process has determined that request has reached or passed its scheduled run time, and has marked it for running, conditional on priority and other restrictions.

Request can be CANCELLED.

COMPLETE

All scheduled runs for the request have completed (successfully or otherwise).

 

CANCELLED

The user has cancelled the request.

Request can be cancelled up to point of completion, using Cancel Request button.

Keeping Track of Jobs

Two forms give more detailed information about the progress of individual jobs within requests:

Request Job Run (JBSF5300)

This form gives details of the progress of individual jobs within a request, including run start and end times, and the statuses of all jobs submitted. These job run statuses (as distinct from request statuses) are WAIT, SCHEDULED, RUNNING, COMPLETE, FAILED, ABORTED and CANCELLED, all but the last of which are maintained by the system. Users can intervene in the running of jobs by cancelling the request, or by cancelling an individual job with a status of WAIT or SCHEDULED.

The table below illustrates the relationship between request statuses and job statuses.

Job Run Statuses for Jobs in a Request

Request Status

Job Run Status

Activity

PLANNED

 

 

WAIT

WAIT

Jobs have not yet been scheduled to run, or have reverted to WAIT status pending another scheduled run.

SCHEDULED

SCHEDULED

RUNNING

COMPLETE

CANCELLED

(FAILED)

(ABORTED)

Jobs are SCHEDULED when request is scheduled by system.
Individual Jobs progress through statuses RUNNING, COMPLETE, (FAILED), (ABORTED) while Request Status is still SCHEDULED.
Any job can be CANCELLED by user.

Status FAILED signifies that job failed to complete successfully.
Status ABORTED signifies that job has been cancelled by the system because a previous job in the request failed to complete successfully, or that it cannot be restarted after a system failure.

COMPLETE

COMPLETE
(FAILED)
(ABORTED)

All jobs have completed (successfully or unsuccessfully), and there are no more runs scheduled for request.

Full details are available in the forms level documentation for Request Job Run (JBSF5300).

Run Log (JBSF5301)

Use the run log to see details of jobs that have finished, whether successfully or unsuccessfully.

More information is given in the documentation for Run Log (JBSF5301).

Troubleshooting - Why Jobs Don't Run

Jobs can fail or be aborted because:

Jobs can be held up, or not run, because

If you suspect one of these problems applies to a job you have attempted to run, you probably need to seek help from IT staff responsible for Callista.

Last Modified on 11 March 2002