Understanding Job Control and Scheduling

Introduction to Job Control and Scheduling

Users interact with Callista in two main ways. They can access data and initiate processes via forms, or by running jobs recorded in this subsystem.

The Job Control and Scheduling (JBS) subsystem controls scheduled report production, large volume database updates and system housekeeping tasks. The programs and commands to perform these activities are initiated from executable files or procedures accessed via 'jobs' which are run by users. These are generally listed on menus. The jobs available to specific users depends on their security access. Jobs may produce output in various forms, such as printed and on-screen reports, email or files.

Some terms and distinctions (defined in the following paragraphs) are especially important for an initial understanding of the subsystem:

Jobs, Parameters and Requests

The term 'executable job' is used in this documentation to describe a set of stored programming instructions. Many executable jobs have an associated parameter file. An existing executable job can be defined in Callista under one or more job names, with different output options and security conditions attached to each job name. Use of the term 'job' in this manual refers to a uniquely named job known to Callista.

When a user wants to run a job in batch mode, it is submitted as part of a request that the user sets to run at, or as soon as possible after, a nominated date and time. Often a request will contain just one job, but it is possible for several different jobs to be included in one request, or for one job to be included several times with different parameters.

Immediate or Batch mode

Users can run jobs and reports in different modes. The ability to run an individual report or job in a particular mode depends on the definition recorded for it (in JBSF4110), the security profile of the user and whether or not the user is defined as a Novice Interface user (in SECF0066). The modes are:

Novice Interface users do not have the option of running 'their' reports/jobs in Batch mode. When reports/jobs are run in Immediate mode it is not necessary for the user to access Job Scheduler. When reports/jobs are run in As Soon As Possible mode, access to Job Scheduler is optional and is required only to monitor progress in JBSF5300. Refer to All About Standard Reports and Jobs for further information about the Novice Interface.

The remainder of this section refers to reports/jobs run in Batch mode. They are handled entirely by the Job Scheduler subsystem (JBS).

Users

Various people will have different responsibilities in maintaining and using the subsystem.

Jobs
Information technology (IT) staff normally create executable jobs and parameter files, and define jobs and related information within the JBS subsystem. They also set up reference data about output destinations such as printers. This is because these tasks require knowledge of the underlying programs, data structures and hardware. IT staff may also be responsible for setting up batch queues.
Security
Student Systems Administrators/Managers will normally decide on security access to jobs, and set up grants in the Security subsystem accordingly.

These two groups are likely to act in consultation.

Requests
A wide range of student administration staff will set up schedules and submit job requests within their areas of expertise, and follow through on the outcomes. Faculty and other staff may have more limited access to job requests.

This introductory section is primarily about creating and submitting job requests. Additional information on setting up the subsystem is available within this online manual in the Overview section of Specialist Functions and in technical documentation provided with the relevant release of Callista.

Note:
If you institution has installed a multi-byte character set, please note that the use of multi-byte characters is designed for and partially supported in specific areas of Callista only: Proposals (user-defined fields), Communication Templates and Items, and Thesis. For more information, see the 'Character Sets' section in the System Wide Features Online Help.
In the JBS subsystem, configuration is required to enable multi-byte characters to be recorded in Job Scheduler Run Log. For more information regarding this, see the latest JBS Installation/Upgrade Guide for the latest Release located in the Callista Release Centre on the wiki.

Factors Influencing the Running of Jobs

Run Windows

'Run window' is the term used for a standard set of times in a week. In its simplest form, there might be one run window for business hours, and a second for after hours. Each request must stipulate a run window, and the job(s) in the request can only start in that window. Some jobs may only be allowed to run in certain windows. Batch queues are also set up to operate in particular run windows.

Dependencies

Some jobs need to run in sequence. For instance, obtaining an up-to-date report might depend on an update job being run first. For any job, it is possible to stipulate the jobs that should run before it (prerequisites) and those that depend on its running (dependents). If a job is set up with mandatory prerequisite and dependent jobs, users need to ensure that all the jobs in the relationship are included in a request.

Conflicts

It is inadvisable to run certain jobs concurrently, especially where they are updating/extracting from the same set of data. A job that conflicts with another in this way can be defined as a 'conflict job'. Callista prevents the two jobs from running at the same time. System jobs for which conflict resolution is critical will already have conflict records created for them.

Priority

All jobs are given a priority number from one to nine. Unless additional constraints (such as dependencies) dictate otherwise, the system will run all scheduled jobs in priority order, irrespective of which requests they come from, starting with priority one jobs. Some users may be given the ability to override, at request time, the designated priorities for particular jobs.

Batch Queues

Before batch jobs can be run, they must be allocated to queues. This is done automatically by the system. How quickly jobs begin once they reach their scheduled time will depend in part on the number of active batch queues, and any restrictions on the running of jobs in particular queues. Jobs in one request may run in different batch queues, but will not run concurrently.

Running a Report Immediately

Refer to All About Standard Reports and Jobs.

Steps in a Job Request

Jobs run in batch mode must be part of a job request. The following paragraphs outline the steps required to set up a job or jobs in a request, enter a schedule and submit the request to run as scheduled. The principal form used in creating job requests is Maintain Request Details (JBSF5210). All other request forms can be accessed from this form either directly or through intermediary forms, and full documentation is given for each form.

Select a Job

Jobs will normally be run from menus. However, if when selected from the menu, a job is able to be run outside Job Scheduler (Immediate or ASAP mode), its parameter form will be displayed. Select the Schedule button on the parameter form toolbar to invoke the Maintain Request Details form (JBSF5210). If the job can only be run in batch mode, on selecting it from a menu the Maintain Request Details form (JBSF5210) is opened automatically. The job name selected from the menu is entered into the form by the JBS Subsystem.

Alternatively, all batch jobs can be run by accessing this central form directly, in which case the user must enter the names of the required jobs.

Associated forms are documented under:

Maintain Request Job Output Destinations (JBSF5230)
Parameter interface (JBSPARAMS)
Maintain Request Schedule Details (JBSF5212)
Maintain Request Job Run (JBSF5300)

Example scenario - creating and submitting a request

In the morning, a staff member wants to set up a request to run a hypothetical job producing class lists. On week days, the run windows operating are a business hours and an after hours window. The list is not required until the next day, so she elects to schedule the request for that evening in the after hours run window, to run after 6 p.m. (18:00).

She selects the job name, ENRR1234, from the Reports menu. The job can only be run in batch mode, so the parameter form opens automatically with the Immediate Mode icon disabled.
She enters the required class as a prameter and clicks on the Schedule icon.
In the central request form, she enters the AFT_HOURS run window, and then clicks in the Request Job block. Job ENRR1234 is displayed there. Although the job has a default of priority 1, there are other more urgent jobs that she has submitted to run that evening, so she sets the priority down to 4, and saves the job.
The job has the faculty printer already set up as a default destination, but she chooses to override this and specify the printer in her office, using the Job Output button to navigate to form JBSF5230 where destinations are maintained.
Back in the central form, she selects the Job Parameters button, which display the parameter interface. She checks the parameters again, realising that she only requires on-campus students. She changes the attendance mode to ON and saves. She exits the parameter form to return to the central form.
Next, she clicks on the Request Schedule button, navigating to form JBSF5212 to enter the date and time when she wants the request to run. She saves the information and exits the form.
In the central form, she clicks the Submit button, and then the Save button.

The request is submitted, and now waits for the system to schedule it to run at the correct time.

Duplicate Requests

Any request in this subsystem can be used as a template for a new job request, with altered request details if required. This gives a quick method to create requests. Full details are given in Maintain Request Details (JBSF5210) form documentation.

Standing Requests

A schedule for a request can be set up in two ways. The more common way is outlined in Steps in a Job Request. An alternative is to set it up as a Standing Request.

With a Standing Request, the system will run the request automatically at specified intervals within a given period, rather than on precise dates designated in advance. For example, a Standing Request can be set up to run every Friday in March, April and May, or on each census date in the Academic Year. The use of Standing Requests is confined to jobs where the original parameters remain valid, whenever the job is run. Security provisions allow for the setting up of Standing Requests to be limited to specific people.

A checkbox in the central form, Maintain Request Details, is used to indicate a Standing Request. If this check-box is selected, a button to navigate to the Maintain Standing Request Frequency form (JBSF5211) takes the place of the button to set up the basic request schedule. If a user does not have the security privilege to set up Standing Requests, this check-box's label is 'greyed out'.

In all other ways, the creation of a Standing Request follows the steps to set up a basic request.

Output From Jobs

A job may produce one or more output files as a source of information to be printed, faxed or sent as email. These outputs are specified with the job, but there are different possibilities in handling output destinations for each output file attached to a job. Destinations may or may not be specified when the job is created, and if created, may or may not be mandatory. Theoretically, therefore, the following situations could be encountered for outputs of jobs in just one request:

An output destination has already been defined for a job (as an 'output option'), and cannot be altered when the job is requested
A destination already defined can be altered when the job is included in the request
A destination is not defined for a job, and the user must add one when creating the request

Provision has been made in Callista for destinations of these types:

Printer
Email

Depending on how the job output is specified, one output file may be directed to different output destination types, or different output files may be created for each type, even where the content is the same.

Further details about adding output destinations at request time are given in the documentation for Maintain Request Job Output Destinations (JBSF5230).

Once a request has run and produced output, this output can be 're-sent' to its original destinations or to new destinations, as long as the person who set up the request has not specified output deletion after sending. See details in the documentation for the form, Inquire on Job Run Outputs (JBSF5410).

Details for the Report Engines to be available for Callista job outputs are defined in JBSF4150 and one of these may be nominated as the default Report Engine.
These defined Report Engines may then be selected for particular jobs in JBSF4110. If no report engine is selected in this form, and there is no default Report Engine selected in JBSF4150, then the default Report Engine for Callista is Oracle Reports.

Checking on the Progress of a Request

Once requests have been submitted, two background processes keep track of their progress and schedule and run the jobs they contain. These processes run at predetermined intervals, so that there may be a slight time delay between submitting a request and subsequent activity. The time intervals for the two background processes are set by information technology staff responsible for the JBS subsystem.

Users can follow the course of their job requests:

By looking at a request's status at any particular time, and
By using forms designed to track individual jobs in a request

Request Statuses

A request's status is maintained by the system, and can be seen in the Request Status field in the central form, Maintain Request Details (JBSF5210). The possible request status values are: PLANNED, WAIT, SCHEDULED, COMPLETE, CANCELLED.

A request is set from PLANNED to WAIT status as soon as it is submitted, and to SCHEDULED when it reaches its scheduled time to run. It can be CANCELLED at any time before completion, and during the WAIT status it can be re-planned. Re-planning can include altering its schedule.

Because a request can be scheduled to run on several occasions, either as a Standing Request or as a normal request with several schedule dates/times, it will revert to WAIT status between runs, and only be set to COMPLETE at the end of the last cycle.

Request Statuses
Status	Progress of request	User intervention possible
PLANNED	Not yet submitted	Most details of request can be changed. A run window can be changed as long as no other request details have been created. Request can be CANCELLED.
WAIT	Request submitted. It may not yet have been picked up by the relevant background process, or has not reached date/time given in its schedule. Requests with more than one schedule date/time revert to WAIT status between runs. The (next) run date/time is decided at the point when the WAIT status is set or re-set.	Request can be: Returned to PLANNED status by using Re-plan Request button. CANCELLED.
SCHEDULED	The relevant background process has determined that request has reached or passed its scheduled run time, and has marked it for running, conditional on priority and other restrictions.	Request can be CANCELLED.
COMPLETE	All scheduled runs for the request have completed (successfully or otherwise).
CANCELLED	The user has cancelled the request.	Request can be cancelled up to point of completion, using Cancel Request button.

Keeping Track of Jobs

Two forms give more detailed information about the progress of individual jobs within requests:

Request Job Run (JBSF5300)

This form gives details of the progress of individual jobs within a request, including run start and end times, and the statuses of all jobs submitted. These job run statuses (as distinct from request statuses) are WAIT, SCHEDULED, RUNNING, COMPLETE, FAILED, ABORTED and CANCELLED, all but the last of which are maintained by the system. Users can intervene in the running of jobs by cancelling the request, or by cancelling an individual job with a status of WAIT or SCHEDULED.

The table below illustrates the relationship between request statuses and job statuses.

Job Run Statuses for Jobs in a Request
Request Status	Job Run Status	Activity
PLANNED
WAIT	WAIT	Jobs have not yet been scheduled to run, or have reverted to WAIT status pending another scheduled run.
SCHEDULED	SCHEDULED	Jobs are SCHEDULED when request is scheduled by System. Individual Jobs progress through statuses RUNNING, COMPLETE, (FAILED), (ABORTED) while Request Status is still SCHEDULED. Any job can be CANCELLED by user. Status FAILED signifies that job failed to complete successfully. Status ABORTED signifies that job has been cancelled by the System because a previous job in the request failed to complete successfully, or that it cannot be restarted after a System failure.
	RUNNING
	COMPLETE
	CANCELLED
	(FAILED)
	(ABORTED)
COMPLETE	COMPLETE	All jobs have completed (successfully or unsuccessfully), and there are no more runs scheduled for request.
	(FAILED)
	(ABORTED)

Full details are available in the forms level documentation for Request Job Run (JBSF5300).

Run Log (JBSF5301)

Use the run log to see details of jobs that have finished, whether successfully or unsuccessfully.

More information is given in the documentation for Run Log (JBSF5301).

Job Notification

Appropriate users can be nominated in JBSF5213 to receive notification by email and/or SMS, when a job request completes or fails. Default details for such users can be recorded in SECF0066.

Troubleshooting - Why Jobs Don't Run

Jobs can fail or be aborted because:

A pre-requisite job has failed.
The person requesting the job does not have access to data or database tables required by the job.
The Abort On Error indicator is on, and a previous job has failed.
There has been a system or database failure, and the job is specified as not restartable.

Jobs can be held up, or not run, because:

There are many higher priority jobs scheduled ahead of them.
They are waiting for a pre-requisite job, or a job specified earlier in the sequence to run.
A pre-requisite or earlier-in-sequence job has a low priority set.
There is no active batch queue for the same run window, or the batch queue is restricted to a different priority or priorities.
There is a run mode (IMMEDIATE or BATCH) restriction against a report job, and the current run window has the opposite mode restriction.

If you suspect one of these problems applies to a job you have attempted to run, you should click on the Job Diagnostics button and view details about a job's progress in the pop-up window displayed.

Last modified on 13 May, 2015 10:25 AM

History Information

Release Version	Project	Change to Document
18.0	2094 - Character Sets	Add note to Intro. regarding the use of multi-byte characters.
10.1.0.0.0.0	1388 - Multiple Output Types	Added details re Report Engines