Assessments For The Subsystem Specialist

Assessments - for the Subsystem Specialist

This section of the user manual provides specialist users (such as system administrators and subsystem specialists) with the information required to set up and maintain the reference data used by the Assessments subsystem, and to perform the specialist functions available in this subsystem.

An overview of the day to day operation of the subsystem is provided in Understanding Assessments.

Information regarding the use of a specific form is obtained by selecting a link to that form in the text below or selecting from the subsystem table of contents.

Topics covered here include:

setting up assessments reference data

setting up assessments calendars

setting up and maintaining assessment items

attaching assessment items to students

setting up and maintaining unit assessment patterns

attaching assessment items to assessment patterns

attaching assessment patterns to students

managing non-examinable assessment items
managing examination details

examination sessions

examination locations and venues

timetabling

supervisors

seat allocations and attendance

maintaining grading schemas

translating grading schemas

recording results
publishing outcomes
assessments jobs and reports

Setting Up Assessments Reference Data

In order for the Assessments subsystem to function effectively, the following reference data must be created and maintained:

Reference Data	Purpose	Optional ?	Maintenance Form	Pre-requisite Setup Dependencies
Assessment Types	used in the definition of assessment items in ASSF2100	Essential	ASSF0110	none
Examination Material Types	Used in the definition of examination materials in ASSF2123	Essential	ASSF01A0	none
Assessor Types	Used to define an assessor assigned to an assessment item in ASSF2100	Essential	ASSF01E0	none
Exam Locations	Venues and supervisors are attached to locations	Essential for examination management	ORGF0131	none
Venues	Supervisors are assigned to venues, examinations are scheduled at venues etc.	Essential for examination management	ASSF4200	Exam Locations (ORGF0131)
Assessment Item variation Cutoff Date Alias	Prevents changes to unit assessment items after a specified date. Particularly applies to jobs ASSR3212 and ASSR3213	Essential. A nominal date could be used where the function is not required.	ASSF01N0	Date alias recorded in this form must be created in CALF0420.
Special Consideration Category	Used to categorise special consideration applications in ASSF4911	Essential for special consideration functions	ASSF01L0	none
Special Consideration Outcome	Used to record sought and final outcomes in ASSF4911	Essential for special consideration functions	ASSF01M0	none
Supervisor Types	Used to categorise supervisors in ASSF4310	Essential	ASSF0150	none
Result Entry Configuration	Controls the entry of student unit attempt outcomes during both manual and electronic recording of results	Essential for electronic upload of results.	ASSF01K0	none
Tracking Types	Used to define tracking items. A tracking type can be mapped to a System defined tracking type of ASSIGNMENT	Essential for Assignment Tracking Items in ASSJ3610.	TRKF1100	none

The documentation for each of the reference data forms above details any functions or rules specific to those forms.

Setting Up Assessments Calendars

Examinations Calendars

In common with most subsystem calendars in Callista, it is most efficient to establish the complete set of exam calendars for a particular academic year and then use the Calendar Rollover process to create calendars for subsequent years. Relevant parts of the recommended examination calendar creation process which follows can then be used to maintain the 'rolled over' calendars.

Using CALF0220 (Maintain Calendar Types), create a calendar type with a calendar category of EXAM which will be the type of all calendars representing exam periods, e.g. EXAM-S1.

In the same form, create instances of the new calendar type to define each required exam period, e.g. a semester 1 exam period could be represented by the calendar instance EXAM-S1 - 01/06/1998 - 12/06/1998. Note: the calendar instance start and end dates MUST reflect the actual exam period start and end dates.

Select the Calendar Relationships button to open CALF0330. Create relationships between the exam period and its superior academic year and its subordinate teaching periods. For example: EXAM-S1 (1999) would have the 1999 academic year as a superior relationship. ALL teaching periods with units to be examined in Exam-S1 (1999) must be set up as subordinate calendars of this exam calendar. For example: SEM-1 (1999), YR-LONG (1999) and S2-E2 (teaching period beginning in semester 2 1998 and ending in semester 1 in semester 1 1999) will be subordinate calendars. Note: where supplementary or special exam calendars are to be used, the calendar relationships should be established as for a standard exam calendar. However, the standard exam calendar should also be set as a subordinate calendar to the supplementary or special exam calendar. For example: EXAM-S1 would be a subordinate of EXAM-SUPP1.

Using CALF0420, create date aliases representing days on which examinations are held. It is recommended that a separate date alias be created for each day in an exam period e.g. EXAM-DAY1, EXAM-DAY2 and so on.

Using CALF0511 (Maintain Date Aliases Instances) (accessed directly via the navigation button in CALF0420), create instances of the date aliases, each instance defining a particular date within an examination period when examinations can occur, e.g. EXAM-DAY1 - 01/06/1998, EXAM-DAY2 - 02/06/1998, EXAM-DAY1 - 12/10/1998.

The examination sessions available on each day of an exam period are then defined according to the process described in managing examination details.

Assessment Calendars

Assessment calendars are used for assessment processes other than examinations. Like examination calendars they are effectively groupings of teaching periods. They are not referenced in any forms but are used mainly as job/report parameters. An assessment calendar defines the set of teaching periods whose associated units are concurrently 'put through' the assessment process. For example, an assessment period ASS-SEM1 may have the teaching periods SEM-1, S1-E1 and S2-E1associated with it. The units associated with these teaching periods will be assessed during the assessment period.

Results for units in these 3 teaching periods are processed concurrently. Thus, when running reports such as Result Sheets, Invalid Grades and Non-enrolled Outcomes, ASS-SEM1/1998 can be specified as a parameter rather than running the report for individual teaching periods.

Assessment calendars for other purposes may be required in later releases of Callista.

Using CALF0220 (Maintain Calendar Types), create calendar types with a calendar category of ASSESSMENT to represent each assessment period in an academic year, e.g. ASS-SEM1, ASS-SEM2.

In the same form, create instances of the new calendar type to define each required assessment period, e.g. a semester 1, 1998 assessment period could be represented by the calendar instance ASS-SEM1 - 01/03/1998 - 30/06/1998.

Select the Calendar Relationships button to open CALF0330. Create relationships between each assessment period and its related teaching periods. Note: teaching periods are subordinate to assessment periods, i.e., the relationship is created between an assessment period and those teaching periods with any units which will be assessed in the assessment period.

Create relationships between assessment periods and their superior academic periods.

Using CALF0420, create a date alias representing a date after which changes should not be made to unit assessment items.

Record this date alias as the Assessment Item Variation Cutoff Date Alias in ASSF01N0.

Setting Up and Maintaining Assessment Items

There are three mechanisms for creating and maintaining assessment items.

The usual approach is to locate a unit offering pattern and then create or maintain assessment items which 'belong' to it (unit assessment items). To do this:

either:

enter ASSF2310 in context from the Maintain Unit Offerings form CRSF2310 (entered via CRSF2210). Other unit offering patterns cannot be queried in ASSF2310 when accessed in this way. Close ASSF2310 and return to CRSF2310 to select another unit offering pattern.

enter ASSF2310 directly from a menu and query to locate the required unit offering pattern

then either:

create a new assessment item by selecting the Assessment Item icon . This invokes ASSF2100

enter basic details of the assessment item in ASSF2100, and other details in forms accessed via navigation buttons in ASSF2100. Exiting ASSF2100 will return you to ASSF2310 and display details of the new assessment item

enter additional details for the unit assessment item (including those restricting its application), as required, in ASSF2310

select an existing unit assessment item and modify it as required in ASSF2310

select the Assessment Item icon to invoke ASSF2100. The existing assessment item is displayed. Modify basic details as required in ASSF2100, and other details in forms accessed via navigation buttons in ASSF2100.

The second approach is to create an assessment item in isolation and then associate it with one or more unit offering patterns. To do this:

create the assessment item and its basic detail using ASSF2100 and other details in forms accessed via navigation buttons in ASSF2100. (or ignore this step if the assessment item has already been created)

open ASSF2310 and query to locate the unit offering pattern to which the assessment item is to be attached

select the Assessment Item icon to invoke ASSF2100 and execute a query here to locate the existing assessment item

exit ASSF2100. The assessment item is returned to ASSF2310.

The third approach is to create a unit assessment pattern and then to create or associate one or more assessment items within the unit offering option assessment pattern. To do this:

open ASSF2320 either in context from ASSF2310 or from the menu
create an assessment pattern; and
create assessment items for the unit assessment pattern by selecting the Assessment Item button to invoke ASSF2100 and create a new assessment item or query existing assessment items
exit ASSF2100. The assessment item is returned to ASSF2320.
Further details on creating assessment patterns can be found in the section on Setting Up and Maintaining Unit Assessment Patterns.

A variation to the above processes involves the use of the Find Assessment Item icon in ASSF2310 and ASSF2320. This icon invokes the Find Unit Assessment Item form (ASSF2311) where a query can be executed to locate a particular assessment item attached to another unit. By default, entering the form displays all assessment items already linked to the context unit. This assessment item is then attached to the current unit by exiting ASSF2311 and returning to ASSF2310 or to the unit assessment pattern by returning to ASSF2320.

ASSF2100 contains buttons which provide access to enable the recording of:

instructions to examination supervisors

announcements to be made to exam candidates at the examination venue

assessment item constraints

comments regarding the assessment item

course types to which the assessment item can be restricted (as text or a list)

assessor details

allowable, non-allowable and supplied examination materials

and the listing of examination materials.

Attaching Assessment Items to Student Unit Attempts

(and Detaching Assessment Items from Students)

When a student enrols, the process ASSJ3212 - 'Automatically Maintain Student Unit Attempt Assessment Item ' is run to attach the default assessment items associated with the unit offering option to the student's unit attempt. Where assessment items are associated with (or deleted from) unit offerings after students have enrolled in them, ASSJ3213 - 'Apply Unit Assessment Item Modification to Students' is run to attach the assessment items to (or detach deleted items from) students.

Note that where:

a student unit attempt assessment item is queried (in ASSF3222) and

a change has been made to the assessment item so that it is no longer valid for the student and

ASSJ3213 has not yet been run to detach the assessment item from the student, then

an 'INVALID' lamp displays against the student unit attempt assessment item.

ASSJ3213 is also used to update students' assessment items where the assessment items attached to a unit have been altered and remove non-default unit assessment items from students where the assessment items have been deleted from the unit.
ASSJ3212 and ASSJ3213 are generally run together in a single job request to ensure that all student records have been updated.

Non-default unit assessment items can be manually attached to, and maintained against, individual students in form ASSF3222.

Setting Up and Maintaining Unit Assessment Patterns

To create and maintain unit assessment patterns:

Locate a unit offering pattern and then create assessment patterns which 'belong' to it. To do this, either:

Enter the Maintain Unit Assessment Patterns form (ASSF2320) in context from the Maintain Unit Assessment Items form (ASSF2310). Other unit offering patterns cannot be queried in ASSF2320 when accessed in this way; or

Enter ASSF2320 directly from a menu and query to locate the required unit offering pattern; then

Create a new assessment pattern. It should be noted that an assessment pattern is meaningless unless it contains assessment items.

To attach assessment items to an assessment pattern:

Navigate to the Assessment Pattern Assessment Item block after creating a new pattern or retrieve an assessment pattern and either:

Enter a known assessment item ID; or

Select the Assessment Item button to invoke the Maintain Assessment Items form (ASSF2100). Create a new item by entering its basic details and other details in forms accessed via navigation buttons in ASSF2100. Exiting ASSF2100 returns you to ASSF2320 and displays details of the new assessment item; or

Select the Find Assessment Item button . This button invokes the Find Unit Assessment Item form (ASSF2311) where a query can be executed to locate a particular assessment item attached to another unit. By default, entering the form displays all assessment items already linked to all offerings of the context unit. The highlighted assessment item is then attached to the current assessment pattern by exiting ASSF2311 and returning to ASSF2320.

Attaching Assessment Patterns to Student Unit Attempts

The processes ASSJ3212 and ASSJ3213 allocate both default assessment patterns and default assessment items to student unit attempts.

Student unit assessment patterns are viewed in the Maintain Student Unit Assessment Patterns form (ASSF3221). The form is also used to manually allocate non-default assessment patterns to individual student unit attempts.

Managing Non-examinable Assessment Items

'Non-examinable assessment item' is the term used in Callista to describe any assessment items other than examinations. This includes student assignments, projects and performances. This function provides mechanisms to manage any non-examinable assessment items which are processed and tracked through a centralised group, such as a group within student administration.

Use of this facility is optional.

The typical assignment management process is:

ASSJ3610 is run through the job scheduling subsystem to create a tracking item for each student assignment. The Tracking subsystem is then used to track the progress of assignments.

In the same job request, ASSJ3500 is run to create an extract file from which an assignment cover sheet is produced for each student assignment. Individual (or groups of) cover sheets can be re-produced as and when required by running ASSJ3500R. Note that the extract file must be processed by an external program which is not included with Callista. Individual institutions need to custom build their own assignment cover sheet creation process, utilising this extract, to produce their version of a cover sheet.

The generation of a cover sheet is automatically registered in the Correspondence subsystem. A single outgoing correspondence item is recorded each time the cover sheet generation process is run. Each student due to receive an assignment cover sheet as a result of the 'run', is recorded as a recipient of the correspondence item.

A number of reports are available to assist in monitoring the assignment management processes. They are:

Assignment Item Ledger Report ASSR3621

Assignment Reminder Letters ASSR3623

Assessor Day Reports ASSR1010

Managing Examination Details

The management of examination details covers all facets of the examination process. Data is recorded progressively to create an information bank upon which timetabling of exams is based. Data is passed to Syllabus Plus via the exam timetabling interface (ETI). Timetabling is performed in Syllabus Plus and exam timetabling details are passed via ETI back to Callista. Supervisors are then allocated to venues and/or individual examinations. A typical approach to examination management follows. It is dependent on the existence of examinable unit assessment items for all units being examined during the examination period in question:

Examination Sessions

Examination calendars are established according to the process described in setting up assessments calendars.

Examination sessions are defined for the date alias instances within the exam period using ASSF4612 (Maintain Examination Sessions) .

Examination Locations and Venues

One or more location types are created using ORGF0117 (Maintain Location Type) each of which is assigned a System Location Type of EXAM_CTR. These are used in the definition of locations.

Examination locations are created and maintained using ORGF0131 (Maintain Locations). Exam locations are assigned one of the location types mapped to the system location type EXAM_CTR. Via navigation buttons in this form:

Location addresses are recorded in ORGF0132 (Maintain Location Addresses).

Examination locations used for on campus unit enrolments should be made subordinates of the campus location(s) to which they relate. If a location is the 'usual' examination location for on campus units at a particular campus location, it should be flagged as the default exam location. If a location is to be the 'usual' examination location for off campus units, it should be flagged as the off campus default exam location. This is achieved via the form ORGF0133 (Maintain Location Relationships). These relationships enable the System to:

Distinguish between internal and external exam locations, for the purposes of the default off campus supervisors process in ASSF4630, and for certain examination related reports (ASSR06F0).

Determine the examination location for on campus student unit attempts. For such unit attempts, this is the exam location flagged as the default subordinate location.

Determine the examination location for off campus student unit attempts. For such unit attempts, this is the exam location flagged as the off camps default subordinate location.

Note: an exam location of NONCENTRAL must be created when using the concept of centrally scheduled but non-centrally managed examinations. Refer to the technical documentation for further details.

The venues associated with a particular examination location are recorded and maintained in ASSF4200 (Maintain Venues). Each exam location must have at least one venue to enable its use for exam timetabling. Venue addresses are recorded in ASSF4201, accessed via a navigation button in ASSF4200.

Venue Session Availability

The form ASSF4212, Maintain Exam Venue Session Availability and the process Default Exam Session Venue Availability, ASSJ4212, are used to record the availability of venues for sessions within an exam period. ASSJ4212 enables all open venues to be defaulted as available for all sessions. ASSF4212 may also be used to manually record session availability for individual venues.

Timetabling

The Examination Timetable Interface (ETI) enables examination and student details to be passed to Syllabus Plus so that Syllabus Plus can generate an exam timetable. Once generated, the timetable details are passed from Syllabus Plus to Callista, again via ETI.

Details passed by Callista to ETI include:

organisational units

exam location details

venue details

unit exam details

student details

student exam details

exam period details

exam period session details

session venue details (i.e. venue availability details)

Once a timetable is generated in Syllabus Plus, details of session, examination instances and student examination instances are passed to Callista. When passing data back to Callista after the initial transfer, the ETI operator is given a list of changes and is able to select which changes to apply to Callista.

For detailed instructions on this process, see the ETI on-line user guide.

Various examination based reports may be run. These include:

Seating allocation report (ASSR06A0)

Exam packaging labels (ASSR06U0)

Once final exam instance and student exam instance details exist in Callista, supervisors can be assigned to either exam session venues or specific exam instances.

Supervisors

Details of examination supervisors are recorded using ASSF4310 (Maintain Examination Supervisor Details). Detail recorded includes the exam location(s) to which the supervisor may be assigned and the supervisor's default supervisor type.
ASSF4630 (Allocate Supervisors to Venues) is used to assign examination supervisors to exam venues within sessions and/or particular examination instances.

Seating, Timeslots and Duration

The form ASSF4642, Maintain Student Examination Details is used to:

Allow the overriding of seating allocations.
Record examination timeslots and duration to allow appointments throughout an examination.

Maintaining Grading Schemas

The grading schemas used by the institution must be recorded and maintained in Callista in order for the Assessments subsystem to function effectively. Grading schemas are recorded and maintained using ASSF5110. Further details of the data comprising a grading schema are contained in the documentation of ASSF5110.

Grading schemas are attached to unit offering options in the Maintain Unit Offerings form (CRSF2500) or to course offering patterns in the Maintain Course Offerings form (CRSF1500). When the outcome of a student's unit attempt is recorded in Callista:

the System checks which grading schema applies to the unit offering option concerned

if a mark is recorded and the applicable grading schema has mark ranges recorded against grades, the System derives the appropriate grade from the grading schema (if configured to do so in ASSF01K0)

if a mark is recorded and there is no mark range in the grading schema into which it falls, the System enforces the entry of a grade

if a grade is entered, it is validated against the available grades for that grading schema

Refer to ASSF01K0 for details of how the use of grading schema can be configured.

Where an overriding course offering pattern grading schema has been recorded, recommended grades are translated form the unit grading schema to the course grading schema.

Future functionality provides for grading schema to be attached to individual student unit attempts.

Translating Grading Schemas

Students enrolled in a unit offering option may also have a grading schema mapped to their course offering pattern. Where this occurs, if the course grading schema has precedence, the student's grade from the unit grading schema must be translated to the grade for the course grading schema.

To translate grading schemas:

Create a relationship between the grading schemas' by mapping the unit grades to the course grades in the Maintain Grading Schema Grade Translations form (ASSF5120).

Then:

Run the Translate Student Unit Attempt Outcomes process (ASSJ5390). This process translates grades for student unit outcomes where the student is enrolled in a course with a grading schema assigned.

The process does not translate student unit attempt outcomes where:

No course grading schema was specified for the course offering pattern.
The Unit Grading Schema Precedence indicator was selected in CRSF2500.
No relationship was created between the unit grading schema grades and the course grading schema grades.

Recording Results

Callista provides two mechanisms for recording student unit attempt outcomes (results):

manual recording

electronic upload

Both mechanisms are configurable using the Maintain Entry Configuration form (ASSF01K0), enabling the result recording practises of the institution to be accommodated.

For example, the manual recording process can be configured to allow the recording of invalid mark/grade combinations which may be corrected later, or to make the entry of marks mandatory. The electronic upload process can be configured to perform different actions depending on the errors or warnings encountered. The section of the manual covering the form ASSF01K0 has full information.

Manual Recording of Results

This method involves the keying of results which have been manually recorded on result sheets. A result sheet is a listing of students enrolled in a particular unit offering location/mode combination and has provision for the recording of marks and/or grades. Typically the process is:

The report ASSR5310 is run to produce the required result sheets (as defined by the report parameters). This report may be run at any time. A report run close to the end of the teaching period gives a more accurate listing of students.

Each result sheet is sent to the responsible academic (for example a lecturer or unit chair for the unit covered by the result sheet).

The responsible academic records marks and/or grades against each student listed on the result sheet and adds details of any students, for whom they have a result, who is not listed.

The result sheet is returned to student administration for keying.

The marks and/or grades for each student are keyed using the Mark/Grade Entry form (ASSF5330)

Students added to the mark sheet by the responsible academic are treated in one of two ways:

Where the student was enrolled in the unit offering after the mark sheet was produced, the student is selected from the list of values displayed when the Add Student button in ASSF5330 is selected. Marks/grades can then be keyed for the student.

Where the student is not enrolled (i.e. not displayed in the Add Student LOV), the Add Non-Enrolled Student Outcome form (ASSF5333) is used to record details for later resolution.

Grades (administrative grades) may be automatically inserted by the System where:

no grade is recorded for a student unit attempt

a unit is non-assessable

a student unit attempt is non-assessed

Results can be amended in ASSF5330, configuration permitting, or in the Amend Student Unit Attempt Outcomes form (ASSF53B0).

Electronic Upload of Results

This method involves the recording of marks and/or grades in electronic format and subsequent uploading into Callista. More detailed instructions for end users are provided in Understanding Assessments. The typical process is:

Responsible staff are provided with the facility to download a file which lists the students for whom unit assessment is to be recorded. The Download Class List process in INQF12E0 is activated by the staff member to produce a comma separated variable file for loading into a database or spreadsheet application.

The staff member records final marks and/or grades (depending on configuration, refer ASSF01K0) against each student listed in the spreadsheet/database file. Students not listed in the file, but for whom the academic has results, may be inserted in the file along with their results.
The staff member enters details of the file to be uploaded in the Validate Outcome Upload File form (ASSF5340) and selects the Validate File button to run the file validation process in accordance with the configuration settings in ASSF01K0. Validated records are uploaded into a batch file within Callista. This process produces an exceptions report, ASSR5340 - Upload Student Unit Attempt Outcomes Exception report, which lists warnings, errors and fatal errors. If fatal errors are encountered, the file is not uploaded. The validation process can be run several times, for the same file, as exceptions are resolved by the academic.

Batch files created in the step above are processed centrally by running the job ASSJ5341. (This job also uses the electronic upload configuration settings from ASSF01K0, as the data is again validated to check for unit discontinuations since the upload.) Records in the batch file may be processed in a number of ways:

Records which were successfully validated are recorded as student unit attempt outcomes. They can be viewed in the Amend Student Unit Attempt Outcomes form ASSF53B0.

Records which were not successfully validated are reported in the Electronic Outcome Upload Exception report ASSJ5341. The report describes the reason each record failed validation and the action applied to the record. Actions include:

Records designated HOLDING (see ASSF01K0) are loaded as non-enrolled student unit attempt outcomes, pending resolution. These can be viewed in ASSF5333. The Non-Enrolled Student Unit Attempt Outcome report (ASSR06J0) is run to assist resolution of these cases.

If a grade already exists for a record, the record is ignored by the upload process and reported in the exceptions report, unless the grade is one which is specified in ASSF5110 as able to be replaced.

Records designated DON'T LOAD or ABORT are ignored by the upload process and reported in the exceptions report. In both cases, remaining records continue to be processed.

Publishing Outcomes

Outcomes are initially set as recommended. After the appropriate authorising body (such as a Board of Examiners) has ratified outcomes and any required amendments have been recorded, outcomes must be finalised before they can be published. This is achieved using the process Finalise Student Unit Attempt Outcomes (ASSJ53A0).

Once finalised, outcomes may be published in a variety of ways. The more traditional methods for publishing outcomes are:

statements of results

academic transcripts (ASSF5600 and ASSJ06V0)

publication in newspapers and on university noticeboards (ASSJ5410).

Results would also be made available via information booth facilities such as the World Wide Web and IVRS. (These facilities are addressed in future releases of Callista)

Assessments Jobs and Reports

The reports in the Assessments subsystem are an integral part of its operation. They are an essential link between the business processes of the institution and Callista. Some reports trigger database processes critical to the operation of the subsystem.

It is expected that a person 'running' any of these reports is experienced in the mechanisms for generating reports in Callista. Full details regarding running reports both on-line and in batch mode are provided under 'All About Standard Reports' and in the Job Control and Scheduling subsystem.