Understanding Assessments

This section of the user manual provides an overview of the Assessments Subsystem. Topics covered include:


Detailed explanations and instructions for the use of individual database forms can be accessed from the Subsystem Table of Contents or via the numerous links provided here.

Information regarding Subsystem Reference Data and Setup is provided separately for the Subsystem Specialist.


The Assessments Subsystem

The Assessments subsystem is used to manage the academic assessment of students enrolled in units of study. Key functions in this management process are:

 

Terms Used in the Assessments Subsystem

To understand the Assessments subsystem it is necessary to understand the following key terms. Explanation of all Assessments terminology is provided in the Subsystem Glossary.

Assessment Item

A set activity intended to evaluate a student's understanding of a unit. For example, a particular examination or Assessment for a given unit.

Assessment Type

A means of classifying Assessment Items. For example, EXAM, ORAL EXAM, Assessment.

 

How the Assessments Subsystem is Used

User Responsibilities

Each institution will determine how the Assessments Subsystem is applied to its own business practises. As a guide, 'non-specialist functions' are limited to Assessment tracking, recording special consideration applications and outcomes and recording and amending Student Unit Attempt Outcomes.

Recording of outcomes can be achieved by direct keying , bulk electronic upload of results, and insertion by the System (where the System inserts a grade as a result of discontinuation of a unit, where the System inserts an administrative grade, if no other grade is recorded, and where a grade is inserted for non-assessable units and non-assessed student units).

The remaining functionality is normally used by Subsystem Specialists or System Administrators. This functionality includes:

These functions are detailed in the Subsystem Specialist section.

 

Assessments Reference Data and Setup

For the Assessments Subsystem to function effectively, it is necessary to setup specific reference data and define certain data values. This setup of the Subsystem is detailed in the Subsystem Specialist section. The setup of reference data will not concern the 'general' user who will see most reference data as a list of values (LOV) within a form.

 

All About Assessment Items

Assessment Items

An Assessment Item describes one of the elements by which a unit of study is assessed.

For example, a student may be assessed in a unit by submitting two Assessments, sitting a written examination and participating in a practical examination. Each of these requirements is an Assessment Item.

The specific requirements for another unit will normally be different and hence that unit will have a different set of Assessment Items.

As an example, both units may have a theory examination. 'Theory examination' might be the institution defined Assessment Type of these Assessment Items. A particular Assessment Item is uniquely identified by a System assigned identification number, with an Assessment Type and description providing its definition.

 

Examples of Assessment Items

Assessment ID

Assessment Type

Description

42

THRY-CTL - Theory Examination centrally administered

Creative Writing 1

57

THRY-FAC - Theory Examination administered by faculty

Psychology 1

72

THRY-CTL- Theory Examination centrally administered

Language 1

121

ORAL-CTL - Oral Examination centrally administered

Indonesian 1

146

ORAL-FAC - Oral Examination administered by faculty

Chinese 1

220

PRAC-FAC - Practical Examination administered by faculty

Preliminary Physics

223

Assessment - Assessment

Myth in the Modern Era

 

Assessment Items are recorded and maintained via the Maintain Assessment Items form (ASSF2100). It is only necessary to record those Assessment Items which are used by specific System functionality.

For example, examinations which are to be scheduled and Assessments which are to be tracked must be recorded.

It is not absolutely necessary to record examinations given by lecturers outside of scheduled Examination Periods or Assessments which are not being tracked. However, it is important to note that, it is not possible to process a special consideration application for a Unit Assessment Item unless the item has been recorded in the System, and assigned to the student's unit attempt. Nor would it be possible to use the System to advise students of all their assessment requirements. It is therefore recommended that the complete set of Assessment Items and Patterns be recorded for each unit.

 

Examinable/Non-Examinable Assessment Items

Assessment Items may be examinable (items 57, 72, 146 and 220 in the table above) or non-examinable, as in the case of an Essay, an Assessment, a theatrical/musical performance or a Practicum. It is only possible for Examinable Assessment Items to be considered for exam time-tabling. The Examinable check box is set (or unset) for each Assessment Type and is inherited by the Assessment Items to which the Assessment Type is assigned.

Examinable Assessment Items may be scheduled (these items are passed to the timetabling module) or not scheduled (these items occur outside the defined examination period and are not passed to the timetabling module). The Scheduled check box is set (or unset) for each Examinable Assessment Item to indicate whether or not it is to be timetabled.

 

Detail Recorded Against Assessment Items

The table below lists additional detail which can be recorded about each examinable Assessment Item using buttons in the Maintain Assessment Items form (ASSF2100). The only additional information able to be recorded against non-examinable Assessment Items is Assessors.

Assessment Item Detail

Description

Supervisor Instructions

Text field for recording instructions to be conveyed to the examination supervisor(s) for this Assessment Item.

Comments

Text field for recording of any comments regarding this Assessment Item.

Assessment Course Types

The Assessment Item is restricted to student units belonging to those course types which are recorded here. E.g. POSTGRAD, UGRAD. (Not: Only if the student is studying a unit in a course attached to one of these course types)

Assessors

Details of the assessor(s) for this Assessment Item. One of the assessors is nominated as the Primary Assessor. Assessors are assigned an assessor type, meaning that a single person can be recorded more than once where they are performing different assessor type roles. The role of an assessor can be refined or restricted down to the level of a Unit Offering option.

Announcements

Text field for recording announcements to be made before, during or after an examination.

Allowable Materials

Text field and/or list for recording the materials permitted to be taken into an examination.

Non-Allowable Materials

Text field and/or list for recording the materials not permitted to be taken into an examination.

Supplied Materials

Text field and/or list for recording the materials supplied by the institution for an examination.

 

Attaching Assessment Items to Units

As mentioned previously, an Assessment Item describes one of the requirements by which a unit of study is assessed. How this relationship between a unit and its Assessment Items is recorded and maintained is described in the Subsystem Specialist section.

When a student enrols in a particular unit of study and the process ASSJ3212 - 'Automatically Maintain Student Unit Attempt Assessment Items' is run, the student 'inherits' the default Assessment Items associated with the unit. When a default Assessment Item is attached to a unit after students have enrolled in it, the process ASSJ3213 assigns the Assessment Item to student units.

 

All About Assessment Patterns

Assessment Patterns

An Assessment Pattern is a group of Assessment Items for a Unit Offering. Creating Assessment Patterns allows a number of Assessment Items to be attached to a Student Unit Attempt as a group. Assessment Patterns are typically used when students are given a choice of assessment methods.

For example, students may be given a choice of two patterns; one comprised of an examination and three Assessments, and the other comprised of an examination and a major project. If no choice of assessment exists, the discrete set of Assessment Items for the Unit Offering may still be set up as an Assessment Pattern.

The advantage of using a pattern is the ability to record the proportion of the total assessment applicable to each item.

 

Examples of Assessment Patterns for Unit AAC131

 Assessment Pattern Code

Assessment Item ID

Assessment Type

AAC131PAT1

10474

EXAM

 

10589

Assessment

 

10592

EXAM

AAC131PAT2

10474

EXAM

 

10589

Assessment

 

10600

PROJECT

 

Assessment Patterns are recorded and maintained via the Maintain Unit Assessment Patterns form (ASSF2320). Unlike Assessment Items, Assessment Patterns cannot be created as stand-alone. Creation of an Assessment Pattern automatically attaches it to the Unit Offering pattern.

 

Attaching Assessment Items to Assessment Patterns

As mentioned previously, Assessment Patterns are groups of Assessment Items. How the relationship between an Assessment Pattern and an Assessment Item is recorded and maintained is described in the Subsystem Specialist section.

 

Attaching Assessment Patterns to Student Unit Attempts

When a student enrols in a particular unit of study and the process Automatically Maintain Student Unit Attempt Assessment Items (ASSJ3212) is run, the student 'inherits' the default Assessment Pattern, and its Assessment Items, associated with the unit. When a default Assessment Pattern is attached to a unit. When a default Assessment Pattern is attached to a unit after students have enrolled in it, the process ASSJ3213 assigns the Assessment Pattern and its items to the Student Unit Attempts.

 

Typical Assessment Processes

The basic requirements for assessment processing are:

The following is typical of the assessment process related to a Student Unit Attempt. The process will vary according to institution practises and the order in which data is created (e.g. whether or not Assessment Items exist for a unit at the time a student enrols in it), the types of Assessment Items assigned to a unit (e.g. Assessments and exams involve different processes), and the methods used for some of the steps in the process (e.g. marks may be submitted by electronic upload or keyed directly in Callista).

A student enrols in a Unit Offering option.

Some student Assessment Items of System Type Assessment can have tracking items created for them, in which case:

Assessment Items may be examinable (Examinable check box set against their Assessment Type in ASSF0110), and examinable Assessment Items may have their Scheduled check box set (in ASSF2100). In this case, (Examination Management describes this process fully):

Result sheets, for the recording of recommended Student Unit Attempt Marks and Grades, are either:

Or

Recommended results are either:

Or

Recorded Marks/Grades may be amended using the Amend Student Unit Attempt Outcomes form (ASSF53B0) (used typically after results have been finalised) or can be overwritten in the Mark Entry form (ASSF5330), configuration permitting. Marks/Grades are initially recorded as recommended Marks/Grades. A recommended Mark/Grade is indicated in ASSF53B0 by the Finalised check box not being selected.
Invalid Mark/Grade combinations may be allowed during upload or keying (this is determined by configuration settings in ASSF01K0), then reported when ASSR5350, Invalid Grades Report is run.

After results have been recorded a suite of jobs/reports is run (ASSJ06V0, ASSR06M0) for approval of results by a Board of Examiners (or similar authority). Marks/Grades are 'finalised' after they have been approved by running the process ASSJ53A0 through Job Scheduler.

ASSR06N0 Unit Review Report is run to produce reports for review (for example by a Board of Examiners) of recommended Student Unit Attempt Outcomes. These reports are used to note changes to recommended outcomes. Changed outcomes are recorded in Callista manually using the Amend Student Unit Attempt Outcomes form (ASSF53B0).

Results may be published at this stage (the detail of which grades are for publication is controlled in ASSF5110) The following jobs produce results for publication:

Results may be amended after publication. For example, where results were incomplete or subject to appeal.

Applications for Special Consideration may be received at any time during this process.

 

Assessment Tracking

Assessment cover sheets are generated by running job ASSJ3500 to produce an extract file and then using a cover sheet production program specific to the institution to produce the cover sheets. This is done in conjunction with ASSJ3610 which creates a tracking item for each cover sheet. Refer to the Subsystem Specialist section for further detail.

 

Examination Management

Refer to Managing Examination Details in the Subsystem Specialist section.

 

Grading Schema

Callista allows the recording of Grading Schemas and the grades associated with each schema. The table below represents a typical Grading Schema.

Grade

HD

D

C

P

F

LF

NR

Translated Grade

P

P

P

P

F

F

-

Full Name

High Distinction

Distinction

Credit

Pass

Fail

Low Fail

Result not available

Default Mark Range

85-100

75-84

65-74

50-64

20-49

0-19

N/A

Result

Pass

Pass

Pass

Pass

Fail

Fail

N/A

Notional Max %

5

10

15

70

N/A

N/A

N/A

Notional Min %

1

2

5

5

N/A

N/A

N/A

Show on official notification

Yes

Yes

Yes

Yes

Yes

Yes

No

Show on noticeboard

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Show in newspaper

Yes

Yes

Yes

Yes

Yes

Yes

No

Show internally

Yes

Yes

Yes

Yes

Yes

Yes

Yes

System only

No

No

No

No

No

No

Yes

Default where outstanding

No

No

No

No

No

No

Yes

 

The Subsystem Glossary contains a description of each element of a Grading Schema.

A Grading Schema is attached to a Unit Offering option (UOO) to define the set of grades which apply to students studying that UOO or to a Course Offering Pattern (COP) for students enrolled in the course. In later releases of Callista, it will also be possible to attach Grading Schema to individual Student Unit Attempts. An order of precedence will apply, with a Grading Schema attached at Student Unit Attempt level overriding all others that might have applied to the student. Where there is no applicable Student Unit Attempt Grading Schema (the 'normal' situation), a Course Offering Pattern Grading Schema (where one exists) will override a Unit Offering Option Grading Schema, unless the Unit Grading Schema Precedence check box has been set for the Unit Offering option in CRSF2500.

Different Grading Schemas may apply to different Cohorts of students. For instance, students from a particular commercial client may require a different Grading Schema to typical undergraduate students. To achieve this, the commercial client's students could be enrolled in their own set of Unit Offering options with their Grading Schema attached or the Grading Schema for their course may take effect where separate Unit Offering Options have not been created for the students.

Where a Course Offering Pattern Grading Schema is to be applied to students in a Unit Offering Option (i.e. when the Unit Offering option Grading Schema precedence check box is not set), results are recorded according to the Unit Grading Schema. Grades are then 'translated' into the Course Grading Schema grades using the Translate Student Unit Attempt Outcomes process (ASSJ5390).

The maintenance of Grading Schemas is described in the Subsystem Specialist section.



Special Consideration

Some students may believe that circumstances beyond their control have had an effect on either their ability to complete an Assessment Item, or on the result they gained for an Assessment Item. These students may apply for special consideration in assessment of their results. Current Callista functionality provides the facility for:

The Register Special Consideration Application Details form (ASSF4911) is used to record applications for special consideration. The same form is used to record the outcome of a special consideration application.

 

Student Assessment

Students are enrolled in particular Unit Offering Options. Each student therefore inherits the Assessment Items which are attached to the unit in which they are enrolled. Assessment Items may be restricted to specific locations, modes or classes, affecting the Assessment Item's which students inherit. The Assessment Items a student inherits then become the means by which the student's progress in a unit is assessed.

When a unit has been assessed, the outcome is recorded against the Student's Unit Attempt. Outcomes can be recorded by direct keying, electronic upload of results (described below) and insertion by the System (where the System inserts a grade as a result of discontinuation of a unit, because a result is blank, for non-assessable units and for non-assessable unit attempts-as indicated in ENRF3000). That outcome may be expressed in terms of a mark (eg. 82%) and a grade (eg. Distinction), or a grade only (determined by the configuration settings for result entry, ASSF01K0). The grade used must exist within the Grading Schema used to record the student unit outcome. Callista can derive a grade from a mark if configured to do so.

The Assessment Processes section provides further context information. Various parts of the Subsystem Specialist section provide further detail about Assessment Items, Grading Schema and student assessment.

 

Reports, Jobs and Inquiries

Various reports and 'jobs' form an integral part of the Assessments Subsystem. Many of the processes described for this Subsystem make reference to a report or job being part of the process. All Assessments reports and jobs are listed in the Subsystem Specialist section. Further information about each is obtained by following the links there.

 

Electronic Upload of Results

This process allows responsible staff to directly input the outcomes of Student Unit Attempts to Callista. To use this process the user must be familiar with a 'helper' spreadsheet or database application, such as Microsoft Excel, Access or similar. In particular, knowledge of importing and exporting files to and from the helper application is required.

Note: When uploading a Results file for multiple units in Staff Connect, ensure that you save the Class List in Excel as a comma delimited text file (.txt).

The Electronic Upload Process

This process typically commences with the download from Callista of a file which lists students for whom results are to be recorded. Typical processes are described below for two scenarios. They assume that Microsoft Excel is being used as the helper application but are equally applicable for other helper applications.

Scenario 1

An academic who electronically records student assessment details throughout the teaching period, using the downloaded file as a master list of students studying the unit.

Use the Class List Inquiry screen (INQF12E0) to retrieve the required class list. Download the Class List to a 'helper' application on your PC. The Help for INQF12E0 describes how this is typically done.

Assuming that the helper application is Microsoft Excel:

Scenario 2

An academic who only records final results electronically.

Assuming that the helper application is Microsoft Excel:

 

Upload File Formats

The results upload process requires that the upload file have a particular format. There are three options for the file format.

Option 1 - Old Upload File Format

Note that this file format, while still usable, is obsolete and requires more user manipulation.

When using this format, the column heading row must be deleted from the spreadsheet prior to uploading.

student ID, Surname, Given Names, Mark, Grade, Unit Code, Location, Attendance Mode, Class

For example:

96882635,BLACK,JOANNE ELIZABETH,95,HD,MLC101,G,ON,D
96534266,DAVIES,ALEXANDER,63,C,MLC101,B,ON,D
97118902,EVANS,JANET BRIDGET,78,D,MLC101,B,OFF,X
97192345,FRANCIS,CHRISTINE MARY,,WN,MLC101,G,ON,D
97204578,GRANT,SEBASTIAN,55,P,MLC101,W,ON,D

Option 2 - New Upload File Format

This format allows a results file to be uploaded in exactly the same format as it was originally downloaded from Callista (using the class list download function).

When using this format, the column heading row must be retainedand the first column heading must contain a non-numeric character (alpha recommended).

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class,Student Attempt Status,Discontin Date,Exam Locn,Course Code,Course Locn,Coures Att Mode,Course Att Type,Course Status,Correspondence Cat,Addr 1,Addr 2,Addr 3,Addr 4,Addr 5,Addr 6,Addr 7,Addr 8,Addr 9,Addr 10,Addr 11,Email

For example:

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class,Student Attempt Status,Discontin Date,Exam Locn,Course Code,Course Locn,Coures Att Mode,Course Att Type,Course Status,Correspondence Cat,Addr 1,Addr 2,Addr 3,Addr 4,Addr 5,Addr 6,Addr 7,Addr 8,Addr 9,Addr 10,Addr 11,Email


98752279,SCOTT,MR,JOE,,WE,1 2000,MAA101,G,ON,D,DISCONTIN,22/05/2000,GEEV,A721,B,N,PT,ENROLLED,,20 GHERINGHAP ST,GEELONG   ,VIC   3217, , , , , ,5227 8812,5227 8866, ,scottie@ozaddress.com.au, 98704008,SMITH,MS,KATHERINE ANNE,,,12000,MAA101,G,ON,D,UNCONFIRM,,GEEV,H900,B,F,FT,ENROLLED,,1 HIGH ST,BELMONT,VIC   3216, , , , ,03 95551234,03 95551235,03 9555 1236, ,kath@work.com.au

Option 3 - New Upload, Minimum Required Columns, File Format

This format allows a results file to be uploaded based on the format in which it was originally downloaded from Callista (using the Class List download function), but with unnecessary (for the upload process) columns removed.

When using this format, the column heading row must be retained and the first column heading must contain a non-numeric character (alpha recommended).

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class

For example:

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class
98724999,ABLEY,MISS,DEBBIE,,,1 2000,MAA101,B,ON,D
98730771,ALEXANDER,MR,CRAIG,,,1 2000,MAA101,G,ON,D

 

The Upload Process

The academic enters details of the file to be uploaded in the Validate Outcome Upload File form (ASSF5340) and selects the Validate File button to run the file validation process. This process is controlled by system administrator maintained settings (ASSF01K0).

Validated records are uploaded into a batch file within Callista. This process produces an exceptions report, 'Upload Student Unit Attempt Outcomes Exception' report, which lists any warnings and details of records that were unable to be uploaded to Callista. The Academic reviews the exceptions and resolves them. The validation process can be run several times, for the same file, as exceptions are resolved by the Academic.

A central area then deals with the uploaded files. This is fully described in the Subsystem Specialist section.

 

Results Upload from a Learning Management System

Student results are able to be uploaded to Callista SMS from an institution's Learning Management System (LMS) via a web service ('resultLoad). The records are transferred in XML format to Callista and uploaded into the same holding area as that used for Staff Connect and SMS forms results upload. A log of all validation messages is kept in Callista. If required, the GENJ1100 can be run to create an extract of the validation messages.

A Callista SMS administrator then runs the ASSJ5341 job to process the uploaded records and create Student Unit Attempt Outcomes. (Note: ASSF5340 is not involved in the process.)

If required, the ASSR06J0 report can be run to identify non-enrolled SUAOs. The 'Add Non-Enrolled Student Outcome' form (ASSF5333) may also be used, if required.

For more information, also see the following Help pages: SMS-LMS Interface.

 


 

Page last modified on 29 May, 2012 3:17 PM

History Information:

Release Version Project Change to Document
14.0.0.2 1738 - SMS-LMS Interface (Grades) Added a new section about upload of results from a LMS
13.1 1323 - Online Help Consolidation Removed VET link from intro.
12.1 Calipso 28849 Added note re: uploading Results files as comma delimited text files.