Understanding Assessments

This section of the user manual provides an overview of the Assessments subsystem . Topics covered include:

·         What the Assessments subsystem does

·         Terms used in the Assessments subsystem

·         How the Assessments subsystem is used:

• user responsibilities
• assessments reference data and setup
• all about assessment items
• assessment items
• examinable/non-examinable assessment items
• detail recorded against assessment items
• attaching assessment items to units
• all about assessment patterns
• assessment patterns
• attaching assessment items to assessment patterns
• attaching assessment patterns to student unit attempts
• typical assessment processes
• assignment tracking
• examination management
• grading schema
• special consideration
• student assessment
• reports, jobs and inquiries

Detailed explanations and instructions for the use of individual database forms can be accessed from the subsystem's table of contents or via the numerous links provided here.

Information regarding subsystem reference data and setup is provided separately for the subsystem specialist.

What the Assessments subsystem does

The Assessments subsystem is used to manage the academic assessment of students enrolled in units of study. Key functions in this management process are:

• The recording of complete assessment pattern and grading schema details, including:
  • the recording of assessment items
  • maintenance of assessment patterns (future development)
  • the assignment of assessment details to students.
• The management of all examination functions (in conjunction with an integrated, external timetable generator), including the organisation of:
  • timetables
  • examination materials
  • examination supervision
  • exam locations & venues
  • exam script tracking (future functionality).
• The management of critical assessment related calendars and dates.
• The recording and publication of results.
• The generation of assessment related correspondence.
• The management of assignments:
  • recording and tracking of assignments
  • assignment due date extensions
  • assignment outcomes.
• The management of applications for special consideration and their outcomes.

Terms used in the Assessments subsystem

To understand the Assessments subsystem it is necessary to understand the following key terms. Explanation of all Assessments terminology is provided in the subsystem glossary.

How the Assessments subsystem is used:

User responsibilities

Each institution will determine how the Assessments subsystem is applied to its own business practises. As a guide, 'non-specialist functions' are limited to assignment tracking, recording special consideration applications and outcomes and recording and amending student unit attempt outcomes. Recording of outcomes can be achieved by direct keying , bulk electronic upload of results, and insertion by the System (where the System inserts a grade as a result of discontinuation of a unit, where the System inserts an administrative grade if no other grade is recorded and where a grade is inserted for non-assessable units and non-assessed student units).

The remaining functionality is normally used by subsystem specialists or system administrators. This functionality includes:

• Creation and maintenance of reference data
• Creation of unit assessment items
• Creation and maintenance of grading schema
• Management of examination data and time-tabling processes.

These functions are detailed in the subsystem specialist section.

Assessments Reference Data and Setup

For the Assessments subsystem to function effectively, it is necessary to setup specific reference data and define certain data values. This setup of the subsystem is detailed in the subsystem specialist section. The setup of reference data will not concern the 'general' user who will see most reference data as a list of values (LOV) within a form.

All About Assessment Items

Assessment items

An assessment item describes one of the elements by which a unit of study is assessed. For example, a student may be assessed in a unit by submitting two assignments, sitting a written examination and participating in a practical examination. Each of these requirements is an assessment item.

The specific requirements for another unit will normally be different and hence that unit will have a different set of assessment items.

As an example, both units may have a theory examination. 'Theory examination' might be the institution defined assessment type of these assessment items. A particular assessment item is uniquely identified by a System assigned identification number, with an assessment type and description providing its definition.

Assessment items are recorded and maintained via the Maintain Assessment Items form (ASSF2100). It is only necessary to record those assessment items which are used by specific System functionality. For example, examinations which are to be scheduled and assignments which are to be tracked must be recorded.

It is not absolutely necessary to record examinations given by lecturers outside of scheduled examination periods or assignments which are not being tracked. However, it is important to note that it is not possible to process a special consideration application for a unit assessment item unless the item has been recorded in the System and assigned to the student's unit attempt. Nor would it be possible to use the System to advise students of all their assessment requirements. It is therefore recommended that the complete set of assessment items and patterns be recorded for each unit.

Examinable/non-examinable assessment items

Assessment items may be examinable (items 57, 72, 146 and 220 in the table above) or non-examinable, as in the case of an essay, an assignment, a theatrical/musical performance or a practicum. It is only possible for examinable assessment items to be considered for exam time-tabling. The examinable indicator is set (or unset) for each assessment type and is inherited by the assessment items to which the assessment type is assigned.

Examinable assessment items may be scheduled (these items are passed to the timetabling module) or not scheduled (these items occur outside the defined examination period and are not passed to the timetabling module). The Scheduled indicator is set (or unset) for each examinable assessment item to indicate whether or not it is to be timetabled.

Detail recorded against assessment items

The table below lists additional detail which can be recorded about each examinable assessment item using navigation buttons in the Maintain Assessment Items form (ASSF2100). The only additional information able to be recorded against non-examinable assessment items is Assessors.

Attaching assessment items to units

As mentioned previously, an assessment item describes one of the requirements by which a unit of study is assessed. How this relationship between a unit and its assessment items is recorded and maintained is described in the subsystem specialist section. When a student enrols in a particular unit of study and the process ASSJ3212 - 'Automatically Maintain Student Unit Attempt assessment Items' is run, the student 'inherits' the default assessment items associated with the unit. When a default assessment item is attached to a unit after students have enrolled in it, the process ASSJ3213 assigns the assessment item to student units.

All About Assessment Patterns

Assessment Patterns

An assessment pattern is a group of assessment items for a unit offering. Creating assessment patterns allows a number of assessment items to be attached to a student unit attempt as a group. Assessment patterns are typically used when students are given a choice of assessment methods. For example: students may be given a choice of two patterns; one comprised of an examination and three assignments, and the other comprised of an examination and a major project. If no choice of assessment exists, the discrete set of assessment items for the unit offering may still be set up as an assessment pattern. The advantage of using a pattern is the ability to record the proportion of the total assessment applicable to each item.

Examples of assessment patterns for unit AAC131

Assessment patterns are recorded and maintained via the Maintain Unit Assessment Patterns form (ASSF2320). Unlike assessment items, assessment patterns cannot be created as stand-alone. Creation of an assessment pattern automatically attaches it to the unit offering pattern.

Attaching assessment items to assessment patterns

As mentioned previously, assessment patterns are groups of assessment items. How the relationship between an assessment pattern and an assessment item is recorded and maintained is described in the subsystem specialist section.

Attaching assessment patterns to student unit attempts

When a student enrols in a particular unit of study and the process Automatically Maintain Student Unit Attempt Assessment Items (ASSJ3212) is run, the student 'inherits' the default assessment pattern, and its assessment items, associated with the unit. When a default assessment pattern is attached to a unit. When a default assessment pattern is attached to a unit after students have enrolled in it, the process ASSJ3213 assigns the assessment pattern and its items to the student unit attempts.

Assessment Processes

The basic requirements for assessment processing are:

• The assessment items for each unit being assessed have been created and their details recorded in Callista (as detailed in the subsystem specialist section)
• The required grading schema have been created and attached to the relevant unit offerings (and/or course offerings and/or student unit attempts - in future functionality) (as detailed in the subsystem specialist section)
• Students being assessed are enrolled (in Callista) in assessable units

  The following is typical of the assessment process related to a student unit attempt. The process will vary according to institution practises and the order in which data is created (e.g. whether or not assessment items exist for a unit at the time a student enrols in it), the types of assessment items assigned to a unit (e.g. assignments and exams involve different processes) and the methods used for some of the steps in the process (e.g. marks may be submitted by electronic upload or keyed directly in Callista).

1. A student enrols in a unit offering option.
   The assessment items associated with the unit offering are automatically assigned to the student, by the System. Attaching Assessment Items to Students describes this process.
2. Some student assessment items of system type ASSIGNMENT can have tracking items created for them, in which case:

• ASSJ3610 is run through Job Scheduler to generate tracking items for those assignments specified in the job parameter form. This job is run in conjunction with job ASSJ3500 which generates cover sheets for the assignment.
• These assignments are tracked through their various stages (such as 'received from student', 'sent to assessor', 'received from assessor', 'returned to student'). Assignment tracking is discussed in detail below.
• Outcomes for the assignment may be recorded in Callista as described under Student Assessment.

3. Assessment items may be examinable (Examinable indicator set against their assessment type in ASSF0110), and examinable assessment items may have their Scheduled indicator set (in ASSF2100). In this case, (Examination Management describes this process fully) :

• Each student's assessment items, which are both examinable and have their Scheduled indicators set, plus examination location and venue details, exam calendars and sessions and exam session availability details, are passed to the examination scheduling software (Syllabus Plus) via the Electronic Timetabling Interface (ETI). (refer to documentation accompanying ETI for further information)
• Syllabus Plus performs the examination scheduling (refer to documentation accompanying that product)
• Schedule data is passed back to Callista via ETI
• Examination timetables are produced
• Supervision of exams is organised and may optionally be recorded in Callista using form ASSF4630.
• Materials for the conduct of exams are supplied to students based on details optionally recorded in Callista using form ASSF2100.
• Optionally, individual notifications of exam details to students are produced from Callista, by report ASSR4622.
• Students sit examinations, attendance is recorded on lists produced by job ASSR06F0.

4. Result sheets, for the recording of recommended student unit attempt marks and grades, are either:

• Generated from Callista by running the report ASSR5310, 'Result Sheets' , and distributed to the responsible person (such as a unit co-ordinator/chair) or
• Downloaded as class lists from the inquiry form INQF12E0 (Refer to Electronic Upload of Results).

5. Recommended results are either:

• Recorded by the responsible person in a spreadsheet or database file and uploaded to Callista via the Validate Electronic Outcome Upload File process in ASSF5340 . (Recording Results describes the electronic upload process fully.) or
• Recorded on result sheets and sent to a central processing point for direct keying into Callista. Direct keying occurs in the Mark/Grade Entry form (ASSF5330). Marks/grades for students not listed on mark sheets may be recorded manually at the bottom of mark sheets and then recorded in Callista using the Add Non-Enrolled Student Outcome form (ASSF5333) or, where they have been enrolled after production of the mark sheet, they are added in the result entry form. Recording Results provides details of how non-enrolled students with a mark/grade recorded are handled by both the electronic upload and direct keying processes.

6. Recorded marks/grades may be amended using the Amend Student Unit Attempt Outcomes form (ASSF53B0) (used typically after results have been finalised) or can be overwritten in the mark entry form (ASSF5330), configuration permitting. Marks/grades are initially recorded as recommended marks/grades. A recommended mark/grade is indicated in ASSF53B0 by the Finalised indicator being unset.
   Invalid mark/grade combinations may be allowed during upload or keying (this is determined by configuration settings in ASSF01K0), then reported when ASSR5350, Invalid Grades Report is run.
7. After results have been recorded a suite of jobs / reports is run (ASSJ06V0, ASSR06M0) for approval of results by a Board of Examiners (or similar authority). Marks/grades are 'finalised' after they have been approved by running the process ASSJ53A0 through Job Scheduler.
8. ASSR06N0 Unit Review Report is run to produce reports for review (for example by a Board of Examiners) of recommended student unit attempt outcomes. These reports are used to note changes to recommended outcomes. Changed outcomes are recorded in Callista manually using the Amend Student Unit Attempt Outcomes form (ASSF53B0).
9. Results may be published at this stage (the detail of which grades are for publication is controlled in ASSF5110) The following jobs produce results for publication:

• ASSR06Q0 Unit Attempt Outcome Noticeboard Report
• ASSJ5410 Results File for Publication

10. Results may be amended after publication. E.g. where results were incomplete or subject to appeal.
11. Applications for special consideration may be received at any time during this process.

Assignment tracking

Assignment cover sheets are generated by running job ASSJ3500 to produce an extract file and then using a cover sheet production program specific to the institution to produce the cover sheets. This is done in conjunction with ASSJ3610 which creates a tracking item for each cover sheet. Refer to the Subsystem Specialist section for further detail.

Automated assignment tracking functionality is not part of release 1.3.x. Manual tracking can be used.

Examination management

Refer to Managing Examination Details in the Subsystem Specialist section.

Grading schema

Callista enables the recording of grading schemas and the grades associated with each schema. The table below represents a typical grading schema.

The Subsystem Glossary contains a description of each element of a grading schema.

A grading schema is attached to a unit offering option (UOO) to define the set of grades which apply to students studying that UOO or to a course offering pattern (COP) for students enrolled in the course. In later releases of Callista, it will also be possible to attach grading schema to individual student unit attempts. An order of precedence will apply, with a grading schema attached at student unit attempt level overriding all others that might have applied to the student. Where there is no applicable student unit attempt grading schema (the 'normal' situation), a course offering pattern grading schema (where one exists) will override a unit offering option grading schema unless the unit grading schema precedence indicator has been set for the unit offering option in CRSF2500.

Different grading schemas may apply to different cohorts of students. For instance, students from a particular commercial client may require a different grading schema to typical undergraduate students. To achieve this, the commercial client's students could be enrolled in their own set of unit offering options with their grading schema attached or the grading schema for their course may take effect where separate unit offering options have not been created for the students.

Where a course offering pattern grading schema is to be applied to students in a unit offering option (i.e. when the unit offering option grading schema precedence indicator is not set), results are recorded according to the unit grading schema. Grades are then 'translated' into the course grading schema grades using the Translate Student Unit Attempt Outcomes process (ASSJ5390).

The maintenance of grading schemas is described in the subsystem specialist section.

Some students may believe that circumstances beyond their control have had an effect on either their ability to complete an assessment item, or on the result they gained for an assessment item. These students may apply for special consideration in assessment of their results.

Current Callista functionality provides the facility for:

• Recording details of special consideration applications for unit assessment items

• Recording the outcome of finalised applications.

The Register Special Consideration Application Details form (ASSF4911) is used to record applications for special consideration. The same form is used to record the outcome of a special consideration application.

Student Assessment

Students are enrolled in particular unit offering options. Each student therefore inherits the assessment items which are attached to the unit in which they are enrolled. Assessment items may be restricted to specific locations, modes or classes, affecting the assessment item's which students inherit. The assessment items a student inherits then become the means by which the student's progress in a unit is assessed.

When a unit has been assessed, the outcome is recorded against the student's unit attempt. Outcomes can be recorded by direct keying, electronic upload of results (described below) and insertion by the System (where the System inserts a grade as a result of discontinuation of a unit, because a result is blank, for non-assessable units and for non-assessable unit attempts-as indicated in ENRF3000). That outcome may be expressed in terms of a mark (eg. 82%) and a grade (eg. Distinction) or a grade only (determined by the configuration settings for result entry, ASSF01K0). The grade used must exist within the grading schema used to record the student unit outcome. Callista can derive a grade from a mark if configured to do so.

The Assessment Processes section provides further context information. Various parts of the subsystem specialist section provide further detail about assessment items, grading schema and student assessment.

Electronic Upload of Results

This process allows responsible staff to directly input the outcomes of student unit attempts to Callista. To use this process the user must be familiar with a 'helper' spreadsheet or database application, such as Microsoft Excel, Access or similar. In particular, knowledge of importing and exporting files to and from the helper application is required.

The Electronic Upload Process
This process typically commences with the download from Callista of a file which lists students for whom results are to be recorded. Typical processes are described below for two scenarios. They assume that Microsoft Excel is being used as the helper application but are equally applicable for other helper applications..

Scenario 1

An academic who electronically records student assessment details throughout the teaching period, using the downloaded file as a master list of students studying the unit.

Use the Class List Inquiry Screen (INQF12E0) to retrieve the required class list. Download the class list to a 'helper' application on your PC. The Help for INQF12E0 describes how this is typically done.

Assuming that the helper application is Microsoft Excel:

·         Open the newly created file from within Excel. (This assumes some knowledge of importing a file into a spreadsheet.)

·         Delete from the spreadsheet any columns not required (e.g. the user may not be interested in exam location). However, retain all columns required for the upload process (see Upload File Format, below).

·         Add any other columns required - optional (e.g. a column could be added for each assessment item, to record the results of assignments, tests, exams, practical tests etc.)

·         Use the spreadsheet throughout the teaching period to record results for each item of assessment , deriving the final result at the end of the period. Can also be used to record other information about the student, e.g. notes.

o        The user may again download the student list at other times during the period (e.g. just after census date) to check for discontinued/added students. These must be manually amended in the original spreadsheet.

o        Formulae may be used in the spreadsheet to derive final results.

o        Students who present for assessment but who are not recorded in the downloaded files can be added to the spreadsheet for later resolution.

·         After final marks and/or grades are recorded in the spreadsheet, prepare a copy of the file for uploading to Callista using the following steps:

o        For upload file format option 1, delete all column headings.

o        For upload file format options 2 & 3, ensure that the first line of the spreadsheet is reserved for column headings (ie. contains no data other than the headings)

o        delete those columns not required for the upload process (see Upload File Formats, below).

o        check that the file contains the required columns in the correct order as specified in the relevant upload file format.

o        re-save a copy of the file as a comma delimited text file.

·         Perform the upload process.

Scenario 2

An academic who only records final results electronically.

·         Use the Class List Inquiry Screen (INQF12E0) to retrieve the required class list. Download the class list to a 'helper' application on your PC. The Help for INQF12E0 describes how this is typically done.

Assuming that the helper application is Microsoft Excel:

·         Open the newly created file from within Excel.

·        Do not alter the row and column structure of the file.

·         Record the final results for the student unit attempts in the spreadsheet.

·         Re-save a copy of the file as a comma delimited text file.

·         Perform the upload process.

Upload File Formats

The results upload process requires that the upload file have a particular format. There are three options for the file format.

<Option 1 - Old upload file format

Note that this file format, while still usable, is obsolete and requires more user manipulation.

When using this format, the column heading<row must be deleted from the spreadsheet prior to uploading.

student ID, surname, given names, mark, grade, unit code, location, attendance mode, class

For example:

96882635,BLACK,JOANNE ELIZABETH,95,HD,MLC101,G,ON,D
96534266,DAVIES,ALEXANDER,63,C,MLC101,B,ON,D
97118902,EVANS,JANET BRIDGET,78,D,MLC101,B,OFF,X
97192345,FRANCIS,CHRISTINE MARY,,WN,MLC101,G,ON,D
97204578,GRANT,SEBASTIAN,55,P,MLC101,W,ON,D

Option 2 - New (release 3.1.0) upload file format

This format allows a results file to be uploaded in exactly the same format as it was originally downloaded from Callista (using the class list download function).

When using this format, the column heading row must be retainedand the first column heading must contain a non-numeric character (alpha recommended).

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class,Student Attempt Status,Discontin Date,Exam Locn,Course Code,Course Locn,Coures Att Mode,Course Att Type,Course Status,Correspondence Cat,Addr 1,Addr 2,Addr 3,Addr 4,Addr 5,Addr 6,Addr 7,Addr 8,Addr 9,Addr 10,Addr 11,Email

For example:

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class,Student Attempt Status,Discontin Date,Exam Locn,Course Code,Course Locn,Coures Att Mode,Course Att Type,Course Status,Correspondence Cat,Addr 1,Addr 2,Addr 3,Addr 4,Addr 5,Addr 6,Addr 7,Addr 8,Addr 9,Addr 10,Addr 11,Email
98752279,SCOTT,MR,JOE,,WE,1 2000,MAA101,G,ON,D,DISCONTIN,22/05/2000,GEEV,A721,B,N,PT,ENROLLED,,20 GHERINGHAP ST,GEELONG   ,VIC   3217,                                        ,                                        ,                                        ,                                        ,                                        ,5227 8812,5227 8866,                                        ,scottie@ozaddress.com.au
98704008,SMITH,MS,KATHERINE ANNE,,,1 2000,MAA101,G,ON,D,UNCONFIRM,,GEEV,H900,B,F,FT,ENROLLED,,1 HIGH ST,BELMONT,VIC   3216,                                        ,                                        ,                                        ,                                        ,03 95551234,03 95551235,03 9555 1236,                                        ,kath@work.com.au

Option 3 - New (release 3.1.0) upload, minimum required columns, file format

This format allows a results file to be uploaded based on theformat in which itwas originally downloaded from Callista (using the class list download function), but with unnecessary (for the upload process) columns removed.

When using this format, the column heading row must be retained and the first column heading must contain a non-numeric character (alpha recommended).

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class

For example:

Person ID,Surname,Title,Given Names,Mark,Grade,Teach Period,Unit Code,Unit Location,Unit Mode,Class
98724999,ABLEY,MISS,DEBBIE,,,1 2000,MAA101,B,ON,D
98730771,ALEXANDER,MR,CRAIG,,,1 2000,MAA101,G,ON,D

The Upload Process

The academic enters details of the file to be uploaded in the Validate Outcome Upload File form (ASSF5340) and selects the Validate File button to run the file validation process. This process is controlled by system administrator maintained settings. Validated records are uploaded into a batch file within Callista. This process produces an exceptions report, ASSR5340 - Upload Student Unit Attempt Outcomes Exception report, which lists any warnings and details of records which were unable to be uploaded to Callista. The academic reviews the exceptions and resolves them. The validation process can be run several times, for the same file, as exceptions are resolved by the academic.

A central area then deals with the uploaded files. This is fully described in the Subsystem Specialist section.

Reports, Jobs and Inquiries

Various reports and 'jobs' form an integral part of the Assessments subsystem. Many of the processes described for this subsystem make reference to a report or job being part of the process. All Assessments reports and jobs are listed in the subsystem specialist section. Further information about each is obtained by following the links there.

Last Modified on 8 March 2002