An examination and critique of the history and method of Tertiary Entrance Ranking in the Australian Capital Territory

Abstract

Since the inception of a separate tertiary ranking scheme for the Australian Capital Territory (ACT) in 1976,several government reports have acknowledged the difficulties of comparing student performance between senior secondary schools in a system that does not have a common, mandated senior secondary school curriculum. For over three decades, these difficulties appear to have been tolerated in deference to a prevailing sentiment of granting schools autonomy in developing and delivering curriculum and assessment at the senior secondary level (years 11 and 12 of schooling). Over that period, there have been several studies and reports investigating bias as a result of the Australian Scholastic Aptitude Test (ASAT),1 being used as the primary scaling instrument for course 2 scores both between courses and between colleges. 3 The reports have resulted in several changes. Principally, the original ASAT has been expanded to measure a broader base of skills and the linear transformation to ASAT has been replaced by the Other Course Score (OCS) method of scaling. It is generally believed that the OCS scaling method addresses previous concerns by introducing a third set of scores (the v scores) against which students‟ course scores are thought to be scaled. However, the role of the v scores is relatively minor and course scores continue to be largely scaled to the AST scores. The OCS method was never designed to address, nor has it addressed, many of the biases that have been of concern.

This study explores the evolution of the ACT tertiary ranking system and describes how the current OCS scaling system for ranking students for tertiary entrance operates. Through simulation it investigates the role of the correlation between AST and course/scaling group scores in providing a fair basis for comparing student performance between colleges. Results for cohorts of students are randomly generated with statistical characteristics based on the annual report figures from the ACT Board of Senior Secondary Studies (BSSS). The aggregates are calculated and then recalculated after removing high AST students from the cohorts and then low AST students. After the removal of students, the difference in the students‟ aggregate is calculated. This process is repeated for three types of cohorts – those with authentic correlations between AST and course scores, those with higher correlations between AST and course score and those with lower correlations. The results indicate that the cohort with whom a student is scaled has a significant impact on the student's final Australian Tertiary Admission Rank (ATAR) and that the impact is lessened with improvements to the AST/course score correlations.

As many senior secondary schools in the ACT now teach a common – albeit not mandated – curriculum in Years 11 and 12 and a national Australian curriculum is likely to be introduced in the near future, this study suggests it would be timely for the ACT government to review its method of ranking students for tertiary entrance.

1Now called the ACT Scaling Test – AST.

2 Throughout this study, courses and scaling groups are used interchangeably in the context of the ACT system. Scaling groups may have students from more than one course placed on a common scale by the college when the student numbers on one or more of the courses are too low to be scaled alone. In the model, each course is its own scaling group.

3"Colleges‟ refer to year 11 and 12 as they are separate educational institutions in the ACT government system. Although non-government schools do not have their senior years in separate institutions, they are included in the term „colleges‟.