Symposium on Professional Standards
This symposium presentation draws on a paper to be published in the July issue of Unicorn and a paper presented to the Australian Association for Research in Education.
Assessment and selection of the Level 3 Classroom Teachers in Western Australia: a case to inform the development of professional standards.
Dr Anne M. Jasman
Australian Institute of Education
Murdoch University
South Street, Murdoch WA 6150
Dr Jasman's current (March 2001) email contact at Melbourne University is: ajasman@unimelb.edu.au
Abstract:
Educational and school reform agendas are now focussing on the improvement of teaching for quality learning. Thus a well-articulated description of what constitutes quality teaching is seen as critical in bringing about improvement to student learning outcomes. It was from this perspective that the Education Department of Western Australia with the support of the State School Teachers’ Union of Western Australia implemented a trial project in 1997 to establish a career path for teachers. This career path was to include standards to be set for transition at three stages: entry to Level 1, transition from Level 1 to Level 2 and from Level 2 to Level 3 as part of the Enterprise Bargaining Agreement.
The competency framework, assessment processes and two sets of standards developed within the Teacher Career Path project are briefly described. The first standards provide the basis for selecting teachers to Level 3 status. The second involves the development of comprehensive professional competency-based standards to support teacher development.
Assessment and selection of the Level 3 Classroom Teachers in Western Australia: a case to inform the development of professional standards.
Dr Anne M. Jasman
Australian Institute of Education
Murdoch University
South Street, Murdoch WA 6150
Dr Jasman's current (March 2001) email contact at Melbourne University is: ajasman@unimelb.edu.au
Introduction:
The development of professional standards and their potential use in the selection of teachers for career progression described here was part of the Teacher Career Path project. The project was a trial process to develop and implement a career progression model for classroom teachers. There were three levels envisaged in the career path: beginning teacher (Level 1), competent teacher (Level 2) and highly accomplished teacher (Level 3). This career path was a major outcome of the 1996 Enterprise Bargaining Agreement negotiated between the State School Teachers Union of Western Australia (SSTUWA) and the Education Department of Western Australia (EDWA).
The purposes described for the teachers’ career path were to
The Teacher Career Path project was contracted out through competitive tender, funded by the Education Department of Western Australia and overseen by a Reference Group comprising representatives of the employer and the union. The tender brief framed the processes for selection of Level 3 Teachers, as it specified the development of a competency framework and the use of portfolios and performance-based assessment exercises to assess these competencies. . The competency framework that was developed comprised five competencies as follows.
Level 3 Classroom teachers:
Each competency was further elaborated with indicators of attainment and potential sources of evidence.
A two stage assessment process was required. We used a teaching portfolio in the first stage and an assessment centre type of exercise called the reflective review at the second stage .
Stage 1 – The Teaching Portfolio
The teaching portfolio (Martin, 1997a) comprised three main parts. An ‘Introductory Statement’ provided an opportunity for teachers to describe the context in which they worked, their rationale and beliefs about teaching. Teachers then demonstrated attainment of the five competencies through a ‘Written Statement’ (maximum two A4 pages for each competency) and presented ‘Evidence’ to support their claims (limited to fifteen A4 pages). 721 applicants submitted teaching portfolios for assessment in the first stage of the selection process.
Assessors were drawn from a pool of experienced classroom teachers who received training in assessment and engaged in moderation exercises to improve the reliability and validity of the selection processes. The role of these assessors was crucial to generating acceptable standards. They were selected from applicants responding to an expression of interest advertisement. Assessors had to have taught for five of the last ten years and were considered by their referees to have demonstrated high quality teaching abilities. They also represented the profile of the teaching population within the state by gender, location and sector - education support, primary and secondary schools. These teachers, as assessors, judged the quality of the evidence provided by the applicants. This judgement then provided the basis for enabling a standard to be set for selection as a Level 3 Classroom teacher.
Two assessors judged each teaching portfolio. Assessors were allocated into pairs during the first training session. The aim was, as far as possible, to pair assessors to get a combination of educational expertise and a balance of demonstrated marking tendencies. In particular, the pairs were allocated to balance primary and secondary experience, geographic location and gender.
Each assessor was given sixteen portfolios to independently rate. The written statement and supporting evidence for each competency was reviewed and assessed by focussing on the validity, reliability, recency, sufficiency and authenticity of the evidence provided by the applicant. Ratings for the quality of evidence provided for each competency were given as follows: 1 – little/no evidence, 2 – very poor, 3 – poor, 4 – fair, 5 – good, 6 – very good and 7 - outstanding.
A moderation exercise was conducted on completion of these initial ratings. The portfolios were then swapped and independently rated by the other assessor of the pair. Assessors also checked the claims in the portfolio with at least one of the nominated referees. If necessary the assessors moderated their assessments until there was only one point difference in their individual ratings. Each portfolio then received a combined score of these two ratings with totals ranging from 14–2 for each of the five competencies. These ratings were used to determine the 313 applicants that would proceed to the second selection stage. The setting of this standard is discussed in more detail later.
Stage 2 – The Reflective Review
Whilst the teaching portfolio provided considerable evidence of performance in the workplace the tender specified the use of a performance-based assessment that could provide additional evidence of attainment of the competencies. Ideally, teachers should be observed over an extended period of time, to gain a full appreciation of the quality of their work. As an alternative to this, the reflective review was designed to provide applicants with an opportunity to demonstrate their competencies in a situation where they were interacting with their peers, in a way not too dissimilar to a work situation.
The reflective review was a group assessment where each of the four or five applicants in the group presented an issue, concern or problem they were currently dealing with for a maximum time of five minutes. The applicant then facilitated a discussion for about twenty minutes before concluding with a brief five-minute reflection on the discussion. After a short break another applicant in the group completed the reflective review in the same manner. Two assessors observed the reflective reviews. First, the introduction was judged as to whether it addressed and provided evidence of Competencies 1, 2 or 5 and second the discussion and reflective review were rated on how well competencies 3 and 4 had been demonstrated. Further details of these assessment methods can be found in Jasman and Barrera (1998).
Assessment at each stage was, therefore, based on the demonstrated attainment of all five competencies. The task remained, however, as to how to set the ‘standard’ for selection as a Level 3 Classroom Teacher: that is, what counted as achieving the standard.
In the Teacher Career Path project the ‘standard’ used for selection arose from identifying generic qualities displayed by those teachers whose portfolios achieved a similar rating in the assessors’ judgements of the evidence provided. A second ‘standard’ was later derived from a detailed analysis of the most highly rated portfolios and the development of professional competency-based standard statements for different groups of teachers .
Standards development based on ratings of quality of evidence.
In order to determine the generic qualities displayed by those teachers whose portfolios achieved a similar rating the researcher conducted an analysis of 160 portfolios. These were selected to represent teacher librarians, education support, primary and secondary teachers whose portfolios had been assessed as providing evidence of attainment of each competency rated in the ranges of outstanding (13/14), very good (11/12), good (10), good to fair (9) or fair to poor (8/7).
Analysis of these portfolios focussed on the teachers’ Introductory Statement and Written Statements for each competency. These provided indications of what was common to these teachers. For example, teachers who were given a rating suggesting an outstanding quality of evidence (13/14)
presented the evidence of achievement of each competency in a coherent, well-integrated and holistic way. Examples cited are often innovative and developed by the teacher to meet specific needs. There is clear emphasis on the learner as the centre of the teaching and learning in the classroom. The examples they include show an understanding of the complexities within a particular situation. These teachers also draw on their experience in different contexts and of using different strategies to illustrate how particular professional judgements have been made. These judgements are situational and take account of the factors evident in a particular case. There is a clear rationale and philosophy for teaching and learning, assessment and leadership that is evident in the way the evidence is presented. (Jasman and Barrera, 1998 p.46)
Similar ‘standards’ statements based on the quality of evidence presented in the portfolios for the ratings 11/12 (very good), 10 (good), 9 (good to fair) and 8 (fair) were developed (Jasman, 1998c). In the absence of more specific standard statements these descriptions of generic teacher qualities derived from the portfolio entries provided a basis for selection at Stage 1. They provided an explicit, if generic, description of the qualities of the evidence and enabled discussions within the Reference Group and project team about the rating that should be set as the ‘standard’ for moving to the next stage of selection.
The project team suggested to the Reference Group that a Level 3 Classroom teacher should have received a rating of ‘good’ (10) for quality of evidence on each competency in order to move to the next stage of selection. We made this recommendation because the descriptions of professional practice found in the portfolios rated at this level and above appeared consistent with understandings about the development of professional expertise available in the literature (Jasman, 1998a).
The Reference Group suggested that a rating of 10 was too high as applicants had only a three month time period to collect and present the evidence of their professional work that was assessed. After much discussion the Reference Group finally decided the rating for the quality of evidence that should be considered sufficient to proceed to the second stage of selection should be 9 (good to fair quality of evidence). All applicants who scored 9 or more on each and every competency proceeded to the next round where the same cut-off was applied. 221 teachers were then appointed as Level 3 Classroom teachers from the original 721 applicants.
Professional competency-based standards
The teaching portfolios also provided the basis for developing these more specific professional competency-based standards. The researcher selected the most highly rated portfolios with a view to developing a statement of standards for Level 3 Classroom teachers from the Introductory and Written Statements with illustrative examples of professional practice. The portfolios were, therefore, selected to represent ‘subject’ specialisations, various age or ability specific teaching as well as generalist teaching perspectives across educational support, senior colleges, primary and secondary schools.
Analysis resulted in these highly rated portfolios being grouped on the basis of the similarities found within teachers’ Introductory and Written Statement for Competency 1, relating to innovative and/or exemplary teaching strategies. As a result of detailed content analysis of the portfolio entries (Jasman, 1998b) standard statements were developed for 16 groups of teachers (see Table 2).
Table 2: Categories for professional competency-based standards.
| SECONDARY TEACHERS: | Arts, English, Health and physical education, Languages other than English, Mathematics, Multi-disciplinary and cross-curricular teaching, Science, Studies of society and environment, Support, Technology and enterprise. |
| PRIMARY TEACHERS: | Early childhood/junior, Generalist, Specialist, and Support. |
| SECONDARY & PRIMARY TEACHERS: | Teacher Librarians and Educational support. |
Fewer differences were found in the written statements for Competency 2, relating to exemplary assessment practices. Standard statements were developed for 4 groups: educational support, library, primary and secondary teachers. The analysis of highly rated teaching portfolios in relation to attainment of Competencies 3, 4 and 5 suggested that there was more in common between teachers’ responses than there were differences. Consequently only one standard statement was developed for each competency although a wide range of examples of professional practices were used to illustrate each standard statement.
Conclusion
Within this research the standards developed were, therefore, both specific and generic. Those standard statements that relate to competencies describing classroom practices typically show the most differentiation and were developed for specific categories of teachers. The differences noted here related to the specific ‘knowledge’ which teachers brought to their professional practice. This was particularly evident in the responses to Competencies 1 and to a lesser extent for Competency 2. Such differences were not unexpected given the literature on pedagogical content knowledge and the importance of ‘subject’ knowledge to effective teaching . However, those competencies associated with professional development and the leadership role of the Level 3 classroom teacher were generic and could be applied to all Level 3 Classroom teachers irrespective of age or subject area taught.
The credibility of these standards within the wider professional community is yet to be determined. However, they do provide a starting point for further discussion and development. We should, however, consider carefully the potential that the profession, particularly the Level 3 Classroom teacher, has to further develop the career path and the professional standards. In this way, the teaching profession can demonstrate their accountability through exercising responsibility for ensuring the quality of teaching and the maintenance of appropriate professional standards.
Within Western Australia the trial processes used for the selection of Level 3 Classroom Teachers are currently under review. There are still some significant issues that remain to be resolved. The question of how best to assess teachers around a particular set of competencies needs to be further addressed. The issues of assessment of teachers’ performance and judgements about attainment of the standards are also problematic. Further research is also needed into the impact of the development of such standards nationally and in the local context and the most appropriate form they might take since we are convinced of their value to the maintenance of quality teaching and thus quality student learning.