|Teaching and Learning Forum 2004 [ Proceedings Contents ]|
Edith Cowan University
The aim of the project was to design, pilot and evaluate self-assessment materials, which would minimise the opportunity for the inappropriate use of academic material and foster the development of reflective learners that is, learners who understand and respect the process of knowledge creation. Educational excellence could be cultivated through the development of an understanding for the process of creating knowledge. The first objective of the project was to develop innovative and authentic tasks for individual students. The second was to develop self-assessment materials in order for students to reflect on their learning and output.
From this research, what became evident is that students who already had very good analytical skills evaluated their strengths and weaknesses. Students who had poor analytical skills tended to over-value the strengths in their work, but often recognised the weaknesses in their work. While all of the students participating in the study were third years, it was apparent that some had not written a theoretical or analytical essay since first year. With any future use of self-assessment, students would need a trial run in which learning is reinforced with evaluation and feedback. Self-assessment is likely to work with students at third year level and later levels who have similar analytical, theoretical and critical thinking background. This paper will briefly discuss the findings of the main writers/researchers on student self-assessment, the method employed in this research and the findings.
Other researchers have found that students are often hostile to different methods of assessment (Elton, 1988; Boud, 1990; Peters, 1996), and for this reason students would need to be advised of the benefits. It was decided to market the additional assessment based on developing critical thinking, reflective practice, and independent learning (Moon, 2002; Mowl & Pin, 1995; Sluijsmans, Dochy & Moerkerke, 1998). Doing the assessment was also considered part of the assignment mark. Students were advised that if they completed the self-assessment, they would not get less than the tutor grade. With regard to assignment weightings, some researchers suggested that early attempts at peer assessment should carry a lower weighting, for example, a 20% overall mark (Ford, 1997; Academic Council Office, 1999). The assignment used for this research had a weighting of 40%; from doing this project, it is evident that as a first assignment this is too high a rating. The weighting for the first assignment in the course has now been changed.
This research project supported previous studies (Ellington, Earl and Cowan, 1997; Moon, 2002) that it is very important to have preparatory meetings to explain to students the rules and marking range for self and peer assessment. Added to that, there is a need to involve the students early on in generating and developing the criteria (Mowl and Pin, 1995). Preparatory meetings were organised and students were involved in developing the marking criteria. Some 26 students (out of 27 third years enrolled in the course) attended the first preparatory meeting and about half participated in developing the marking criteria.
Some researchers (Butcher, Stephanie, and Tariq, 1995; Topping, Smith and Swanson, 2000) stated that it is necessary to have a follow-up discussion about the difference in marks between self or peer and tutor, and how that mark will be moderated. Follow-up discussions were offered, however, not always taken-up. The coordinator of the course offered to moderate the marks. Students however were promised that they would not get less than the tutors' grade, as it is important that any variations in marks due to assessors' differences be evaluated and adjusted in the final mark (Butcher, Stephanie, and Tariq, 1995). Keith Topping (1998) suggests there needs to be an awareness of gender and cultural differences. He points to the example of mature women who tend to identify their weaknesses opposed to their strengths. As we only had two mature age women (over 30 years) enrolled in the course, this was not evident in the sample. With regard to cultural difference, only two people participating in the research were from a non-Anglo background.
From this literature review, it was evident that third year students would be the best undergraduate group to participate in the research. They needed to be involved in the development of the assessment and marking criteria as well as being clearly advised of the benefits of using self-assessment.
Third year students were chosen because from reviewing the literature researchers such as Ron Oliver (2002) argued that third years and above have a "greater capacity to look after themselves" and in being able to assess their work. In the first organised meeting, students were consulted about questions were to be included in the self-assessment, and the marking criteria. A self-assessment questionnaire (see appendix one) and marking criteria form was designed. An electronic form of the questionnaire was made available on the school website. In the second meeting, the lecturer discussed with students the grading of a HD, D, Cr, C, N, and appropriate criteria allocated to each grade. A handout on grading was given to students. Overall, 24 essays were handed in with a completed self-assessment questionnaire. The sample was 24 third year bachelor of communication students (24 out of 27 completed the self-assessment) with the majors listed in Table 1.
|Advertising/Public Relations (x2)|
|Mass Communications (x2)|
|Media Studies (x4)|
|Media Studies/minor Drama studies|
|Media Studies/minor English|
|Advertising/Film and Video|
|Advertising/Media Studies (x2)|
|Public Relations/Media Studies (x2)|
|Film and Video|
|Health Promotions/Media Studies|
|Interactive Multimedia/Media Studies|
Students submitted their self-assessment at the same time as their essays. Tutors did not look at the self-assessments until after the essays had been marked. The assignment was worth 40% and the topic was a detailed analysis of the ways in which gender roles are constructed in one Australian media culture (such as television, film, magazines, video clips etc).
|Students whose grade matched Tutors Grade||4|
|Students who came within 5% Tutors Grade||9|
|Students who came within 10% Tutors Grade||1|
|Students where there was a major discrepancy||9|
|Students who did not allocate a grade||1|
Students who already had very good analytical skills evaluated their strengths and weaknesses. Students who had poor analytical skills tended to over-value the strengths in their work, but often recognised the weaknesses in their work. The first piece of assessment in the course was changed the following year in order to assist students in understanding the theory and developing critical thinking skills. The marking criteria developed during the research project was effective in assessing that all aspects of the question had been addressed. This marking criterion has been put into the course outline so it can be openly discussed and referred to.
The findings of this research did not support previous research that all third year students have the capacity to be self-directed and self-reflective learners who can assess their work at the same level as the tutor. Although students in this study were third years, what became evident is that they entered the course with differing analytical abilities. Those with better critical and analytical skills were able to assess their work at a similar standard as the tutor; in other words, they were reliable assessors. Students with less developed critical and analytical skills, however, were not able to assess their work reliably. Some students had written few analytical and theoretical essays since first year. It was concluded that students within this demographic would need a trial run with self-assessment materials in which learning is reinforced with evaluation and feedback.
Adams, C. & King, K. (1995). Towards a framework for student self-assessment. Innovations in Education and Training International, 32(4), 336-43.
Brown, S., Rust, C. & Gibbs, G. (1994). Section 5: Involving students in the assessment process. Strategies for Diversifying Assessment in Higher Education. p.1-4. Oxford: Oxford Centre for Staff Development, 1994. [verified 20 June 2004] http://www.lgu.ac.uk/deliberations/ocsd-pubs/div-ass5.html
Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher Education, 15(1), 101-111.
Butcher, A.C., Stephani, L.A. & Tariq, V.N. (1995). Analysis of peer, self and staff assessment in group project work. Assessment in Education: Principles, Policy and Practice, 2(2), 165-185.
Catalyst Tools (2002). Introduction to peer review. University of Washington. [viewed 17 Jan 2002, verified 20 June 2004] http://catalyst.washington.edu/tools/peer_review.html
Clifford, V.A. (1999). The development of autonomous learners in a university setting. Higher Education Research and Development, 18(1), 115-128.
Donnelly, A. & Ridgway, E. The Mathew Brady bunch - self assessment and peer evaluation. Library of Congress: Americam memory. [viewed 11 Jan 2002 at http://memory.loc.gov/ammen/ndlpedu/lessons/98/brady/selfeval.html, verified 20 Jun 2004 at http://learning.loc.gov/learn/lessons/98/brady/selfeval.html]
Eanes, R. (undated). Honors 3382: Political Action Letter Peer Review: Self assessment rubric. St Edward's University, Austin, Texas. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.stedwards.edu/educ/eanes/rubrics/letter_peer_self.htm
Eanes, R. (undated). Honours 3382: Self-evaluation of achievement of course goals. St Edward's University, Austin, Texas. [viewed 29 Jan 2002, verified 20 June 2004] http://www.stedwards.edu/educ/eanes/self-assess_fa98.htm
Ellington, H., Earl, S., & Cowan, J. (1997). Making effective use of peer and self assessment. The Robert Gordon University. [viewed 11 Jan 2002 at http://led.gcal.ac.uk/ciced/Ch26.html, not found 20 June 2004; see http://www2.rgu.ac.uk/subj/eds/resources/documents/tltbooks.htm]
Elton, L (1988). Teaching in higher education: Appraisal and training. London: Kogan Page.
Falchikov, N. (1995). Peer feedback marking: Developing peer assessment. Innovations in Educations and Training International, 32(2), 175-187.
Ford, A. (1997). Peer group assessment: Its application to a vocational modular degree course. Journal of Further and Higher Education, 21(3), 285-296.
Gatfield, T. (undated). Group project and peer assessment evaluation study: An investigation into Australian and international student perceptions. [viewed 4 Feb 2002, verified 20 Jun 2004] http://www2.auckland.ac.nz/cpd/HERDSA/HTML/Workshop/GATFIELD.HTM
Gibson, B. (1996). A peer review activity for essay organization. The Internet TESL Journal, 2(3). [viewed 17 Jan 2002, verified 20 Jun 2004] http://iteslj.org/Techniques/Gibson-PeerReview.html
Global SchoolNet Foundation (2002). Sample peer review evaluation rubric. International Schools Cyberfair. [viewed 17 Jan 2002, verified 20 Jun 2004] http://gsh.lightspan.com/gsh/cf/rubric/evalrubric.html
Healy, M. (1999). Using peer and self assessment for assessing the contribution of individuals to a group project. University of Gloucestershire. [viewed 11 January 2002 at http://www.chelt.ac.uk/el/phil/gdn/abstracts/a69.htm, verified 20 Jun 2004 at http://www.chelt.ac.uk/gdn/abstracts/a69.htm]
Isaacs, G. (1998). Brief briefing: Peer and self assessment. Effective Assessment at University: A conference for staff of the University of Queensland and local universities. University of Queensland, 4-5 November. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.tedi.uq.edu.au/conferences/A_conf/papers/Isaacs.html
Knoy, T., San-Ju, L., Zhi-Feng, L., & Shyan-Min, Y. (2001). Networked peer assessment in writing: Copyediting skills instruction in an ESL technical writing course. National Chiao Tung University, Hsinchu, Taiwan. [viewed 24 Jan 2002, verified 20 June 2004] http://www.cc.nctu.edu.tw/~tedknoy/html/News_2001_NPAW.htm
Moon, J. (2001). Peer and self assessment in student work. Teaching Quality Assurance Manual, University of Exeter. [viewed 7 Jan 2002, verified 20 June 2004] http://www.ex.ac.uk/admin/academic/tls/tqa/peerassess1.htm
Moursund, D. (2002). PBL Part 7: Assessment. University of Oregon. [viewed 17 Jan 2002, verified 20 Jun 2004] http://www.uoregon.edu/~moursund/PBL/part_7.htm
Mowl, G. & Pin, R. (1995). Using self and peer assessment to improve students' essay writing: A case study from geography. Innovations in Education and Training International, 32(4), 324-335.
Murdoch University (2002). Code of practice - assessment: Advice on group, self and peer assessment and on assessment of participation. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.murdoch.edu.au/vco/secretariat/admin/codes/advice.html
Peat, M., Franklin, S. & Lewis, A. (2001). A Review of the use of online self-assessment modules to enhance student learning outcomes: Are they worth the effort of production? [verified 20 Jun 2004] http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/peatm1.pdf
Peters, M. (1996). Student attitudes to alternative forms of assessment and to openness. Open Learning, 11(3), 48-50.
Platt, J. (2000). Assessment strategies and standards in sociology section two: Self and peer assessment for sociology? University of Sussex. [viewed 11 Jan 2002 at http://www.bathspa.ac.uk/socassess/sectiontwo.htm, verified 20 Jun 2004 at http://users.bathspa.ac.uk/socassess/sectiontwo.htm]
Pond, K., et.al. (1995). Peer review: A precursor to peer assessment. Innovations in Education and Training International, 32(4), 314-323.
Oldfield, K. & MacAlpine, J.M.K. (1995). Peer and self-assessment at tertiary level: An experimental report. Assessment and Evaluation in Higher Education, 20(1), 125-132.
Oliver, R. (2001). Exploring the development of critical thinking skills through a Web-supported problem-based learning environment. In J. Stephenson (Ed.), Teaching and Learning Online: Pedagogies for New Technologies (pp. 98-111): Kogan Page. [viewed 24 Jan 2002 at http://www.lle.mdx.ac.uk/iclml/newsite/js/chap8.html, not found 20 Jun 2004]
Oliver, R. & McLoughlin, C. (1999). Using web and problem-based learning environments to support the development of key skills. Proceedings ASCILITE'99, Brisbane. http://www.ascilite.org.au/conferences/brisbane99/papers/olivermcloughlin.pdf
Restructuring for urban student success (1999). [viewed 31 Jan 2002 at http://www.universitycollege.iupui.edu/Russ/Home.asp. URL not found 20 Jun 2004, see also http://www.universitycollege.iupui.edu/UC/Assessment/RUSS_Rpt_Final.pdf]
RMIT University (2001). Assessment: Frequently asked questions. Learning Technology Services. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.lts.rmit.edu.au/renewal/assess/faq2.htm
Sambell, K., McDowell, L. & Brown, S. (1997). "But is it fair?": An exploratory study of student perceptions of the consequential validity of assessment. Studies in Educational Evaluation, 23(4), 349-371.
Schell, E. E. (undated). Student peer review. Writing Program, Virginia Polytechnic Institute and State University. [viewed 17 Jan 2002 from http://www.edtech.vt.edu/uwp/online2cb.htm, not found 20 June 2004, see version at http://www.uwp.vt.edu/html/online_resources/teaching/olr_menu_03_sub_2.htm]
Sluijsmans, D., Dochy, F. & Moerkerke, G. (1999). Creating a learning environment by using self, peer and co-assessment. Learning Environments Research, 1, 293-319.
Sluijsmans, D., Dochy, F. & Moerkerke, G. (1998). The use of self, peer and co-assessment in higher education: A review of literature. Educational Technology Expertise Centre, Open University of the Netherlands. [viewed 24 Jan 2002 at http://www.ouh.nl/otec/, article not found 20 Jun 2004; see http://www.ou.nl/info-alg-english-r_d/OTEC_research/publications/index.htm]
Sluijsmans, D. & Moerkerke, G. (1999). Student involvement in performance assessment: A research project. OTEC, Open University of the Netherlands. [viewed 24 January 24, 2002 at http://www1.nks.no/eurodl/shoen/Assessment%20issue/sluijsmans/, not found 20 June 2004]
Topping, K.J. (1998). Peer assessment between students and colleges and universities. Review of Educational Research, 68(3), 249-276.
Topping, K.J., Smith, E.F., & Swanson, I. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment and Evaluation in Higher Education, 25(2), 149-166.
Tutorial Presentation: Peer assessment (undated). Educational and Inspirational Tools for Educators. [viewed 17 Jan 2002, verified 20 Jun 2004] http://www.catholic-forum.com/catholicteacher/tutorial_presentation_novel_PEER_ASSESSMENT.html
Zariski, A. (1996). Self and peer assessment as a means of teaching professional responsibility. School of Law, Murdoch University. [viewed 11 Jan 2002, verified 20 Jun 2004] http://law.murdoch.edu.au/academics/zariski/peer2.html
Zariski, A. (1996). Student peer assessment in tertiary education: Promise, perils and practice. Teaching and Learning Within and Across Disciplines. Eds. J. Abbott, and L. Willcoxson, pp.189-200. http://lsn.curtin.edu.au/tlf/tlf1996/zariski.html
Title of Detailed Analysis:
|Author: Debbie Rodan, Media Studies, Edith Cowan University. Email: email@example.com
Please cite as: Rodan, D. (2004). Seeking Educational Excellence: Developing Self-Assessment for Analytical Essays. In Seeking Educational Excellence. Proceedings of the 13th Annual Teaching Learning Forum, 9-10 February 2004. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf2004/rodan.html