Teaching and Learning Forum 2004 Home Page
Teaching and Learning Forum 2004 [ Proceedings Contents ]
Seeking educational excellence: Developing self assessment for analytical essays

Debbie Rodan
Edith Cowan University

The aim of the project was to design, pilot and evaluate self-assessment materials, which would minimise the opportunity for the inappropriate use of academic material and foster the development of reflective learners that is, learners who understand and respect the process of knowledge creation. Educational excellence could be cultivated through the development of an understanding for the process of creating knowledge. The first objective of the project was to develop innovative and authentic tasks for individual students. The second was to develop self-assessment materials in order for students to reflect on their learning and output.

From this research, what became evident is that students who already had very good analytical skills evaluated their strengths and weaknesses. Students who had poor analytical skills tended to over-value the strengths in their work, but often recognised the weaknesses in their work. While all of the students participating in the study were third years, it was apparent that some had not written a theoretical or analytical essay since first year. With any future use of self-assessment, students would need a trial run in which learning is reinforced with evaluation and feedback. Self-assessment is likely to work with students at third year level and later levels who have similar analytical, theoretical and critical thinking background. This paper will briefly discuss the findings of the main writers/researchers on student self-assessment, the method employed in this research and the findings.


Introduction

In order to become a self-directed and self-reflective learner students need to be able to assess their work; one way they can do this is with self-assessment materials. There has been a body of research on the use of self and peer assessment materials for projects, oral presentations and as a way of teaching professional responsibility however very little on analytical essays (Oliver, 2000; Platt, 2002; Zariski, 2002). The aim of this project was to design, pilot and evaluate self-assessment materials for analytical essays. It was envisaged that self-assessment would minimise the opportunity for the inappropriate use of materials and foster the development of reflective learners who understood and respected the process of knowledge creation. The first objective was to develop innovative and authentic tasks, which would be used by individual students. The second was to develop self-assessment materials in order for students to reflect on their learning and output. Educational excellence could be cultivated through the development of an understanding for the process of creating knowledge and the ability to reflect on one's theoretical/analytical work.

Review

A literature review of 45 articles (print and electronic) was carried out. One of the key issues that emerged from the literature review was that few researchers had used self-assessment materials for analytical essays. Graham Mowl and Rachel Pain (1995) had used this form of assessment for essay writing. Third year students in the theory course Media and Identity at Edith Cowan University have three assessment components in which they are required to do analytical work. This group of students were asked to participate in the project, because previous research has shown self-assessment materials are more appropriate for third and fourth year students (Ford, 1997; Oliver, 2002). From the review of the literature, it was evident that it would be best to do the research with the same year group, third years.

Other researchers have found that students are often hostile to different methods of assessment (Elton, 1988; Boud, 1990; Peters, 1996), and for this reason students would need to be advised of the benefits. It was decided to market the additional assessment based on developing critical thinking, reflective practice, and independent learning (Moon, 2002; Mowl & Pin, 1995; Sluijsmans, Dochy & Moerkerke, 1998). Doing the assessment was also considered part of the assignment mark. Students were advised that if they completed the self-assessment, they would not get less than the tutor grade. With regard to assignment weightings, some researchers suggested that early attempts at peer assessment should carry a lower weighting, for example, a 20% overall mark (Ford, 1997; Academic Council Office, 1999). The assignment used for this research had a weighting of 40%; from doing this project, it is evident that as a first assignment this is too high a rating. The weighting for the first assignment in the course has now been changed.

This research project supported previous studies (Ellington, Earl and Cowan, 1997; Moon, 2002) that it is very important to have preparatory meetings to explain to students the rules and marking range for self and peer assessment. Added to that, there is a need to involve the students early on in generating and developing the criteria (Mowl and Pin, 1995). Preparatory meetings were organised and students were involved in developing the marking criteria. Some 26 students (out of 27 third years enrolled in the course) attended the first preparatory meeting and about half participated in developing the marking criteria.

Some researchers (Butcher, Stephanie, and Tariq, 1995; Topping, Smith and Swanson, 2000) stated that it is necessary to have a follow-up discussion about the difference in marks between self or peer and tutor, and how that mark will be moderated. Follow-up discussions were offered, however, not always taken-up. The coordinator of the course offered to moderate the marks. Students however were promised that they would not get less than the tutors' grade, as it is important that any variations in marks due to assessors' differences be evaluated and adjusted in the final mark (Butcher, Stephanie, and Tariq, 1995). Keith Topping (1998) suggests there needs to be an awareness of gender and cultural differences. He points to the example of mature women who tend to identify their weaknesses opposed to their strengths. As we only had two mature age women (over 30 years) enrolled in the course, this was not evident in the sample. With regard to cultural difference, only two people participating in the research were from a non-Anglo background.

From this literature review, it was evident that third year students would be the best undergraduate group to participate in the research. They needed to be involved in the development of the assessment and marking criteria as well as being clearly advised of the benefits of using self-assessment.

Method

An ethics application was submitted, the process for an ethics clearance took six weeks, so it is very important to start early. In the lectures for week one and two of the course the project aims and benefits were advertised to the students.

Third year students were chosen because from reviewing the literature researchers such as Ron Oliver (2002) argued that third years and above have a "greater capacity to look after themselves" and in being able to assess their work. In the first organised meeting, students were consulted about questions were to be included in the self-assessment, and the marking criteria. A self-assessment questionnaire (see appendix one) and marking criteria form was designed. An electronic form of the questionnaire was made available on the school website. In the second meeting, the lecturer discussed with students the grading of a HD, D, Cr, C, N, and appropriate criteria allocated to each grade. A handout on grading was given to students. Overall, 24 essays were handed in with a completed self-assessment questionnaire. The sample was 24 third year bachelor of communication students (24 out of 27 completed the self-assessment) with the majors listed in Table 1.

Table 1: Sample of students

Advertising/Public Relations (x2)
Mass Communications (x2)
Media Studies (x4)
Media Studies/minor Drama studies
Media Studies/minor English
Advertising/Film and Video
Public Relations
Advertising/Media Studies (x2)
Public Relations/Media Studies (x2)
Film and Video
Health Promotions/Media Studies
Interactive Multimedia/Media Studies
Exchange Student
Journalism
Journalism/Media Studies
Journalism/Photomedia

Students submitted their self-assessment at the same time as their essays. Tutors did not look at the self-assessments until after the essays had been marked. The assignment was worth 40% and the topic was a detailed analysis of the ways in which gender roles are constructed in one Australian media culture (such as television, film, magazines, video clips etc).

Discussion

Overall, in terms of peer assessment, previous researchers (Topping, Smith and Swanson, 2000) found that with tutor and peer/self assessment, generally the marks are similar and students can be reliable assessors (Butcher, Stephanie, and Tariq, 1995). This was not born out in this research using self-assessment material. Just over half of the marks (13 out of 24) were similar (within 5%) to the tutors grade. In terms of major discrepancies 4 people submitted material, which appeared to be from another course, 1 person had mainly cut and pasted text from the Internet, 1 person had a 10% variance and 3 people had not sufficiently met the criteria. I have outlined this in the table below.

Table 2: Overview of student assessment grades

Students whose grade matched Tutors Grade4
Students who came within 5% Tutors Grade9
Students who came within 10% Tutors Grade1
Students where there was a major discrepancy9
Students who did not allocate a grade1
Total24

Students who already had very good analytical skills evaluated their strengths and weaknesses. Students who had poor analytical skills tended to over-value the strengths in their work, but often recognised the weaknesses in their work. The first piece of assessment in the course was changed the following year in order to assist students in understanding the theory and developing critical thinking skills. The marking criteria developed during the research project was effective in assessing that all aspects of the question had been addressed. This marking criterion has been put into the course outline so it can be openly discussed and referred to.

Conclusion

One of the aims of the project was to develop self-assessment in order to minimise the opportunity for the inappropriate use of materials and foster the development of reflective learners who understand and respect the process of knowledge creation. Self-assessment did not minimise forms of plagiarism as 4 students used material from another course and 1 student copied and pasted material from the Internet.

The findings of this research did not support previous research that all third year students have the capacity to be self-directed and self-reflective learners who can assess their work at the same level as the tutor. Although students in this study were third years, what became evident is that they entered the course with differing analytical abilities. Those with better critical and analytical skills were able to assess their work at a similar standard as the tutor; in other words, they were reliable assessors. Students with less developed critical and analytical skills, however, were not able to assess their work reliably. Some students had written few analytical and theoretical essays since first year. It was concluded that students within this demographic would need a trial run with self-assessment materials in which learning is reinforced with evaluation and feedback.

Acknowledgment

I would like to thank Robyn Quin for suggesting the project and the Teaching and Learning Office at ECU for providing the grant. Also thanks to Ron Oliver as well as Joe Luca for valuable suggestions during the project and Zoe Chambers for assisting in the research and sharing her ideas.

References

Academic Council Office (1999). Section M9: Student Peer and Self Assessment. Edith Cowan University, Perth.

Adams, C. & King, K. (1995). Towards a framework for student self-assessment. Innovations in Education and Training International, 32(4), 336-43.

Brown, S., Rust, C. & Gibbs, G. (1994). Section 5: Involving students in the assessment process. Strategies for Diversifying Assessment in Higher Education. p.1-4. Oxford: Oxford Centre for Staff Development, 1994. [verified 20 June 2004] http://www.lgu.ac.uk/deliberations/ocsd-pubs/div-ass5.html

Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher Education, 15(1), 101-111.

Butcher, A.C., Stephani, L.A. & Tariq, V.N. (1995). Analysis of peer, self and staff assessment in group project work. Assessment in Education: Principles, Policy and Practice, 2(2), 165-185.

Catalyst Tools (2002). Introduction to peer review. University of Washington. [viewed 17 Jan 2002, verified 20 June 2004] http://catalyst.washington.edu/tools/peer_review.html

Clifford, V.A. (1999). The development of autonomous learners in a university setting. Higher Education Research and Development, 18(1), 115-128.

Donnelly, A. & Ridgway, E. The Mathew Brady bunch - self assessment and peer evaluation. Library of Congress: Americam memory. [viewed 11 Jan 2002 at http://memory.loc.gov/ammen/ndlpedu/lessons/98/brady/selfeval.html, verified 20 Jun 2004 at http://learning.loc.gov/learn/lessons/98/brady/selfeval.html]

Eanes, R. (undated). Honors 3382: Political Action Letter Peer Review: Self assessment rubric. St Edward's University, Austin, Texas. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.stedwards.edu/educ/eanes/rubrics/letter_peer_self.htm

Eanes, R. (undated). Honours 3382: Self-evaluation of achievement of course goals. St Edward's University, Austin, Texas. [viewed 29 Jan 2002, verified 20 June 2004] http://www.stedwards.edu/educ/eanes/self-assess_fa98.htm

Ellington, H., Earl, S., & Cowan, J. (1997). Making effective use of peer and self assessment. The Robert Gordon University. [viewed 11 Jan 2002 at http://led.gcal.ac.uk/ciced/Ch26.html, not found 20 June 2004; see http://www2.rgu.ac.uk/subj/eds/resources/documents/tltbooks.htm]

Elton, L (1988). Teaching in higher education: Appraisal and training. London: Kogan Page.

Falchikov, N. (1995). Peer feedback marking: Developing peer assessment. Innovations in Educations and Training International, 32(2), 175-187.

Ford, A. (1997). Peer group assessment: Its application to a vocational modular degree course. Journal of Further and Higher Education, 21(3), 285-296.

Gatfield, T. (undated). Group project and peer assessment evaluation study: An investigation into Australian and international student perceptions. [viewed 4 Feb 2002, verified 20 Jun 2004] http://www2.auckland.ac.nz/cpd/HERDSA/HTML/Workshop/GATFIELD.HTM

Gibson, B. (1996). A peer review activity for essay organization. The Internet TESL Journal, 2(3). [viewed 17 Jan 2002, verified 20 Jun 2004] http://iteslj.org/Techniques/Gibson-PeerReview.html

Global SchoolNet Foundation (2002). Sample peer review evaluation rubric. International Schools Cyberfair. [viewed 17 Jan 2002, verified 20 Jun 2004] http://gsh.lightspan.com/gsh/cf/rubric/evalrubric.html

Healy, M. (1999). Using peer and self assessment for assessing the contribution of individuals to a group project. University of Gloucestershire. [viewed 11 January 2002 at http://www.chelt.ac.uk/el/phil/gdn/abstracts/a69.htm, verified 20 Jun 2004 at http://www.chelt.ac.uk/gdn/abstracts/a69.htm]

Isaacs, G. (1998). Brief briefing: Peer and self assessment. Effective Assessment at University: A conference for staff of the University of Queensland and local universities. University of Queensland, 4-5 November. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.tedi.uq.edu.au/conferences/A_conf/papers/Isaacs.html

Knoy, T., San-Ju, L., Zhi-Feng, L., & Shyan-Min, Y. (2001). Networked peer assessment in writing: Copyediting skills instruction in an ESL technical writing course. National Chiao Tung University, Hsinchu, Taiwan. [viewed 24 Jan 2002, verified 20 June 2004] http://www.cc.nctu.edu.tw/~tedknoy/html/News_2001_NPAW.htm

Moon, J. (2001). Peer and self assessment in student work. Teaching Quality Assurance Manual, University of Exeter. [viewed 7 Jan 2002, verified 20 June 2004] http://www.ex.ac.uk/admin/academic/tls/tqa/peerassess1.htm

Moursund, D. (2002). PBL Part 7: Assessment. University of Oregon. [viewed 17 Jan 2002, verified 20 Jun 2004] http://www.uoregon.edu/~moursund/PBL/part_7.htm

Mowl, G. & Pin, R. (1995). Using self and peer assessment to improve students' essay writing: A case study from geography. Innovations in Education and Training International, 32(4), 324-335.

Murdoch University (2002). Code of practice - assessment: Advice on group, self and peer assessment and on assessment of participation. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.murdoch.edu.au/vco/secretariat/admin/codes/advice.html

Peat, M., Franklin, S. & Lewis, A. (2001). A Review of the use of online self-assessment modules to enhance student learning outcomes: Are they worth the effort of production? [verified 20 Jun 2004] http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/peatm1.pdf

Peters, M. (1996). Student attitudes to alternative forms of assessment and to openness. Open Learning, 11(3), 48-50.

Platt, J. (2000). Assessment strategies and standards in sociology section two: Self and peer assessment for sociology? University of Sussex. [viewed 11 Jan 2002 at http://www.bathspa.ac.uk/socassess/sectiontwo.htm, verified 20 Jun 2004 at http://users.bathspa.ac.uk/socassess/sectiontwo.htm]

Pond, K., et.al. (1995). Peer review: A precursor to peer assessment. Innovations in Education and Training International, 32(4), 314-323.

Oldfield, K. & MacAlpine, J.M.K. (1995). Peer and self-assessment at tertiary level: An experimental report. Assessment and Evaluation in Higher Education, 20(1), 125-132.

Oliver, R. (2001). Exploring the development of critical thinking skills through a Web-supported problem-based learning environment. In J. Stephenson (Ed.), Teaching and Learning Online: Pedagogies for New Technologies (pp. 98-111): Kogan Page. [viewed 24 Jan 2002 at http://www.lle.mdx.ac.uk/iclml/newsite/js/chap8.html, not found 20 Jun 2004]

Oliver, R. & McLoughlin, C. (1999). Using web and problem-based learning environments to support the development of key skills. Proceedings ASCILITE'99, Brisbane. http://www.ascilite.org.au/conferences/brisbane99/papers/olivermcloughlin.pdf

Restructuring for urban student success (1999). [viewed 31 Jan 2002 at http://www.universitycollege.iupui.edu/Russ/Home.asp. URL not found 20 Jun 2004, see also http://www.universitycollege.iupui.edu/UC/Assessment/RUSS_Rpt_Final.pdf]

RMIT University (2001). Assessment: Frequently asked questions. Learning Technology Services. [viewed 11 Jan 2002, verified 20 Jun 2004] http://www.lts.rmit.edu.au/renewal/assess/faq2.htm

Sambell, K., McDowell, L. & Brown, S. (1997). "But is it fair?": An exploratory study of student perceptions of the consequential validity of assessment. Studies in Educational Evaluation, 23(4), 349-371.

Schell, E. E. (undated). Student peer review. Writing Program, Virginia Polytechnic Institute and State University. [viewed 17 Jan 2002 from http://www.edtech.vt.edu/uwp/online2cb.htm, not found 20 June 2004, see version at http://www.uwp.vt.edu/html/online_resources/teaching/olr_menu_03_sub_2.htm]

Sluijsmans, D., Dochy, F. & Moerkerke, G. (1999). Creating a learning environment by using self, peer and co-assessment. Learning Environments Research, 1, 293-319.

Sluijsmans, D., Dochy, F. & Moerkerke, G. (1998). The use of self, peer and co-assessment in higher education: A review of literature. Educational Technology Expertise Centre, Open University of the Netherlands. [viewed 24 Jan 2002 at http://www.ouh.nl/otec/, article not found 20 Jun 2004; see http://www.ou.nl/info-alg-english-r_d/OTEC_research/publications/index.htm]

Sluijsmans, D. & Moerkerke, G. (1999). Student involvement in performance assessment: A research project. OTEC, Open University of the Netherlands. [viewed 24 January 24, 2002 at http://www1.nks.no/eurodl/shoen/Assessment%20issue/sluijsmans/, not found 20 June 2004]

Topping, K.J. (1998). Peer assessment between students and colleges and universities. Review of Educational Research, 68(3), 249-276.

Topping, K.J., Smith, E.F., & Swanson, I. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment and Evaluation in Higher Education, 25(2), 149-166.

Tutorial Presentation: Peer assessment (undated). Educational and Inspirational Tools for Educators. [viewed 17 Jan 2002, verified 20 Jun 2004] http://www.catholic-forum.com/catholicteacher/tutorial_presentation_novel_PEER_ASSESSMENT.html

Zariski, A. (1996). Self and peer assessment as a means of teaching professional responsibility. School of Law, Murdoch University. [viewed 11 Jan 2002, verified 20 Jun 2004] http://law.murdoch.edu.au/academics/zariski/peer2.html

Zariski, A. (1996). Student peer assessment in tertiary education: Promise, perils and practice. Teaching and Learning Within and Across Disciplines. Eds. J. Abbott, and L. Willcoxson, pp.189-200. http://lsn.curtin.edu.au/tlf/tlf1996/zariski.html

Appendix One: Self-Assessment Questionnaire

Part 1

Name:
Personal Tutor:
Title of Detailed Analysis:

Part 2

  1. Do you think you have used enough evidence from the relevant primary material to support your points?



  2. Do you think your detailed analysis shows the ability to develop a personal viewpoint? Does your research support the conclusions you make?



  3. Do you demonstrate an adequate understanding of terminology and gender theories relevant to the subject?



  4. Have you structured the assignment to meet the requirements of the project? And have you presented your ideas clearly?



  5. How do you think your detailed analysis could be improved?
    - What do you think are the strong points of this assignment?
    - What do you think are the weaknesses?



Allocate a grade, which you believe represents a fair appraisal of your overall performance on this project, bearing in mind the projects instructions and agreed criteria for assessment:



Author: Debbie Rodan, Media Studies, Edith Cowan University. Email: d.rodan@ecu.edu.au

Please cite as: Rodan, D. (2004). Seeking Educational Excellence: Developing Self-Assessment for Analytical Essays. In Seeking Educational Excellence. Proceedings of the 13th Annual Teaching Learning Forum, 9-10 February 2004. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf2004/rodan.html


[ Proceedings Contents ] [ Teaching and Learning Forum 2004 ]
This URL: http://lsn.curtin.edu.au/tlf/tlf2004/rodan.html
Created 20 June 2004. Last revision: 20 June 2004.