in association with
Jennifer Barlow, Cherry Barlowe, George Borzyskowski, Andrew Brown, Robina Crook, Renee Dubruin, George Green, Helen Richards, Steven Rubeck, Pamela Stephenson, Sue Taylor
Assessment has been a problem area in the School of Design for a long time. Issues include: the diversity of things to be addressed, the degree of precision that is possible/desirable, the amount of time that can/should be devoted to assessment, questions of subjectivity and objectivity, fairness, competition and cooperation and the contribution that assessment, as a form of feedback, can make to the learning process.
Following our recent programme review, and in the context of Quality Assurance, the School had prepared a strategic plan with the help of an external facilitator. A high priority has been given to the development of standard assessment processes. This task has been undertaken by a team of staff and students.
Of all the systems used in the school the ones which seem most promising are those based on feedback sheets. These sheets list all the criteria relevant for a given project. A student's performance according to each criterion in recorded on a scale. It is then possible to derive from these records an overall grade or mark for each student. Team members are looking at variations on the theme of feedback sheets and preparing an inventory of criteria. Our underlying objective is establishment of a process which is manageable while being clearly focussed on support for learning.
We will present our conclusions at the Forum and are looking forward to feedback which may help us finalise our proposals for adoption by the School.
We used to award marks - sometimes out of 100, usually out of 10 - for solutions to design problems, folios of drawings, essays etc. We have also experimented with the A, B, C grading systems. And we have tried a simple Pass/Fail system and then an elaborated one where P for Pass could be modified with a + of a -. The mark or grade was reinforced by feedback: written comments on essays and verbal comments on design work and studio exercises. In some situations we found ourselves needing to make the same kinds of comments over and over again so we began to develop standard forms for the purpose. These evolved into feedback sheets which, in turn, became a basis for the marks or grades.
As far as the learning process was concerned our old systems, especially those systems of marks which perpetuated the high school model, were in many ways counter productive. Students tended to focus on getting good marks rather than developing as designers. This meant that some would keep doing what they could already do well rather than explore new media and that they would choose safe solutions rather than take risks. Marks were received as a kind of payment for effort. Emotional energy, which could have been devoted to learning, was often used up in brooding over what were felt to be unfair marks. A contributing factor was often the different priorities attached by students and tutors to the diversity of things that might be marked: ideas, skills, knowledge, understanding, effort, professionalism, participation etc. Also the illusory precision of marks gave some students a false sense of their own position on an imaginary scale of worth. Some of the more competitive students were reluctant to share ideas or make any suggestions that might help another student up this scale.
Meanwhile demands on tutors' time has been increasing and the task of ensuring that marks are indeed fair has become more demanding as has the task of providing comprehensive feedback to students. Our problem, therefore, is part of the larger problem of "how to teach better for less". At the 1993 Teaching and Learning Forum, Richard Berlach (1993:210) was probably speaking for most of us when he said that ".... a goal worth striving for is one which ensures that high-quality feedback is maintained while at the same time making the whole exercise of marking a time-efficient one."
It was agreed that we should develop a standard system for use throughout the School. This Teaching and Learning Forum provided the catalyst. It gave us a deadline to work towards and offered an opportunity to run our proposals past an impartial audience who, as we hope, are both experienced and sympathetic. We are also sending our proposals to the members of our Advisory Committee. Suggestions from that group, as well as from the present audience, will be reviewed and may be incorporated in the system we plan to introduce in first semester.
Six student members of our teaching/learning team were able and willing to work with School staff over the holiday period. We have had four sessions of up to five hours each. Problems were identified, issues debated and proposals discussed.
A survey of some existing systems had been conducted. Student answers to a questionnaire confirmed our view that the feedback sheet system shows most promise of being able to deliver "the mostest for the leastest". Sheets used by different members of staff were reviewed as well as one from another discipline (Faculty of Education) and one from overseas (HNC).
We also looked at other models for possible insights. Figure skating and gymnastics are two activities where judgements of performance must be quick, consistent and fair, made in pubic and under pressure. In both activities there are many dimensions of performance which must be judged simultaneously. Significantly for our purposes both have aesthetic as well as technical dimensions and both admit the possibility of a perfect score (notably achieved by Torvill and Dean and Nadia Comenici). In both cases the criteria are very clearly spelt out and are under constant review, especially those dealing with degrees of difficulty. (We were interested to learn that the routines which earned Nadia Comenici perfect 10s in Montreal are no longer considered particularly difficult and would not be worth much more than eight today. And we saw at Lillehammer how the rest of the World has caught up with Torvill and Dean.) We are very grateful for the information supplied by international figure skating judge Tony Jonikis (1994) and the chairman of the WA Gymnastics Judging Assembly, Bill Barr (1994).
In our workshop sessions there was early disagreement about whether or not it should be possible to get 100% for a solution to a design problem. The winning argument was that other marks, like 80%, would be meaningless if 100% were impossible. The key for design, as for skating and gymnastics, is the principle of degrees of difficulty. So a solution which might earn 100% for a first year student would not earn so much for a student in third year.
A great deal of discussion was devoted to establishing an inventory of assessment criteria. Some criteria were controversial, seeming to depend on the subjective opinion of the tutor. But we agreed that whatever it is that makes one design solution work better than another it should be on the list. At the 1993 Teaching Learning Forum, Paul Green-Armytage (1993) argued against allowing ease or reliability of measurement to determine what should count in students' work. We agreed that there could be cases where a desirable quality in a design solution would be "magic". Rather than exclude this as unmeasurable we agreed that a degree of "magicness" could be established by the consensus of an impartial group of people, preferably representative of the intended audience. And even this consensus would not be the average subjective opinion. We are impressed by the arguments put forward by David Best for the objectivity of artistic appreciation: "If an activity is not objective, then the notion of standards cannot be applicable to it, since to set a standard necessarily implies the possibility of citing reasons which refer to observable, objective phenomena" (Best, 1980:14). Even as we say that a design solution has magic we must be able to point to those features which give it that quality.
All courses at Alverno College focus attention on a set of eight core abilities. We have adopted a set of our own, similar but not identical, and have grouped our assessment criteria as we see them relating to these abilities:
Feedback sheets will be given to students for each component of each unit in the course. On these sheets will be recorded judgements of the students on a five step scale for each criterion listed. The five steps equate with the five levels of achievement recognised by the University: High Distinction, Distinction, Credit, Pass, and Fail. From the judgements recorded on the feedback sheets students will learn where their strengths and weaknesses lie and where they need to direct their efforts for improvement.
The records on the feedback sheets can also be turned into numbers for the purposes of establishing a mark or grade: 0 for something missing, 1 for a Fail, 2 for a Pass, etc. With several criteria being scored like this the spread of numbers could be very large and it would be possible to produce a list of students in finely ranked order in accordance with these records of their performance. For two reasons we will not do this but will convert the overall record for each student's performance in each component into a simple grade which will correspond to one of the five steps.
The first reason is that while the judgements may be objective they will still be personal. The possibility of disagreement, however slight, means that absolute precision of marking is impossible. (We see this in the judges' scores for skating and gymnastics. Rarely are all judges in perfect agreement. Their scores may range from, say, 5.2 - 5.6, but it would be an extreme aberration if one judge's score was a long way outside that range.) A finely ranked order, therefore, would be misleading. Furthermore a large range of marks would take us back to the old situation of counter productive competition.
The second reason (only admissible because of the first) is that a long time would be needed to count up each score. We have found that it is quite easy to scan the pattern of marks on the sheets and determine where the total would belong if there was a spread of marks corresponding to each step on the scale. It is possible to determine an overall grade quite quickly and without doing the arithmetic. At the same time the students could still do the sums themselves and could get grades altered if they could show that a mistake had been made.
Producing the final marks for each unit is also reasonably straightforward. A unit might have major components and minor ones; grades for the components would be weighted accordingly. We have linked particular numbers to the grades. When a final mark is worked our from those numbers it comes within the correct range for one of the levels of achievement recognised by the University. The way it works is spelt out in detail in Assessment Explained.
It is true that in the list of marks for each unit at the end of semester the students are likely to be in a finely ranked order. But at that stage competition is not an issue. During the semester several students would get the same grade for a project and for most of the students that would be a Credit. Students would know where they stood generally. They would also have the more detailed information. A Credit would be a boring grade to get but on the feedback sheets a student might find one or two High Distinctions, and feel well rewarded. They might also see one or two Fails and feel suitably chastened.
We hope this system will be quick and easy to manage and that it will provide effective feedback to support the students' learning. Perhaps at a future Teaching and Learning Forum we will be able to present a report on how it has worked out in practice.
Berlach, Richard (1993). The Value of Writing Comments on Student Assignments: A Case for Reducing Marker Expectations. In Sharing Quality Practice. Curtin University of Technology, Perth.
Best, David (1980). Objectivity in artistic appreciation. In Arts in Cultural Diversity. Holt, Reinhart and Winston, Sydney.
Faculty of Education, Curtin University of Technology. Assignment Cover Sheet and Holistic Scoring Guide. Perth.
Green-Armytage, Paul (1993). The Tyranny of Measurement. In Sharing Quality Practice. Curtin University of Technology, Perth.
HNC (Higher National Certificate). Design Common Skills Assessment. London.
Jonikis, Anthony (1994), Personal communication, December, Perth.
Loacker, Georgine, Cromwell, Lucy, and O'Brien, Kathleen (1986). Assessment in Higher Education: To Serve the Learner. In Assessment in American Higher Education, US Department of Education, Washington.
O'Connor, Moira (Chairperson) et al (1994). Report of the Program Review Committee for the School of Design. Curtin University of Technology, Perth.
|Please cite as: Shaw, J. and Green-Armytage, P. (1995). Feedback sheets as a form of assessment to support learning. In Summers, L. (Ed), A Focus on Learning, p234-238. Proceedings of the 4th Annual Teaching Learning Forum, Edith Cowan University, February 1995. Perth: Edith Cowan University. http://lsn.curtin.edu.au/tlf/tlf1995/shaw.html|