Teaching and Learning Forum 95 [ Contents ]
Giving away the marks... or taking the 'ass' out of assessment
School of Social Sciences and Asian Languages
Note to readers: The following set of instructions for running a workshop on assessment is offered in the hope of stimulating similar (or even different!) sessions by other practitioners and staff developers. The only way we can make a general improvement in the way that we assess our students is to unpack and challenge and rethink our founding assumptions. If you do use my approach, or a variant of it, I would be interested to hear from you about what your group came up with!
Ever wondered at the end of a semester, after all those essays and exams have rolled in, just what exactly you are doing with all the marks collected? How do you reconcile your general policy on assessment with all those special cases you deal with? Are you objectively assessing the academic worth of students? Are you grading the ability of each person relative to other students in the class? To the improvement each student has made? To the standards of classes in other years? To the standards of classes elsewhere in the department / university? How many of your students fail? Are you just following academic routine and not really worrying? Maybe we should just give the marks away. By this I mean two things: firstly, perhaps assessment is no longer valid or appropriate, given the teaching approaches which more and more of us are using. In this sense we should just give away marking altogether and think through a new process of bringing each semester to a close! But I also mean - perhaps a little more seriously - that maybe marks should not be distributed according to criteria of ability, but of effort; that is, students who work the hardest at their learning should do better than those who cruise through. With this in mind, basically it is over to you! I will facilitate, but I would ask that your work be driven by your own experiences and approaches and that you learn from one another.
Program (see below for more details)
- Why are we here: roundtable introductions of participants, plus discussion of why we are at this workshop and our experiences of assessment [10 minutes]
- Setting the norm of assessment: small group discussions of 'normal' assessment - sources, methods, aims and outcomes [15 minutes]
- Radical departures: continue in small groups to come up with new methods of assessment which are radically different from the 'normal' - how to do it, why, will it work, problems of implementation [20 minutes]
- Sharing ideas: plenary to share ideas from small group work [15 minutes]
At the end
You will find a two page description of my own innovative approach, in which I 'give away' the marks - rewarding effort more than performance and encouraging independent learning.
Why are we here
Roundtable introductions, in which we introduce ourselves and briefly speak about why we are at this workshop. We'll do this as quickly as possible, but with due regard to the need to establish the numerous agendas and assumptions which people bring with them to such discussions.
Setting the norm of assessment
We need to have some ideas about why we assess students in the way we do. This is not simply a matter of discussing what methods to use - though that is usually the level of discussion in which academics engage. Rather, we need to explore where assessment as we understand it comes from; why it works as it does; how methods fit with these sources and aims; and consider what's right and wrong about assessment. At this stage, I want you to discuss a hypothetical 'normal' assessment (very difficult, I know) - rely on your experiences, but try to establish some picture of the dominant ideas of assessment are in the Australian (world?) university system. In small groups, please discuss the following questions - you may concentrate on one or other, or look at all of them, or rephrase them as suits you. There's no need to report on your discussion, since it is a preliminary stage for the main discussion [see below], but you may like to take notes for the next activity
What are the aims of assessment?
What methods do we use?
Where do these methods and aims come from?
What are the strengths and weaknesses of them (in various contexts...)?
It may be that you are quite happy with assessment as we have conceived it in the above exercise - but I am assuming that your participation in this workshop is dependent on a desire to do something different - perhaps radically different. I would like you to discuss, in the same small groups, some aspect of assessment that you believe needs a radical overall (it can be the whole system, or the way essays are marked, or the balance between continuous assessment / exams - please take assessment in its broadest context) and, after discussion, to present a brief statement to the other groups outlining your rationale, your methods, and your intended outcomes. Briefly explore what resistance / difficulties of a practical nature would be encountered in implementing it.
How to proceed
You may like to start simply by brainstorming things you do already, want to do, have read about - then, after five minutes, find a consensus and write it up for the other groups, using OH slides, butchers paper etc.
Try being bold and provocative! Assessment reforms seem to me to suffer from a tendency towards incremental changes, and modifications which miss the opportunity for a really radical alternative. If you don't want to be radical, and maybe even like 'normal' assessment - present a defence of current practices, or indicate the problems of being too radical in making changes.
Keep in mind the need to present a rationale for the change - "aims and outcomes"; and, in particular, note problems - ie not enough time, too few resources - these practical issues need to be addressed.
A learning journal for logic and reasoning
A radical approach to assessment
In 1994, the School of Social Sciences and Asian Languages (SSAL) offered for the first time a new unit on critical thinking, called Logic and Reasoning 200. It was designed to meet two objectives. Firstly, SSAL had agreed to a request by the School of Accounting that it provide a unit which students studying accounting could take to improve their general analytical and reasoning skills. The School of Accounting had formed the view that such a unit would be allow direct enhancement of the skills being developed within its own offerings to meet the needs of the profession. Secondly, SSAL offered the unit to all students in the university, and especially its own, so as to provide them with a similar opportunity to build more directly on the basic development of reasoning skills which tertiary study fosters indirectly.
The pedagogic approach was one of active learning, with a heavy emphasis on independent activity by the students. The nature of critical thinking is such that it cannot be learnt by rote memorisation, nor by student reading and discussion (though this activity is important for understanding the finer details). The main learning activity is in doing practice exercises and problems and then checking the answers and doing some more! Because of the opportunity offered by a new unit and the particular nature of the curriculum, Logic and Reasoning took a highly innovative approach, both to its subject matter and, especially, to its learning style.
The main innovation was in assessment. In first semester, 50% (rising to 70% in second semester) of the assessment was based on the work (short exercises, taking notes etc) students did week by week in preparation for tutorials, in review of lectures and in completing each topic. This assessment was concerned not what students were learning (which assessed via an essay - only in first semester - and exam). Instead, marks were awarded on the basis of how students were learning: were they doing the work effectively? were they self-checking their answers? were they finding extra materials to supplement those provided? were they conscientiously working week-by-week? were they being innovative themselves (ie writing summaries of tutorials)? were they being 'scrappy' in their approach - ie taking a do the minimum possible approach - or were they trying to work with the unit to extend themselves?
This work was collected, week by week, in a large ring-binder called the 'Learning Journal'. It was not a reflective journal, but more of a workbook with some elements of reflective commentary. Students were able to obtain up to a grade of 65% for the learning journal simply by completing compulsory exercises (such as "do these problems", "take notes from the text book, chapter 1"); the actual answers were not assessed, but rather the effort put in - in this sense a correct answer without explanation was seen as worse than a wrong answer in which the student had made an effort to explore the concepts involved. Discrimination between students was based on the effort put in and not on the right or wrong answers. A higher mark came from completion of extra work - either as suggested by the teachers or (and this scored highest marks) on their own initiative.
There were some problems! The amount of work which students produced was staggering and made it impossible to review all of it in detail. Teachers in the unit took a sampling approach which seemed to work fine in terms of grading (some journals were subjected to more detailed analysis after initial grading and the initial grade was confirmed); however some students were less than happy with the amount of feedback they received. What's interesting is that, in actually submitting work (such as lecture notes, tute summaries, self-directed exercises) which would not normally be looked at by a lecturer (and thus be given no feedback), their expectations were raised such that they assumed they would get as much feedback as on, say, an essay. Teachers generally considered that the time taken to mark the work over the whole semester was about the same as for more traditional forms of assessment (particularly when the essay was removed from the list of assessment in second semester), but there were considerable administrative problems in collecting and returning 200 large ring binders.
Feedback problems were addressed to some extent through the distribution of answer sheets but, again, students often failed to follow through as independent learners and check their own work. Those students who did were rewarded with extra marks. In 1995, tutorials will be more directly linked to the journal exercises, ensuring that problems can be looked at in detail after students have attempted them.
Another problem was that some students simply could not cope with being told to think of things to do for themselves, even when shown examples of what extra work they might do. My conclusion is that the Learning Journal is so different from other forms of assessment that they could adjust their mindsets to the new challenge. Nevertheless, it was heartening that many students (including Asian students who are 'notoriously' 'poor' independent learners) did adjust and produced outstanding work.
But there were advantages
There were many advantages. Firstly, it was gratifying to see just how hard students work and to be able to reward them directly for their efforts. Some went to extraordinary lengths, learning much along the way (and there was no need for the teachers to 'check' this - simply by doing the work they learned something). They also provided materials which have been used in later semesters - little examples, cuttings from newspapers, references to other books on reasoning, articles on why students should study reasoning. Looking directly at what students wrote in their lecture notes was also revealing (!)
The journal was also a very good way of organising a wide variety of activities into a single, coherent, continuous whole, easing students' worries about what they had to do and when. Students also appreciated the continuous assessment approach, which took the pressure off them at exam time. While they may have wanted more feedback, they did receive more than if they were assessed through large, end-of-semester exams. The main advantage was that students who worked hard and tried, did receive good marks. In both semesters, the marks in the final exam corresponded closely with the journal marks. Only in the case of students studying off-shore in Hong Kong did the journal not serve as an accurate predictor of success. This can be put down to language problems and the different marking approach of the teacher in Hong Kong.
Overall, the judgement of the four teachers involved in the unit was positive - and the students, in confidential questionnaires also gave a big thumbs up to the Learning Journal.
|Please cite as: Allen, M. (1995). Giving away the marks... or taking the 'ass' out of assessment. In Summers, L. (Ed), A Focus on Learning, p1-4. Proceedings of the 4th Annual Teaching Learning Forum, Edith Cowan University, February 1995. Perth: Edith Cowan University. http://lsn.curtin.edu.au/tlf/tlf1995/allen.html|
[ TL Forum 1995 Proceedings Contents ]
[ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [email@example.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1995/allen.html
Last revision: 16 Apr 2002. © Edith Cowan University
Previous URL 11 Feb 1997 to 15 Apr 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf95/allen1.html