Teaching and Learning Forum 96 [ Contents ]

Can altering the assessment for a unit encourage students to engage with content and processes, and be reflective about their learning, rather than focus on superficial detail?

Alison Bunker
Lecturer, Kurongkurl Katijin
Edith Cowan University

The aim of this paper is to describe an attempt to improve student learning in a course bridging to Tertiary studies. The course is for students from a sector of the community where typically there are negative educational experiences and, associated with this, low levels of educational achievement in formal education. Consequently this sector of the population is very under-represented in the university student population. The bridging course aims to empower students with the skills and attitudes required for successful tertiary study.

Because many of the students on the course have had unsatisfactory school experiences, it is important that they have the opportunity to learn in a meaningful and enjoyable way. Quality in the learning process involves intellectual engagement with the material and a personal commitment of time and effort. Learners that engage in this way have qualitatively better learning outcomes, and experience a sense of empowerment (Biggs & Moore, 1993).

The unit, Foundations of Statistics, is one of the last units of the course. It is based on the manipulation of complex abstract concepts, and in the past students have found the unit very difficult. The teaching/learning process involved exposition, algorithms, worked examples and practice examples. Students were assessed on their ability to apply the algorithms to further examples. For most students this unit was "one more hoop", a test of their staying power and motivation to complete the bridging course. The unit focussed the students on performing calculations and drawing graphs, skills which, with the wide availability of statistics software packages are no longer required for researchers. In addition, it failed to highlight the fundamental principles of statistics which are essential to all research. From conversations with students it is clear that few felt the unit had any relevance, and no-one indicated enjoying it. Further their assignments and examination answers suggested little understanding of the content.

The main assumption underlying my work as a teacher is that students actively construct their own knowledge. Students are not "empty vessels" to be filled with knowledge, but actively engage with the learning material to construct new knowledge. It follows then that: "If students are to learn desired outcomes in a reasonably effective manner, then the teacher's fundamental task is to get students to engage in learning activities that are likely to result in their achieving those outcomes ... It is helpful to remember that what the student does is actually more important in determining what is learned than what the teacher does." (Shuell 1986 in Biggs & Moore 1993.) This has a number of implications. Firstly what students believe about learning will affect they way they go about their learning, and this in turn will affect the learning outcome (van Rossum & Schenk).

Marton, Dell'Alba & Beatty (1993) have identified six qualitatively different conceptions of learning which can be broadly grouped into two. The first group includes the conceptions which focus on learning as a quantitative increase in knowledge and are concerned with reproducing what has been learned. The other group involves conceptions of learning which focus on meaning and coherence. Students who consider learning to be a quantitative increase in knowledge are concerned with remembering and reproducing that knowledge in some way. They tend to focus on surface features in the learning material. In contrast, students who view learning as a process of developing meaning expect to have to delve deeply into the learning material in order to understand. Students with this second view achieve higher grades and retain the knowledge longer (Watkins and Hattie 1981).

Another factor affecting students' learning is their motivation to learn in a particular context. Students may be motivated by extrinsic consequences, for example, in this context students may be extrinsically motivated by the benefits associated with successful completion of the course. Alternatively, students may be motivated by the increase in self-worth which comes from successful achievement. Or finally, students may be intrinsically motivated through the subject matter itself, these students find the challenge of learning enjoyable and therefore motivating.

A third factor affecting the learning outcome is a student's approach to learning. This can be considered a combination of "the will [motive] and the skill" (Pintrich & de Groot in Biggs & Moore 1993, p. 310). The skill in learning involves being able to choose an appropriate strategy from a range of strategies which best matches both the purpose in learning and the learning task.

Three different types of approach have been identified, they are differentiated by the intention of the learner (Marton & Saljo 1976, Entwistle & Ramsden 1983, in Biggs & Moore 1993). A surface approach is associated with learning strategies which focus on remembering facts, with cramming for an examination and a superficial understanding of the content. A student choosing this approach will do the minimum required for a satisfactory outcome. The approach is typically corner-cutting and is used where students feel overloaded by content or limited by time. It is a sensible "fall back" position (Biggs & Moore 1993) where the learner feels he has little control over the learning process. Students choosing a surface approach often "miss the point" of a learning activity (Crooks 1988) simply because they aren't looking for it. This is often the chosen approach where the motivation to learn is external to the learning process.

In contrast, the student who chooses a deep approach is interested in the subject matter, actively seeks meaning, employs strategies aimed at understanding and is prepared to invest time and effort into it (Biggs & Moore, 1993). Students choosing this approach tend to see understanding as a necessary part of their learning either for their own satisfaction, or in order to achieve their external or achievement goals. Typical deep strategies involve seeking meaning, relating new material to what is known, searching for main ideas, and posing questions.

An achieving approach is linked to the achieving motive. A student choosing this approach will do whatever is necessary to get the high marks desired. Therefore they will engage deeply with the content if that is what is required. This approach is very goal orientated and strategies tend to concentrate on effective use of time and other resources.

The choice of approach to learning affects the quality of the learning outcomes (Biggs & Moore 1993). Students who choose a deep approach perform at a higher qualitative level, but do not always achieve higher marks (Trigwell & Prosser in Biggs & Moore, 1993), this of course depends on what the marks are awarded for. In addition, because they do not have a strong external goals, "deep" learners may negotiate their own way through the content and not "arrive" at the destination they are supposed to. A good combination for success in learning institutions is the deep-achieving approach.

Thus we can consider a student's approach to learning as a description of the quality of interaction with the learning task. We have seen how it is affected by personal factors such as motivation and conceptions of learning, but it is also context dependent.

Most students on the bridging course are motivated to study by the opportunity to gain entry to university, or to be personally successful at study. Thus they are largely externally motivated to study the Foundations of Statistics unit, and so we could expect a surface approach to be the chosen approach. Other students with a stronger personal achievement motivation could be limited by access to surface strategies only. Further this unit previously had a large content component and involved teacher exposition, demonstration and practice examples. The content and pathway through the content were tightly prescribed and students were not required to relate the new knowledge to their existing knowledge. These elements of the teaching context would have the effect of reinforcing any quantitative views of knowledge, encourage surface strategies such as rote learning of key points and algorithms, and minimal searching for understanding. There is a "blind faith" about this approach to learning, the student has little sense of control over the learning process and hopes that accurate reproduction of content will get them through the assessment procedures.

If the existing teaching context did much to encourage pre-existing surface strategies, and to reinforce surface approaches and quantitative views of learning, how can it be changed in order to encourage students to use deeper approaches to their learning. Ramsden, Martin & Bowden (1989) have indicated that it is the relationship between the learner and the task which needs to be examined. Biggs' analysis (1993) of the classroom as an open system clearly illustrates the factors which affect this relationship. They include student characteristics and the teaching context with interactive feedback between the components. Within the teaching context, "the quickest way to change student learning is to change the assessment system." (Elton & Laurillard in Biggs & Moore, 1993 p. 459).

My aim for students in this unit was to improve the learning experience. Given that most students approach learning superficially with the intention of "‘getting the right answers' to the exclusion of knowing how to get it and of what it means when it has been obtained" (Ramsden, 1988 p 20), they look to the assessment part of a unit to determine what is required to pass the unit. The content, format, time frame, marking scheme and feedback all give messages to the learner about what is considered important and determine how students set about the learning task (Marton and Saljo, 1984 p 48). The learning intended by the teacher and the actual learning can be two different things.

"It is the student's subjective perception of the requirements of teachers - the context of learning - that is the driving force behind much of their learning" (Ramsden, 1988 p 21)
Conversations with tertiary students also indicate that it is the assignments in a university unit which focus the pragmatic student on the material to be learned. Therefore assignments have the potential to be the point at which students begin to engage meaningfully with the content and when learning starts to happen. However, because of time constraints, in practice the work carried out for an assignment is often only a superficial tour round the perceived content. The challenge to a teacher is to design assessment activities which match the aims and objectives of a course and encourage the kind of attitudes to learning which the deep approach is symptomatic of.

The previous assessment for this unit was through more examples of a similar kind to those already practiced. To be successful, students had to follow the "recipe" presented in the class, to be successful in the examination they had to be able to remember and reproduce the recipe. This focus on superficial details without understanding would encourage a superficial approach.

The new assessment was designed to address the following aims of the unit and course: students should develop (i) an understanding and retention of some basic statistical concepts and processes, (ii) an awareness and some experience of deeper strategies and approaches to learning. This second aim is an important aim on a bridging course to tertiary study. Students who engage more deeply with the learning material have more control over their learning, are more likely to get better results, and are more likely to develop an intrinsic and continuing motivation to learning.

"The deep approach is associated with personally valued subjects"(Biggs & Moore 1993 p 321) that is, students have to care. They have to be interested in the topic, want meaningful results, have an interest in the mistakes they make and be motivated to get it right. They have to be prepared to invest the time and effort required to engage meaningfully with the content of the course. They must believe it will be worth their while. These characteristics are typical of students who choose deep approaches.

For the new assessment, students would be required to carry out a mini research project. This involved designing and using a questionnaire, summarising and analysing the results, and drawing conclusions. Some fundamental ideas in statistics had to be dealt with, for example population, sample, and bias. Students were free to choose what they investigated, they were encouraged to choose something that reflected a genuine interest. It was hoped that a genuine interest in the topic of their choice would engage students and the processes involved in getting answers would therefore be perceived as relevant and not trivial. The factual learning necessary to carry out the project would become a means to an end. The topics investigated ranged from, for example, "What are the best things to serve at morning tea?" to "Does the Aboriginal Medical Service provide an adequate service?"

Since order and organisation, coherence and goal direction, as perceived by pupils are constantly associated with higher levels of achievement (Fraser 1986 in Ramsden et al 1989), the assignments task were presented in a clearly structured way. Students were given clear guidelines about the purpose of the assignment, how to tackle it and how it would be assessed. The goal was a finished piece of research and an understanding of the processes involved. The project was made up of three assignments corresponding to three phases of the research process, they cumulated to form the total project, giving order, organisation and coherence to the work. Discussions with students indicated that being able to see were the work would lead them was motivating.

Generally, students often submit work which is in effect a first draft and rarely have time to revisit it. To encourage students to remain involved with the content in an ongoing way - a deep strategy, work was marked and returned to students with feedback as quickly as possible. Students were encouraged to rework and resubmit the assignment for extra marks. Each assignment became the first part of the next assignment and was remarked if it had been reworked. The students have commented that found the formative rather than summative assessment encouraging. Students' engagement with the task was also maintained through their initial ownership of the problem and through ongoing dialogue between students, tutors and the lecturer about the task.

Feedback to students was either in the form of questions where insufficient detail had been given, or a discussion of the implications of a particular statement to verify the writers intentions. The feedback was imagined as one side of a dialogue. The questions asked were from a list of generic questions compiled by King (1990) designed to elicit explanations either by making connections between the new ideas, or linking new knowledge to existing knowledge. Explaining material helps "learners to construct and elaborate on their representations" (King 1990 p 667). Students were given this list of generic questions and encouraged to use them themselves. Follow up discussions by ‘phone were had with the student and tutor. The intention of the dialogue in these discussions was clearly for making meaning, rather than transmitting messages (Evans and Nation 1989 in Morgan 1993 p 84). Through assessment feedback students were encouraged to see problems and asked to link new knowledge with prior knowledge in a critical and reflective way, for example by relating "common sense" to what they were finding out. They were also encouraged to reflect on the validity of their findings.

Marks were awarded on the basis of the structure of a student's response. Following Biggs & Collis' (1989) work on the SOLO taxonomy which looks at the structure and complexity of student responses, a very simple ad hoc scheme was introduced where for each subsection of the assignments, students could get one mark for a relevant point, two marks for listing a number of points, and three marks if they related the points and made connections between ideas. Students were told that they could get three out of three and that to do this they should write at a level which relates ideas together and shows they understand why the basic concepts are so important to statistics. Mistakes were interpreted as learning experiences, by allowing students to make mistakes, the importance of some of the basic concepts could be focused on. Students got marks for showing understanding of why there might be a "right" way to go about research rather than loosing marks by getting something "wrong".

By hooking into students' motivation to succeed and rewarding the results of deeper strategies through the marking scheme, meaningful engagement with the content was encouraged. Since meaningful learning is itself enjoyable, this experience allows for the possibility that students will come to enjoy learning for intrinsic reasons and move towards a more qualitative conception of learning.

Being an intentional learner and watching over the process gives the learner control and direction. It is an important component of the deeper approach, critical and realistic "reflection transforms error into a positive learning experience" (Biggs & Moore 1993 p 308). With this in mind, the final part of the project was a 500 word guided reflection on the mini-research project. This was marked on the same basis as other work: related ideas scored higher marks.

For students to maximise the benefit from the experiences offered in this unit, there should be consistency with regards to the central message about meaningful engagement with the content. With this in mind, the message was carried through into the exam questions and to the way the students were introduced to the exam. The exam is an important component of the bridging course, since exams still feature largely in tertiary studies. The exam would also have value in terms of learning for this unit by offering one more opportunity to attend to the topic. An "open-book" exam was considered inappropriate because this could easily reinforce a surface approach with students looking for the answer in the book whereas the central themes for this unit have to be constructed by each student. Students were shown a "mock" exam paper which introduced the format of the exam, the type of questions and weighting of the marks. The exam questions required students to demonstrate understanding of the processes involved in collecting and analysing data. The final exam question asked them to write reflectively on what they had learnt about these processes. This was marked in terms of understanding of the fundamental concepts and relationships.

Having control over one's learning is important in encouraging a deep approach: "Assuming control over oneself ... is a prerequisite to metalearning activity" (Biggs & Moore 1993 p 317). Where possible throughout the process, the locus of control was with the students. The content, which was to be a medium for their learning of fundamental statistical ideas, was their choice. Students had to accept responsibility for finding something worthwhile spending their time on. They were seen as partners in the learning process bringing with them a "common sense" statistical knowledge which they were encouraged to use and re-evaluate as they made sense of the unit.

Tasks which are too difficult or too easy encourage a surface approach to learning , and there is an optimal level of challenge for each student (Ramsden et al. 1989). Students were able to select for themselves the level of difficulty that they approached this task with. Anecdotal evidence suggests that the students who found it difficult where the ones choosing to tackle it most deeply, however they were also highly motivated, and performed the best.

The encouragement of intrinsic and continuing motivation in learning were considered important outcomes of the unit. Through an interesting assignment, student interest and commitment was engaged. Most students reported a positive learning experience, which was hard work but worthwhile. These effects are important on a bridging course. In addition, the examination answers this semester indicate a much better grasp of the fundamental concepts of statistics.

There are still a number of issues to be dealt with in this unit. For example, there is a lot of "background noise" in the unit in the form of the readers. These readers were written for the old assignments and therefore contain a lot of detail irrelevant to the new course. They present a content overload which mitigates against a deep approach. When the course runs again, two more modules will be removed. However, in spite of all the measures taken to increase the likelihood of a deep approach being taken, it is still possible for students to tackle the course superficially by following all the cues about what is required and to achieve a reasonable mark without engagement with the content. Such students are expert at doing the minimum required to get a satisfactory grade.

In conclusion, the key to changing how students tackle learning lies in the teaching context. It is the quality of the engagement with the learning material which is critical in developing quality learning outcomes. Students bring past experiences and present intentions to the situation, the teacher brings expertise. Many factors in the teaching context work against a deep approach, but changing how a unit is assessed can offer an effective mechanism for changing students approaches to learning.

References

Biggs, J. & Collis, K. (1989). Towards a mode of school-based curriculum development and assessment using the SOLO taxonomy. Australian Journal of Education, 33, 151-163.

Biggs, J. & Moore, P. (1993). The process of learning. Sydney: Prentice Hall.

Crooks, T. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

King, A. (1990). Enhancing peer interaction and learning in the classroom through reciprocal questioning. American Educational Research Journal, 27, 663-687.

Marton, F. & Saljo, R. (1984). Approaches to learning. In Marton, F., Hounsel, D. & Entwistle, N. The Experience of Learning. Gt Britain: Scottish Academic Press (Ch. 3 pp 36-55).

Marton, F., Dall'Alba, G. & Beaty, E. (1993). Conceptions of learning. International Journal of Educational Research, 19, 277-300.

Morgan, A. (1993). Improving your students learning. London: Kogan Page.

Ramsden, P. (1988). Studying learning: Improving teaching. In Ramsden, P. (Ed), Improving learning: new perspectives, (Ch.2 pp13-31). Great Britain: Kogan Page.

Ramsden, P. Martin, E. & Bowden, J. (1989). School environment and sixth form pupil's approaches to learning. British Journal of Educational Psychology, 59, 129-142.

Watkins, D. & Hattie, J. (1981). The learning processes of Australian university students: Investigations of contextual and personological factors. British Journal of Educational Psychology, 51, 384-393.

van Rossum, E. J. & Schenk, S. M. (1984). The relationships between learning conception, study strategy and learning outcome. British Journal of Educational Psychology, 54, 73-83.

Please cite as: Bunker, A. (1996). Can altering the assessment for a unit encourage students to engage with content and processes, and be reflective about their learning, rather than focus on superficial detail? In Abbott, J. and Willcoxson, L. (Eds), Teaching and Learning Within and Across Disciplines, p33-39. Proceedings of the 5th Annual Teaching Learning Forum, Murdoch University, February 1996. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf1996/bunker.html


[ TL Forum 1996 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1996/bunker.html
Last revision: 12 Apr 2002. © Murdoch University
Previous URL 21 Dec 1996 to 12 Apr 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf96/bunker33.html