Category: Professional practice
|Teaching and Learning Forum 2013 [ Refereed papers ]|
Sandra E Carr, Zarrin Seema Siddiqui, Diana Jonas-Dwyer and Susan Miller
The University of Western Australia
Email: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com
How much feedback is enough for undergraduate students in medicine, dentistry and health science? Feedback is meant to provide students with sufficient information on their performance in a given activity so it guides their future performance in similar activities. Different models of feedback have been described in the literature, some more comprehensively than others, with formative assessment and reflective practice being the underlying themes for all of the models. Data gathered from the undergraduate courses of Medicine, Dentistry, Health Sciences and Podiatric Medicine raised awareness of the common issues related to insufficient and inadequate feedback for students about their performance. These findings shaped the development of five targeted pilot projects. The pilot projects looked at ways to improve verbal and written feedback through formal and informal mechanisms. This paper describes how one Faculty is approaching the difficult task of shifting the established culture of offering limited feedback to students and enabling students to ask for feedback that is appropriate and useful for them. In doing so it offers ideas for other Faculties wanting to enhance the feedback mechanisms for students.
Different models of feedback have been described in the literature, some more comprehensively than others; however formative assessment and reflective practice are the underlying themes for all of the models (Brukner, Altkorn, Cook, Quinn & McNabb, 1999; Milan, Parish & Reichgott, 2006; Mohanna, Wall & Chambers, 2004; Nicol & McFarlane-Dick, 2006; Oxley, 1997; Pendleton, Schofield, Tate & Havelock, 1984). The Milan, Parish and Reichgott model (2006) is based on the Trans-theoretical model of change (see Figure 1). This model incorporates a learner's readiness to change in response to feedback. Learners move through stages from pre-contemplation to contemplation, preparation and action. The model recognises that movement through the stages may be an iterative process and is not always a smooth progression from one stage to another. To apply the model the teacher determines which stage the learner is at so that they can use the suggested interventions for that stage. For example, if a student is not performing well, and at the same time is not aware of the issue, or is in denial, the student is in the pre-contemplation stage. This implies that the intervention will be directed towards making them aware of the issue and hence move them on to the contemplation stage. However, if a student is already aware of the issue and requires guidance to improve their performance he/she has already demonstrated a commitment to change and is in the preparation stage. The final stage is when the student takes action to resolve the issue.
Other models relate more to the process of providing feedback and define steps that can be followed. These include Pendleton's rules (Pendleton et al., 1984), the Standard Committee of Postgraduate Medical and Dental Education (SCOPME) model (Oxley 1997), the Chicago model (Brukner et al., 1999), the six-step problem solving model (Mohanna, K., et al., 2004) and the Guidance and Feedback loop (Hounsell, McCune, Hounsell & Litjens, 2008).
Figure 1: Stages of the transtheoretical model (Milan, Parish & Reichgott, 2006)
Nicol and McFarlane-Dick's Model of Formative Assessment and Feedback is based on the students' central and active role in the feedback processes and recognises that the student must have internalised and understood the feedback in order for change to occur (Nicol & McFarlane-Dick, 2006).
The effectiveness of feedback is reported to be reliant on a number of factors which include the mode, environment, timing, abilities and skills of the staff in providing feedback, and the readiness to accept and act on feedback by students (Ende, 1983). There are different modes in which feedback can be provided; for example from self, peer or teacher. Self feedback is characterised by reflective activities and may include information from formative assessments. Peer feedback occurs when students either provide feedback to other students about their work or receive feedback from fellow students on their own work (Brinko, 1993). Both teacher and peer feedback may be provided in one-to-one, small group or large group situations. Similarly, teacher feedback may take the form of face to face or remote interaction through the use of technology, between students and the faculty. When students are able to select the way in which feedback is conveyed it is more effective (Brinko, 1993). The environment in which feedback is given also plays an important role as it can impact the effectiveness of the feedback (Farnill, Gordon & Sansom, 1997). This implies a climate of trust, safety and confidentiality, based on a supportive relationship.
Feedback can be immediate, delayed or reinforced. Brinko (1993, p.580-581) states that "feedback is more effective when given as soon as possible after performance" and "it is more effective when it is considered as a process and not a one time quick fix."
Hattie (1997) reviewed 87 meta-analyses of studies on student achievement and found that feedback produced the most powerful single effect on achievement for student learning. Carless (2006) in his study, which spanned over eight publicly funded universities in Hong Kong, found that students and tutors had differing perceptions about the level of detail, and the usefulness of feedback provided; eg, the tutors believed they were providing detailed feedback but the students thought differently. This finding is supported by our own experiences. The results of the Course Experience Questionnaire (CEQ) for undergraduate courses in the Faculty of Medicine, Dentistry and Health Sciences (FMDHS) at The University of Western Australia (UWA) showed low scores for 'Good Teaching' and 'Learning Community'. Extensive program evaluation for medicine, dentistry and health sciences over four years has noted that students do not feel they received adequate feedback from their teachers during the course.
This issue therefore raises the question "How much feedback is enough for undergraduate students in our Faculty?"
The researchers followed an action research approach. Action research is defined by Stringer as
... a collaborative approach to inquiry or investigation that provides people with the means to take systematic action to resolve specific problems (2007 p.8).Action research methodologies pursue action or change, and research outcomes at the same time. Action research is cyclical in nature, participatory (is conducted by the people running the program) responsive to the emerging findings and encourages critical reflection (Atweh, Kemmis & Weeks, 1998). The steps involved in action research are to PLAN --> ACT --> OBSERVE --> REFLECT --> and then PLAN, and so on.
This paper describes how the Faculty (FMDHS) modified the processes used to provide feedback to students about their academic performance.
General understanding of feedback
Feedback was defined by staff and students in a number of ways, and in general definitions were considerably different between staff and students. Staff tended to see feedback as an opportunity to inform students about their progress and performance of their learning, whereas students were more inclined to define it as an opportunity to receive suggestions for ways to improve in subsequent assessments and performance. By the end of the student focus-groups, students realised that feedback is a complex area that required both staff and students to have a shared understanding of the purpose and intention of giving and receiving feedback. Both recognised their lack of understanding about feedback. Students also noted that there were distinct types of feedback; negative and positive, with negative feedback being provided to them most frequently. Negative feedback was described as not being constructive and lacking in the provision of strategies for improving performance.
Specific perceptions of feedback mechanisms
Based on the data collected, feedback is provided by staff in a range of settings including one-to-one verbal feedback. However most commonly, students receive feedback when assignments or examination results are handed back, in the written form or verbally in a large group setting. Students would like feedback to be more timely, as in many instances it is provided too late, if at all. Staff and students lacked an awareness of the different types of feedback available and how to implement these in their current situation. Staff were not aware of the different models of feedback available and many staff welcomed the information provided on models of feedback, and the opportunity to receive assistance with the implementation of different feedback models.
Personal skill level
Most students agreed they are not very good at eliciting or providing feedback to their teachers. Students identified the need to receive more training in how to elicit feedback. Many staff were comfortable with the amount and models of feedback they were using, but commented that they would prefer to be able to give more individualised feedback to students and were not able to easily identify the steps they use to provide feedback for students.
Gathering the data revealed the complex nature of student feedback. Factors that required consideration included the models and modes of feedback, the feedback environment, the timing of feedback, and the skills and abilities needed by staff and students to be able to give and receive effective feedback. Several exemplars were identified while gathering data. One example from the Year 5 Obstetrics and Gynaecology unit in the medical course is described. Halfway through the ten week attachment, the students are expected to meet with a clinical staff member. The aim of the meeting is to discuss the student's progress in the clinical learning environment and to highlight, in a reflective manner, their strengths and areas of their performance that they need to work on. The clinical teacher keeps a record of the conversation and the actions the student will take, but no marks are allocated.
Another example is from a Year 2 Health Science unit where students receive structured written feedback on a significant written assignment. The feedback is designed to enhance the students' skills in writing and presenting an argument around a topic. The students submit their written work, receive documented written feedback and are able to incorporate their learning into the final summative assessment.
Baseline data around students' perceptions of their ability to provide and receive feedback were gathered from the students before each of the pilot projects and are summarised in Table 1. There is a variation in agreement (5 point scale, 1 = excellent, 5 = poor) across the different undergraduate courses. Generally, students rated their skills in providing and eliciting feedback as "good", with Dentistry students rating their pre-intervention skills the highest and Medicine students feeling least comfortable to ask for feedback.
Feedback sessions in simulated learning environment: Operative dentistry
In the second year Operative Dentistry unit, the clinical assessment forms were redesigned to make the assessment rating criteria explicit. This included the provision of articulated, standardised verbal feedback for the students. This unit had identified a lack of transparency in the assessment process and had high rates of student failure. A new unit coordinator was keen to make efforts to improve the student experience and the process was seen to be very positive by the students and course director. The coordinator played a key role and took the initiative to define explicit criteria for each of the exercises undertaken during the simulated laboratory setting, and ensured students were assessed and provided with feedback about their performance in a timely manner.
|Skill providing feedback to peers||2 (2, 3)||3 (2, 3)||3 (2, 3)||2 (2, 3)||3 (2, 3)|
|Skill eliciting feedback from staff||2 (2, 3)||2 (2.25, 3.75)||3 (3, 4)||3 (2, 3)||3 (2, 3)|
|Skill providing feedback to staff||3 (2, 3)||4 (2.25, 4)||3 (2.75, 4.25)||3 (2,3)||3 (2.5,3)|
|Receiving feedback helped me learn||1 (1, 2)||2 (1.5, 2)||2 (1, 2)||2 (1, 2)||2 (1, 2)|
|Who should provide feedback in this unit (most frequently cited answer)||Peers and Teachers||Peers, teachers and other health professionals||Peers, teachers and other health professionals||Peers, teachers and other health professionals||Peers and teachers|
|Preferred forms of feedback||Verbal and written||Verbal and written||Verbal and written||Written and online||Verbal, written and online|
|How often should feedback be received in this unit||Daily||Weekly||Daily||Weekly||Weekly|
|How often will you receive feedback in this unit||Between daily and weekly||Weekly||Middle of unit||Monthly to middle of unit||Weekly|
|I am comfortable asking feedback from peers||1 (1, 2)||2 (2, 2.5)||2 (2, 2)||2 (2, 2)||2 (2, 2)|
|I am comfortable asking for feedback from teachers||1 (1, 2)||2 (2, 4)||3 (2, 4)||2 (2, 2)||2 (2, 2)|
|* 5 point scale: 1 = excellent, 5 = poor. IQR = inter-quartile range.|
There was an opportunity for students to self-assess their performance, as well as receive feedback verbally and in the form of a numerical score. Because the feedback was provided by several tutors, staff development sessions were conducted outlining assessing the students using predefined criteria.
Feedback in small group sessions: Psychiatry problem based learning (PBL) sessions
As a facilitator of one of the PBL groups, the unit coordinator agreed to trial a model of providing feedback in a small group setting. A paper-based guide was prepared for the coordinator by one of the project team. Both students and the unit coordinator reported that it was productive for learning. However, both students and the tutor found it difficult to say something in every session. This may have been because they were new to the process and were not yet comfortable with areas in which feedback could be offered. This experience highlighted the need to orient students, as well as facilitators, towards the feedback process and its' utility, before they begin PBL sessions.
Podiatric Medicine was selected to conduct a pilot, as the unit coordinator was new to teaching and the written assignment needed to be reviewed. One of the project team assisted the unit coordinator in developing the criteria for the assignment. The newly created rubric contained explicit criteria which helped students to understand what was required. A paper-based resource containing examples of the types of written feedback that could be given was also provided. Each student received written feedback on their assignment and was provided with an opportunity to discuss their feedback with the unit coordinator.
|Feedback in simulated learning environment||Dentistry||Operative Dentistry||10||52||Assessment criteria defined for each exercise.|
|Small group feedback||Medicine||Psychiatry||1||10||Feedback in PBL sessions implemented and resource for tutors created.|
|Written feedback||Podiatric Medicine||Podiatric Medicine||1||38||Modified assessment criteria and created marking guide.|
|Large group face to face feedback||Health Science||Foundations of Epidemiology||2||50||Recorded common errors and strengths and held a class discussion about them. Placed notes on the web for students with an opportunity for online discussion.|
|Verbal feedback on clinical learning||Medicine||Medicine||10||40||Student and staff training on how to elicit and give feedback.|
This part of the project highlighted a need to discuss the type of feedback that will be provided about an assignment with first year students, prior to them completing the task, and the need to provide opportunities for students to discuss their feedback with the unit coordinator.
Large group face to face: Health science
The unit coordinator for Foundations of Epidemiology agreed to trial a face-to-face large group setting, providing feedback on a written assignment. The purpose of the feedback was two-fold; to provide formative feedback on the assignment to allow re-submission, and to assist students with applying the skills and knowledge learned in the assignment in their final assessment, the end-of-semester exam. The written comments and common points raised when marking the student's individual assignments were recorded. After all of the students had received their assignments back, and prior to the exam preparation week, a face-to-face discussion about the common strengths and weaknesses found in the assignments was held with the group. Following the discussion, an exemplar was placed online via WebCT, for students to access. A resource for teachers that outlined the process and described the experience was also produced, to encourage wider use of the method.
Feedback on clinical learning: Clinical medicine
Staff and student training sessions were offered about giving, receiving and eliciting feedback in the clinical setting. Students in the first rotation were included in the development of the guidelines regarding eliciting feedback. These guidelines were used in subsequent discussions with another group of students. The process of writing the guidelines meant students engaged more actively in the process, valuing the opportunity to develop a resource for themselves and others. Students in the second rotation were more passive recipients of information. Staff were reluctant to attend the training sessions, despite it being offered in the hospital setting, with lunch provided and the Head of School's endorsement.
As part of the situation analysis, we asked staff and students about the current feedback mechanisms and models being used in the Faculty. The complex nature of student feedback was revealed during this analysis. Factors that required consideration included the models and modes of feedback; the feedback environment; the timing of feedback, and the skills and abilities needed by staff and students to be able to give and receive effective feedback. These findings are supported by the literature, which documents the disconnect between the understanding of feedback by students and by staff, and the students desire to receive objective, non-judgemental feedback more frequently, individually and with explicit examples on how to improve (Brinko, 1993).
From the data collected, the project team identified common issues across the Faculty related to insufficient and inadequate feedback for students about their performance. These findings shaped the development of five targeted pilot projects. The pilot projects looked at ways to improve verbal and written feedback through formal and informal mechanisms using self, peers and unit coordinators, and were trialled in a range of settings across the undergraduate courses in the Faculty.
We succeeded in raising awareness of the issues surrounding the provision of adequate feedback for students on their performance. Initially it proved difficult to engage staff in training, so we revised the structure of the staff development programs to support learning, and focused on three areas of effective staff development: (1) the needs and characteristics of staff; (2) the program characteristics of purpose, structure, content, process and follow-up; and (3) the organisational characteristics that contribute to or support effective staff development. A recurring series of centrally run faculty development sessions have now been introduced which attract more staff. In addition, one School has implemented local training for their clinical academic staff. Difficulty engaging the staff proved to be the biggest limitation of the project. It was also difficult to follow up students over a longer period of time, thereby limiting the response to subsequent surveys.
The process of improving feedback was also meaningful for the students. Students enjoyed the experience in the pilot projects and increased their awareness of the use of feedback in the learning environment. The year after the initial project, Year 4 medical students undertook training in "how to elicit feedback in the clinical learning environment"; Year 6 medical students have access to the resource developed - "how to elicit feedback", and Year 3 dentistry students and their teachers are participating in a project aimed to increase the level of reflective feedback for students using an audience response system. Faculty course evaluations also reflected an increasing level of student satisfaction with the provision of helpful feedback over the following two year period. The Faculty committed ongoing funding to implement the pilot projects within other units in the undergraduate courses and established a continuing staff development program related to giving and receiving feedback. The relevance of this approach has been noted by the wider university community and the Centre for the Advancement of Teaching and Learning (CATL) at UWA has adopted a similar approach to develop targeted pilot projects in each Faculty.
Suggestions for future work in this area include, evaluating the effectiveness of different types of feedback and developing validated tools for measuring the effectiveness of giving feedback. The motivation of students to seek feedback about their performance, and influences on staff behaviour related to giving feedback are of interest and could be areas for future research. Why do some staff offer regular, structured, specific feedback and others offer little or no feedback? Does this link to their teaching philosophy or their own preferred learning style or experiences, or workload? Further work is also needed to explore how gender differences affect giving (language used etc.) and receiving feedback, and how these differences should be incorporated into staff and student development programs.
The process of shifting the culture of the learning environment to one where students are actively encouraged to engage in the feedback process has begun in our Faculty, and the next cycle of change is underway.
Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. http://dx.doi.org/10.1080/0969595980050102
Brinko, K. T. (1993). The practice of giving feedback to improve teaching: What is effective? The Journal of Higher Education, 64(5), 574-593. http://www.jstor.org/stable/2959994
Brukner, H., Altkorn, D. L., Cook, S., Quinn, M. T. & NcNabb, W. L. (1999). Giving effective feedback to medical students: A workshop for faculty and house staff. Medical Teacher, 21(2), 161-165. http://dx.doi.org/10.1080/01421599979798
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233. http://dx.doi.org/10.1080/03075070600572132
Ende, J. (1983). Feedback in clinical medical education. The Journal of the American Medical Association, 250(6), 777-781. http://dx.doi.org/10.1001/jama.1983.03340060055026
Farnill, D., Gordon, J. & Sansom, D. (1997). The role of effective feedback in clinical supervision. Australian Journal of Clinical and Experimental Hypnosis, 25(2), 155-161.
Hattie, J. A. (1997). Identifying the salient facets of a model of student learning: A synthesis of meta-analyses. International Journal of Educational Research, 11(2), 187-212.
Hounsell, D., McCune, V., Hounsell, J. & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research and Development, 27(1), 55-67. http://dx.doi.org/10.1080/07294360701658765
Milan, F. B., Parish, S. J. & Reichgott M. J. (2006). A model for educational feedback based on clinical communication skills strategies: Beyond the "feedback sandwich". Teaching and Learning in Medicine, 18(1), 42-47. http://dx.doi.org/10.1207/s15328015tlm1801_9
Mohanna, K., Wall, D. & Chambers, R. (2004). Teaching made easy: A manual for health professionals (2nd ed.). Abingdon: Radcliffe Medical Press.
Nicol, D. & McFarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. http://dx.doi.org/10.1080/03075070600572090
Oxley, J. (1997). Appraising doctors and dentists in training. British Medical Journal, 315, S2-7116. http://dx.doi.org/10.1136/bmj.315.7116.2
Pendleton, D., Schofield, T., Tate, P. & Havelock, P. (1984). The consultation: An approach to learning and teaching. Oxford: Oxford University Press.
Stringer, E.T. (2007). Action research (3rd ed.). California: SAGE Publications Inc.
|Please cite as: Carr, S. E., Siddiqui, Z. S., Jonas-Dwyer, D. & Miller, S. (2013). Enhancing feedback for students across a health sciences faculty. In Design, develop, evaluate: The core of the learning environment. Proceedings of the 22nd Annual Teaching Learning Forum, 7-8 February 2013. Perth: Murdoch University. http://ctl.curtin.edu.au/professional_development/conferences/tlf/tlf2013/refereed/carr.html|
Copyright 2013 Sandra E Carr, Zarrin Seema Siddiqui, Diana Jonas-Dwyer and Susan Miller. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.