Teaching and Learning Forum 2000 [ Proceedings Contents ]

Course evaluation on the web (CEW): A 'learning community' tool to enhance teaching and learning

Leon Straker and Ray Smith
School of Physiotherapy
Curtin University of Technology
    Course Evaluation on the Web (CEW) is a web based system for collecting, synthesising and reporting student and teacher perceptions on aspects likely to influence teaching and learning. It aims to facilitate student, teacher and course manager reflection on a course. This paper will describe the development and current use of CEW. The elements in the "Learning Community" model on which CEW is based will be defined including the student, their peers, the teacher(s), the unit/subject material, the unit/subject organisation, the local environment and the broader environment. The technical and administrative systems established to support CEW within the School of Physiotherapy will be outlined together with how CEW meshes with the other administrative systems established with a role in course development. The nature of CEW will be described including the ratings feedback by students, the qualitative comments feedback by students, the unit/subject perceptions of teachers and the response of unit/subject coordinators. Coordinator responses cover planned changes to the unit/subject and why some suggestions will not be implemented. The use of CEW will be elucidated with instantiations of the feedback gathered, the teaching staff responses made and examples of how the system has enhanced teaching and learning. Finally, suggestions for how the system could be developed and implemented in the future will be presented.
Teaching and Learning Forum 2000 Home Page


Course evaluation on the web (CEW)

The School of Physiotherapy at Curtin University has a commitment to ongoing improvement of its courses. A cycle of evaluation and improvement based on student feedback is seen as essential to the process of quality improvement (Brown et al 1997) and has been the feature of effective schools in the UK (Ramsden 1998). We wished to remain an effective school and placed a high value on both the process and the outcomes of teaching and evaluation. According to Weimar (1990), excellence in the more significant aspects of teaching and learning is not easily achieved and requires more than just a passing participation in improvement activity. Such paradigm shifts are more likely when teachers are supported through a process of reflective dialogue based on student feedback (Brockman 1998).

The School of Physiotherapy therefore wanted to gather student feedback on various aspects of their experience as they go through the course. A number of systems were trialled over the last few years, and whilst providing good information they have tended to be laborious for both students (to provide their feedback) and staff (to collate and interpret the feedback). The authors had considerable experience with the World Wide Web (including distance education and conducting international virtual conferences) and saw the potential to use the Web to facilitate students providing timely and confidential feedback to the tutors and managers of the Courses offered by the School of Physiotherapy.

A pilot version of CEW was developed in 1998 and used with one class (4th year undergraduate research class). Based on this experience the system was refined and piloted in 1999 on all 350 undergraduate Physiotherapy students. We now have a workable mechanism to support the 3 aims of CEW:

  1. encourage student reflection on learning,
  2. encourage teacher reflection on teaching, and
  3. enable better management of teaching and learning resources.
The School of Physiotherapy has an expectation of reflective practice by students. Reflection implies student responsibility for their own learning (Brockbank 1998). Whilst course content design can encourage some higher order learning, we believe transformative learning of relativist/constructed knowledge (Brockbank 1998) can be facilitated by students reflecting on their own learning through participation in course review. This process not only develops critical ability during participation (Brockman 1998), but also by observation of the reflective practice modelling by staff. Meyers (1993) suggests modelling actions are more important than words about wanting students to be more reflective in their learning. By providing reflective practice opportunities "we hope students will become self directed and collaborative, critically reflective, politically savvy, empathic and fair minded as well as competent in the skills that are essential to meaningful lives and careers" (Meyers 1993). CEW will assist us develop graduates who are used to a quality improvement process and can contribute to the improvement of their profession.

The School of Physiotherapy also has an expectation of reflective practice by staff. However, in the past when academic staff have attempted to reflect on their teaching practice they have found this process difficult due to a failure in support mechanisms. Reflective teaching practice is an ongoing, cyclical endeavor which Weimar (1990) colourfully describes as 'tinkering'. To support this 'tinkering' we see the need for changes in culture, dialogue and information.

Culture

The academics' environment profoundly affect their work processes, morale and productivity (Ramsden 1998). The School of Physiotherapy has tried to create an appropriate teaching culture through various initiatives including monthly teaching seminars and small teaching development grants. However developing a non threatening culture is not easy. To support reflective practice by staff we need to establish a culture of building on faculty strengths (Ramsden 1998) based in part on dialogue about student feedback (Brown 1997).

Dialogue

Without dialogue, critical reflection may not happen (Brockman 1998). Whilst peer review is the norm in most aspects of academic work, this is not the case with teaching (Moses 1988). Whilst staff may be willing to engage in dialogue about their teaching, there has not been the established culture and mechanisms to require and support this. Dialogue with peers can be threatening, and the culture needs to develop to be supportive and non judgmental. Rather, staff all need to recognize that most aspects of teaching are neither right or wrong absolutely and what may work in one situation may not work in another situation or time (Weimar 1990). Developing such a culture is important as effectiveness of teaching and learning is related to a supportive environment where dialogue about teaching is encouraged (Ramsden 1998). Higher work satisfaction by academics is also reported in collaborative culture environments (Ramsden 1998). CEW supports dialogue by requiring regular peer discussion on student expectations.

Information

Information is needed as the basis for the reflective dialogue on teaching and learning. Previously we have either had no data, or data in a format which is not easily used as information (typically an intimidating pile of paper questionnaires). Yet student feedback information is probably an important factor in improving teaching and learning (Marsh 1994) and is essential to reflective practice by staff (Weimar 1990). CEW provides student feedback in forms easily used by teaching staff.

With CEW, student expectation information is collected each semester in the form of quantitative and qualitative feedback on units (using a specially developed questionnaire) and the course to date (using the Course Experience Questionnaire). The questionnaires are completed on line making data immediately available for analysis by unit/subject coordinators. CEW automatically processes the rating score information and collates qualitative information enabling easier synthesis of feedback. Unit coordinators discuss the student feedback and their response to this feedback with a peer. A written response is then put on line to 'close the feedback loop' and show students how the School is responding to their comments. [Further details of CEW are available at http://physio.curtin.edu.au/cew/].

Learning Community Model for CEW

CEW is based on a 'Learning Community' model, as illustrated in Figure 1. The model suggests that student learning is a result of the interaction among several key elements (the student themselves, their student peers, the unit/subject learning materials and the academics). This interaction is further influenced by how these elements are managed and by the School's physical and psychosocial environment and the broader University and geo-cultural environment. Learning is used to encompass changes in knowledge, skill, attitudes and behaviour.

Figure 1

Figure 1: Model of a Learning Community

Thus a student's learning is influenced by their intellectual capacity, previous experience, concurrent commitments such as part time work and family responsibilities, knowledge, attitudes to learning in general and specifically to the areas currently being studied and the time and energy a student commits to a unit/subject.

A student's learning is also influenced by their student peers, both by the same influences as for the individual student (intellectual capacity, experience etc.) and also by the group dynamics.

The unit/subject learning materials provided to the student also influence their learning. The presentation, content, tone and relevance of the unit/subject Plan, Guide, Reader, any textbooks and references used and any electronic resources used influence a student's learning.

A student's learning is also influenced by the academics and clinicians/practitioners teaching the unit/subject. The teacher's knowledge, attitudes, clinical experience and teaching experience are important influences.

The unit/subject management influences a student's learning. The structure of the unit/subject in terms of class size, number and type of class contacts, type of assignments and assessments are important. Similarly the organisation of the unit/subject is important, for example the sequencing of lectures and practical classes, the proportionate allocation of time to various topics, the workload, the clarity of expectations.

The local / School environment within which the student studies also influences their learning. The physical facilities such as rooms (thermal, noise, lighting environment) and equipment (numbers, modernness, maintenance state) and general impression of the physical facilities (drab, unvalued, spaciousness, well cared for, brightness, humane) influence learning, as does the local (year, course, School) psychosocial environment.

A student's learning is also influenced by the broader environment. For example aspects of the University including its administrative efficiency, library resources, student support services, student social activity provision, culture and reputation are important influences. The student's family and home also have important influences on a student's learning. Finally, the culture and lifestyle of Australia influences a student's learning.

Naturally a School does not have control over all these influences on a student's learning. However it does have a large degree of control over many of these influences and to provide the best learning environment would like to be aware of those aspects over which is has no control. By understanding these other influences the School will be better able to support students.

Technical and administrative support structures for CEW

CEW is not an isolated mechanism; rather it supports many other mechanisms within the School of Physiotherapy aimed at improving courses. Aside from CEW, student feedback is also provided through two formal student representative mechanisms (one reporting to the academic coordinator of each year/course and one reporting to either an undergraduate or postgraduate student staff committee). Academic feedback is also provided directly to teaching staff. Grievances and allegations of harassment have separate processes established for students. Students also have representation on School Advisory Boards and involvement in curriculum development processes.

The nature of CEW

In weeks 3 and 4 of semester students are allocated a number of unit/subjects from the previous semester to provide feedback. For small groups all students provide feedback, whilst pseudo random sampling is used for larger groups. The students rate agreement with statements under the headings: student, peers, learning materials, teaching staff, local environment, broader environment and unit management. The three best aspects of the unit/subject are requested as well as the three poorest aspects and three suggestions for change. Space is provided for additional comments and students provide an overall rating of satisfaction.

Rating results and keyword sorted comments are immediately available to unit/subject coordinators.

Unit/subject coordinators prepare a summary of student comments, together with a response. Both the summary of student comments and the teaching staff response are discussed with an academic colleague. This discussion is to encourage reflective teaching practice and provide support for teaching staff.

Following peer discussion, the graphic summary of student ratings, the summary of student comments and the teaching staff response are posted onto the Web for access by all School of Physiotherapy students and staff. Also available are all the individual student comments.

Future directions for CEW

Over the next three years we will be developing CEW in a number of ways, supported by the School of Physiotherapy and LEAP funding by Curtin University. CEW will be used with local and off campus students in postgraduate and undergraduate programs. We also have a list of other courses and Schools interested in trialling CEW with their programs. The broadening use of CEW will be used to develop and evaluate its utility, whilst specific evaluations of its test-retest reliability and face, content and criterion validity will also be conducted.

Our aim is to have a flexible system that satisfies student desires to be an active part in their learning, satisfies staff needs to be reflective teachers and enables course managers to know what is happening in courses to provide appropriate resources.

References

Brockbank, A. and McGill, I. (1998). Facilitating reflective learning in higher education. Open University Press.

Brown, S., Race, P. and Smith, B. (1997). 500 tips for quality enhancement in universities and colleges. Kogan Page.

Chalmers, D. and Fuller, R. (1996). Teaching for learning at university. Kogan Page.

Cherns, A. (1976). The principles of sociotechnical design. Human Relations, 29, 783-792.

Cooper, R. and Sawaf, A. (1997). Executive EQ: Emotional intelligences in business. Orion.

Marsh, H. and Roche, L .(1994). The use of students' evaluation of university teaching to improve teaching effectiveness. Australian Government Publishing Service.

Meyers, C. and Jones, T. (1993). Promoting active learning: Strategies for the college classroom. Macmillan.

Moses, I. (1988). Academic staff evaluation and development: A university case study. University of Queensland Press.

Ramsden, P. (1998). Learning to lead in higher education. Routledge.

Weimar, M. (1990). Improving college teaching. Jossey-Bass.

Please cite as: Straker, L. and Smith, R. (2000). Course evaluation on the web (CEW): A 'learning community' tool to enhance teaching and learning. In A. Herrmann and M.M. Kulski (Eds), Flexible Futures in Tertiary Teaching. Proceedings of the 9th Annual Teaching Learning Forum, 2-4 February 2000. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2000/straker.html


[ TL Forum 2000 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf2000/straker.html
Last revision: 20 Feb 2002. Curtin University of Technology
Previous URL 28 Dec 1999 to 20 Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2000/straker.html