Teaching and Learning Forum 95 [ Contents ]

UCLES: A questionnaire for evaluating portfolio cultures in postgraduate teaching

Peter Charles Taylor
Key Centre for School Science and Mathematics
Curtin University
As a university teacher educator working in the field of science and mathematics education I am concerned with reforming the epistemology that underpins postgraduate university teaching. In this paper, I present an account of my current attempts to create a constructivist learning environment in one of my postgraduate classes, and describe a newly-developed questionnaire (UCLES) that I am using to evaluate the efficacy of my teaching reforms.

Introduction

Traditionally, universities have been exemplars of the transmissionist paradigm typified by large lecture theatres in which a single perspective has been dominant -- the lecturer's (note the appropriateness of the title). In these forums, it is almost inevitable that knowledge is regarded as a commodity which, metaphorically speaking, can be transmitted from the lecturer's mind to the minds of the mass of students in attendance. Communicating (i.e. lecturing) serves the purpose of conveying seemingly objectified knowledge in as efficient way as possible. Consequently, learning is rendered as little more than the memorisation of uncontestable facts and, especially in science/ mathematics-related fields, the memorisation of standard problem types. This impoverished image of learning is reinforced by the predominance of the traditional practice of summative assessment. End-of-course examinations have the effect of isolating learning (i.e. the process of coming to know) from assessment (i.e. judging the quality of learning processes and outcomes) and generating in the minds of students an implicit belief in an external locus of control of learning.

Little wonder then that new postgraduate students bring with them historically-grounded expectations for more of the same -- more lecturing, more examinations, more absorption of expert knowledge, more external control of learning. In this teaching and learning culture of consumption, the quality of the learning 'product' (i.e. students' knowledge) is judged by criteria established entirely by others (i.e. academic 'quality contollers'). Little opportunity exists for students to develop skills of judging the quality of their own learning, and less opportunity exists for students to learn how to generate criteria of quality.

As a university teacher, it is my concern that unless we reform the prevailing transmissionist epistemology of university teaching, our students will remain trapped in an unhealthy culture of uncritical, unreflective and reproductive thinking that is intellectually and emotionally disempowering. For me, the starting point for epistemological reform is my own postgraduate teaching in which I must model exemplary teaching practice to my postgraduate students who are, themselves, professional school teachers. It is no exaggeration to say that, left to their own devices, most of these mature-age students would readily re-adopt the passive and impoverished learning roles of their undergraduate years. Such is the power of enculturation and habituation.

Constructivism: A referent for teaching reform

Constructivism is an epistemology that asserts that knowledge is constructed (or developed) within the mind of the individual learner (von Glasersfeld, 1993). Of course, learners are inextricably linked with others by means of language and shared cultural assumptions and practices. So, constructivism accounts for the influence of the socio-cultural world by regarding it as both an important source of stimulation for the learner's sense-making process and a constraining influence on what counts as viable knowledge.

The press for basing my postgraduate teaching on a constructivist epistemology has arisen from a number of sources. Fifteen years of concerted research by science and mathematics educators worldwide has resulted in constructivism achieving an international status as the recognised alternative epistemology for curriculum reform of school science and mathematics (Tobin, 1993). During this period, constructivism also has come to serve as a powerful referent for framing questions that research addresses and ways that research is conducted. In the field of education, it has made research much more accesssible as the metaphor of research as learning has been widely adopted by teachers (Denzin & Lincoln, 1994).

My own research has shown, however, how difficult it can be for well-intentioned and well-supported school teachers to create constructivist learning environments based only on 'cold' rational considerations of how to reform their teaching methods (Taylor, 1992, 1994). It seems to me that if teachers are to understand the rich implications of constructivism as a referent for the reform of teaching then they must be given opportunities to experience first-hand what it means to be a learner in a classroom environment framed by a constructivist epistemology. I am motivated, therefore, to provide opportunities in my own postgraduate classes for teachers to reflect on their own 'hot' experiences of their struggles to become empowered learners.

Principles of constructivist teaching

For my Curriculum class, which is a core Unit in our MSc (Science Education) program, the following constructivist principles serve as an epistemological framework for my design of teaching and learning activities. The descriptors in parentheses are related to the scales of the UCLES questionnaire whose role is explained later in the paper.
  1. Knowledge is a transformative growth process shaped by the learner's sense of purpose, rather than a product to be absorbed from external sources (Reflexivity, Relevance, Management).

  2. The teacher is a crafter and facilitator of knowledge growth, rather than a disseminator, and modifies and adapts learning activities, rather than adheres rigidly to a prescribed curriculum (Accountability).

  3. Students interactively construct their knowledge in social and cultural contexts (Negotiation).

  4. The curriculum goals are concerned with how and why we know what we claim to know, and with knowledge growth and explanation (Reflexivity).

Organisation and structure of course

Each week, the class is divided into two two-hour sessions. In the first session, we discuss a chapter from the course text book, Curriculum: Product or Praxis? (Grundy, 1987). I chose this book because it provides a structured approach to the development of a constructivist perspective which can be used to challenge students' underlying epistemological assumptions. I find that it is essential for me to adopt (somewhat ironically) a central role in establishing a student-centred learning environment. This enables me to encourage students to participate in a discourse that is both 'open' and 'critical'. Open discourse involves self-disclosure of valued beliefs and assumptions, and requires a caring and sharing atmosphere. Critical discourse involves reflecting critically on one's own taken-for-granted beliefs and practices, particularly from an autiobiographical perspective, and requires an atmosphere conducive to the critical self-examination of 'cherished icons' and 'holy cows'.

In the second session, students assemble in self-selected special interest groups (or SIGs) for the purpose of examining in detail a particular issue related to the overall theme of curriculum. Initially, I assist groups to form by suggesting a range of topics and then I 'kick start' each group by providing preliminary readings. Currently, the range of SIGs includes: Ethics, Technology, Culture, and Assessment. During the ensuing weeks, I impose on groups the requirement to submit in writing their emerging personal and negotiated group learning goals. During my visits to each SIG, I listen carefully, respond to questions, and stimulate thinking about the connection between the work of the SIG and the epistemological framework that is the subject of the first session. Each SIG is required to submit a report which is to be included in each student's individual portfolio for the purpose of formal course assessment.

Portfolio assessment

As a means of promoting a developmental model of student learning, I have adapted the portfolio approach to learning and assessment advocated by Duschl and Gittomar (1993). In a general sense, portfolios constitute a culture because they reconfigure a closer nexus amongst the trilogy of learning, assessment and teaching. In a more specific sense, they constitute a collection of samples of students' work that provides authentic and rich evidence for evaluating learning. As well as providing students with my own assessment criteria, I encourage them to develop their own criteria upon which to base self-evaluation of their own learning. In the course outline, I set out the following guidelines for students' portfolios.
A portfolio is a coherently organised collection of work completed during the semester and should provide evidence of: (1) your process of learning, especially changes in your understanding that result from critical self-reflective thinking; and (2) your understanding of selected key issues in the domain of curriculum that reflect your own learning goals. My assessment of your portfolio will be guided by your self-assessment report, and will be based on the following four criteria.

The teacher researcher

As a teacher-researcher I am committed to evaluating the efficacy of my own teaching. This is not a new role for school teachers, especially those who are familiar with the well-established action research paradigm (Kemmis & McTaggert, 1988). However, in higher education the self-study of teaching is in its infancy, as evidenced by the recent establishment of a Special Interest Group of this title within the American Educational Research Association.

I have decided, therefore, to generate data that are useful for evaluating my attempts to transform the epistemology of my teaching of the Curriculum course. My experience as an interpretive researcher (Erickson, 1986) focusses my attention on the need for multiple sources of data. For this study, I draw on (1) students' portfolios, (2) student interviews, (3) my own reflections that I record in a journal at the end of each class, and (4) a questionnaire designed to obtain measures of students' perceptions of key dimensions of the classroom learning environment (UCLES).

The University Constructivist Learning Environment Survey

The University Constructivist Learning Environment Survey is a questionnaire designed for use in universities, particularly in postgraduate classrooms where constructivist epistemological reform is intended. It has arisen from research that combines the fields of learning environment research (Fraser, 1989) and constructivist research on teaching (Tobin, 1993). An earlier version of the questionnaire was designed for use in high school science and mathematics classrooms and has been found to be very useful for enabling teacher-researchers to evaluate epistemological transformations of their own classrooms (Taylor, Fraser & White, 1994; Taylor, Dawson & Fraser, 1995).

The UCLES comprises 30 statements arranged in five scales each of which focuses on a key aspect of a constructivist learning environment. The five scales are termed 'Relevance', 'Reflexivity', Accountability', ' Management', and 'Negotiation'. Table 1 presents scale descriptions and a sample item of each scale. The UCLES has a 5-point Likert-type frequency response scale which comprises the categories: almost always (5 points), often (4), sometimes (3) seldom (2), and almost never (1). Therefore, the maximum possible mean score of each 6-item scale is 30 and the minimum possible scale mean score is 6. The UCLES is available in two forms.

Table 1: UCLES Scales and Sample Items (Student Perceived)


ScaleSample Item
In this class. . .

Relevance: perceived relevance of learning to students' professional lives.. . . what I learn is related to my professional life.
Reflexivity: perceived press for reflecting critically on established concepts, values and assumptions. . . . I feel a need to examine critically my preconceptions.
Accountability: perceived legitimacy of holding the teacher accountable for learning opportunities. . . . it's OK to question the way I'm being taught.
Management: perceived involvement in planning, conduct and assessment of learning. . . . I have a role in planning what I learn.
Negotiation: perceived involvement with other students in assessing viability of new ideas. . . . I ask other students to explain their ideas.

Conclusion

Because of the small size of my postgraduate classes (N<15), I am using the UCLES as a heuristic device for exploring issues rather than as a psychometric instrument for measuring statistically changes in the learning environment. The UCLES provides a useful interpretive framework for examining students' perceptions and preferences, and enables me to construct not only a numerical profile of the whole group, but also provides a platform for constructng case studies of individual students.

References

Denzin, N. & Lincoln, Y. (eds.) (1994). Handbook of Qualitative Research. Thousand Oaks, CA: Sage.

Duschl, R. A and Gitomar, D. H. (1993, April). Emergent Questions About Portfolio Assessment. Paper presented as part of the symposium 'The Intellectual Activity of Science Teaching' at the annual meeting of the American Educational Research Association, Atlanta, GA.

Erickson, F. (1986). Qualitative Methods In Research On Teaching. In Wittrock M. (ed.), Handbook of Research On Teaching (3rd. ed.). New York, NY: Macmillan.

Fraser, B. J. (1989). Twenty Years On Classroom Environment Work: Progress and Prospect. Journal of Curriculum Studies, 21, 307-327.

Grundy, S. (1987). Curriculum: Product or Praxis? East Sussex, UK: The Falmer Press.

Kemmis, S. and McTaggert, R. (eds.), (1988). The Action Research Planner (3rd ed.). Geelong: Deakin University Press.

Taylor, P. C. (1992). An Interpretive Study Of The Role Of Teacher Beliefs In The Implementation Of Constructivist Theory In A Secondary Mathematics Classroom. Unpublished doctoral thesis, Perth: Curtin University.

Taylor, P. C. (1994). Mythmaking And Mythbreaking In The Mathematics Classroom. SMEC, Curtin University of Technology, Perth, Australia.

Taylor, P. C., Fraser, B. J. and White, L. R. (1994, April). CLES: An Instrument For Monitoring The Development Of Constructivist Learning Environments. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Taylor, P. C., Dawson, V. and Fraser, B. J. (1995, April). Classroom Learning Environments Under Transformation: A Constructivist Perspective. Paper to be presented at the annual meeting of the American Educational Research Association, San Fransisco, CA.

Tobin, K. (ed.) (1993). The Practice of Constructivism in Science Education. Washington, DC: Association for the Advancement of Science (AAAS) Press.

Von Glasersfeld, E. (1993). Questions and Answers about Radical Constructivism. In Tobin, K. (ed.), The Practice of Constructivism in Science Education. Washington, DC: Association for the Advancement of Science (AAAS) Press.

Please cite as: Taylor, P. C. (1995). UCLES: A questionnaire for evaluating portfolio cultures in postgraduate teaching. In Summers, L. (Ed), A Focus on Learning, p252-256. Proceedings of the 4th Annual Teaching Learning Forum, Edith Cowan University, February 1995. Perth: Edith Cowan University. http://lsn.curtin.edu.au/tlf/tlf1995/taylor.html


[ TL Forum 1995 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1995/taylor.html
Last revision: 19 Apr 2002. Edith Cowan University
Previous URL 8 Mar 1997 to 19 Apr 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf95/taylo252.html