Teaching and Learning Forum 2001 Home Page
[ Teaching and Learning Forum 2001 ] [ Proceedings Contents ]

Expanding what and how we assess: Going beyond the content

Alex Radloff
Faculty of Life Sciences
RMIT University

Barbara de la Harpe
Curtin Business School
Curtin University of Technology


Going beyond the content

As university teachers we all hope that our students will leave our courses not only knowledgeable about the content but also possessing a range of learning skills and the ability to be metacognitive about learning and themselves as learners. We also hope that they leave with a positive attitude to learning and the discipline and the motivation to continue learning. Thus, we hope that our graduates will have the cognitive, metacognitive, motivational and affective characteristics which research suggests play an important role in effective university study, achievement and learning for life (Biggs, 1999; Pintrich, Smith, Garcia & McKeachie, 1993; Zimmerman, 1994). Only once university graduates have developed these, will they truly be lifelong learners, a goal to which universities, professional and employer groups and graduates themselves aspire (ACNielsen Research Services, 2000; Johnson, 1998; Institute of Chartered Accountants in Australia, 1994).

Traditionally, however, most university courses tend to focus more on discipline content than on the 'skill and will' of learning. This is not surprising since many staff perceive themselves to be content experts and may find their role as teachers challenging. As Sutherland (1996, p. 91) points out,

[t]he reasons that faculty find it difficult to assess non-content outcomes are the same as the reasons they find it difficult to consider using new teaching approaches. Faculty are experts in their field of study. They have spent their professional lives developing skill and confidence in their abilities as chemists, sociologists, rhetoricians, and art historians. Their training and focus has been on content, and few have been supervised or mentored in teaching and evaluating students.
This emphasis on discipline content is often reflected in the types of assessment tasks teachers set for their students to complete. These tasks focus mainly on "...the products of learning" rather than on the "...how and why of student learning" (Anderson, 1998, p. 8). Moreover, many assessment tasks fail to provide feedback to students and teachers about how students are developing the skills they need for lifelong learning. As Cross (1998, p. 7) reminds us,
[w]e don't pay a lot of attention right now to giving students feedback on their progress as learners. Mostly, students get grades that tell them how they have done relative to their classmates. That information is not useful feedback on their progress as learners, nor does it do anything to help students develop skills for self assessment.
Without a focus on assessing these characteristics, neither students nor teachers can make informed decisions about how they learn or teach. It is, therefore, critical that we pay attention to what aspects of learning are being assessed since, as Ramsden (1992) among many others, has pointed out, it is assessment that actually drives the curriculum and both student and teacher behaviour (Biggs, 1999).

Expanding assessment

Teachers can, as part of their normal subject teaching, expand their assessment to focus on the aspects of learning that contribute to development of lifelong learning skills. Assessment can include as well as content, a focus on the following aspects of learning: These aspects of learning underpin lifelong learning skills such as for example, oral and written communication, problem solving, analytical thinking, decision making, information literacy and team work.

Research suggests that the skills for lifelong learning are best developed in context by the discipline teacher with special attention being paid to the role of assessment. In fact, as McKeachie, Pintrich, Lin and Smith (1986, p. 1) point out,

...every course should help students become aware of strategies for learning and problem solving. An explicit goal of education throughout the curriculum should be to facilitate the development both of learning strategies and problem solving skills and of effective strategies for their use.
Thus, such help should be provided by teachers as an integral part of every university course. Assessment tasks can be expanded in a number of ways to include feedback on both content and aspects of the learner and the learning process. In the next section, we provide some examples of how this can be done as part of regular assessment.

Examples of assessment tasks

One way of assessing what learning strategies students are using and their motivation for learning is to ask them to complete a survey such as the Motivated Learning Strategies Questionnaire (MSLQ) at the start of a course and again, after the mid-semester test. Students can then analyse their responses and write a self reflective commentary including what changes they will make to their learning based on their feedback from the MSLQ and their level of understanding of the subject as well as their performance on the test. Such an assessment task could carry a weighting of 10 to 15 percent of the total mark for the course.

The MSLQ (Pintrich, Smith, Garcia & McKeachie, 1991) is a standardised 81 item Likert-type self report instrument designed to measure students' motivational orientations and learning strategy use. It consists of two sections - Motivation and Learning. The Motivation section is made up of three scales namely, value, expectancy and affective components. The Learning Strategies section is also made up of three scales namely, cognitive, metacognitive, and resource management. The scales are designed to be modular and thus, can be used together or singly, to fit specific needs. The full MSLQ takes approximately 20-30 minutes to complete. See Appendix 1 for examples of items.

A way to assess metacognitive aspects of learning is to ask students to keep a personal learning log in which they describe their learning and reflect on their learning experiences in term of how they have changed or intend to change or not change the way they learn. Students can be helped to do the task by being given opportunities to discuss with peers and the teacher the value of keeping a log, their goals for their logs, and criteria for assessing their logs. This activity can be assessed in a number of ways. For example, students can submit three extracts from their log together with a reflective comment justifying the selection and its significance for their understanding of the subject and themselves as learners. This task could carry up to 20 percent of the total mark for the course. Another way of assessing the activity is to include a question in the final exam asking students to describe what they have learned about effective ways of learning the subject and/or how they have changed as learners over the semester.

Learning logs have been used to foster and assess students' cognitive and metacognitive strategy use (Alderman, Klein, Seeley, & Sanders, 1993; Dart & Clarke, 1991). Learning logs or diaries have been found to help students evaluate their learning, develop more sophisticated conceptions of learning, and stimulate critical thinking (Hartley, 1998).

A way to assess how students feel about a learning task and themselves as learners is to ask students, as part of an assignment that assesses some aspect of the content, to complete the Zuckerman Affect Adjective Checklist (AACL) at the start, during and end of the assignment. In a section of the assignment, students include a graph plotting their feelings and a brief comment explaining their feelings and any changes over time.

The AACL (Docking & Thornton, 1979; Zuckerman, 1960) is a self report instrument designed to measure anxiety. It consists of 21 key adjectives embedded in a total of 60 with various affective connotations, arranged in alphabetical order. Respondents select the words that describe how they generally feel about a particular situation. The AACL takes about two minutes to complete and is easy to use. See appendix 2 for the AACL.

These examples show how, with some minor adaptation of existing assessment tasks and a bit of creative thinking, teachers can expand their assessment to include a focus on content as well as the skill and will involved in learning. There are many other assessment tools that can be used to expand the focus of assessment. Some of these are shown below. For a more in depth discussion of such tools, see de la Harpe and Radloff (2000).

Cognition, metacognition, motivation and/or affectLearning and Study Strategy Inventory (LASSI)Weinstein, Zimmerman and Palmer (1988)
Approaches to Study (ASI)Entwistle and Ramsden (1983)
Study Process Questionnaire (SPQ)Biggs (1987)
One or more of cognition, metacognition, motivation and affectClassroom Assessment Techniques (CATs)Angelo and Cross (1993)
Cognition and metacognitionSelf Regulated Learning Interview Schedule (SRLIS)Zimmerman and Martinez-Pons (1986)
Learning statementsBoulton-Lewis, Wilss and Mutch (1996)
MetacognitionMetacognitive Awareness Inventory (MAI)Schraw and Dennison (1994)
Perceived Self Efficacy for Writing Scale (PSEWS)Zimmerman and Bandura (1994)
AffectState-Trait Anxiety Inventory (STAI)Spielberger (1983)

Importance of expanding assessment

Expanding assessment to include a focus on content as well as the skills needed for learning, work and life helps both students and their teachers. Engaging students in the types of assessment activities outlined above encourages them to be more informed about learning and themselves as learners. It also helps them to develop valued skills.

Using these types of assessment tasks also helps teachers, since, as Pintrich and Johnson (1990, p. 83) note, "...the information generated by these instruments can enlighten faculty members about the general cognitive and motivational characteristics of their students". It provides teachers with information about how students develop as learners and about the impact that the curriculum, and their instructional and assessment activities have on students. This information, in turn, can be used in designing and planning curricula that foster and value both content knowledge and skills.

In this paper, we have presented a rationale for expanding assessment to include aspects of learning that are valued, shown that it is possible for teachers in any discipline to adapt their assessment tasks to do so, and highlighted the benefits for students and staff. It is now up to us as teachers to expand the assessment horizon and to harness its power in order to enhance teaching and learning.

References

ACNielsen Research Services (2000). Employer satisfaction with graduate skills: Research report. (Evaluations and Investigations Report 1999/7). Canberra, ACT: Commonwealth of Australia. [verified 17 Dec 2000] http://www.detya.gov.au/highered/eippubs/eip99-7/eip99_7pdf.pdf
[exec summ] http://www.detya.gov.au/highered/eippubs/eip99-7/execsum99_7.htm

Alderman, M. K., Klein, R., Seeley, S. K., & Sanders, M. (1993). Metacognitive self-portraits: Preservice teachers as learners. Reading Research and Instruction, 32(2), 38-54.

Anderson, R. S. (1998). Why talk about different ways to grade? The shift from traditional assessment to alternative assessment. In R. S. Anderson & B. W. Speck (Eds.), Changing the way we grade student performance: Classroom assessment and the new learning paradigm (pp. 5-16). San Francisco: Jossey-Bass.

Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass.

Biggs, J. (1999). Teaching for quality learning at university. Buckingham: SRHE and Open University Press.

Biggs, J. (1987). Student approaches to learning and studying. Hawthorn, Victoria: ACER.

Boulton-Lewis, G. M., Wilss, L., & Mutch, S. (1996). Teachers as adult learners: their knowledge of their own learning and implications for teaching. Higher Education, 32, 89-106.

Dart, B. C., & Clarke, J. A. (1991). Helping students become better learners: A case study in teacher education. Higher Education, 22, 317-335.

de la Harpe, B., & Radloff, A. (2000). Informed teachers and learners: The importance of assessing the characteristics needed for lifelong learning. Studies in Continuing Education, 22(2), 169-182.

Docking, R. A., & Thornton, J. A. (1979, November). Anxiety and the school experience. Paper presented at the Annual Conference of Australian Association for Research in Education, Melbourne.

Entwistle, N. J., & Ramsden, P. (1983). Understanding student learning. London: Croom Helm.

Hartley, J. (1998). Learning and studying: A research perspective. London: Routledge.

Institute of Chartered Accountants in Australia (1994). Chartered accountants in the 21st century. Sydney: Author.

Johnson, T. (1998). The 1997 Course Experience Questionnaire. (A report prepared for the Graduate Careers Council of Australia). Parkville, Australia: Graduate Careers Council of Australia Ltd.

McKeachie, W. J., Pintrich, P. R., Lin, Y., & Smith, D. A. F. (1986). Teaching and learning in the college classroom. A review of the research literature. (Technical report No. 86-B-001.0). National Centre for Research to Improve Postsecondary Teaching and Learning, University of Michigan.

Pintrich, P. R., & Johnson, G. R. (1990). Assessing and improving students' learning strategies. In M. D. Svinicki (Eds), New directions for teaching and learning: The changing face of college teaching (pp. 83-91). San Francisco: Jossey-Bass.

Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813.

Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: National Center for Research to Improve Postsecondary Teaching and Learning, University of Michigan.

Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475.

Spielberger, C. D. (1983). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press Inc.

Sutherland, T. E. (1996). Emerging issues in the discussion of active learning. In T. E. Sutherland & C. C. Bonwell (Eds.), Using Active Learning in College classes: A range of options for faculty. San Francisco: Jossey-Bass.

Weinstein, C. E., Zimmerman, S. A., & Palmer, D. R. (1988). Assessing learning strategies: The design and development of the LASSI. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 25-40). San Diego: Academic Press.

Zimmerman, B. J. (1994). Dimensions of academic self-regulation: A conceptual framework for education. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulation of learning and performance. Issues and educational applications (pp. 3-21). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory influences on writing course attainment. American Educational Research Journal, 31(4), 845-862.

Zimmerman, B. J., & Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23, 614-628.

Zuckerman, M. (1960). The development of an affect adjective checklist for the measurement of anxiety. Journal of Consulting Psychology, 24, 457-462.

Appendix 1

The scales, subscales and examples of items in the
Motivated Strategies for Learning Questionnaire (MSLQ)

ScaleSubscale Item
Motivation
Value componentintrinsic goalThe most satisfying thing for me in this course is trying to understand the content as thoroughly as possible.
extrinsic goalGetting a good grade in this class is the most satisfying thing for me right now.
task valueI like the subject matter of this course.
Expectancy componentcontrol of learningIt is my own fault if I don't learn the material in this course.
self-efficacyI'm confident I can do an excellent job on the assignments and tests in this course.
Affective componenttest anxietyI have an uneasy, upset feeling when I take an exam.
Learning strategies
CognitiverehearsalWhen studying for this class, I practice saying the material to myself over and over.
elaborationWhen I study for this class, I pull together information from different sources, eg. lectures, readings, discussions.
organisationWhen I study the readings for this course I outline the material to help me organise my thoughts.
critical thinkingI often find myself questioning things I hear or read in this course to decide if I find them convincing.
Metacognitivemetacognitive self-regulationBefore I study new material thoroughly I often skim it to see how it is organised (planning).
I ask myself questions to make sure I understand the material I have been studying in this class (monitoring).
If course materials are difficult to understand I change the way I read the material (adapting).
Resource managementtime and studyI make good use of my study time for this course.
effort regulationEven when course materials are dull and uninteresting, I manage to keep working until I finish.
peer learningWhen studying for this course, I often try to explain the material to a classmate or a friend.
help-seekingI ask the instructor to clarify concepts I don't understand well.

Appendix 2

Sixty adjectives showing the 20 embedded key words on the
Affect Adjective Checklist (AACL)
absorbed
aware
challenged
contented (-)
disappointed
fortunate
incapable
loving (-)
overloaded
pushed
secure (-)
terrified (+)
afraid (+)
bored
cheerful (-)
creative
efficient
frightened(+)
inspired
miserable
panicky (+)
refreshed
serious
thoughtful (-)
aimless
calm (-)
cheated
curious
entertained
happy (-)
interested
misplaced
pleasant (-)
regretful
shaky (+)
upset (+)
ambitious
careless
comfortable
dedicated
excited
hopeless
joyful (-)
nervous (+)
pleased
rewarded
steady(-)
weary
annoyed
cautious
confused
desperate (+)
fearful (+)
impatient
lazy
organised
productive
satisfied
tense (+)
worried (+)

Please cite as: Radloff, A. and de la Harpe, B. (2001). Expanding what and how we assess: Going beyond the content. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2001/radloff.html


[ Abstract for this article ] [ TL Forum 2001 Proceedings Contents ] [ All Abstracts ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf2001/radloff.html
Last revision: Feb 2002. Curtin University of Technology
Previous URL 17 Dec 2000 to Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2001/radloff.html