Teaching and Learning Forum 98 [ Contents ]

How should we assess and report student generic attributes?

Rick Cummings
Teaching and Learning Centre
Murdoch University
University graduates are expected to have attributes such as critical thinking, ethical practice and communication skills and their university experiences are intended to contribute significantly to the development of these attributes. A review of the literature reveals that it is an international trend to describe these attributes explicitly and direct resources towards measuring their achievement. Australian universities recently have been swept up in this international trend.

Although there is general agreement on the areas of generic attributes which are desirable, universities have generally reserved the right to develop their own specific descriptions and many have done so. However, much less has been achieved in incorporating generic attributes into curriculum and even less progress has been made in developing means to assess whether graduates have acquired these attributes. There is also a lack of knowledge about how employers use evidence of generic attributes and the extent to which they desire reports on the generic attributes of university graduates.

This paper raises several questions which should be considered when describing generic attributes, deciding how to incorporate them into curriculum, and developing assessment and reporting processes for student acquisition of generic attributes.


Introduction

The concept that students develop generic attributes through education and that these should be measured and reported has become a world wide phenomenon in the past ten years. It has developed as the result in part of three major trends: The interrelated influence of these three trends has led to a search for measurable generic attributes which can demonstrate the acquisition of interdisciplinary knowledge, skills and attitudes important to the individual, society and employment.

Common generic attributes

In a previous paper, Cummings, Ho & Bunic (1997) reviewed the development of generic attributes in Australia, including employers views of them, and research on how generic attributes might be assessed and reported. The review found that there are a common core of attributes identified by universities and these overlap the Mayer Key Competencies used in secondary schools and TAFE, although the former have a greater focus on attitudes and values. Table 1 compares the most common attribute areas used by Australian universities and the Mayer Key Competencies.

Table 1

Common generic attributes
identified by Australian universities*
Mayer Key Competencies

A body of knowledge
Communication skills
Lifelong learning
Critical thinking/problem solving skills
Creativity
Independent and team worker
International perspective
Aesthetic appreciation
Citizenship
Professionalism
Collecting, analysing and organising information
Communicating ideas and information
Planning and organising activities
Solving problems
Using mathematical ideas and techniques
Working with others and in teams
Using technology

* These attributes are summarised in a few words to identify attribute areas. Most universities describe their generic attributes in more detailed behavioural terms.

Clanchy & Ballard (1993) argue that universities should be able to certify that graduates have mastered a body of knowledge and have acquired certain generic attitudes and the following three distinctive generic skills; thinking (reasoning), research (including methods of inquiry and management of information) and communication (oral and written). Although many universities have published list of generic attributes, Clanchy & Ballard found that nowhere has this resulted in any serious curriculum revision.

Employer views on generic qualities

As mentioned earlier, improving the employability of graduates is one of the reasons supporting the development of generic attributes. Several studies have been done into which generic attributes employers are looking for in prospective employees and how they think the attributes should be assessed and reported (Cummings & Ho, 1996, Goddard & Ferguson 1996a, 1996b, McCurry, 1996, NIEF, 1996). Much of this work has focussed on the Mayer Key Competencies but Cummings & Ho found the following findings are also relevant to generic attributes:
There is widespread use of generic attributes, explicitly and implicitly, in the recruitment and promotion of employees; Employers use their own terms to describe generic attributes, and these overlap considerably with externally identified generic attributes; and Firms would like to align position descriptions with generic attributes.
In their review, Cummings & Ho concluded that there was a remarkable similarity in the generic attributes identified by employers and the secondary school, vocational education and training, and university sectors. A major difference is that attitudes are included in the generic attributes defined by universities and employers. Cummings & Ho also found evidence that employers' expectations of university graduates are higher both in terms of the level of importance placed on generic attributes and the number of attributes universities expected graduates to exhibit. Finally, employers were found to have clear and fairly consistent views about how generic attributes should be assessed and how best to report the assessment evidence to assist in staff recruitment.

Five essential questions relating to assessing and reporting generic attributes

There has been little research into the process of assessing and reporting on generic attributes and even less experience in practice. This is the next big issue to be addressed by organisations wishing to pursue generic attributes as measurable outcomes of education and training.

Question 1: How to describe each generic attribute?

The manner in which generic attributed are described varies considerably. However, it is crucial that there is a common interpretation of the attribute by the staff of the university, the students and potential employers. Examples of three different approaches are given below:
Communicating ideas and information: the capacity to communicate effectively with others using the range of spoken, written, graphic and other non-verbal means of expression (Mayer Key Competencies).

Communication: Make connections that create meaning between yourself and your audience. Learn to speak, read, write and listen effectively, using graphics, electronic media, computers and quantified data (University of Canberra).

Graduates will be able to communicate effectively in professional practice and community activity. Some indicators include demonstrate oral, written, mathematical and visual literacies as appropriate to the discipline or professional area, display sensitivity to their audience in organising and presenting ideas, and communicate appropriately with professional colleagues and the public (Murdoch University draft attribute).

These three examples have considerable differences which would significantly affect the teaching and assessment of this attribute. It is still very much an open question as to how to describe generic attributes so they are easily incorporated into curriculum and most appropriate for assessment and reporting.

Question 2: How to incorporate generic attributes into curriculum?

The recency of the development of generic attributes in each of the three sectors of education and training has meant that there has been little time to work on the problems of incorporating them into curriculum. However, in their paper, Cummings, Ho & Bunic (1997) identify two general issues gleaned from the limited work done to date; whether generic attributes should be expressed implicitly or explicitly in curriculum, and the extent to which generic attributes are context or discipline bound. There is insufficient space to discuss these issues in this paper and the reader is referred to the discussion in Cummings, Ho & Bunic (1997).

Based on experience in the VET sector, two general approaches to the incorporation of generic skills into curriculum have developed, adjunct and integrated.

An adjunct approach separates aspects of generic attributes from technical aspects of the course. Separation can occur at a number of levels within the curriculum:

Rather than integrated into the course, generic attributes remain a separate and identifiable component. While this ensures that generic attributes are not overlooked in delivery or assessment, the danger arises that they are not seen as relevant or important by students due to their lack of content.

In an integrated approach, the generic attributes become an integral part of the course philosophy, are expressed in course descriptions, course objectives or course outcomes and opportunities for their development are identified explicitly in the curriculum. Supports of this approach argue that the explicit identification of generic attributes facilitates their development by increasing the opportunity for the delivery, assessment and reporting of the skills.

Important opportunities for the development of generic attributes occur in the selection of delivery methods. Teaching contexts can provide an explicit focus on the development of generic attributes, thus providing students with opportunities to develop them. The development of generic attributes will be promoted where there is the opportunity for students to practice these attributes within learning activities.

Delivery methods which have been used with some success include:

Although the benefits and disadvantages of using an adjunct or integrated approach to incorporating generic attributes into curriculum are not yet known, the integrated approach appears to be more popular at present.

Question 3: Can each generic attribute be measured?

This has been a topic of considerable debate in the higher education sector. However, little if any research has been carried out in Australia to empirically test the assessibility of different generic attributes. The VET sector has done considerable work on how to assess skill competencies and the Mayer Key Competencies. There is some empirical evidence that each of the Mayer Key Competencies can be assessed validly, reliably and in a way which is useful to the training provider, the student and the employer (McCurry, 1996). There is considerably more uncertainty in the higher education sector due both to a less enthusiastic endorsement of generic attributes by university staff and a broader range of skills, attitudes and values incorporated in generic attributes identified by universities.

The question of whether a generic quality can be measured is intimately linked to the reasons why the assessment is being done and for whom it is being done. There are at least three reasons linked to specific stakeholders in the assessment process which are important to consider. First, assessment might be done to provide individual students with evidence of whether they have the necessary attributes to contribute to society and to continue to learn throughout their lives. Second, assessment of generic attributes could provide students and employers with evidence that the student has the necessary skills, knowledge, and values to gain employment. Third, assessment could measure the extent to which universities contribute to a student's generic attributes and produce the graduates they aim to produce. There is further discussion under Question 4 about the relationship between assessing attributes and the users of that assessment.

Question 3a: If a generic attribute can be measured, how should it be assessed?

This question could be restated to 'How to arrive at a judgement that an individual student has acquired a particular attribute to a prescribed level?'

Evidence from survey conducted by McCurry (1996) and Cummings & Ho (1996) indicate that employers believe a single mark, as provided for academic subjects, is not appropriate as evidence of generic attributes. Instead, they would like to have available one or more of the following; evidence of the student's level of generic attribute against some criterion, a description of what constitutes this level of performance, and examples of the student's work which demonstrate this attribute.

Research in the secondary school and VET sectors (McCurry, 1996; NIEF, 1996) advocates the use of a portfolio approach incorporating assessment by several teachers, self-assessment by the student and examples of work produced by the student which demonstrates each attribute.

There is some concern that a portfolio approach may be better suited to secondary schools where teachers have a more personal, pastoral care role, than universities where the lecturer-student relationship is more formal. It may be that lecturers would not have sufficient experience of individual students to make judgements about generic skills. There is at present very little empirical research on the appropriateness of different approaches to assessing generic attributes.

Question 4: How should the assessment result be reported?

The answers to this question are likely to be strongly influenced by who the reporting is intended for. Students, on the one hand, may want a single normative mark, which enables them to compare their results to their peers, as is currently done in reporting subject results. Employers, on the other hand, have expressed a desire for descriptive information and demonstration of the students work.

Universities, in turn, may want the same reports as for students and employers but also seek evidence which is comparable across disciplines and provides information about the quality of course of study.

McCurry (1996) compares the strengths and weaknesses of several approaches to assessment and reporting and concludes that it is possible to develop an assessment framework which meets the criteria of good assessment, addresses employers requirements for reporting, uses a subject based approach, with staff as assessors, and can be monitored by central agencies.

Two other criteria which might be applied to the assessment of generic attributes are its useability, that is, how well and to what extent the assessment results will be used by the individual, future employers, and society, and its credibility. Clearly these are related because if the results have low credibility their level of use is likely to be diminished.

Summary

In line with recent moves to identify the generic attributes of graduates from secondary schools, vocational education and training institutions and universities, a number of issues arise about how to identify these attributes, how to incorporate them into curriculum and how to assess and report on the extent to which graduates have acquired them. This paper has argued that there is considerable similarity of generic attributes across Australian universities. The important issues now are whether to identify the attributes explicitly or implicitly in the curriculum, how to ensure the generic attributes are taught to students, and how to construct appropriate assessment and reporting processes. This paper raises a number of important questions to be asked when including generic attributes in higher education curriculum. Further work on incorporating generic attributes in curriculum and the efficacy of different approaches to assessing and reporting generic attributes is now required in the higher education sector.

References

Bowden, J. A. & Masters, G. N. (1992). Implications for Higher Education of a Competency-Based Approach to Education and Training. Canberra: Department of Employment, Education and Training.

Clanchy, J. & Ballard, B. (1993). Generic Skills in the Context of Higher Education. A discussion paper on aspects of 'Generic Skills' as they affect teaching and learning within ANU. Study Skills Centre.

Cummings, R. & Ho, R. (1996). Survey of Key Competencies in Selected Industries in Western Australia. Perth: Murdoch University.

Cummings, R., Ho, R. & Bunic, S. (1997). Assessing and Reporting of Student Generic Qualities in Higher Education, Paper presented at the HERDSA Conference, Adelaide, SA.

Goddard, D. & Ferguson, K. (1996a). Small Business Owners, their Employment Practices and Key Competencies. Department of Employment, Education, Training and Youth Affairs and Business in the Community, Ltd.

Goddard, D. & Ferguson, K. (1996b). The Southern Province Enterprise Network Project: Enterprise - Go For It. Department of Employment, Education, Training and Youth Affairs and Business in the Community, Ltd.

Golding, B., Marginson, S. & Pascoe, R. (May 1996). Changing Context, Moving Skills: Generic skills in the context of credit transfer and the recognition of prior learning, Canberra: Australian Government Publishing Service.

Hill, P. W. (undated). Key Competencies: Which Way Forward? Melbourne: National Industry and Education Forum.

Mayer, E. (Chair) (1992). Putting general education to work: The Key Competencies. The Australian Education Council and Ministers for Vocational Education and Training.

McCurry, D. (1996). Approaches to Assessing the Key Competencies. Melbourne: Australian Council for Educational Research.

McCurry, D. and Bryce, J. (1997). The School Based Key Competencies Levels Assessment Project, Final Report. Melbourne: Australian Council for Educational Research.

National Industry and Education Forum (1996). Business, Industry, Key Competencies and Portfolios. Melbourne: NIEF.

Secondary Education Authority (1996). Key Competencies Project Report. Perth: SEA.

Schwartz, S., Piper, D. W. & Crook, N. (1993). Defining, Assessing, and Improving the Outcomes of Higher Education: An Annotated Bibliography. Brisbane: The University of Queensland.

Werner, M. C. (1994). Australian Key Competencies in an International Perspective. South Australia: National Centre for Vocational Education Research Ltd.

Please cite as: Cummings, R. (1998). How should we assess and report student generic attributes? In Black, B. and Stanley, N. (Eds), Teaching and Learning in Changing Times, 85-90. Proceedings of the 7th Annual Teaching Learning Forum, The University of Western Australia, February 1998. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1998/cummings.html


[ TL Forum 1998 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1998/cummings.html
Last revision: 5 Mar 2002. The University of Western Australia
Previous URL 29 Dec 1997 to 5 Mar 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf98/cummings.html