Against a background of accumulating research evidence, this paper presents a conceptual framework of how evidence of variation in student learning can be used to initiate and inform a variety of teacher-focused, and learner-focused, interventions aimed at improving the quality of teaching and learning.
Within this framework the attributes of some of contemporary models of student learning, or aspects thereof, are discussed. Examples of manifestations of student learning to which such models, or discrete aspects thereof, are 'fitted' for interpretative or evaluative purposes are presented and examined in terms of practical utility.
An example is presented of how a discipline-specific model of student learning can be developed that is sensitive to individual differences in the manner in which students engage the content and context of learning. It is argued, in particular, that a sufficiently sensitive model can form a basis for (a) initiating a process of change in terms of assisting teachers to reconceptualise their teaching and make it more student-focused, (b) identifying and assisting students who are potentially 'at risk' by virtue of what have technically been referred to as dissonant forms of study orchestration.
To lay the foundation for an introductory model of student learning it may be observed that, for most students, 'learning' represents a purposeful activity, the outcome of which has to be externalised for scrutiny in some formally recorded sense. Teachers often use such formally observed outcomes (such as examination results) as a basis for speculating why, or how, students differ in relation to one another given essentially similar exposures to, say, a course of study in some subject. Attributions may thus typically be made in terms of variation in other unrecorded observables such as effort, enthusiasm, interest, and so on. So, intuitively, most teachers are able to admit the idea that some of the variation in observed outcomes has a possible explanation in terms of other unobserved sources of variation. An immediate difficulty is that only a few such explanatory sources of variation (how hardworking a student is, for example) are amenable to direct observation, and then so only to a limited degree.
|Observable||As operationalised in responses to questions like||Dimension(s) of variation|
|Intention||What are you trying to do?||Seek understanding - 'make personal sense'|
Memorise in order to understand
Memorise in order to reproduce
Understand in order to memorise
'Play the system'
Seek high levels of achievement
|Motivation||Why are you doing it?||Intrinsic interest in object of study|
Pass examinations with minimal effort
Perceived future relevance or application
Need to meet external demands or fulfill 'duty'
Enjoyment of peer competition
|Process||How are you going about it?
What are you doing?
|Explore subject beyond formal requirements|
'Interrogate' subject - ask questions of it
Relate new knowledge to prior knowledge
Repeated rehearsal or recitation
Practise on previous examination papers
Various forms of study 'methods'
|Context||How do you perceive the learning environment?
What does the learning environment require of you?
|Perceived criteria for success - task demands|
Attributes of learning materials
Teacher's intention(s) - signalling of 'cues'
Freedom of choice
|What do you think 'learning' means?||Accumulating information|
Transforming information into knowledge
Changing as a person
Sources of variation thus exist that need to be externalised by students themselves in order to become accessible to an outside observer. The process of externalisation may take place via written or spoken responses to appropriate stimuli and, if carried out in a cooperative spirit, the process can yield data about the students, and the context of response (a course, say) that can inform a variety of actions.
The focus here is on such data expressed in numerical form, and as typically captured in terms of inventory responses. A simple example of how a single source of variation such as 'disorganised studying' may be operationalised, and how variation associated with this source may be exhibited, is presented in Figure 1 in the form of a frequency distribution. The context of response for the data in Figure 1 is a course in one of the professions allied to medicine. The respondents are a small group of first year students (n=34) who volunteered data as part of a pilot project on student learning.
The shape of the distribution in Figure 1 conveys the presence of inter-individual variation in the data that, even in this simple descriptive example, can serve an evaluative function. At face value at least one student identified as 'Case 34' is exhibiting a high level of response (a score of 24 on a scale of 5-25) relative to the rest of the group. Viewed in isolation, a general pattern of disorganised studying is not cause for concern, but it could represent one feature of a more problematic pattern of learning engagement. In contrast, the student identified in Figure 1 as 'Case 24' is exhibiting a low level of 'disorganised studying'. So, on one dimension of conscious reflection, these two students are highlighting at least one difference between themselves.
In any modelling process there is an interest in relationships between sources of variation. For the same student sample as in Figure 1, consider the scatterplot presented in Figure 2. In this case responses to 'memorisation' and 'fear of failure' exhibit a moderate linear association (positive correlation).
Thus, even the most basic representations of interrelationships between observed data can convey potentially useful information about the 'real people' that the data represent. But there are also limitations in terms of the inferences that can be made from such data, especially where there is a concern for the learning well-being of students. For example, it is clear from Figure 2 that the student identified earlier as 'Case 34' is, once again, signalling a pattern of response that is potential cause for concern. Concern might also be expressed in respect of 'Case 2'.
But when should such concern be translated into action? For an individual student, 'fear of failure' can represent either a positive or negative motivational influence depending on what else it is temporally associated with. An undifferentiated process of 'memorisation' can similarly be regarded as contributing to either the simple storage and recall ('cramming and dumping'), or to the internal transformation ('understanding'), of information. The difficulty here is that that there is insufficient modelling information available to determine the possible consequences of what 'Cases 2 and 34' have disclosed thus far. Theoretically, though, it could be speculated that a high level manifested combination of 'disorganised studying', 'memorisation', and 'fear of failure', as exhibited so far by 'Case 34', is a sufficient enough cause for concern; albeit one informed by a limited range of diagnostics. How much further multivariate complexity to admit really depends on how sensitive the model needs to be to individual differences.
In referring again to Figure 2, it may be noted that 'Case 2' performed well in the final examination and that 'Case 34' performed poorly. So there is a question as to where the real difference of consequence between these two students lies in terms of what they are capable of declaring about themselves. A partial answer may be found in an examination of the scatterplot, presented in Figure 3, of two more sources of variation that represent learning pathologies. There is again a clear separation between the responses of 'Cases 24 and 34' (as in Figure 1) and, at this stage of disclosure, there is no doubt that the overall pattern of learning behaviour exhibited by 'Case 34', compared to 'Case 2', represents a valid distress signal.
Similarly, an intention to memorise, a motivation based on fear of failure, a lack of process (inability to integrate new information), and a narrow focus on the syllabus, constitutes what may referred to as a broader composite 'reproductive' dimension of learning. On theoretical grounds it can thus be hypothesised that, although there are multiple observables here, the underlying conceptual model basically consists of just two more complex dimensions of variation.
The degree to which such a hypothetical two-dimensional model can be fitted to observed data in a strict statistical sense can also be determined. Given that there are linear relationships in the data (as shown, for example, in Figures 2 and 3) it is possible to statistically determine, via a process known as confirmatory factor analysis, and subject to certain distributional assumptions, what the underlying linear (covariance) structure in any chosen set of observed data would need to look like in this case in order to satisfy the conceptual specification of the model. ('Covariance' can be thought of as a more generalised form of 'correlation'.)
The conceptual specification, in essence, is that the observables must unambiguously define two linear structures that are (negatively) correlated with one another. The degree to which the resultant mathematically specified structure corresponds to the observed structure essentially determines how well the conceptual model fits the data, and therefore the degree of confidence that can be placed in the observed model for inferential purposes. Note that the distinction being made here is that the conceptual model represents the theory, while the observed model represents the responses of 'real people'. In the example under discussion the degree of fit is in fact 'very good' in statistical terms as determined by a variety of 'goodness of fit' indices (for example, Steiger-Lind RMSEA = 0.009 as per SEPATH module in Statistica/w 5.0).
The observed model (that is, the empirical manifestation of the conceptual model based on the observed data) is presented in Table 2; this is what a compact multivariate model 'looks like'. The observed model in this case literally communicates an abstract conceptual 'picture' within which all the students differ from one another in varying degrees. Such 'pictures' can serve as a basis for evaluation in their own right; for example, in a dynamic sense, successive 'pictures' can be compared over time within courses to obtain an overall impression of the impact of the course on students' learning (Meyer and Scrivener, 1995). In this case the 'picture' is a static one; interpretations of the two composite dimensions of variation in the 'picture' (the factors labelled F1 and F2) are unambiguously indicated by the magnitudes of the tabulated numbers (technically called the factor loadings) which can be thought of as determining the relative 'emphasis' of the corresponding source of variation within each of the dimensions.
|Source of variation||Illustrative items within each source of variation||F1||F2|
|use of evidence||I am usually cautious in reaching conclusions unless they are well supported by evidence||77||.|
|relating ideas||I try to relate ideas in this course to other ideas in this course, or to ideas in other courses||73||.|
|deep approach||I often find myself questioning things that I hear in class or read in books||70||.|
|intrinsic motivation||I find that parts of this course can often be really exciting||42||.|
|fear of failure||I am scared that I might fail this course||.||71|
|memorisation||I often find I have to memorise things that I don't really understand||.||69|
|syllabus boundness||I tend to read very little beyond what's required for completing assignments||.||54|
|disorganised study||My habit of putting off work leaves me with far too much to do before tests or exams||.||37|
More technically stated, a conceptually validated observed model presents a structural framework within which all the students contributing to the analysis vary relative to one another; in this case all the students vary simultaneously within two dimensions that, in conceptual terms, are basically incompatible. Furthermore, in terms of what has been observed in this case, and by virtue of the degree of 'fit' of the conceptual model, each student can be located within the structure of the model via estimates of what are technically referred to as 'factor scores'. A student may thus, for example, exhibit a low factor score on the 'meaning' dimension (F1) and a high factor score on the 'reproducing' dimension (F2). This is, in fact, the case for the student identified as 'Case 34'. On theoretical grounds such a student, and others with similar patterns of response (who would thus constitute an 'individual-similarity' subgroup), would be expected to exhibit a qualitatively poor learning outcome compared to a student with an opposite score pattern. There is thus an introduction to the concept of 'risk' in learning behaviour and, from a teacher perspective, a possible interest concerning the distribution of 'risk' within a given context of response.
More fundamentally, concerned teachers can differentially respond to the learning needs and problems of students as an integral part of academic practice. Responses generally need to address individual, or individual-similarity subgroup, needs in terms of (a) supportively assisting students to become aware of, and sensitive to the possible consequences of, their engagement of learning and, (b) the assumption of some (teacher) responsibility in assisting students to take control over any indicated process of change.
Teacher-based responses that focus on individual students can assume different forms that range from relatively benign learning conversations in which an informed interest is shown in how students are engaging learning (based on what students have disclosed), to interventions that are either implicitly or explicitly intended to assist students to alter 'high risk' patterns of learning engagement. Studies by Meyer and Kaschula (1994), and Meyer, Cliff and Dunne (1994) describe practical examples of how some such teacher-based interventions have been approached and implemented.
Two sources of variation specific to the diagnostic process in clinical medicine are represented in the scatterplot presented in Figure 4, which is based on the responses of fifth-year medical students (n=110). 'Premature closure' refers here to a diagnostic outcome that has been arrived at prematurely; for example a diagnosis based on a history taking without adequately considering the findings of the physical examination of the patient. 'Difficulty with likelihoods', as the title suggests, refers to an inability to assess diagnostic information in terms of likelihoods.
There is a considerable degree of inter-individual variation represented in Figure 4 that, in this case, can be interpreted and responded to strictly within the discipline. Students represented in the top right hand region of Figure 4 are legitimate objects of professional concern and curiosity. Bluntly put, there is no point in pretending that the individual-similarity subgroups at opposite ends of the regression line are similar in this example. Given additional sources of variation that are also discipline-specific it becomes possible to construct discipline-specific model forms that, as Meyer and Cleary (1997) have illustrated, can facilitate the identification of students exhibiting potentially 'high risk' forms clinical reasoning.
Cleary, E. G. and Meyer, J. H. F. (1997). The Conceptions and Experiences of Diagnosis Inventory. Department of Pathology, University of Adelaide.
Meyer, J. H. F. (1998). A medley of individual differences. In B. Dart and G. Boulton-Lewis (Eds), Teaching and Learning in Higher Education: From Theory to Practice. Camberwell: Australian Council for Educational Research, in press.
Meyer, J. H. F. and Cleary, E. G. (1997). Towards an 'interference' model of student learning in medicine. Symposium paper, Seventh European Conference for Research on Learning and Instruction, Athens, August 26-30.
Meyer, J. H. F., Cliff, A. and Dunne, T. T. (1994). Impressions of disadvantage. II - monitoring and assisting the student at risk. Higher Education, 27, 95-117.
Meyer, J. H. F. and Kaschula, W.A. (1994). Helping engineering students to learn better: The concept and creation of a learning 'hot seat'. In A. J. Smith (Ed.), Engineering Education: Increasing Student Participation. Sheffield: Sheffield Hallam University, 294-300.
Meyer, J. H. F. and Scrivener, K. (1995). A framework for evaluating and improving the quality of student learning. In G. Gibbs (Ed), Improving student learning through assessment and evaluation. Oxford: OCSD, 44-54.
Meyer, J. H. F. and Watson, R. M. (1991). Evaluating the Quality of Student Learning. II - study orchestration and the curriculum. Studies in Higher Education, 16, 251-275.
|Please cite as: Meyer, J. H. F. (1998). Variation in student learning: A basis for improving teaching and learning. In Black, B. and Stanley, N. (Eds), Teaching and Learning in Changing Times, 214-222. Proceedings of the 7th Annual Teaching Learning Forum, The University of Western Australia, February 1998. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1998/meyer.html|