Teaching and Learning Forum 99 [ Contents ]

Changing conceptions with an "intelligent tutor"

Shelley Yeo, Robert Loss, Marjan Zadnik,
Physics Education Research and Development Group, Department of Applied Physics
David Treagust, Science and Mathematics Education Centre
Curtin University of Technology
This paper reports results of a study of students' learning of qualitative physics concepts while using the intelligent computer-based instructional (ICBI) program, Freebody (Oberem, 1996). Students' force and motion conceptions, which underpin their ability to construct free-body diagrams (abstract representations of forces acting on an object), were assessed before and after use of the program. The program helped students to confront inconsistencies in their reasoning about Newtonian physics but results suggest that new knowledge was about classifying the context rather than conceptual change.

Introduction

Freebody is a computer-based intelligent tutoring program based on the natural language processing system, ALBERT (Oberem, 1994). It assists students to identify forces and practice drawing free-body diagrams. Underpinning students' ability to construct free-body diagrams are their conceptions of force and motion. Students are given physical situations and asked to draw using the mouse, a free-body diagram for a named object. The program 'discusses' students' diagrams with them in plain English dialogue. It recognises certain 'misconceptions' about forces and attempts to correct them. Finally, if the student's diagram is inconsistent with either the problem description or their text description, the program tries to negotiate a correct resolution. In this way, Freebody attempts to model a human tutor.

An 'intelligent' tutoring program actively builds up a picture of its user and adapts or designs instructional strategies to meet their ever-changing needs and proficiency. Such a program consists of three 'models', the Expert Model, Student Model and Instructional Model. The Expert Model contains a database of relevant subject-based knowledge. The Student Model contains the framework for identifying a user's misconceptions and sub-optimal performance. It contains a database of misconceptions and missing conceptions. A missing conception is an item of knowledge which the expert has but the student lacks; a misconception is an item of knowledge that the student has but the expert lacks. The Instructional Model contains the program's pedagogical strategies.

Background

Many of the basic concepts and ideas of Newtonian physics are at variance with children's understandings of the world (Carey, 1985). Children's alternative conceptions about force and motion persist into adulthood and prove highly resistant to change (Driver, 1989). Bliss and Ogborn (1994) suggest that an infant's whole concept of 'cause and effect' is learned through and embodied in the conception 'effort produces motion'. Many students rote learn Newton's Laws in school science, assimilating this new knowledge alongside existing discrepant or contradictory information. At some time later, or given a new context, they revert to their pre-existing intuitive beliefs (Posner, Strike, Hewson & Gertzog, 1982).

Conceptual change may occur following student's realisation that their existing beliefs can longer satisfactorily explain their observations, and provided that a new belief is plausible and proves fruitful is its further application (Hewson, 1981). The process of disequilibration in which the learner realises that beliefs do not match observations is characterised by both rational and emotional responses (Gorsky & Finegold, 1994). Several researchers have attempted to describe the nature of conceptual change, from students' development of synthetic models (Vosniadou, 1994) to change resulting from ontological changes in concepts (Chi, Slotta & DeLeeuw, 1994 ; Dykstra, Boyle & Monarch, 1992; Thagard, 1992).

Earlier research (Yeo, Loss, Zadnik, Harrison & Treagust, 1998) with students learning conceptually difficult physics using an interactive multimedia program, found that students proceeded too rapidly through the program, often ignoring instructions and key details. It was suggested that students placed their own, often incorrect, interpretations on what they looked at and ignored that which contradicted their prior conceptions. Although learner control is increasingly favoured in educational multimedia programs, user control in this example enabled users to avoid confronting and/or resolving their incorrect alternative conceptions.

Aim

The aim of the study was to evaluate the effectiveness of Freebody in changing students' force and motion conceptions and to examine the ways in which the conceptual change processes occurred.

Method and data collection

A pretest/posttest was used to assess the change in students' conceptions. An interpretive methodology focusing on students' dialogue and actions, and analysis of subsets of pretest/posttest questions, was used to investigate the conceptual change process.

Subjects
The study involved 58 students, 26 females and 32 males, from five different classes, four at high school and one at university. All school students (N=59) were in Year 12 Physics classes and had studied physics in Year 11. The university students (N=5) were studying a service physics unit and had less formal physics backgrounds. Most students worked in pairs.

Pretest and posttest
The pretest/posttest was developed from a force-concept inventory, originally produced by Hestenes, Wells and Swackhamer (1992), and described by Mazur (1997). The selected items were modified: some contexts made more familiar to local students, the multiple choice format modified requiring students to agree or disagree with each of a number of statements about a given context and students asked to give a written explanation for some answers. There were 45 questions in 19 different items, 26 about identifying forces acting in given situations and 19 about the relationship between force(s) and motion.

Other data
The software recorded all student/computer interactions and dialogue in a text-based 'user history'. In addition, four pairs of students were videoed so that their spoken dialogue and physical interactions with the program were recorded simultaneously with the computer screen, on a picture-in-picture videotape image.

Procedure
Prior to using the program, each student completed the pretest and then after the program were given the same questions as a post-test. Students took about 25 minutes to complete each test. Most had 75 minutes to work on the program, although Group 3 students had only an hour. Not all students completed the 10 Freebody exercises.

Results

Table 1 lists the pretest and posttest results for all participants and Table 2 shows the results by class groups. The average improvement of 3.8 (14%) out of a total of 45 is significant (p<0.01%). Four of the groups (1,2,4 and 5) made a similar improvement (see Table 3), although one of these (group 5) cannot be shown to be statistically significant because of the small number in the group. Group 3 showed some improvement, however this was not statistically significant.

The Spearman-Brown coefficients for pretest and posttest were 0.89 and 0.84 respectively suggesting that the test was a reliable measuring instrument.

Table 1: Data for all students on pretest and posttest (N=58).

Pretest meanStDevPosttest meanStDevDifference
between means

27.25.8 31.05.33.8*

* Denotes significant at 0.01% confidence level.

Table 2: Pretest and posttest results for the five different groups

GroupNPretest meanStDevPosttest meanStDevDifference
between means

1
2
3
4
5
12
15
13
13
5
27.5
28.2
28.8
24.8
25.6
5.1
6.2
5.5
5.1
8.8
32.0
32.1
31.0
29.4
29.8
5.2
5.9
4.7
5.4
5.2
4.5*
3.9*
2.2
4.6*
4.2

* Denotes significant at the 0.01% confidence level.

Discussion

This paper examines the nature of the conceptual change in students using Freebody. Two earlier papers (Yeo, Loss, Treagust & Zadnik, 1998 a,b) discuss the quantitative results and student collaboration aspects of this study.

The instructional design of the program adopts a constructivist rather than instructivist approach, discussing students' ideas with them and encouraging them to resolve inconsistencies or contradictions in their beliefs. Two students, Sam and Adam (S&A), had difficulty with Exercise 8, which shows a boy having thrown a rock in the air. They (incorrectly) drew an upward 'propelling' force on the rock. Freebody (C) suggested they consider an inconsistency in their reasoning.

C:What kind of force is force #1?
S&A:Normal force
C:What exerts this normal force?
S&A:Boy
C:On what is this force being exerted"
S&A:Rock
C:A normal force is a contact force. The boy is not in contact with the rock. Can the boy exert a normal force on the rock?
S&A:No
C:So what is exerting this normal force which you drew?
S&A:Boy
C:A normal force is a contact force. The boy is not in contact with the rock. Can the boy exert a normal force on the rock?
S&A:Yes
C:I don't see how this can be. A contact force implies contact between the objects.
Sam and Adam's belief in the existence of the force was strong enough for them to assume that their problem was inability to name the force, rather than it not existing. Sam and Adam then called it a "non-contact force", a "velocity force", a "vertical force", "up" and finally "contact force". Eventually, they changed it and drew a gravitational force. Sam was still unhappy about their diagram which had only one force on the rock. These two could not agree to reject the upward force and Freebody was unable to negotiate a satisfactory resolution. Adam had reached a conclusion about which he was not sure, but the alternative was at least plausible. Sam remained unconvinced; for him the alternative was not sensible.

Why did Sam and Adam eventually delete the 'upward' force? Either because there was nothing which could exert a force upwards or because it became the only solution to the dilemma that they faced. If the former, it represents an important cognitive step. If the latter, it may simply be a rote-learned response for the particular context, and not necessarily transferable to another context. Four questions in the pretest/posttest examined the alternative conception: force of motion (A moving object has a 'force of motion' in it. A body slows down and stops as its force is used up. McCloskey, 1983). These two students were incorrect on two of the four questions which directly tested for this conception on the pretest but were correct for all four on the posttest.

Such incidents lead to an examination of sets of questions which examined a single concept or conception. The following describes one such group of three questions.

Questions 13 and 14 were about identical situations; an aeroplane flying at constant speed and height. Question 13 provided a diagram of an aeroplane and showed four balanced forces, thrust, drag, weight and lift; part 13C asked if the velocity of the plane was constant. Question 14 referred to a text description of an aeroplane with motion consistent with the diagram in Question 13. Part 14A asked if the net force acting on the plane was zero. Question 19 gave a situation of constant speed under the effect of several forces - a person pushing a box at constant speed. Part 19A asked if friction was less than the [forward] applied force. In effect, Questions 13C, 14A and 19A were all based on equivalent situations - a body moving with constant speed. The number of students who answered correctly are shown in Table 3.

Table 3: Number of students who answered Questions 13C, 14A and 19A correctly (N=58).

Number of students correct
Q 13CQ 14AQ 19A

Pretest53216
Posttest584111

An explanation for the variation in pretest results is that students have learnt to recognise the diagram used for Question 13C, that of a vehicle having four balanced forces moving at constant speed, and most answered using a rote-learned response. Students learn to apply Newton's first law in similar contexts. It appears that many students did not recognise that Question 14A applied to an identical situation as Question 13C and so applied naive reasoning, most probably a common alternative conception: A constant force acting on a body produces a constant speed (McCloskey, 1983). Why then, did only six students answer Question 19A correctly since it was (as far as physics principles are concerned) similar to Question 14A? We suggest that, unlike Question 14A, fewer students had encountered the context for Question 19A and hence the context was treated by most with intuitive reasoning alone.

Following use of Freebody, all students correctly answered 13C and there was an increase of 20 answering 14A correctly. It is likely that many of these students came to see that Questions 13C and 14A referred to identical situations. Nevertheless, 19 students still used their original force-motion conception for Question 14A despite answering Question 13C correctly. Very few students changed their minds about Question 19A.

Many students held conflicting conceptions, both before and after use of the program. This is explainable if a conception is inextricably linked to a context and does not exist without a context in which it applies. All of the students had studied Newton's laws and would have written or talked about their Newtonian understandings in textbook or teacher-generated contexts. It may be that the context itself is what students retain in memory, perhaps categorised as 'situations consistent with Newton's first law.' Only contexts which closely resemble those categorised thus will be recognised by students as ones to which Newton's first law applies. The limited improvement in Question 19A was possibly because most students still did not recognise the situation as one to which Newton's first law applies.

Two other groups of four questions also support the ideas developed here, that conceptual change is linked to students' learning about contexts. This study, therefore, suggests that in teaching conceptually difficult ideas, students need to become familiar with many applications of the conception as possible if they are to start to believe the universal applicability of the ideas.

Conclusion

Both before and after use of the program Freebody, year 12 students simultaneously held conflicting conceptions about force and motion. Although interaction with Freebody appeared to effect a degree of conceptual change, students' new ideas seemed linked to their learning about classifying (or re-classifying) situations to which Newton's specific laws can be applied rather than a knowledge change about concepts and the causal links between them.

References

Bliss, J & Ogborn, J. (1994). Force and motion from the beginning. Learning and Instruction, 4, 7-25.

Carey, S. (1985). Conceptual Change in Childhood. Cambridge, MA: MIT Press.

Chi, M. T. H., Slotta, J. T., & De Leeuw, N. (1994). From things to processes: A theory of conceptual change for learning science concepts. Learning and Instruction, 4, 27-43.

Driver, R. (1989). Students' conceptions and the learning of science. International Journal of Science Education, 11(5), 481-490.

Dykstra, D. I., Boyle, C. F., & Monarch, I. A. (1992). Studying conceptual change in learning physics. Science Education, 76, 615-652.

Gorsky, P. & Finegold, M., (1994). The role of anomaly and cognitive dissonance in restructuring students' concepts of force. Instructional Science, 22, 75-91.

Hewson, P. W. (1981). A conceptual change approach to learning science. European Journal of Science Education, 3, 383-396.

Mazur, E. (1997). Peer instruction. Englewood Cliffs NJ: Prentice Hall.

McCloskey, M. (1983). Naive theories of motion. In Gentner, D and Stevens, S. (Eds), Mental Models. Hillsdale: L. Erlbaum.

Oberem, G.E., (1994). Transfer of a natural language system for problem-solving in physics. In Proceedings of ED-MEDIA 94: World conference on Educational Multimedia and Hypermedia, Vancouver.

Oberem, G.E. (1996). Freebody. Computer software available Physics Academic Software.

Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66, 211-227.

Thagard, P. (1992a). Conceptual revolutions. Princeton, NJ: Princeton University Press

Vosniadou, S. (1994). Capturing and modelling the process of conceptual change. Learning and Instruction, 4, 45-69.

Yeo, S., Loss, R., Zadnik., Harrison, A. & Treagust, D. (1998). Learning physics using interactive multimedia: Research and practice. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching held in San Diego, April, 1998.

Yeo, S., Loss, R., Zadnik, M. & Treagust, D. (1998b). Learning with Freebody: Importance of student collaboration. Submitted for publication in proceedings of 23rd Western Australian Science Education Association conference held at Curtin University of Technology, Perth, 1998.

Yeo, S., Loss, R., Zadnik, M. & Treagust, D. (1998a). Students learning about force with Freebody. Submitted for publication in proceedings of OzCUPE 1998, sub-meeting of the AIP National Congress held in Perth, 1998.

Please cite as: Yeo, S., Loss, R., Zadnik, M. and Treagust, D. (1999). Changing conceptions with an "intelligent tutor". In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context, 474-483. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/yeo.html


[ TL Forum 1999 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1999/yeo.html
Last revision: 1 Mar 2002. The University of Western Australia
Previous URL 26Jan 1999 to 1 Mar 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/tz/yeo.html