Teaching and Learning Forum 98 [ Contents ]

Web-based tutorial systems in use at UWA

Dr Nathan Scott and Prof. Brian Stone
Department of Mechanical and Materials Engineering
The University of Western Australia
Since March 1995 all of our tutorial classes in Engineering Dynamics have been held in large computer rooms, using a monitored, diagnostic, online tutorial system. The form of this system has changed over the years but the basic ideas have remained the same: We have published widely on this system and it is now attracting attention from around the world; the idea has also been taken up and used in several other courses. In this demonstration session we invite you to visit a large teaching space at UWA and try the software for yourself.

Venue: The Maths Computing Laboratory, first floor, Mathematics Department


1. Introduction: traditional tutorials in engineering

In the years before 1995 we taught Engineering Dynamics in small problem classes of about 20 students. Students were expected to submit a written assignment each week but there was no formal penalty for not completing these and only the best students submitted them throughout the year. Attendance at the tutorials was not enforced in any way and we observed that only a few students were coming at the end of each year. We felt that students were not able to make an appropriate decision about using these resources and so were setting themselves up for failure in the exams.

2. A computer-based tutorial system

After examining computer-based tutorial systems used elsewhere we developed one of our own. This system has the following features:
  1. Students log in using a password (See Figure 1a);
  2. A long series of engineering problems is solved, usually by entering a numerical answer (See Figure 1b);
  3. The software attempts to provide instant feedback to student misunderstandings about each problem;
  4. The system records all student activity and responses (see Figure 2);
  5. There are deadlines and assessed problems which count for credit towards the engineering degree;
  6. There is a messaging system integrated with the problem set, so that students can quickly get in touch with staff if no other help is available.
  7. A complete set of lecture notes, with interactive animations, are available at the computer terminals. The notes are also given to the students in printed form.
Initially (in 1995 and 1996) this system was implemented for Macintosh only. In 1997 our students used a completely Web-based version of the same system, and it is this version that will be discussed here.

(a)
Figure 1a

(b)
Figure 1b

Figure 1: (a) A screen shot of the login page for Dynamics 100; (b) A typical engineering problem presented by the system, showing the numerical answer.

In Dynamics 100 students had to complete about 200 problems, of which 37 were assessed. The problems were divided into 18 problem sets. Students had to complete the non-assessed problems in each set before they could attempt the assessed problems, and each set had to be completed before the next set could be started.

The introduction of this kind of computer-based tutorial led to a new kind of classroom culture. Students worked in a large computer room in the Mathematics Department (although there were other sites that would also work). The regular deadlines, along with the 20% credit for the assignments, meant that students worked consistently during the year. In turn this meant that students seemed to understand more in the lectures.

Figure 2

Figure 2: A visual display maintained by the server, showing the progress of each student.

When students made errors we had expected them to make the computer system responded with a useful diagnostic message, and this was then something which students did not need to ask a human tutor. The learning atmosphere in the computer room was one of active collaboration. Students worked closely together while solving the problems, and for most students this was a productive and valid learning activity. Some students abused this working approach by taking too much knowledge from friends - but this happens even under traditional tutorial methods. These two features of the system: the automated diagnostics and the collaborative environment, together led to remarkable efficiencies in teaching. A tutor was present in the computer room during advertised tutorial hours, but they were not always in high demand even though there were often 50 or more students.

If students were unable to get useful help from peers, from the automatic diagnostics, or from the tutors, there was an additional source of help. In 1997 each problem had an anchor or "hot link" that would take the student to the "forum" for the problem. Each forum was a text-only Web page which students could read or add text to. See Figure 3.

Wednesday, 3 September 1997; 2:13:58 PM; Adrian Norris
is the distance AO the same as OC, even though it does not look like 
it? If not, what is the distance AO?
          insert new text here

Wednesday, 3 September 1997; 2:16:02 PM; Nathan Scott
AO is NOT equal to OC. OC is the radius of the wheel! 
          insert new text here

Wednesday, 3 September 1997; 3:05:34 PM; Gareth Blakey
why isn't AO the radius of the wheel
          insert new text here

Wednesday, 3 September 1997; 4:40:22 PM; Nathan Scott
The problem clearly states that OA is [given value] and that the 
radius is [a different, larger value]. point A is NOT on the edge of 
the wheel but somewhat closer to the centre. 
          insert new text here

You can add text to this file: type your message in the box below and 
then click the button marked "Send". Remember that staff read all 
messages posted here.
Figure 3

Figure 3: Part of the Forum for the problem of Figure 1b.

The principle of the Forum was very simple but it proved to be an effective learning resource. It should be noted that students were anonymous to one another: if a student were to view the forum of Figure 3 then the names associated with the messages (except for Nathan Scott, a staff member) would be simply "Anon.". Generally the pattern of the interaction was [student question, staff response] but there were also cases where one student would answer the questions of another. In either case the forum became an enduring record of conceptual difficulties students had had with the engineering problem. Essentially staff were often able to answer a given question once. One student even made the comment that "[in the forum] I saw whole aspects of the problem that I did not realise were important". In other words, by seeing the conceptual difficulties of other students, this student had come to know about "holes" in her own understanding that she was not aware of.

It is important to note that each Forum was a separate record. If a student decided to visit a Forum this meant that he or she would only see information pertinent to a specific problem, and not a confusing mixture of (possibly irrelevant) ideas.

3. Student satisfaction and performance

During the introduction of the computer-based tutorial system we have monitored the examination performance of the students closely. See Figure 4.

In 1995 (the first year of computer tutorials) there was an unusually high failure rate, but in 1996 the rate was back to its historical level. We attribute this high failure rate to early system difficulties, coupled with difficult examinations. At the time of writing the examination marks for 1997 were not yet available.

For many years we have also taken anonymous surveys of student opinion. These surveys show that (even in 1995) student approval of the computer system was high.

Figure 4

Figure 4: Failure rates in Engineering 100 (Dynamics) (from Faye 1997)

4. Extensions to calculus and statistics

In 1996 Dr Kevin Judd of the Dept. of Mathematics at UWA extended our tutorial system. He developed diagnostic packages based around the commercial algebra engine Mathematica, which were able to detect misconception in several calculus subjects.

(a)
Figure 5a

(b)
Figure 5b

(c)
Figure 5c

Figure 5: Diagnostic software in Calculus. (a) a problem in differentiation; (b) the result of clicking the "Show" button; (c) automatic diagnosis of the student's attempt (b).

The power of these diagnostics should not be underestimated. This software is able to give the student precise, English feedback about errors in an expression, even if there are several or if they are deeply embedded (see Figure 5). This software has been used for several successful courses in both calculus and statistics, and it is clear that it is a powerful tool for learning. Students can 'drill' in mathematical skills at quite a high level, and receive instant diagnostic feedback. This is normally only achievable using intensive human tuition.

We will demonstrate Kevin's software during the session in the Maths Lab. The URL is http://calmaeth.maths.uwa.edu.au/

For further information contact Kevin: kevin@maths.uwa.edu.au

5. Conclusion: efficiency in teaching

Some of our early work in computer-aided education was to produce animation software, set 'movies' and lecture notes in hypertext format. We did this because it was then our view that students were failing dynamics through an inability to visualise the motion of objects. The resources that we generated were not really used by the students in a meaningful way, and we were not able to detect any improvement in either exam performance or teaching efficiency through using them.

We are now of the view that our first-year students actually needed a more structured tutorial environment; essentially that they needed 'parental' monitoring and guidance throughout the year. Our effort in producing animations and other 'multimedia objects' was not entirely wasted since they form the core of a valuable printed textbook for the course, and we use them as illustrations in lectures.

Efficient computer teaching can only occur when the system meets the real needs of the students.

Bibliography

Faye, C. (1997). Evaluating and comparing traditional and computer-based tutorials with respect to quality and cost: A Case Study of the Engineering 100 (dynamics) unit at UWA. Unpublished Honours thesis of the Dept. of Mechanical & Materials Engineering, The University of Western Australia.

Further reading about the tutorial system

  1. Devenish, D. G, Entwistle, R. D., Scott, N. & Stone, B. J. (1994). A computer package for teaching curvilinear motion. Australasian Association for Engineering Education 6th Annual Conference, University of Technology Sydney, 11-14 December 1994, pp 718-721.

  2. Devenish, D. G, Entwistle, R. D., Scott, N. & Stone, B. J. (1995). An assessment package with diagnostic facilities. Proceedings of International Conference on Computers in Education, Selected Applications Track Papers, Singapore, December 1995, pp 101-107.

  3. Scott, N., Devenish D. G., Entwistle R. D. & Stone B. J. (1994a). Dynamic Teaching Solutions. IEEE First International Conference on Multi-Media Engineering Education, The University of Melbourne, pp 419-427. [This paper has been published in a special edition of the International Journal of Engineering Education. It can be viewed at http://www.rzbt.fh-hamburg.de/ihw/tobepub/

  4. Scott, N. W. (1994b). Simple computer assessment of a sketched graph. Proc. ASEE '94 Conference, Edmonton, Canada, pp 2571-2574.

  5. Scott, N. W. & Stone, B. J. (1995). Student behaviour near a deadline as a predictor of academic success. Proceedings of the twelfth annual conference of the Australian Society for Computers in Learning in Tertiary Education (ASCILITE), The University of Melbourne, 46 December, pp 462-467.

  6. Scott, N. W & Stone, B. J. (1996a). Continuous monitoring and class performance. Proceedings of ASEE '96, Washington DC, 23-26 June, session 2220.

  7. Scott, N. W. & Stone, B. J. (1996b). An integrated computer environment for collaborative learning in engineering dynamics. Proceedings of HERDSA '96, The University of Western Australia 8-12 July.
Please cite as: Nathan Scott and Brian Stone (1998). Web-based tutorial systems in use at UWA. In Black, B. and Stanley, N. (Eds), Teaching and Learning in Changing Times, 300-304. Proceedings of the 7th Annual Teaching Learning Forum, The University of Western Australia, February 1998. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1998/scott.html


[ TL Forum 1998 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1998/scott.html
Last revision: 14 Mar 2002. The University of Western Australia
Previous URL 7 Jan 1998 to 14 Mar 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf98/scott.html