Teaching and Learning Forum 99 [ Contents ]
Some observations about the transfer of educational technology
Nathan W Scott and Brian J Stone
Department of Mechanical and Materials Engineering
The University of Western Australia
We have been developing and using computer-based tutorial systems since 1995. Teachers who see the software in use have often approached us to find out about extending it for use in a new subject area or installing it a new site. As a result we now have five examples where staff have taken our innovations and either used them or improved upon them. We have noticed that
These findings should be of interest to university administrators, since this type of intensive support will only occur if it is allowed for in planning new development.
- A single enthusiastic person (the enthusiast) can do all the necessary work in setting up a new system, but if there is no-one like this involved at the new site then the project is likely to fail;
- It takes time and close interpersonal work to transfer the technology. The enthusiast is usually willing to learn but even so it takes several weeks of intensive cooperative work to bring this person up to speed;
- Continued technical support is critical to the ongoing success of the innovation at the new site. As the software is extended in new ways there are inevitably new technical problems that must be resolved.
As the twentieth century draws to a close the world of education is expending significant resources on developing and utilising computer based teaching, tutoring and assessment. The reasons for this are varied but may include some or all of the following,
Most academics are not the originators of innovative computer based teaching. Academics who are forced to use new technologies are well able to ensure that it is not a successful experiment.
- To be 'High Tech' and up to date.
- To improve efficiency.
- To improve outcomes.
- Distance education.
- Flexible Program Delivery (FPD)
- To do things that cannot be done any other way.
The most frequently asked question at seminars given on our work is "How long does it take to write such material?" In general, academics apparently do not have the time needed to change to a new method of teaching. Also in most institutions there is no incentive to improve teaching as promotion to senior levels rarely is achieved on the basis of teaching. The other main problem that restricts the development of new approaches to teaching is the uncertainty of the life of the initiative. Will the delivery platform remain stable? Will the course be taught by someone else? The time commitments required for developing such innovations require some expectation that the developer will benefit from the effort expended.
In this paper we present five case studies of institutions who have either taken some of our software or ideas. First there are three cases in the field of Engineering Dynamics:
Then there is the case of our UWA Dept. of Mathematics and the work of Dr Kevin Judd in Calculus, Statistics and Linear Algebra. Finally there is the case of Dr Moira Maley also at UWA, who created new tutorial problems in the field of Pathology.
- Overseas Prestige University A
- Australian Technical University B
- Australian East Coast University C
2. Description of the technologies
This discussion of the transfer of technology will make no sense unless the nature of the technology itself is explained. In each year since 1995 we have taught large first-year classes in Engineering Dynamics using a networked, monitored tutorial system. The system is unusual in that multiple-choice questions are NOT used. Instead we have written a long sequence of engineering problems that require the student to enter a numerical answer with units, e.g. "3.2 m/s". The software is somewhat diagnostic; we try to give the student meaningful feedback for incorrect answers. Each student has slightly different numerical parameters to each problem and so enters a different answer (although the words and pictures for each problem are the same for all students).
We have found that, if the problem set is carefully chosen, the usual student difficulties in the subject can be drawn out and addressed. The monitored nature of the system, the credit for assessed work and the deadlines, help students to allocate an appropriate amount of time to the work during the year. The result is an improvement in the standard of learning overall since students actually know some of the material before they arrive at the lectures or the examinations. We have seen a gradual improvement in pass rates and academic standards over the past four years, although this may be partly attributed to other factors such as better lecturing.
We encourage students to work collaboratively in solving the problems, even those which are assessed. We believe that the negative effect of the small amount of "cheating" behaviour that goes on is more than offset by the large positive effect of students working in groups. We also reason that "cheating" happens even with paper-based traditional tutorial methods.
Our tutorial systems also usually have "forums", which are bulletin boards about narrow subject areas, usually a single problem. These have proven to be a very efficient learning resource because once a student has posted a question on a forum,. and a staff member has answered it, the exchange remains as a record for all subsequent students. The forums become like FAQ files and the students find them very helpful.
By using these technologies we have reduced staff teaching time by a factor of three, without compromising either academic standards or student satisfaction. We see the new technologies as reducing the most mundane labour of teaching and multiplying the effectiveness of staff time. These features make our software very attractive to outsiders - hence the drive to transfer technology. A key point in the following discussion is that there may be features of our work which are not resident in the software and cannot be "transferred" in the same way, for example our enthusiasm for teaching.
3. Case histories
3.1 Overseas Prestige University A
OPUA invited Stone for several visits during the period 1995-98 and he taught several short courses there in addition to pursuing research in Vibration. He gave seminars about computer-based tutorial systems in 1995 and 1996 that were poorly attended. Then, in 1997, there was a change in policy at the highest levels of the University: teaching was given new importance. Additional funds were also released to support initiatives in teaching provided they were framed in terms of research proposals. Stone's seminars in 1997 and 98 were well attended and some higher-level staff showed interest. As a result a collaborative research project was set up.
The research project had several related aims:
A research officer was employed and Scott made several visits during 1997 and 98, to install software.
- a pilot project in engineering dynamics, to prove the technology;
- establishment of standards for web pages to deliver lecture notes with animations;
- transfer of technology to OPUA in the area of computer-based tutorial systems;
- the development of four new problem sets in new subject areas.
The pilot project was successful but it used entirely imported technology and personnel. Stone taught a short course in Dynamics using his own lecture notes and the course server computer was literally brought from Australia. When Stone had finished teaching his part of the course a local and highly experienced teacher took over. This teacher was sympathetic to the use of the software and (at some personal risk) chose to continue to use the tutorial system even though the support staff (Scott & Stone) had returned to Australia. Thankfully the system functioned well and his faith was justified.
The remaining aims of the project remain largely unfulfilled. It is hard for us as Australians and outsiders to fully understand the situation but we perceive that
The work at OPUA is not finished and there is still some chance that the project will succeed. However at this stage there is no local expert/enthusiast to carry the work forward when the Australians are not around.
- there are factions at work who seem to value their own ends more highly than those of the students or the university. Any innovative action done under the project attracts quick criticism from one or another of these parties;
- academic staff, who have detailed knowledge of student needs and difficulties through years of teaching, have been reluctant to get involved. The research officer is expected to create software but is not given the essential academic input. No staff member has come forward to have a set of tutorial problems converted to the computer-based format. One staff member has had some lecture notes converted but there were serious time constraints and the result was poor.
3.2 Australian Technical University B
A close associate of Stone, charged with teaching an almost identical Dynamics course, chose to install and use a "copy" of a successful system in use at UWA. This was achieved with only minimal technical support from Scott and is an example of efficient use of (if not true transfer of) technology.
3.3 Australian East Coast University C
This university saw a demonstration of the tutorial system at a conference and decided to try it on a pilot basis. They decided not to install a local server for the course but to attempt to have students log in "across the country" to a server in Western Australia. A subset of the usual two-semester Dynamics problem set at UWA was chosen to make a one-semester course.
Some technical difficulties were experienced early in the semester (second half of 1998) because the students were logging in from UNIX machines and there were some unexpected incompatibilities. These were resolved by moving the students to a Macintosh lab. The most damaging difficulties were to do with the content and type of the problems themselves. It became clear that the subset of problems chosen was too hard for the target class. It was no secret that the problem set was in use at UWA and resentment arose in the class at AECUC. Somehow the AECUC students failed to notice that they were doing half as many problems as their UWA counterparts, in half the time. There was a widespread misconception that they were being asked to do the same number of problems in half the time. The degree of resentment became apparent when AECUC and UWA students met at an interstate event, and the UWA students were verbally abused for being "too clever".
AECUC intends to try the system (in modified form) in 1999. They will use a local server and will reduce the amount of work required of the students. Some access restrictions will also be relaxed so that students can progress through the problem set even if some work is not completed.
3.4 UWA Dept of Mathematics
In 1995 the Dynamics classes at UWA were held in a computer laboratory jointly owned with the Department of Mathematics. As a result Mathematics staff saw the system in action and were sufficiently impressed that they asked Dr Kevin Judd to create something similar for Calculus. Judd went much further than his brief and created a sophisticated diagnostic package for Calculus based on Mathematica(tm). He adapted the server in use by Scott & Stone so that students could enter answers in the form of mathematical expressions. His diagnostic packages were able to provide quick, concise and accurate feedback, even if several errors were made.
Judd has since written several other diagnostic packages and even a new version of the server package. It should be noted that Judd is a career mathematician and has programmed computers for many years.
Student feedback about Judd's work was fairly critical in the first year, when there were some technical problems. Since then, however, he has refined his packages to the point where they are faster and more accurate than human tutors, and student satisfaction is now typically high. There is also evidence that the rigorous standards of correctness enforced by the software have led to better student work in later years. Students in second and third year seem to be better with the fundamental techniques of mathematics such as taking the derivative of an expression.
3.5 UWA Faculty of Medicine
In 1998 Dr Moira Maley heard a presentation by Scott & Stone and was inspired to do something of her own. In consultation with Scott a new problem type was invented: the "drag-and-drop micrograph". This innovation is described in detail in another paper in these proceedings, Drag & Drop Pathology: first experiences with a new computer-based tutorial system in the Faculty of Medicine. Once the initial examples of the new problem type had been created, Maley worked with several other academics in the Faculty of Medicine to create a library of several hundred problems. The other academics supplied interesting micrographs and assisted in the generation of diagnostic feedback messages. Maley's medical background enabled her to interpret the ideas of the other academics; her familiarity with the computer system enabled her to turn them into software. The new learning resource has proven to be popular with the students, who are often overly busy and appreciate efficient forms of learning.
Technology is really just an elaborate set of learned behaviours; it does not reside only in the artefacts made but in those who make them. Technology can only be transferred when a new person learns to apply a practical idea at a new site. Most real learning is hard-won and socially-based: it takes hard work by the learner, in the right context, over many years. So it is not surprising that we have seen success in only some cases. We have seen technology transferred when
Teaching technology can only take root at a new site if it addresses real student needs. Students seem to decide very quickly how best to invest time. Consequently teaching software must be
- there was an enthusiastic and somewhat knowledgeable person at the receiving site.
- the technology happens to work perfectly without modification at the new site, the receiving person need not know the detail of how it works (as in the case of Australian Technical University A). However, if the technology needs to be adapted, the receiving person must go at least half-way in producing the adaptations. An outsider cannot know exactly what is needed at the new site, and the receiving person may not have all the skills necessary to make progress. Vision and support are both required.
- in the case of teaching technology, there must be a close relationship between the students who are to use the technology and the inventors. Several forms of relationship have been seen to work:
- ideally the innovator is also the teacher - although if this were the only model possible, teaching technology would rarely spread since few teachers change disciplines.
- in the cases of Judd and Maley the innovator worked alongside the actual teachers and also had some contact with the students (e.g. taking one tutorial class in the subject). In this way they maintained contact with the student body and could respond quickly to fix errors in the software.
Ironically, staff are not well equipped to judge the quality of teaching innovations, because they are by definition "successful" rather than "struggling" students. They cannot know what students will find hard because they are the sort of people who learn quickly and have good strategies for coping with new knowledge. They are also removed from the place of the learner by many years of focussed study so that the subject matter is now "obvious". Therefore the views of students must be sought often and honestly while introducing new teaching approaches. It is not enough to take a survey at the end of each semester - daily attention to student comments and anecdotal evidence is needed. It is interesting to note that final-year projects which develop new teaching materials for earlier years tend to produce valuable approaches that would be missed by established lecturers.
- It should preferably do something that cannot be done in any other way.
- It must always be available when required (i.e. the hardware must be reliable and available).
- It must support social (i.e. collaborative) learning.
Truly valuable new software can only be written under a narrow set of circumstances. It is possible to "drive" a development forward (as at the Overseas Prestige University) but the quality and quantity of the resulting material cannot be compared to that generated by enthusiasts. Therefore the best way to support quality innovations is to bring enthusiasts together and let them find common ground.
Scott, N. W. & Stone, B. J. (1998). A flexible Web-based tutorial system for engineering, maths and science subjects. Global Journal of Engineering Education, 2(1), 7-16.
|Please cite as: Scott, N. W. and Stone, B. J. (1999). Some observations about the transfer of educational technology. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context, 367-372. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/scott.html|
[ TL Forum 1999 Proceedings Contents ]
[ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [email@example.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1999/scott.html
Last revision: 1 Mar 2002. © The University of Western Australia
Previous URL 22 Jan 1999 to 1 Mar 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/ns/scott.html