A current area of teaching and learning research within our School is in reviewing and incorporating existing methods as well as developing new methods to produce a large scale software-based mentoring system which will benefit students as they progress through their university course.
The mentoring system is being developed in Java and incorporates the disciplines of network communication, database management, user interface design and software agents and a pedagogy from the Computer Aided Instruction (CAI) and Intelligent Tutoring System (ITS) fields of education.
The system is Web-aware and ultimately, will operate cognizant of the large number of online units available from our School. The system will employ re-usable communicating software agents and enable a student to "program" new information retrieval or management tasks via the software agents to help them in their university courses.
The Mentor System is designed to address the following elements:
- to determine if the resulting system is beneficial to students using it.
- to determine if the system can be scaled and applied to a real-world, unconstrained environment.
This vignette details the Mentor System and discusses the potential benefits of software-based mentoring.
The final Mentor System (shown in figure 1) requires the development and implementation of:
The entire system - a suite of cooperating network based computer programs, databases and user interfaces runs on a network of Unix workstations and PCs. Core to this is the central Mentor daemon or server - a Java based mini operating system in its own right.
Java has been particularly successful in educational areas due to its Web presence in online teaching material. However, this small scale limited use does not indicate how Java may scale up for real world performance in real world educational situations. Similarly, although the literature indicates that software agent systems and intelligent tutoring systems are successful, research in these areas has been limited to small scale, restricted environments. There is a need for the development and analysis of the software-based Mentor System applied in a real-world environment. It is only very recently that researchers have examined applying agent technology in learning environments and no case studies or large scale applications of this technology have been reported. Given that software agents, Java and the World Wide Web (WWW) have all been subjected to excessive hyperbole, it is essential to understand what Java and Java based software agents can accomplish, especially in the field of education.
A good introduction to and overview of agents can be found in the Communications of the ACM (CACM 1994a) as well as at http://www.cs.bham.ac.uk/~amw/agents/. Cheong (1996a) is a good general referenced work covering agents and many related aspects of the Web. The collection Designing Autonomous Agents by Maes (1990a) gives little practical information on designing agents but is a standard work in the field. The University of Maryland-Baltimore County maintains an AgentNews Webletter which contains varied current material about agents as well as links to other sources of agent information. However, Foner (1993) gives a refreshingly insightful and amusing sociological study of an agent.
Intelligent Tutoring Systems (ITS) are well established - a good source of online information is available from the reference section of the Masters thesis by Farrow (1995) with similar information from Inlärning (1995) and from the Tap Project (1995). McArthur (1992) is a well researched and referenced paper on the role of Artificial Intelligence in education with specific sections outlining the MicroWorld concepts. Eklund (1993) (Eklund 1995) looks at the problems for ITS given the new Hypermedia Systems (HMS) and how a new cognitive model is needed for our educational material and its presentation. Brusilovsky (1995) and Brusilovsky (1995) reports on research into ITS and HMS within a Web framework and Mark and Greer (1993) report on evaluation methodologies for a modern ITS.
Currently, only a few researchers (Cheikes 1995, Cheikes 1995, Chan and Chou 1995) have put agents and ITS together. Cheikes (1995) asks and to some extent answers the question: "Should ITS Designers Be Looking For A Few Good Agents?". The Gia system (Cheikes 1995) uses a minimal set of KQML and KIF to meet the needs of arbitrary distributed, collaborating heterogeneous intelligent agents and explores the advantages of using agent technology in an ITS environment.
Johnson (1992) (Johnson and Rickel 1996, Rickel and Johnson 1997, Rickel and Johnson 1997) reports on research into using agents in virtual learning environments where functions such as student modelling and coaching are assigned to individual agents rather than catered for by the overall ITS. The article by Johnson (1992) provides valuable references for those interested in agents and ITS.
Other innovative uses of the Web and associated technology have been reported by Philippe Duchastel (Duchastel and Spahn 1996). The use of an agent as a student model in an ITS has been hinted at by some (Hubler and Assad 1995, Major 1993, Project 1995) and advanced CAI Systems are starting to emerge (Ritter and Blessing 1995, Woolf and Hall 1995, Centinia et al 1996, Lesgold et al 1996, Reichgelt et al 1996). Some ITS researchers are investigating using Java as the development language.
Various (online) journals are now publishing articles on (Java) Agents-in-Education : Compute-Ed, Australian Journal of Educational Technology, Journal of Educational Multimedia and Hypermedia, Journal of Interactive Learning Research, Journal of Technology and Teacher Education and the Journal of the Asynchronous Learning Network. Some journals, such as reported in Kahle (1995) and Shulman (1995) are also becoming aware of the role of Web and Agent technology in the education arena.
Many articles concerned with Java, Software Agents and ITS can be searched for and retrieved from the Web page http://www.cs.curtin.edu.au/~raytrace/phd_papers.html
If a user has never used the system, then he/she is introduced to it via a series of explanatory windows:
Figure 2: The user does not have a User Model to start with although the Mentor System will know what class - Staff (Technical, Administrative or Academic), Student (Undergrad, Honours or PostGrad) or Guest - he/she is.
These help determine the initial capabilities of the user model. It is assumed that the user understands the point and click paradigm and that he/she may choose not to use the system. The "Tell me later" button simply closes the interface, and "Help" will run a Web browser to give hyper and multi media help.
As the user progresses through the panels above, user model requirements are built up (these can be added to later) and at some stage, the actual UserModel is created. This user model is a Java class that is updated dynamically - user "raytrace" is characterised by the Java file called "raytrace.class". The source code for this file is available to the user - we assume that advanced users will want to take control of their user model to add other tasks such as specific data mining, interaction with other environment utilities, etc.
As the Mentor System learns about the user's preferences, it will update the user model and recompile it as appropriate. At some stage the user may want to take control of their model and maintain it themselves. This poses a problem for subsequent updating of user information; what units the person may be enrolled in, who they mean when they say "geoff", what their current research interests are, etc. A possible solution under investigation is via the Current Context mechanism that is decoupled from the user model and will be updated as appropriate and available to the user model regardless of whether it is maintained by the user or the Mentor System.
The main text interface between the user model and the Mentor System is via a Query client type interface integrated into the user model. An actual but contrived dialogue is shown on the following page and shows some of the functionality of the system. The user input is in the top window and the Mentor System response is in the bottom one.
The Mentor System does not have Natural Language Parsing but has a limited ability to understand user requests. It will greet the user in a non-deterministic fashion occasionally using various parts of the user's full name. In a similar manner it will respond to various requests with varying phrasing such as "what is the weather", "what is the time", etc. Occasionally, in response to a weather update request, it will data mine a weather Web site to report accurate meteorological information.
The system makes heuristic guesses about the subject of enquiries. In the example shown, the system has accurately guessed the identity of "geoff". With the implementation of the Current Context mechanism, the system will be able to be corrected if it guesses incorrectly so that it will learn and not make the same mistake in the future. Notice as well that the system knows the sex of the subject (and of any requested individual) and adjusts the output accordingly. Currently and unfortunately, the system only knows about typical English names. Since many of our students are from South East Asia, further work needs to be done to accurately identify the sexes from their Asian names. With the implementation of the Current Context mechanism, the system will also understand that having asked "who is on lillee" and then asking to talk to "architec", that the user means the person reported from the first request.
The Mentor System is mainly a mechanism for mentoring but it is also hoped that it will foster a communication environment between users. This may typically be between students or between student and lecturer. Ultimately, the communication channel is hoped to be multi-media in nature. The user may ask the Mentor System to "tell" a friend something. If the person is not currently using the system or does not want to be seen, then the system can send email or store a message for later transmission when the friend starts using the system again. Messages are received and the user is notified, asynchronously. The standard user model has the facility to screen incoming requests and also the displaying of messages
As a user's capabilities increase and they change to maintaining their own user model, they may also want to "program" various user model tasks such as specific data mining to help in their research. Details of Java in the Software Agent area have been reported elsewhere (see http://www.cs.umbc.edu/agents/news/). The Mentor System will provide hooks to easily enable standard data mining tasks.
The Mentor System is still being developed but is stable. It has been used by a limited number of people as it evolves and seems to be able to meet their limited needs. The real test, as always, is whether the system can still work with increased functionality and scale up to real world conditions such as a School full of students and lecturers interacting. Trials are scheduled within the School of Computing for 1999.
Whether it is an educational benefit is another matter!
Figure 3: Another Mentor Client which is the base-level querying part of the User Model. See the text for an explanation of the example dialogue between the user and the Mentor System.
|Author: Andrew Marriott|
School of Computing
Curtin University of Technology
Please cite as: Marriott, A. (1999). A lifelong mentor system. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/marriott.html