Teaching and Learning Forum 97 [ Contents ]

Tertiary examination conducted on the Internet

Timo Vuori and Raj Gururajan
Department of Computer Science
Edith Cowan University

Motivation

The Department of Computer Science at Edith Cowan University is in the process of offering its traditional program to certain overseas countries. To reduce cost and other overheads, the delivery mode chosen is Internet. The Department is in the process of producing unit materials in the form that is suitable for the Internet access.

The process of developing electronic media based material lead to the thinking of producing a net based examination. The idea was strengthened by the fact that there are considerable delays in exam script handling with respect to external mode students of the Department.

The researchers established certain guidelines for such an examination so that the process can be successful. The objectives of this delivery mode can generally be categorised in to:

Based on the guidelines, the researchers are motivated to set an examination paper. The process took into account the objectives set, and yet prepare an electronic examination, that is possible to administer with minimum complexity.

An important point that needs to be noted is that the examination is dependent on factors like computing resources, network access and other security procedures. The researchers were motivated further by the fact the cost associated with printing, posting and receiving exam papers can completely be eliminated using this mechanism. Moreover, the marking guidelines can be set to a higher level of accuracy so that human interpretation and error can be minimised to a certain extent.

The process involved certain human intervention in marking. However, there is abundant scope to generate a completely automated exam, that can include conducting the exam on the Internet, marking and result generation. The complete automation is yet to be trialed.

Preparation

To prepare the students for the examination process, workshops were conducted to familiarise students on the search tools available and how to best utilise these tools to find specific information, on the Internet. The external students were provided with an exercise book which covered the same topics. A week prior to the actual examination, these tools were revived and specific issues related to the actual examination process were explained to the students. Similarly, an information package was send to the external students explaining how they could practice for the examination. An Internet page with the same access controls as in the actual examination was set-up so that students could practice and make sure that they can access the electronic page during the examination situation. Students were strongly advised to test their ability to access this test page and resolve any problems by contacting their tutor/supervisor prior to the examination.

Security considerations

In a normal paper based examination the question papers are securely stored after printing and then distributed to students in the beginning of the examination. The answer sheets are collected at the end of the examination period. In the examination utilising electronic medium similar measures need to be put in place to ensure the identity of the student and the confidentiality of the answers. The following discussion looks into the practical requirement set to examination performed using the electronic medium.

The availability of the electronic examination papers in these examinations were controlled in two ways. One, by making the examination paper available at a specific time to the internal students electronically on the Edith Cowan University's LAN. Note, this network is only accessible from inside the university premises and is not available for the external students. Two, making the examination paper available on the Internet for external students for a specific duration.

For the internal students there were no further need to control the access to the examination paper as the examination was supervised and limited to two hours duration. The same did not apply to the external students as their examination was self supervised. The access to the Internet page was controlled with two main measures. The first measure utilised the normal Username and Password combination to verify users identity. In this case the username was selected to be students ID number and the password was a combination of student first and second name. The second measure to control the access related to the availability of the examination paper on the Internet.

For practical reasons it would have been unreasonable to make the examination paper available for only the specified examination period of time. In this case it was made available for nine hours duration with 2 hours of availability for each student. The main reason for this arrangement was the limited number of modem lines available in the university's modem pool. These modems needed to be shared with the other users in the first in first served basis. Knowing this, it was actually recommended to use alternative Internet provider if possible to gain access to the examination paper.

The above mentioned arrangement to allow access to examination paper over long period of time created a new problem. How to restrict and supervise the time taken to answer the examination questions? In this instance, the students were let to know, that when they first login to the examination page by using their individual Username and Password indicates the start of the examination. This time is recorded by the Internet server. As the examination was two hours of duration the students were required to email their answers to a given email address on completion. It was made clear to the students that it is their responsibility to time their own examination. When the email is received, the time, clearly indicated in the message heading ensures that students have complied with the time limit of the duration of the examination.

To ensure that the internal students did not share their responses electronically, by utilising email, Internet Relay Chat (IRC) or other methods, the use of IRC was banned and their emails were logged, that is, all emails sent from the examination rooms were recorded for later analysis. This arrangement was explained prior to the examination process.

The same arrangements were not made for the external students as they were self supervising their exam. It would have unfairly disadvantaged the external students if they would have to use any other computer facilities than the one they are familiar with for the examination purposes. The supervision and practical arrangements of electronic examination for external students are issues to be resolved.

Special arrangements were put in place to safeguard against accidental loss of examination responses due to possible problems associated with the email. First of all the students were instructed to keep a copy of their responses in their send folder. This was done in case the student accidentally send their response to a wrong email address. Secondly, email responses were routed through a special email server where a copy of all responses were kept before forwarded to individual tutors for marking the examination. This was to safeguard against accidental deletion by a tutor or a failure in email service. In case of external students a copy was also send to External Studies for their records. This was the only occasion when the examination responses were actually printed out.

Feedback

In general, the examination seem to have gone without any major difficulties. Some minor difficulties were experienced in the internal examination process. These included the slowness of network when many students tried to access the same network services at the same time. In some cases there was a limit on the number of concurrent connections available on a particular Web page thus creating a denial of service when a large number of students tried to access it simultaneously. This problem was resolved by trying repeatedly to access the particular service a short time later.

Some problems were directly linked to the students' preparation of the examination process. Regardless of the training and clear instructions given prior to the examination, that all answers need to be submitted using the email system some students experienced difficulties. These students were allowed to submit their answers in a floppy disk.

A similar situation occurred with some external students when their computer system either crashed or the email server was not accessible. It is difficult to truly assess the actual nature of these problems experienced by external students.

In some cases students had not set up their electronic mail system properly and the examiners were unable to send their comments and examination marks back to the students electronically. In these cases the students had to contact their tutor to receive their marks.

Marking the examination papers

Marking of the electronic examination papers differs from the traditional paper based examination marking. This is assuming that the answers are marked in their electronic form and not printed out and then marked. First of all, it is easy to make a copy of the original answer sheet and add comments and correct or model answers to each question. These marked and commented answers can then be instantly send back to the student using email. In addition, both the original answer and the marked and commented answers can be retained in the electronic records in the case of student inquiry or complaint.

The second major difference is imposed by the tools available to work with electronic documents, that is, the need to use a computer to view and add comments/marks to answer documents.

Observation

Once the process was completed, the marks were compiled for submission. It was observed that the distribution of marks fell under the normal curve, without any scaling. The observation was interesting because the researchers anticipated that this electronic examination scheme to cause some abnormalities. The researchers expected students face some initial difficulties in the electronic system due to the newness of the concept. However, the feedback received and the marks scored by the students did not reflect any of the researchers views.

Issues

The electronic examination conducted raised issues that need to be investigated for further refinement of the system. The following issues are raised:
  1. How to better prepare students for electronic examination system ?
  2. How to assure uniform quality of this system for a multicampus situation ?
  3. How to improve the supervision of external students ?
  4. How to prepare exams to accommodate a variety of students ?

Further Research

The examination conducted was an initial effort. Even though basic security measures were taken care of, the researchers feel that there is scope to improve and refine the overall security aspects. For instance, the security aspects can include adaptation of authentication techniques.

The marking was done manually by the unit coordinator. However, the electronic system can be refined in such a way that the marking process can be semi automated to incorporate certain elements of the examination to be marked as and when a student responds to the question.

Conclusion

The research conducted to set and run an electronic examination is only the beginning. The process opened certain new concepts that can be tested in future research projects. One of the benefits experienced was minimisation of time delay and cost. The researchers felt that the Internet based examination is affordable and of a high quality.

Please cite as: Vuori, T. and Gururajan, R. (1997). Tertiary examination conducted on the internet. In Pospisil, R. and Willcoxson, L. (Eds), Learning Through Teaching, p341-344. Proceedings of the 6th Annual Teaching Learning Forum, Murdoch University, February 1997. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf1997/vuori1.html


[ TL Forum 1997 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1997/vuori1.html
Last revision: 11 Apr 2002. Murdoch University
Previous URL 15 Jan 1997 to 11 Apr 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf97/.html