|[ Teaching and Learning Forum 2001 ] [ Proceedings Contents ]|
A project was designed to evaluate and implement online assessment procedures and tools that can be utilised by both on campus and distance students. The primary instruments used to achieve this are online quizzes that were trialed for three units in Semesters 1 and 2 in 2000 within the Geographic Information Science program. The quizzes were developed using the WebCT courses management software and feedback was obtained from the students using written and online surveys. Additionally, virtual field trips were designed, primarily to cater for distance students.
A core component of the geographic information science program is the development of student skills using practical sessions in computer based laboratories. The practical sessions complement the underlying concepts and foundational knowledge that has traditionally been the domain of the lecture (Thomas 1995). It is necessary and important for students to develop their knowledge and skills in tandem by maintaining a consistent study program across both the practical sessions and the lectures. This has been reflected in the assessment procedures involving laboratory sessions, assignments, mid-semester tests and final examinations.
Over the past decade, there has been very little change in the assessment procedures other than minor variations. Yet, the main issues regarding assessment still remain, namely, providing timely and prompt feedback to students, and using assessment to motivate students to keep up to date with their study program. Most of the existing assessment takes place in the latter half of the semester. Hence, students don't receive their first assessed work and feedback until well into the unit. Further, in the case of the practical sessions, it is difficult to monitor the students' progress on a weekly basis unless considerably more resources are utilised to grade and return feedback more frequently. The Internet and the online learning environment have opened up new and exciting opportunities to address these issues (Forer 1997, Kemp and Unwin 1997, Veenendaal 1999, Veenendaal 2000, Winship and Carter 1999).
The aim of this project was to develop and evaluate online assessment procedures. The specific objectives were to:
The online quizzes were trialed in the first semester for the Geographic Information Systems (GIS) first year unit and the Spatial Analysis (SA) third year unit. In order to test the appropriate procedures for delivering the assessment, a different approach was taken for each unit. For the GIS unit, a quiz was provided every fortnight on average, each quiz being approximately 30 minutes in duration. For the SA unit, the frequency and duration of the quizzes were altered, being held every 4 weeks and 40 minutes in duration.
In both cases, the quizzes were available to the students for a limited one (1) week window. The students could commence the quiz, available from the Internet, anytime within this window, although once they had commenced the quiz, it had to be completed in the given time. Only one attempt was provided, hence it was imperative that the students were prepared prior to completing the quiz. The quizzes were comprised of a series of multiple choice, matching, and calculation questions that could be graded automatically to facilitate a fast response to students, particularly for large classes. A sample of the first part of a quiz indicating the response to the student is illustrated in Figure 1.
Figure 1: Sample of an online quiz and feedback
Once all students had submitted their quiz, the results were released to students. This occurred prior to the following quiz being available. The students could utilise the feedback to assist them in understanding their strengths and weaknesses, and preparing them for the following quiz.
Although the same quizzes were initially used for both on campus and distance students, it was soon realised that this was not possible when there were variations in the practical session work (primarily). It was decided fairly early on to develop separate targeted quizzes for on campus and distance students. In terms of the implementation of this, separate Online Quiz links had to be generated for the on campus and distance students within the unit web site. In addition, an additional link of Deferred Quiz was made available for both on campus and distance students who had a valid reason for receiving a deferred quiz. This option was used quite frequently by the distance students who had valid reasons for deferring their quiz based on other commitments.
The students in both classes were surveyed at the end of the semester (using an online survey) regarding the usefulness and procedures used for the online quizzes. The feedback was used to implement a series of quizzes for the Geographic Data Analysis (GDA) unit in the second semester. Based on this feedback, quizzes were held every fortnight and were restricted to 30 minutes in duration. On campus students were given a one week window to complete the quiz, whereas distance students were given two weeks. The reason for this was to provide flexibility for the distance students who were often inundated with other work and travel commitments. The results were then released after the two week window was closed.
The second semester quizzes involved random questions where students would not all receive the same set of questions. This measure was taken to reduce and avoid the temptation for students to collaborate with each other regarding solutions. Emphasis was placed on the need for students to take responsibility in their own understanding of the material and developing their own skills rather than relying on others. Overall, the online quizzes accounted for no more than 10% of the final assessment, again to avoid any unnecessary collaboration simply to achieve the maximum possible marks.
The online student survey was again conducted at the end of the second semester to obtain student perceptions regarding the usefulness and organisation of the online quizzes.
To create a virtual field trip, a video was taken of the field trip activities of the on campus students. Still images were then obtained from this video, placed online, and made available to on campus and distance students. Approximately 30 to 50 images were utilised for each field trip. By placing the field trip "online", the distance students could "see" the physical environment as well as "follow" the students performing the activities required for the field trip. The data obtained by the on campus students was provided to the distance students who could then complete the same assessed work as the on campus students. Figure 2 illustrates a sample of part of the online virtual field trip.
Figure 2: Sample of an online virtual field trip
Figure 3: Semester 1 distribution of students based on program of study
The students were surveyed regarding the value of the quizzes, the feedback that they provided, the extent to which the quizzes were a source of motivation, the marking scheme utilised and how this unit compared with others. A scale of 1, representing "poor", to 5, representing "excellent" was used. The results, as illustrated in Figure 4, indicate that the online quizzes were highly accepted and viewed by the students as an excellent source of feedback and motivation. The comparison to other units indicates that this was probably the only instance in which students participated in online assessment.
Figure 4: Results of Semester 1 student survey
Students were also able to provide open ended comments as part of the survey. The following is a sample of responses:
"Online assessment provides flexibility within a set time and access from home or work, freedom in own time management"Overall, the students enjoyed the flexibility of the quizzes and the value that it added to their study program. They did prefer the shorter but more frequent quizzes. This was particularly true for the SA students who experienced the longer but less frequent quizzes. This was therefore put into place for the second semester trial.
"Please convince others to use web and online quizzes for units"
"It would be good to have a quiz after each lecture"
"Quizzes provide good motivation to keep up to date with studies, indicates is studying on track, highlights weaknesses"
"I found the feedback really useful and it will be great as a study tool for the exam"
"Ongoing assessment such as quizzes very good to keep the learning curve going. Gets learning happening early!"
The distribution of the second semester students and the associated survey results are indicated in Figures 5 and 6. Of 55 students in the GDA unit, 20% were distance students. There was a greater proportion of students in GIScience students and fewer from other degree programs.
Figure 5: Semester 1 distribution of students based on program of study
Figure 6: Results of Semester 2 student survey
The results of the second semester survey, as for the first semester, showed that the quizzes were very well accepted by the students. There is, however, a greater variation across the spectrum. Keeping in mind that most of the students undertook the first semester units with the online quizzes, the variation is most likely due to the fact that the initial honeymoon period of online quizzes has worn off. Some students realised that they had to provide an effort in order to keep up with the regular quizzes, and for a few of these, the quizzes were viewed negatively.
The open ended feedback was similar to the first semester feedback, indicating student appreciation for the flexibility, feedback and motivation that the quizzes offered. Students provided useful constructive feedback requesting more targeted questions, more information on the quiz structure and method of grading, and more specific information indicating what contents were targeted by the quizzes.
The second semester survey also requested feedback from the students regarding the virtual field trip. Because of the small number of distance students, all students were surveyed and the results aggregated across both on campus and distance groups. The students were surveyed regarding the usefulness of the role of the virtual field trip in the unit, the usefulness of the images displayed and the relevance to the unit. The results are indicated in Figure 7.
Figure 7: Results of student survey regarding virtual field trips
Overall, students found the virtual field trip very useful and helpful. Via informal feedback, it was ascertained that the on campus students, for obvious reasons, did not find the virtual field trip as useful as the distance students. The open ended comments received indicated that "The virtual field trip really helped to understand the environment; it almost felt like that I was there" and "They are excellent; especially for rural students for something to show your parents" (!).
The virtual field trip was especially useful for the distance students who had no opportunity, as did the on campus students, to attend the real field trip. These students were able to grasp the concepts and utilise that data that was captured in order to complete their assigned work. The on campus students, since they were physically present at the field trip, found it less useful but did appreciate the information which served to remind them of the activities they undertook.
The project clearly indicated the success of online quizzes and field trips in providing flexible assessment to students. While the methodologies for implementing them require further development, focus and refinement, the online assessment tools will continue to be used, certainly within the GIScience program at Curtin University. Online assessment is an important component in the continuing shift to student centred learning and flexible learning environments that are being embraced in the higher educational sector.
Forer, Pip (1997). Flexible Delivery and Social Learning: Seeking a new geography of education for GIS and GIS in education. Transactions in GIS, 2:2, Pearson Professional Limited.
Kemp, Karen K. and David J. Unwin (1997). From Geographic Information Systems to Geographic Information Studies: An agenda for educators. Transactions in GIS, 2:2, Pearson Professional Limited.
Thomas, Diana (1995). Learning to be Flexible. In Flexible Learning Strategies in Higher and Further Education, Edited by Diana Thomas, Cassell Education, London.
Veenendaal, Bert (1999). Cyber-learning for GIS: assistant or replacement? Conference Proceedings of AURISA 99, Australasian Urban and Regional Information Systems Association, 22-26 November, Fairmont Resort, Blue Mountains, NSW, Australia, ISBN 0 9586233 1 7.
Veenendaal, Bert (2000). Flexible and web-based delivery of GIScience in higher education. Proceedings of the International IT Conference on Geo-Spatial Education, Hong Kong, 6-8 July.
WebCT (2000). WebCT.com: The e-learning hub. http://www.webct.com/ [accessed 12 December 2000]
Winship, John and David Carter (1999). Alternative Delivery Modes of Teaching and Learning Using New Information Technology. A Strategic Initiative Project, Office of Teaching and Learning, Curtin University of Technology, July.
|Please cite as: Veenendaal, B. (2001). Flexible assessment in GIScience education. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2001/veenendaal.html|