Teaching and Learning Forum 2001 Home Page
[ Teaching and Learning Forum 2001 ] [ Proceedings Contents ]

Flexible assessment in GIScience education

Bert Veenendaal
Department of Spatial Sciences
Curtin University of Technology

Introduction

The geographic information science (GIScience) program at Curtin University of Technology has been in existence for over 12 years and provides a range of offerings at both the undergraduate and postgraduate levels. Over the past two years, the Internet and the WebCT teaching and learning resources (see WebCT 2000, Curtin WebCT 2000) have been developed to the point where a Graduate Certificate could be available in 2000 via distance learning. Initially, the Internet was utilised for electronic communication, distribution of unit outlines, and placing lecture notes online. However, the online resources have been expanded considerably to include a study plan, study guide, work guide, etc. that can be utilised in distance studies.

A core component of the geographic information science program is the development of student skills using practical sessions in computer based laboratories. The practical sessions complement the underlying concepts and foundational knowledge that has traditionally been the domain of the lecture (Thomas 1995). It is necessary and important for students to develop their knowledge and skills in tandem by maintaining a consistent study program across both the practical sessions and the lectures. This has been reflected in the assessment procedures involving laboratory sessions, assignments, mid-semester tests and final examinations.

Over the past decade, there has been very little change in the assessment procedures other than minor variations. Yet, the main issues regarding assessment still remain, namely, providing timely and prompt feedback to students, and using assessment to motivate students to keep up to date with their study program. Most of the existing assessment takes place in the latter half of the semester. Hence, students don't receive their first assessed work and feedback until well into the unit. Further, in the case of the practical sessions, it is difficult to monitor the students' progress on a weekly basis unless considerably more resources are utilised to grade and return feedback more frequently. The Internet and the online learning environment have opened up new and exciting opportunities to address these issues (Forer 1997, Kemp and Unwin 1997, Veenendaal 1999, Veenendaal 2000, Winship and Carter 1999).

The aim of this project was to develop and evaluate online assessment procedures. The specific objectives were to:

In particular, a number of online quizzes were developed for three (3) units in the GIScience program, and a number of virtual field trips were designed for one of these units. This paper discusses the methodology used for the design and evaluation, and describes the results obtained from student surveys and feedback.

Methodology used for online assessment

The aim of online assessment is to provide students with a flexible means of receiving timely feedback as well as motivating them in maintaining a consistent and up to date study program. Within the GIScience program, it was important to cater, not only for on campus students, but also for distance students who don't have access to the campus and study remotely, using the Internet as the primary medium. For this project, this was achieved using online quizzes and virtual field trips.

Online quizzes

As part of the assessment, it was decided to use online quizzes on a regular basis. The quizzes would test the students on both their content knowledge and practical skills (ie. the lectures and practical sessions for on campus students; distance students utilised the online Study Guide and Work Guide resources). By holding the quizzes regularly and releasing the results after the submission date, students would be able to obtain feedback fairly early on in the unit and thereafter on a regular basis. Further, the regularity of the quizzes would prompt the students to maintain their study program (knowledge and skills) since they would have to be prepared for the next quiz.

The online quizzes were trialed in the first semester for the Geographic Information Systems (GIS) first year unit and the Spatial Analysis (SA) third year unit. In order to test the appropriate procedures for delivering the assessment, a different approach was taken for each unit. For the GIS unit, a quiz was provided every fortnight on average, each quiz being approximately 30 minutes in duration. For the SA unit, the frequency and duration of the quizzes were altered, being held every 4 weeks and 40 minutes in duration.

In both cases, the quizzes were available to the students for a limited one (1) week window. The students could commence the quiz, available from the Internet, anytime within this window, although once they had commenced the quiz, it had to be completed in the given time. Only one attempt was provided, hence it was imperative that the students were prepared prior to completing the quiz. The quizzes were comprised of a series of multiple choice, matching, and calculation questions that could be graded automatically to facilitate a fast response to students, particularly for large classes. A sample of the first part of a quiz indicating the response to the student is illustrated in Figure 1.

Figure 1

Figure 1: Sample of an online quiz and feedback

Once all students had submitted their quiz, the results were released to students. This occurred prior to the following quiz being available. The students could utilise the feedback to assist them in understanding their strengths and weaknesses, and preparing them for the following quiz.

Although the same quizzes were initially used for both on campus and distance students, it was soon realised that this was not possible when there were variations in the practical session work (primarily). It was decided fairly early on to develop separate targeted quizzes for on campus and distance students. In terms of the implementation of this, separate Online Quiz links had to be generated for the on campus and distance students within the unit web site. In addition, an additional link of Deferred Quiz was made available for both on campus and distance students who had a valid reason for receiving a deferred quiz. This option was used quite frequently by the distance students who had valid reasons for deferring their quiz based on other commitments.

The students in both classes were surveyed at the end of the semester (using an online survey) regarding the usefulness and procedures used for the online quizzes. The feedback was used to implement a series of quizzes for the Geographic Data Analysis (GDA) unit in the second semester. Based on this feedback, quizzes were held every fortnight and were restricted to 30 minutes in duration. On campus students were given a one week window to complete the quiz, whereas distance students were given two weeks. The reason for this was to provide flexibility for the distance students who were often inundated with other work and travel commitments. The results were then released after the two week window was closed.

The second semester quizzes involved random questions where students would not all receive the same set of questions. This measure was taken to reduce and avoid the temptation for students to collaborate with each other regarding solutions. Emphasis was placed on the need for students to take responsibility in their own understanding of the material and developing their own skills rather than relying on others. Overall, the online quizzes accounted for no more than 10% of the final assessment, again to avoid any unnecessary collaboration simply to achieve the maximum possible marks.

The online student survey was again conducted at the end of the second semester to obtain student perceptions regarding the usefulness and organisation of the online quizzes.

Virtual field trips

In addition to the online quizzes, virtual field trips were designed to cater primarily for distance students. Because the distance students are not able to attend the field trips as the on campus students do, it was decided to utilise online resources to provide distance students with a much information as is possible concerning the environment and procedures used in the field trips. The GDA unit in the second semester was targeted for this purpose. Two field trips were undertaken as part of this unit, one involving the sampling of vegetation and the collection of related data, and the other involving navigation using Global Positioning System (GPS) receivers. The students were to complete an assignment based on the results of the field trips.

To create a virtual field trip, a video was taken of the field trip activities of the on campus students. Still images were then obtained from this video, placed online, and made available to on campus and distance students. Approximately 30 to 50 images were utilised for each field trip. By placing the field trip "online", the distance students could "see" the physical environment as well as "follow" the students performing the activities required for the field trip. The data obtained by the on campus students was provided to the distance students who could then complete the same assessed work as the on campus students. Figure 2 illustrates a sample of part of the online virtual field trip.

Figure 2

Figure 2: Sample of an online virtual field trip

Results of utilising flexible assessment

The results of the student surveys showing overwhelming support for the online quizzes and virtual field trips. For the first semester, the breakdown of students across the various programs of study (Bachelor of Science, Graduate Diploma, Graduate Certificate, other degree programs and Extension studies) is illustrated in Figure 3. The majority are undergraduate and graduate GIScience students. Approximately 10% of the 110 students were distance students.

Figure 3

Figure 3: Semester 1 distribution of students based on program of study

The students were surveyed regarding the value of the quizzes, the feedback that they provided, the extent to which the quizzes were a source of motivation, the marking scheme utilised and how this unit compared with others. A scale of 1, representing "poor", to 5, representing "excellent" was used. The results, as illustrated in Figure 4, indicate that the online quizzes were highly accepted and viewed by the students as an excellent source of feedback and motivation. The comparison to other units indicates that this was probably the only instance in which students participated in online assessment.

Figure 4

Figure 4: Results of Semester 1 student survey

Students were also able to provide open ended comments as part of the survey. The following is a sample of responses:

"Online assessment provides flexibility within a set time and access from home or work, freedom in own time management"

"Please convince others to use web and online quizzes for units"

"It would be good to have a quiz after each lecture"

"Quizzes provide good motivation to keep up to date with studies, indicates is studying on track, highlights weaknesses"

"I found the feedback really useful and it will be great as a study tool for the exam"

"Ongoing assessment such as quizzes very good to keep the learning curve going. Gets learning happening early!"

Overall, the students enjoyed the flexibility of the quizzes and the value that it added to their study program. They did prefer the shorter but more frequent quizzes. This was particularly true for the SA students who experienced the longer but less frequent quizzes. This was therefore put into place for the second semester trial.

The distribution of the second semester students and the associated survey results are indicated in Figures 5 and 6. Of 55 students in the GDA unit, 20% were distance students. There was a greater proportion of students in GIScience students and fewer from other degree programs.

Figure 5

Figure 5: Semester 1 distribution of students based on program of study

Figure 6

Figure 6: Results of Semester 2 student survey

The results of the second semester survey, as for the first semester, showed that the quizzes were very well accepted by the students. There is, however, a greater variation across the spectrum. Keeping in mind that most of the students undertook the first semester units with the online quizzes, the variation is most likely due to the fact that the initial honeymoon period of online quizzes has worn off. Some students realised that they had to provide an effort in order to keep up with the regular quizzes, and for a few of these, the quizzes were viewed negatively.

The open ended feedback was similar to the first semester feedback, indicating student appreciation for the flexibility, feedback and motivation that the quizzes offered. Students provided useful constructive feedback requesting more targeted questions, more information on the quiz structure and method of grading, and more specific information indicating what contents were targeted by the quizzes.

The second semester survey also requested feedback from the students regarding the virtual field trip. Because of the small number of distance students, all students were surveyed and the results aggregated across both on campus and distance groups. The students were surveyed regarding the usefulness of the role of the virtual field trip in the unit, the usefulness of the images displayed and the relevance to the unit. The results are indicated in Figure 7.

Figure 7

Figure 7: Results of student survey regarding virtual field trips

Overall, students found the virtual field trip very useful and helpful. Via informal feedback, it was ascertained that the on campus students, for obvious reasons, did not find the virtual field trip as useful as the distance students. The open ended comments received indicated that "The virtual field trip really helped to understand the environment; it almost felt like that I was there" and "They are excellent; especially for rural students for something to show your parents" (!).

Conclusions

The use of online quizzes and virtual field trips in providing assessment within the GIScience program, was very well received by the students. The flexibility afforded to students in terms of when and where they could access the online quizzes was very much appreciated. The quizzes provided earlier and regular feedback to students as well as an incentive to motivate students in keeping up with their study program. From the student feedback, it was ascertained that the quizzes should maintain a very targeted focus and should be relatively short but frequent.

The virtual field trip was especially useful for the distance students who had no opportunity, as did the on campus students, to attend the real field trip. These students were able to grasp the concepts and utilise that data that was captured in order to complete their assigned work. The on campus students, since they were physically present at the field trip, found it less useful but did appreciate the information which served to remind them of the activities they undertook.

The project clearly indicated the success of online quizzes and field trips in providing flexible assessment to students. While the methodologies for implementing them require further development, focus and refinement, the online assessment tools will continue to be used, certainly within the GIScience program at Curtin University. Online assessment is an important component in the continuing shift to student centred learning and flexible learning environments that are being embraced in the higher educational sector.

Acknowledgment

This work was funded through the competitive LEAP (Learning Effectiveness Alliance Program) 2000 Grant in the Division of Engineering and Science at Curtin University of Technology.

References

Curtin WebCT (2000). WebCT Environment at Curtin University of Technology. http://webct.curtin.edu.au/ [accessed 12 December 2000]

Forer, Pip (1997). Flexible Delivery and Social Learning: Seeking a new geography of education for GIS and GIS in education. Transactions in GIS, 2:2, Pearson Professional Limited.

Kemp, Karen K. and David J. Unwin (1997). From Geographic Information Systems to Geographic Information Studies: An agenda for educators. Transactions in GIS, 2:2, Pearson Professional Limited.

Thomas, Diana (1995). Learning to be Flexible. In Flexible Learning Strategies in Higher and Further Education, Edited by Diana Thomas, Cassell Education, London.

Veenendaal, Bert (1999). Cyber-learning for GIS: assistant or replacement? Conference Proceedings of AURISA 99, Australasian Urban and Regional Information Systems Association, 22-26 November, Fairmont Resort, Blue Mountains, NSW, Australia, ISBN 0 9586233 1 7.

Veenendaal, Bert (2000). Flexible and web-based delivery of GIScience in higher education. Proceedings of the International IT Conference on Geo-Spatial Education, Hong Kong, 6-8 July.

WebCT (2000). WebCT.com: The e-learning hub. http://www.webct.com/ [accessed 12 December 2000]

Winship, John and David Carter (1999). Alternative Delivery Modes of Teaching and Learning Using New Information Technology. A Strategic Initiative Project, Office of Teaching and Learning, Curtin University of Technology, July.

Please cite as: Veenendaal, B. (2001). Flexible assessment in GIScience education. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2001/veenendaal.html


[ Abstract for this article ] [ TL Forum 2001 Proceedings Contents ] [ All Abstracts ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf2001/veenendaal.html
Last revision: Feb 2002. Curtin University of Technology
Previous URL 22 Dec 2000 to Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2001/veenendaal.html