Teaching and Learning Forum 2001 Home Page
[ Teaching and Learning Forum 2001 ] [ Proceedings Contents ]

Communication skills for online students: An evaluation of a website

Maria Northcote
Kurongkurl Katitjin, School of Indigenous Australian Studies
Edith Cowan University

Amanda Kendle
Education Centre, Faculty of Medicine and Dentistry
The University of Western Australia


Background

With the growing prevalence of online education, students' academic success is being increasingly determined by their ability to communicate and operate effectively in the online environment. While the range of communication skills inherent in face to face teaching are well recognised and often specifically addressed as study skills within many courses, the unique skills relevant to online communication have been somewhat neglected. The mechanical skills such as those involving the surface use of email or bulletin boards are often explained to students who are new to the online environment by university technical or administrative staff. Additionally, procedures for the use of these online communication tools are frequently included in courseware instruction documents provided to students at the beginning of the academic year or semester. However, more specific instruction about appropriate ways to communicate online, including suitable language, sentence structure and use of abbreviations, is less common. Generally, this deeper knowledge and familiarity with online communication protocols is what is required by contemporary students who are either enrolled in online courses or use online resources as part of face to face or external classes.

On a more specific level, how effectively a learner is able to communicate online can be an important influence on their general study techniques and processes. Online communication skills can also impact on how students' assignments are completed and the depth of their learning experience in general. Despite this inherent need for online communication skills, Kearsley reports that "many students and teachers have little experience with online learning/teaching and find themselves uncomfortable with the whole idea" (1997, p. 1). Teaching staff and students require some form of instruction or presentation of exemplars and communication samples to further understand, use and become skilled in the new style of online discourse. As universities continue to increase their online unit offerings (Bauer et al, 1999), the online communication skills of educators and students will become more essential to teach and learn in the context of Internet education.

The website, "Communication Skills for Online Students", attempts to address this skill disparity by presenting fundamental information, authentic examples and practical activities in a stand alone module which is easy to reach within the online environment. At a technical level, the website is highly accessible. No passwords, plugins or special software applications are required to view or operate this online module. From a pedagogical standpoint, the development of the website was directed by such instructional design issues as the analysis of predicted learners, the current online context, navigational structures and the inclusion of supporting media. Such considerations took into account the users' technical needs and learning requirements.

Overarching theoretical framework

The website was designed by two instructional designers who were determined to create a practical resource that was supported by sound instructional design principles, the details of which are mentioned above. The broader educational theories of situated cognition and constructivism were the two main guiding pedagogical influences on this project in conjunction with a widespread recognition of the growing importance of online communication skills for tertiary students. Recent research supports the case for specific training in online communication skills (Cuskelly, Danaher, & Purnell, 1997). In fact, Tornow (1997) reports that new conventions of literacy are evolving and distinctions between various types of communication are becoming more blurred. Davis and Brewer (1997) further suggest that the characteristics of online "conversations" are quite unique to the online environment. A recognition of current student demands along with recent supporting research provided the development of this website with a systematic development strategy.

In line with traditional and recent instructional design models (Dick & Carey, Smith & Ragan, Leshin, Pollock & Reigeluth, cited in Gustafson & Branch, 1997), the analysis stage of the online module began by taking into account the predicted target audience. These users were expected to be tertiary students with some level of computer literacy skills but with a lack of experience communicating online, especially at the level required to become a successful online student. Furthermore, the instructional design of the website was based on a non-linear structure. This structure was adopted to enable the creation of a resource which users from a variety of backgrounds could access either during a scheduled on campus class or in an entirely self directed manner. By recognising the "pedagogical benefits of hypermedia and multimedia to be that they allow for a tighter integration of subject material, learning activities and assessment" (Benyon et al., cited in Evans & Edwards, 1999, p. 152), the website was based on a networked organisational structure which allows it to be approached by users who may only require interaction with components of the content rather than being required to work through a step by step process. This preliminary consideration of user needs indicated that a friendly, non formal interface was best suited to this resource in order to reduce learner apprehension and to increase user enjoyment.

The website utilised constructivist principles to guide the presentation of content, the provision of learning activities and feedback. Constructivism focuses on the importance of the student in the learning process and stresses the necessity of active learning (Fineman & Bootz, 1995). It recognises an individual's prior experience and emphasises the importance of active student involvement in the ongoing learning process:

The teacher interacts with the learner in line with the assumption that learning involves active construction of meaning by the student and is not something that is imparted by the teacher. (Biggs & Moore, 1993, p. 25)
By presenting students with a variety of resources in this website, their learning patterns are able to directly reflect knowledge construction processes incorporating generative learning activities instead of simple knowledge interpretation (Wittrock, 1991). An online learning environment based on communication and choice also further promotes a socio-cognitive atmosphere (Fetherston, 1998), one where meaning is negotiated, challenged and retained in authentic contexts. These aspects of online learning have been incorporated into this website.

In addition to this constructivist theoretical foundation, the principles of situated cognition were also utilised. Situated cognition is based on the premise that individuals learn best in authentic contexts. By presenting tasks and examples set in real "live" settings using active online contexts, the website offers a resource which adopts these principles of situated cognition by emphasising the links between learning and cognition (Brown, Collins & Duguid, 1989). Oliver, Herrington and Omari describe situated learning as being "based heavily on constructivist learning principles which encourage learners to construct their own meaning for knowledge and information in the learning process" as well as recognising "the importance of interaction and socialisation among learners as a critical element in the learning process" (1998, p. 2). This website has attempted to create an environment where users can create their own knowledge against a backdrop of social interaction.

The website is driven by learning activities and each activity is based on online specific issues such as netiquette (online communication etiquette) and the appropriate use of email, bulletin boards, chat and online forms. In this way, the content presentation method used is complementary and appropriate to the actual content represented by the topics and themes in the website.

Evaluation methodologies and results

Despite the recent proliferation of multimedia and website resource development, "systematic evaluation of the implementation and efficacy of these systems is sadly lacking" (Reeves, 1997b, p. 14). Taking heed of this advice, the process involved in creating the "Communication Skills for Online Students" website was planned and implemented with evaluation methodology in mind. After an early trial period, the website was evaluated from two angles: from the users' perspective and from the perspective of a number of expert educators. Instead of just relying on one perception of this online resource, a variety of methodologies were examined and combined to develop a flexible evaluation tool, taking into account the advice of Mulholland, Au and White:
Traditional evaluation methodologies used in courseware evaluation have been criticised for inconsistency, inappropriateness and lack of methodological validity ... A more thorough understanding of the existing courseware methodologies will allow educators to choose discerningly. (1998, p.1)
To ensure that our evaluation was as valid, consistent and appropriate as possible, users' feedback and expert feedback was sought.

Evaluation by users from "within" the website

The website itself contains three forms of evaluation. The users are encouraged to use evaluation tools to provide the website creators with feedback about how the site was used. At various strategic sections, the website creators' email addresses are presented as active hyperlinks which users can activate to submit their anecdotal comments. Secondly, five different evaluation forms are situated throughout the website in context specific locations. As well as including a form which requests users to record their constructive feedback comments about the website as a whole, individual evaluation forms are also provided within each of the four main content components of the module: email, bulletin boards, chat and online forms. Lastly, a component of one of the learning activities in the email section requests users to forward an email to one of the website authors which includes anecdotal evaluative comments about the website in general.

As a result of the initial trial of the website (which began in September 2000), the use of these intra-website evaluation tools has been minimal. The website has been designed in a non-linear manner to provide an educational resource which can be "dipped into" at the user's leisure. It is thought that this may have contributed to the fact that the online forms were not as successful as predicted for obtaining user feedback in that users were not accessing all the sections of the website and therefore missing the forms.

Despite the lack of direct feedback from users of the site via the online feedback forms, verbal and anecdotal comments have indicated that the site is "easy to understand", "well structured", "simple" and "easy to follow". Other users described it as "great", "informative", "easy to use" and "comprehensive". One user noted that "I even learnt things I didn't know about before and very easily, too". These comments indicate that aspects of the site's four main objectives were met (see Table 1 below). Users appear to be able to use the site easily to gain information (objectives 1 and 2) and there do not seem to be any comments relating to users being stressed or confused about the site or the use of online communication skills (objective 4). Perhaps the only objective that isn't reflected in the anecdotal comments is the one relating to practising the online communication skills, the evaluation of which may be best done in a second round of evaluation procedures. To date, less positive evaluative comments have indicated that a small number of users have experienced difficulties with browser clashes relating to the use of Netscape Navigator as opposed to Microsoft Explorer.

Table 1: Student objectives for Communication Skills for Online Students module

By working through the activities in the online module, the students will be able to:
1Understand the main components of online communication (email, chat, bulletin boards and forms).Knowledge
2Understand and recognise the various forms, styles and purposes of online communication.Knowledge
3Communicate effectively in an online context using email, chat, bulletin boards and forms.Skills
4Reduce anxiety and increase confidence about using computers and communicating onlineAttitudes

Expert evaluation from "outside" the website

To complement the evaluation results collected from students and users using tools located within the website, a number of instructional design and tertiary education experts were enlisted to complete an evaluation tool which focused on the quality of the website in terms of its pedagogical base. This evaluation process took place "outside" the website in that it involved experts examining it from a more detached position compared to users who were immersed in the activities while completing the evaluation tools.

Overall, the evaluation tool created for the purposes of evaluating this website incorporated a mixture of checklists, opportunities for short anecdotal comments as well as responses to open ended questions. The choice of evaluation tools was directed by the theoretical framework which guided the development of the website. The expert evaluation tool was developed by combining elements of the courseware evaluation tools used by Reeves (1997) and Nicholson (1997), and Herrington, Sparrow & Herrington's (2000) instructional design guidelines.

Reeves' (1997a, 1997b) pedagogical and interface dimensions have been widely applied to multimedia materials in all levels of education across many discipline areas. They are steeped in constructivist and situated cognition learning theories and were selected as the central evaluation tools. They have been recognised and utilised as valuable, valid tools for the evaluation of interactive multimedia programs, courses and materials. Furthermore, Reeves' evaluation tools are especially relevant to educators as they serve to assess the worth of educational materials in terms of the principles of instructional design, learning and education technology. Their popularity and integrity can be traced to the fact that they recognise issues associated with both learning theory and technical efficiency.

Since the website was centred around authentic learning activities, Herrington's (2000) guidelines for writing learning activities, which are particularly relevant to situated cognition, were used to guide the website's design. Aspects of Nicholson's CERT (A Courseware Evaluation and Review Tool) were especially appropriate to assess the achievement of the website's objectives which were designed to develop the students' knowledge, skills and attitudes about online communication.

The expert evaluation process is still underway, the results of which will be discussed during the presentation of this conference paper.

Conclusion

The results of the two above mentioned evaluation processes (by users and experts), will be analysed and considered in light of the development of the second version of the website. By gaining feedback from learners and educators alike, a collection of suggested modifications, additions and deletions will be compiled. Such changes will inform the specific future development of the website in terms of navigation, graphic design, content structure and interactivity levels. A detailed explanation of the evaluation process will be presented at the conference, the results of which will be linked to the proposed modifications which are planned to be implemented into the next version of the website.

References

Bauer, C., Berkhout, J, Chang, V., Chin, K.L., Glasson, B., & Tauber, J. (1999). Exploring online education: A research framework. In K. Martin, N. Stanley & N. Davison (Eds.), Teaching in the disciplines/ Learning in context, 28-34. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth, Western Australia: UWA. http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/ac/bauer.html

Biggs, J., & Moore, P. (1993). The process of learning (3rd ed.). Sydney: Prentice Hall.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-41.

Cuskelly, E., Danaher, P., & Purnell, K (1997). Just which technology do distance students really want? Results of focus group research. In Osborne et al (Eds.), Open, flexible and distance learning: Education and training in the 21st century, 88-93. Paper presented at the 13th Biennial Forum of ODLAA, Launceston, Tasmania.

Davis, B. H., & Brewer, J. P. (1997). Electronic discourse: Linguistic individuals in virtual space. Albany, NY: State University of New York Press.

Evans, C., & Edwards, M. (1999). Navigational interface design for multimedia courseware. Journal of Computers in Mathematics and Science Teaching, 8(2), 151-174.

Fetherston, T. (1998). A socio-cognitive framework for researching learning with IMM. Australian Journal of Educational Technology, 14(2), 98-106. http://cleo.murdoch.edu.au/ajet/ajet14/fetherston.html

Fineman, E., & Bootz, S. (1995). An introduction to constructivism in instructional design. University of Texas. [verified 8 Jan 2001] http://www.coe.uh.edu/insite/elec_pub/html1995/197.htm

Herrington, J., Sparrow, H., & Herrington, A. (2000). Instructional design guidelines for authentic activity in online learning units. Paper presented at EdMedia 2000 Conference.

Kearsley, G. (1997). A guide to online education. Florida: Nova Southeastern University. [viewed 21 May 2000, verified 8 Jan 2001] http://gwis.circ.gwu.edu/~etl/online.html

Mulholland, C., Au, W., & White, B. (1998). Courseware evaluation methodologies: Strengths, weaknesses and future directions. ACEC98 Conference Proceedings, University of South Australia. [verified 8 Jan 2001] http://www.cegsa.sa.edu.au/acec98/papers/p_mulh1.html

Oliver, R., Omari, A. & Herrington, J. (1998). Developing converged learning environments for on and off-campus students using the WWW. In R. Corderoy (Ed.), Conference Proceedings: ASCILITE '98 (pp. 529-538). Wollongong, Australia: University of Wollongong. [verified 8 Jan 2001] http://www.ascilite.org.au/conferences/wollongong98/asc98-pdf/oliver0051.pdf

Reeves, T. C. (1997a). A model of the effective dimensions of interactive learning on the World Wide Web. Proceedings of Interaktiivinen Koulutuksessa (ITK '97) (pp. 23-31). Hameenlinna, Finland.

Reeves, T. C. (1997b). Evaluating what really matters in computer-based education. [verified 8 Jan 2001] http://www.educationau.edu.au/archives/cp/reeves.htm

Tornow, J. (1997). Link/age: Composing in the online classroom. Logan, UT: Utah State University Press.

Wittrock, M. (1991). Generative teaching of comprehension. Elementary School Journal, 92, 169-184.

Please cite as: Northcote, M. and Kendle, A. (2001). Communication skills for online students: An evaluation of a website. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2001/northcote1.html


[ Abstract for this article ] [ TL Forum 2001 Proceedings Contents ] [ All Abstracts ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf2001/northcote1.html
Last revision: 8 Feb 2002. Curtin University of Technology
Previous URL 9 Jan 2001 to 8 Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2001/northcote1.html